sockets - Erlang server, Java client - TCP messages get split?

Question

As the title says, I have a server written in Erlang, a client written in Java and they are communicating through TCP. The problem that I am facing is the fact that gen_tcp:recv apparently has no knowledge of when a "complete" message from the client has been received, and is therefore "splitting" it up in multiple messages.

This is an example of what I'm doing (Incomplete code, trying to keep it to only the relevant parts):

Code

Erlang server

-module(server).
-export([start/1]).

-define(TCP_OPTIONS, [list, {packet, 0}, {active, false}, {reuseaddr, true}].

start(Port) ->
   {ok, ListenSocket} = gen_tcp:listen(Port, ?TCP_OPTIONS),
   accept(ListenSocket).

accept(ListenSocket) ->
    {ok, Socket} = gen_tcp:accept(ListenSocket),
    spawn(fun() -> loop(Socket) end),
    accept(ListenSocket).

loop(Socket) ->
    case gen_tcp:recv(Socket, 0) of
        {ok, Data} ->
            io:format("Recieved: ~s~n", [Data]),
            loop(Socket);
        {error, closed} ->
            ok
    end.

Java client

public class Client {
    public static void main(String[] args) {
        Socket connection = new Socket("localhost", Port);
        DataOutputStream output = new DataOutputStream(connection.getOutputStream());
        Scanner sc = new Scanner(System.in);

        while(true) {
            output.writeBytes(sc.nextLine());
        }
    }
}

Result

Client

Hello!

Server

Received: H
Received: el
Received: lo!

I have been searching around and if I understand it correctly, TCP has no knowledge of the size of messages, and you need to manually set some kind of delimiter.

What I don't get though, is that the messages never seem to split up if I write the client in Erlang instead, like this:

Erlang client

-module(client).
-export([start/1]).

start(Port) ->
    {ok, Socket} = gen_tcp:connect({127,0,0,1}, Port, []),
    loop(Socket).

loop(Socket) ->
    gen_tcp:send(Socket, io:get_line("> ")),
    loop(Socket).

Result

Client

Hello!

Server

Received: Hello!

This makes me wonder if it is something that can be fixed on the Java side? I have tried several combinations of different output streams, write methods and socket settings on the server side, but nothing solves the problem.

Also, there are loads of Erlang (chat) server examples around the net where they don't do any delimiter things, although those are often written in Erlang on both ends. Nevertheless, they seem to assume that the messages are received just like they are sent. Is that just bad practice, or is there some hidden information about message length when both the client and server are written in Erlang?

If delimiter checks are necessary, I am surprised I can't find much information on the subject. How can it be done in a practical way?

Thanks in advance!

Answer 1

This makes me wonder if it is something that can be fixed on the Java side?

No, absolutely not. Regardless of why you don't happen to see the problem with an Erlang client, if you aren't putting any sort of "message boundary" indication into the protocol, you will not be able to reliably detect whole messages. I strongly suspect that if you send a very large message with the Erlang client, you'll still see split messages.

You should either:

• Use some sort of "end of message" sequence, e.g. a 0 byte if that wouldn't otherwise come up in your messages.
• Prefix each message with the length of the message.

Aside from that, you aren't clearly differentiating between bytes and text at the moment. Your Java client is currently silently ignoring the top 8 bits of each char, for example. Rather than using DataOutputStream, I would suggest just using OutputStream, and then for each message:

• Encode it as a byte array using a specific encoding, e.g.

  byte[] encodedText = text.getBytes(StandardCharsets.UTF_8);
  

• Write a length prefix to the stream (possibly in a 7-bit-encoded integer, or maybe just as a fixed width, e.g. 4 bytes). (Actually, sticking with DataOutputStream would make this bit simpler.)

• Write the data

On the server side, you should "read a message" by reading the length, then reading the specified number of bytes.

You can't get around the fact that TCP is a stream-based protocol. If you want a message-based protocol, you really do have to put that on top yourself. (I'm sure there are helpful libraries to do this, of course - but you shouldn't just leave it up to TCP and hope.)