c - How to handle buffering serial data

Question

I am trying to figure out a nice solution to reading serial data, and what to do when a read() is done but it contains an incomplete message.

The expected messages between devices have a defined start and end byte so its easy to see when a message starts and ends.

I can open a serial port fine and read from the serial port. But I am encountering the computer is reading faster than data coming through and I get an incomplete message.

For this example, lets say the message expected is

0x10 0xFF 0xFF 0xFF 0xFF 0x11

With 0x10 the start, 0x11 the end, and 0xFF is the data bytes

I am new to C so I may be missing something obvious, My current solution

int main() {
     /* Ommited serial port opening and checking*/
     char read_buffer[80];
     char message_buffer[80];
     int message_buffer_index = 0;
     int start_index = -1;
     int end_index = -1;
     int read_bytes;
     read_bytes = read(serial_port, read_buffer, sizeof(read_buffer) - 1);
     /* Now lets say read_bytes returns 3 and read buffer is {0x10, 0xFF, 0xFF} */
     /* What should I do with the read_buffer? Currently appending to message buffer*/
     memcpy(&message_buffer[message_buffer_index], &read_buffer[0], read_bytes);
     /* Now check the message buffer for a full message */
     for (int i = 0; i < 80; i++) {
          if (message_buffer[i] = 0x10) {
               start_index = i;
               continue;
          }
          if (message_buffer[i] = 0x11) {
               end_index = i;
          }
          if (start_index != -1 && end_index != -1) {
               /* Found a message, do something with it, not super important here */
               process_message();
               /* Now how to erase the full message from the 
               buffer and push any non processed data to the 
               front? */
               remove_message();
          }
    }
}

int process_message();  
int remove_message();

Answer 1

To minimize the overhead of making many read() syscalls of small byte counts (e.g. the misguided solution of reading a byte at a time), use an intermediate buffer in your code.
The read() of the serial terminal should be in blocking mode to avoid a return code of zero bytes.

#define BLEN    1024
unsigned char rbuf[BLEN];
unsigned char *rp = &rbuf[BLEN];
int bufcnt = 0;

/* get a byte from intermediate buffer of serial terminal */
static unsigned char getbyte(void)
{
    if ((rp - rbuf) >= bufcnt) {
        /* buffer needs refill */
        bufcnt = read(fd, rbuf, BLEN);
        if (bufcnt <= 0) {
            /* report error, then abort */
        }
        rp = rbuf;
    }
    return *rp++;
}

For proper termios initialization code for the serial terminal, see this answer. You should increase the VMIN parameter to something closer to the BLEN value or at least the length of longest expected message, and a VTIME of 1.

Now you can conveniently access the received data a byte at a time with minimal performance penalty.

#define MLEN    1024  /* choose appropriate value for message protocol */
int main() 
{
    unsigned char mesg[MLEN];
    ...

    while (1) {
        while (getbyte() != 0x10)
            /* discard data until start found */ ;
    
        length = 0;
        while ((mesg[length] = getbyte()) != 0x11) {
            /* accumulate data until end found */ 
            length++;
        }

        /* process the message */
        ...

    
    }  /* loop for next message */
...
}

Note that your definition for a message frame is not robust.
If the data is binary and therefore can use the same values as the start and end bytes, then this parsing of the received data is prone to misaligned message frames.