arduino - Maximizing serial communication speed for live plotting data from Teensy 3.2 using Python

Question

I'm trying to plot data as quickly as possible with Python (PyQtGraph) received from a Teensy 3.2 which is sending analog data over a serial communication. The code can sufficiently plot a test waveform of higher frequencies (sine wave of about 5kHz), but it takes nearly 30 seconds for the plot to show a change in frequency. For example, if the test waveform is turned off, it continues to plot the sine wave for an additional half minute.

I've tried performing a "serial flush" to clear out the buffer on both the Python side and Teensy side, however, that seriously slows down the plot and the frequency response of my plot goes down to single hertz.

Python (Plotting) Side:

# Import libraries
from numpy import *
from pyqtgraph.Qt import QtGui, QtCore
import pyqtgraph as pg
import serial
import re

# Create object serial port
portName = "COM8"                      
baudrate = 115200 
ser = serial.Serial(portName,baudrate)

### START QtApp #####
app = QtGui.QApplication([])           
####################

win = pg.GraphicsWindow(title="Signal from serial port") # creates a window
p = win.addPlot(title="Realtime plot")  # creates empty space for the plot in the window
curve = p.plot()                        # create an empty "plot" (a curve to plot)
windowWidth = 100                       # width of the window displaying the curve - this is the time scale of the plot
Xm = linspace(0,0,windowWidth)          # create array of zeros that is the size of the window width   
ptr = -windowWidth                      # set first x position

# Realtime data plot. Each time this function is called, the data display is updated
def update():

    global curve, ptr, Xm    
    Xm[:-1] = Xm[1:]                    # shift data in the temporal mean 1 sample left 
    if ser.isOpen():                    # make sure there is data coming in
        b1 = ser.read(1)                # read the first byte of data
        b2 = ser.read(1)                # read the second byte of data
        data = b1 + b2                  # concatenate the two bytes
        data_int = int.from_bytes(data, byteorder='big')

        Xm[-1] = data_int                     # stack the data in the array
        ptr += 1                              # update x position for displaying the curve
        curve.setData(Xm)                     # set the curve with this data
        curve.setPos(ptr,0)                   # set x-y position in the graph to 0 and most recent data point - this creates the scrolling of the plot 
        QtGui.QApplication.processEvents()    # process the plot

### MAIN PROGRAM #####    
# this is a brutal infinite loop calling realtime data plot
while True: update()

### END QtApp ####
pg.QtGui.QApplication.exec_() 
##################

Teensy 3.2 Side:

const int sensorPin = A9;
uint16_t sensorValue = 0;
byte b1;
byte b2;
int flag = 0;
IntervalTimer heartBeatTimer;

void setup() 
{
  analogReadRes(12);
  Serial.begin(115200);
  heartBeatTimer.begin(heartBeat, 140); // (1 / 115200 Baud) * 16 bits / integer = 139us per 16 bits sent. Interrupt at 140 us to synchronize with baud rate.
  pinMode(13, OUTPUT);
}

void heartBeat() 
{
  flag = 1;                           // Interrupt routine every 140us
}


void loop() 
{
  if (flag == 1) {
    sensorValue = analogRead(sensorPin);  // read the analog pin as a 16 bit integer
    b1 = (sensorValue >> 8) & 0xFF;       // break up the reading to two bytes
    b2 = sensorValue & 0xFF;              // get the second byte
    Serial.write(b1);                     // write the first byte (trying to speed things up by sending only strictly necessary data)
    Serial.write(b2);                     // write the second byte
    digitalWrite(13, HIGH);               // just to make sure we're interrupting correctly
    flag = 0;                             // wait for next interrupt
  }
  digitalWrite(13, LOW);                  // just to make sure we're interrupting correctly 
}

Does anyone have any suggestions on how to speed things up?

Answer 1

As M.R. suggested above you'd be probably better off if you pack more data before sending it through instead of sending a two-byte packet at a time.

But the horrible performance you see has more to do with the way you read data on your computer. If you read just two bytes from your serial port and attach them to the plot the overhead you end up with is huge.

If you instead process as many bytes as you have available on your RX buffer you can get almost real-time performance.

Just change your update function:

def update():

    global curve, ptr, Xm    

    if ser.inWaiting() > 0                         # Check for data not for an open port
        b1 = ser.read(ser.inWaiting())             # Read all data available at once
        if len(b1) % 2 != 0:                       # Odd length, drop 1 byte
            b1 = b1[:-1]
        data_type = dtype(uint16)
        data_int = fromstring(b1, dtype=data_type) # Convert bytes to numpy array
        data_int = data_int.byteswap()             # Swap bytes for big endian
        Xm = append(Xm, data_int)              
        ptr += len(data_int)                              
        Xm[:-len(data_int)] = Xm[len(data_int):]   # Scroll plot
        curve.setData(Xm[(len(Xm)-windowWidth):])                     
        curve.setPos(ptr,0)                   
        QtGui.QApplication.processEvents()   

After toying a bit with the idea of iterating the bytes two at a time I thought it should be possible to do it with numpy, and coincidentally I found this question, which is very similar to yours. So credit goes there for the numpy solution.

Unfortunately, the battery of my portable scope died so I could not test the code above properly. But I think a good solution should be workable from there.

I did not check the Teensy code in detail, but at a quick glance, I think the timer for the interrupt you're using to give the tempo for the ADC might be a bit too tight. You forgot to consider the start and stop bits that travel with each data byte and you are not accounting for the time it takes to get the AD conversion done (I guess that should be very small, maybe 10 microseconds). All things considered, I think you might need to increase the heartbeat to be sure you're not introducing irregular sampling times. It should be possible to get much faster sampling rates with the Teensy, but to do that you need to use a completely different approach. A nice topic for another question I guess...