The last time I wrote, I talked about how the Arduino seemed to be reading serial information slower than the Raspberry Pi was writing it.

After reading the TLDP Serial HOWTO, I learnt that my suspicion on the serial buffer is true: it is a FIFO queue. That makes it quite clear why my Arduino was lagging behind: because I intentionally added a few hundred milliseconds of delay after every read statement. As a result, the buffer is getting clogged with writes from the Raspberry Pi during this few hundred ms, and following that the Arduino reads the buffer just once.

To recap: previously my Arduino code was something like this:
char buffer = '\0';

void setup(){
  Serial.begin(9600);
}

void loop(){
  if (Serial.available()){
    buffer = Serial.read();
  }
  /*** Do some stuff here with buffer char ***/
}
Now loop() is more like this:
void loop(){
  while (Serial.available()){
    buffer = Serial.read();
  }
  /*** Do some stuff here with buffer char ***/
} 
If you couldn't tell the difference, I simply changed the if keyword to while, such that every time loop() runs, the buffer is read till the last byte. Then we do things with that byte. Previously, only the head of the buffer queue was read for each run of loop(), and that was the cause of the problems.

The downside of this is that even if there was nothing left in the buffer, the buffer variable will still contain the value from the previous Serial.read() (since it's a global variable), and the /*** Do some stuff ***/ part will still run with this previous. This may not be the behaviour that you want, but for me it's fine.

And now I no longer have the lag issue.

On a separate note, I ordered a few more electronic parts from Element 14, including an Arduino motor shield. It should be arriving today. I'm pretty excited about getting my robot up and running. The next challenge I face now is getting a good power supply for the Raspberry Pi, Arduino and the DC motors. I at least will need some form of voltage regulation for the Pi. I'm reading up on that now. Fingers crossed!