How to store a large data from serial port without loosing any data?

I have a GPRS module for reading data from a web page. I am receiving the data through a serial port, but I cannot store all data at once. When I have a loop for reading and storing the data, I loose some of the data between iterations.
I’m wondering why this is being happened and what is the best way for storing the data?

Thank you

Ideally I’d like more info on the serial data? Is it a particular protocol or a custom stream of data? What baud rate? Any hardware hand shaking? etc.

But in general:

  1. Buffering is important so have you set your serial receive buffers big enough? You need a buffer big enough to hold all the data that might arrive between your attempts to read it.

  2. If your program is implemented as a single task you might consider splitting it up into multiple tasks either using the costates functionality of Dunamic C or using the uCos II real time operating system that comes with the Dynamic libraries.

  3. Use serial ports E or F if your data rate is high as these have an inbuilt 4 byte receive buffer which helps to reduce dropping of data (ports A to D have a single byte receive buffer).


1 Like

It’s HTTP, at 115200 bps with no handshaking at the moment. I also use port B for communication. I guess to fix my problem I need to use (XON/XOFF) software handshaking to control my receiving data, but I’m not sure how to do it.

The data I need to receive is about a few thousands of characters. I’m using a char array to store the read data, but when I make my buffer to be quite large, it still cannot store more than 128 data.

Are you using PPP to connect to the GPRS module or getting a raw data feed from the module? I.e. is the module being used as a proper network connection or as an autonomous system which is simply providing a serial data stream to the Rabbit system?

In the first instance the Dynamic C TCP/IP stack should manage the connection and throttle things/retry etc.

In the second instance, 115200 bps is a lot to ask without handshaking. I’d be inclined to add handshaking and also drop the baud rate if possible.