Autonomo with GPRSbee FTP upload issue

Hi,
We have a project that uploads data to an FTP server at a rate of 100Kb every hour. We currently use an mbili to do this and it works perfectly uploading a file from the SD card once an hour. We decided to upgrade the unit to an autonomo as it is faster, smaller and uses less power however we are finding that although the unit uploads data, it stops half way through the upload with an error although the error doesn’t give any info as to why but it fails in the gprsbee.sendFTPdata method. Initially we thought the problem was due to running the GPRSBee at 115200 baud as it had the same problem but reducing the baudrate back to the default 57600 showed no improvement.

Any help with this is much appreciated.

The code for the upload is based no the FTP example “test_FTP” sketch (see below):

void doFTP() {
  SerialUSB.println("Sending data via FTP");

  SerialUSB.println("Switching on..");
  bool retval = gprsbee.on();
  SerialUSB.print( "...done. Retval = " );
  SerialUSB.println(retval);
  
  if (!gprsbee.openFTP(APN, SERVER, USERNAME, PASSWORD)) {
    SerialUSB.println("Failed to open connection");
    gprsbee.off();
    return;
  }

  if (!gprsbee.sendCommandWaitForOK("AT+FTPPUTOPT=\"APPE\"")) {
    SerialUSB.println("Failed Append command ");
    gprsbee.off();
    return;
  }

  if (!gprsbee.openFTPfile(dateString, FTPPATH)) {
    SerialUSB.println("Failed to open file on FTP server");
    gprsbee.off();
    return;
  }

  char myData[1360];
  tempFile = SD.open("temp.txt",FILE_READ);

  if (tempFile) {
    dataSize = tempFile.size();
    SerialUSB.print(F("\nFileSize> ")); SerialUSB.println(dataSize);

    //while (dataSize > 0) {
      while (dataSize >= 1358) {
        for (int i = 0; i < 1358; i++) {
          myData[i]=tempFile.read();
          //dataSize -= i;
          dataSize--;
        }
        if (!gprsbee.sendFTPdata((uint8_t *)myData, strlen(myData))) {
          SerialUSB.println("Failed to upload data");
          gprsbee.off();
          return;
        }
        SerialUSB.print(F("\nFileSize> "));SerialUSB.println(dataSize);
      }

      //send the remaining data
      SerialUSB.println("sending remaining data");
      memset(myData,0,sizeof(myData));
      dataLeft = dataSize;
      for (int i = 0; i < dataLeft; i++) {
        myData[i]=tempFile.read();
        dataSize--;
      }
      
      if (!gprsbee.sendFTPdata((uint8_t *)myData, strlen(myData))) {
        SerialUSB.println("Failed to upload data");
        gprsbee.off();
        return;
      }
      SerialUSB.print(F("\nFileSize> ")); SerialUSB.println(dataSize);
  }
  else {
    SerialUSB.println(F("Cant Open File"));
  }  
      
  if (!gprsbee.closeFTPfile()) {
    SerialUSB.println("Failed to close file. The file upload may still have succeeded.");
    gprsbee.off();
    return;
  }

  if (!gprsbee.closeFTP()) {
    SerialUSB.println("Failed to close the connection. The file upload may still have succeeded.");
    gprsbee.off();
    return;
  }

  gprsbee.off();

}

Can you try sending the debug data to SerialUSB:

gprsbee.setDiag(SerialUSB);

Then see what messages you get when it stops sending the FTP data.

Hi Gabriel,
Thanks for the reply.

The Debug data was already being sent to SerialUSB (see setup code below);

void setupModemComms() {
  Serial1.begin(57600);
  gprsbee.initAutonomoSIM800(Serial1, BEE_VCC, BEEDTR, BEECTS);
  gprsbee.setDiag(SerialUSB);
}

When the unit first fires up it gets the time from the NTP server (works perfectly) then it attempts to upload data which fails part way through, sometimes it gets almost the whole upload done but most of the time it is only a few kilo bytes like the sample output from serial console below. The same code works perfectly in the Mbili.

Any help with this is much appreciated. Cheers.

Below is an excerpt from console output;
>> AT+CREG?
+CREG: 0,1
OK
>> AT+SAPBR=3,1,“CONTYPE”,“GPRS”
OK
>> AT+SAPBR=3,1,“APN”,“internet”
OK
>> AT+SAPBR=1,1
OK
>> AT+SAPBR=2,1
+SAPBR: 1,1,“49.226.229.187”
OK
>> AT+FTPCID=1
OK
>> AT+FTPSERV=“ftp.drivehq.com
OK
>> AT+FTPUN=“niwaeel”
OK
>> AT+FTPPW=“XXXXXXX”
OK
>> AT+FTPPUTOPT=“APPE”
OK
>> AT+FTPPUTNAME=“18-3-25.txt”
OK
>> AT+FTPPUTPATH="/test/"
OK
>> AT+FTPPUT=1
OK
+FTPPUT: 1,1,1360
FileSize> 60906
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
OK
+FTPPUT: 1,1,1360
FileSize> 59548
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
OK
+FTPPUT: 1,1,1360
FileSize> 58190
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
OK
+FTPPUT: 1,1,1360
FileSize> 56832
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
OK
+FTPPUT: 1,1,1360
FileSize> 55474
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
OK
+FTPPUT: 1,1,1360
FileSize> 54116
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
OK
+FTPPUT: 1,1,1360
FileSize> 52758
>> AT+FTPPUT=2,1358
+FTPPUT: 2,1358
ERROR
Failed to upload data

I’m not sure what is causing this exactly.

Could you try lowering the baud rate on Serial1 to something like 9600.

Hi Gabriel,
Thanks for the reply.

OK i lowered the baud-rate to 9600 and it seems to work but upload is obviously very slow. i find this a bit odd as it works at a baudrate of 57600 when using the mbili with essentially the same code. Is this something to do with the autonomo itself or could it be code related (header files?). i will run it over the weekend to make sure that it continues to upload correctly and maybe try slightly higher baud-rates.

We would prefer to use the autonomo for the current project instead of the mbili if possible.

Any help is much appreciated.

Cheers.

My theory is that there is a send buffer on the GPRSbee module for the FTP data. I believe you may be trying to send data to the GPRSbee at a faster rate than the network upload speed, that in combination with the amount of data you are sending, is resulting in an overflow of that buffer.

On the Mbili, the process of loading the data from the SD card probably takes longer and this is why you don’t see the same issue. Additionally, using a slower baud rate to send the data to the GPRSbee, also seems to resolve the issue.

You could try a couple of things. You could try increasing the baud rate to the GPRSbee to find the maximum speed that avoids this issue (this however will probably be dependent on the network connection speed). Alternatively, you could add a small tuned delay after sending each 1358 byte chunk to the GPRSbee.

For full reliability you might need to assume that the network connection is running at the slowest upload speed, or alternatively query the connection to find the actual connection speed.

Hi Gabriel,
Thanks for the reply.

I don’t quite understand, do you mean that the send buffer is not being cleared before more data is arriving so that when a slower upload baud-rate is used (which takes longer to upload the chunk of data) this gives more time for the buffer to be cleared? So if i slow down the rate at which i send data to the GPRSbee (via a delay) this may allow higher baud-rates to be used? However if this was the case surely this type of error would have occurred before and would have been noticed by other people.

I will give the delay approach a go and see what happens.

Your help with this is much appreciated.

Cheers.

Yes, I believe that is what is causing the issue. You are attempting to upload the data at a faster rate than the GPRSbee can send it over the network. You could test this further by trying a range of baud rates and seeing how quickly it fails. In the above output it seem to fail after about 8kb using 57600.

I’m not sure if there is any benefit to adding a delay over using a lower baud rate for the GPRSbee. I don’t know how much extra time it will take to run the connection sequence at 9600 vs 57600. Most of the time spent is waiting for responses or specific states to be achieved.

It is strange that this issue hasn’t shown up before. I believe it is probably due to a number of factors. I believe the FTP functionality is probably one of the least used with the library. Additionally, the problem doesn’t manifest itself when sending smaller files or using a lower baud rate.

It may be useful to know which exact module you are working (including revision number) and the firmware version.

Hi Gabriel,
Thanks for the reply.

it looks like the maximum baud-rate i can use that doesn’t cause incomplete uploads is 19200. However the time to upload the data at a baud-rate of 19200 is slower than the time to upload the data using the mbili running at 57600. Does this suggest that a delay may work?

The GPRSbee i am using is REV. 6b and the autonomo is the latest REV.

However all the examples sketches i have seen to have the GPRSbee with the autonomo running at 57600 baud so i am not convinced that the speed of data going from the autonomo to the GPRSbee is the problem here even though it does seem that way - could it be my code - as you can see in the upload code above i take a chunk of data from the temp file and put it into a char array which has a size that is less than the max upload packet size (1360 byte) then transfer this to the GPRSbee. Is there a better way? Or should i reduce the size of the initial data chunk to say 1000 bytes.

As a point of interest i have used a SIM900 shield module (normally used for arduino) being driven by a pyboard (STM32F405RG microcontroller with on-board python3 interprter) using similar code (direct AT Commands over uart) that runs at 115200 Baud with no error. However the autonomo is still the better option for us due to all the other options that it has.

Any help is much appreciated.

Cheers.