Problems with pachube update script

[sandeen]

I'm using a sketch like this to update a pachube feed; lots more happens in the loop of course, but I've turned that all into a "delay(1000)" for simplicity:
(to make this actually work you'll need to fill in a real pachube api key, feed id, SSID, and AP password of course, and fix up local IP addresses)

Code: Select all

#include <WiServer.h>

#define WIRELESS_MODE_INFRA   1
#define WIRELESS_MODE_ADHOC   2

#define PACHUBE_API_KEY "<MY PACHUBE API KEY>"
#define PACHUBE_FEED_ID "<MY FEED ID>"
#define PACHUBE_UPDATE_INTERVAL (30L * 1000L)

// Wireless configuration parameters ----------------------------------------
unsigned char local_ip[]   = {10,0,0,82};      // IP address of WiShield
unsigned char gateway_ip[]   = {10,0,0,1};      // router or gateway IP address
unsigned char subnet_mask[]   = {255,255,255,0};   // subnet mask for the local network
const prog_char ssid[] PROGMEM   = {"MY_SSID"};      // max 32 bytes

unsigned char security_type = 3;   // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2

// WPA/WPA2 passphrase
const prog_char security_passphrase[] PROGMEM = {"my_password"};   // max 64 characters

prog_uchar wep_keys[] PROGMEM = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d,   // Key 0
                  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   // Key 1
                  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   // Key 2
                  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00   // Key 3
                };

// setup the wireless mode
// infrastructure - connect to AP
unsigned char wireless_mode = WIRELESS_MODE_INFRA;
unsigned char ssid_len;
unsigned char security_passphrase_len;
// End of wireless configuration parameters ----------------------------------------

// Function that prints data from the server
// prints data --received--
void printData(char* data, int len) {
  unsigned long printtime = millis();

  Serial.print(printtime);
  Serial.println(": got data.");
}

//body data function, provides data for the POST command in comma separated values (CSV)
//currently POSTs one value but more can be added by separating with commas (no spaces)
// Sends data -out-
//
void feedData() {
   char buf[32];
  unsigned long feedtime = millis();
  Serial.print(feedtime);
  Serial.println(": sent data.");
   sprintf(buf, "%d,%d,%d", 0, 0, 0);
   WiServer.print(buf);
}

// IP Address for api.pachube.com
uint8 ip[] = {173,203,98,29};
char hostName[] = "www.pachube.com\nX-PachubeApiKey: " PACHUBE_API_KEY "\nConnection: close";
char url[] = "/api/" PACHUBE_FEED_ID ".csv?_method=put";

// A request that POSTS data to Pachube
POSTrequest postPachube(ip, 80, hostName, url, feedData);

unsigned long updateTime = 0;    // next time to post data

void setup()
{
   Serial.begin(57600);

   // Initialize WiServer (we'll pass NULL for the page serving function since we don't need to serve web pages)
   WiServer.init(NULL);
   
   // Enable Serial output and ask WiServer to generate log messages (optional)
   WiServer.enableVerboseMode(true);
   // Have the printData function called when data is returned by the server
   postPachube.setReturnFunc(printData);

  // First update 30s from now
  updateTime = millis() + PACHUBE_UPDATE_INTERVAL;
}

void loop()
{

delay(1000);
Serial.println("loop");

if (millis() >= updateTime) {
   unsigned long timeNow = millis();
   Serial.print(timeNow);
   Serial.print(" >= ");
   Serial.print(updateTime);
   Serial.print(": Updating pachube; ");
   // Do another update 30s from now
   updateTime += PACHUBE_UPDATE_INTERVAL;
   Serial.print("updateTime now ");
   Serial.println(updateTime);
   postPachube.submit();
}


// Run WiServer
WiServer.server_task();
delay(10);
}


The problem is, things start off just fine; I see on the serial port:

Code: Select all

64869 >= 94506: Updating pachube; updateTime now 94506
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
loop
Connected to www.pachube.com
X-PachubeApiKey: XXXX
Connection: close
84136: sent data.
TX 269 bytes
RX 0 bytes from www.pachube.com
X-PachubeApiKey: XXX
Connection: close
loop
loop
loop
loop
RX 164 bytes from www.pachube.com
X-PachubeApiKey: XXX
Connection: close
88240: got data.
loop
RX 164 bytes from www.pachube.com
X-PachubeApiKey: XXX
Connection: close
89280: got data.
loop
loop
loop
loop
loop
loop
loop
95360 >= 124506: Updating pachube; updateTime now 124506
RX 39 bytes from www.pachube.com
X-PachubeApiKey: XXX
Connection: close
95393: got data.
Ended connection with www.pachube.com
X-PachubeApiKey: XXX
Connection: close
95421: got data.


... i.e. it does seem to be sending and receiving requests, but eventually this stops; all I see is:

Code: Select all

875286 >= 904506: Updating pachube; updateTime now 904506
loop
...
904650 >= 934506: Updating pachube; updateTime now 934506
loop
...
935028 >= 964506: Updating pachube; updateTime now 964506
loop
...


and I never get any debug message indicating I've hit the feedData() function.

Any idea what I'm doing wrong, or how to debug this?

(I have a hunch that I'm sending a new request before the old one is complete..?)

Thanks,
-Eric

[sandeen]

Digging more, it seems that one of the connections never ends. Filtering the output a bit:

Code: Select all

514317 >= 514070: Updating pachube; updateTime now 634070
Connected to www.pachube.com
Ended connection with www.pachube.com
634752 >= 634070: Updating pachube; updateTime now 754070
Connected to www.pachube.com
Ended connection with www.pachube.com
755094 >= 754070: Updating pachube; updateTime now 874070
Connected to www.pachube.com
Ended connection with www.pachube.com
874419 >= 874070: Updating pachube; updateTime now 994070
Connected to www.pachube.com
Ended connection with www.pachube.com
994686 >= 994070: Updating pachube; updateTime now 1114070
Connected to www.pachube.com
1114777 >= 1114070: Updating pachube; updateTime now 1234070
1234750 >= 1234070: Updating pachube; updateTime now 1354070
1354711 >= 1354070: Updating pachube; updateTime now 1474070
1474662 >= 1474070: Updating pachube; updateTime now 1594070
1594617 >= 1594070: Updating pachube; updateTime now 1714070
1714573 >= 1714070: Updating pachube; updateTime now 1834070
1834777 >= 1834070: Updating pachube; updateTime now 1954070
1954879 >= 1954070: Updating pachube; updateTime now 2074070


I get one chunk of the reply back but no more:

Starting with printData, len 164
HTTP/1.1 200 OK
Server: nginx/0.6.35
Date: Mon, 20 Sep 2010 21:09:45 GMT
Content-Type: text/plain; charset=utf-8
Connection: close
X-Runtime: 228
Content-Leng
Done with printData

<nada>

it just goes away mid-reply and never comes back... shouldn't this time out & reset somehow?

[sandeen]

Talking to myself here

If I run WiServer.server_task(); in a loop after I submit the request:

Code: Select all

   postPachube.submit();

   // see if we can just sit tight until pachube is done (1s enough!?)
   do {
     WiServer.server_task();
     delay(10);
   } while (millis() < (timeNow + 1000));

things seem to run smoothly again. Is there some minimum frequency at which the server_task() must run to not drop packets?

Thanks,
-Eric

[GregEigsti]

Yes, the WiShield needs to be serviced quickly and often. I don't have any real data to offer but I do know that if you sleep/delay/whatever for a second between giving the WiShield processor time it can/will drop off of the network.

Was going to mention this to you but you beat me to it

Greg

[sandeen]

Greg, thanks. I'm now running the server task like -mad- and giving it extremely concentrated love after doing a post when we know we'll get a response:

Code: Select all

do {
     WiServer.server_task();
     delay(10);
} while (!response_done);


and response_done is set in the callback function when a 0-byte response is received...

But still I get disconnections every day or so Greg, looking over the forums I see various hacks & changes to the library to allow the wishield to reconnect if the connection is lost ... is there a "best practice" for this now? Any wiki page pointing at it? I'm about to give up and go to a wired connection.... I get the sense some people have workarounds but I'm not sure what the latest/best approach is...

Thanks,
-Eric

[GregEigsti]

There are various hacks around WiServer disconnect (search out posts by aerodyno- he has been very active with this). The biggest problem is that the various hacks is that they work for some folks but not others.

We definitely should at least list the hacks on a wiki page and I should implement them (or the best one) in the user contrib branch. Problem is that WiServer seems to be broken in the user contrib branch and I don't use it so I have a hard time getting interested.

Have you tried the non-WiServer web server example to see how that works?

Forgive me for leaving out the names of those who have worked hard on this - its a rough morning and aerodyno was the only name that came to mind.
Greg

[sandeen]

Ok, thanks for that info.

I take it that the asynclabs guy(s) no longer care(s) about this? Seems to be a big problem.

-Eric

[GregEigsti]

WiServer is user contributed code which has not been maintained recently. I personally do not use it so I have not made much effort. Maybe you would like to champion its upkeep? Someone, anyone?

Greg

[sandeen]

"WiServer is contributed code" - hum, I thought that's all there was. So what is the bare-bones interface, is there a sample sketch which shows that so I know what you're talking about? I really appreciate your time, btw.

-Eric

[GregEigsti]

There is a webserver sample sketch (not the simple one - which is WiServer based). Or you can go down to interacting directly with the uIP stack - which is where I like to play.

Greg