WiServer.print() and webpage limits

Discussions related to the WiServer add-on code for WiShield. WiServer is a friendly, easy-to-use front end for webserver and webclient usages.

Moderator: shard7

WiServer.print() and webpage limits

Postby rock » Wed May 26, 2010 8:37 pm

Hi. I'm working on a project where we are controlling two servos based on URL calls from a webpage. The end goal is to be able to remotely control the servos via an iPhone ad-hoc connection to the WiShield. I'm using the latest arduino and WiShield 2.0 (just bought both about a month ago). I've done some recent refactoring and the performance is pretty nice (page reloads quickly, responds well to link clicks, etc). The issue I'm running into now seems to relate to the length of the text for the webpage html. I originally had just a simple list of links, and the total character count was about 400. I figured I could do something a little more interesting and started adding some CSS styling, but once the string got above ~650-700 characters, things started getting weird. The page would become non-responsive, lag would be introduced, etc. If I keep the string length below 600 characters, everything works fine.

So, what I was wondering, is there some kind of a limit in the WiServer.print()? Would I be better off with multiple WiServer.print() statements instead of one large one? I've seen other posts during my research that suggest lots of WiServer.print()s can be problematic, but the Wiki suggests that a page of essentially any size should be doable.

I'm also trying to better understand exactly how the WiServer.server_task() works. If the buffer is 200bytes and the page is 600bytes, when the server_task() gets called in loop(), does it do only one call to sendPage(), processing the first 200bytes and then the next time it gets called in loop() is when it processes the next 200? Or does server_task() make multiple calls to sendPage() within the first time it's called in loop()?

This is code that works and has the shorter WiServer.print() string.
Code: Select all
#include <WiServer.h>
#include <string.h>
#include <Servo.h>

#define WIRELESS_MODE_INFRA   1
#define WIRELESS_MODE_ADHOC   2

//http://www.asynclabs.com/forums/viewtopic.php?f=18&t=104&p=505&hilit=Toggle+LED#p505

// Wireless configuration parameters ----------------------------------------

unsigned char local_ip[]    = {169,254,0,11};   // IP address of WiShield
unsigned char gateway_ip[]  = {169,254,0,1};   // router or gateway IP address
unsigned char subnet_mask[] = {255,255,0,0};   // subnet mask for the local network
const prog_char ssid[] PROGMEM = {"ASYNCLABS"};      // max 32 bytes

unsigned char security_type = 0;   // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2

// WPA/WPA2 passphrase
const prog_char security_passphrase[] PROGMEM = {"12345678"};   // max 64 characters

// WEP 128-bit keys
// sample HEX keys
prog_uchar wep_keys[] PROGMEM = {   0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d,   // Key 0
               0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   0x00,   // Key 1
               0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   0x00,   // Key 2
               0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   0x00   // Key 3
            };

// setup the wireless mode
// infrastructure - connect to AP
// adhoc - connect to another WiFi device
unsigned char wireless_mode = WIRELESS_MODE_ADHOC;

unsigned char ssid_len;
unsigned char security_passphrase_len;

// Servo Control Parameters ------------------------------------------------

Servo servoX;   // create servo object to control a servo
Servo servoY;   // a maximum of eight servo objects can be created

// Limits for servo positions
int minRange = 20;
int maxRange = 160;
int median = 90;
int posX = 90;         // Start tail in center position

int incomingByte = 90;   // value inbetween wags and curls
int wagStep = 1;      // Increment stepper value
boolean positive = true;// Flag for managing wag direction

int wagPattern = 90;   // Stores byte value for wagging tail
int curlPattern = 90;   // Stores byte value for curling tail

// Methods ------------------------------------------------------------------

void setup() {
   // Initialize WiServer and have it use the sendPage function to serve pages
   Serial.begin(115200);
   Serial.println("setup() ... ");
   servoX.attach(2);  // attaches the servo on pin 2 to the servo object
   servoY.attach(4);  // attaches the servo on pin 4 to the servo object
   servoX.write(median);
   servoY.write(median);
   WiServer.init(sendPage);
}

boolean sendPage(char* URL) {
   if (URL[1] == '?') {
      // Set wagPattern or curlPattern based on URL
      incomingByte = URL[2];
      if(incomingByte < 91) {
         wagPattern = incomingByte;
      }else if(incomingByte > 96) {
         curlPattern = incomingByte;
      }
   }

   Serial.print("serving ... ");
   // Serial.println(counter, DEC);
   WiServer.print("<html><head><meta name='viewport' content='width=300'><style type='text/css'>html{width:300px;font-family:sans-serif;text-align:center}body{margin:0px}a{display:block;margin:3px;padding:3px;text-decoration:none;border:solid 1px black;}</style></head><body>Wags<a href='?A'>no swing</a><a href='?B'>light swing</a><a href='?C'>medium swing</a><a href='?D'>heavy swing</a><a href='?E'>jittery swing</a>Curls<a href='?a'>straight down</a><a href='?b'>slight curl back</a><a href='?c'>full curl back</a><a href='?d'>slight curl front</a><a href='?e'>full curl front</a></body></html>");
   Serial.print("served ... ");
   Serial.println(incomingByte, DEC);

   return true;
}

void stepServoX(int delayVal,int max,int min) {
   if(posX >= max){ positive = false; }
   if(posX <= min){ positive = true; }
   if(positive){
      posX += wagStep;
   }else{
      posX -= wagStep;
   }
   servoX.write(posX);
   delay(delayVal);
}

void wag() {
   switch (wagPattern) {
      case 66:
         // B. light wag
         stepServoX(50,maxRange,minRange);
         break;

      case 67:
         // C. medium wag
         stepServoX(25,maxRange,minRange);
         break;

      case 68:
         // D. heavy wag
         stepServoX(5,maxRange,minRange);
         break;

      case 69:
         // E. shiver
         stepServoX(5,median+10,median-10);
         break;

      default:
         // A. no wag
         servoX.write(median);
         positive = true;
   }
}

void curl(){
   switch (curlPattern) {
      case 98:
         // b. light curl back
         servoY.write(median+30);
         break;

      case 99:
         // c. heavy curl back
         servoY.write(maxRange);
         break;

      case 100:
         // d. light curl front
         servoY.write(median-30);
         break;

      case 101:
         // e. heavy curl front
         servoY.write(minRange);
         break;

      default:       
         // a. straight down
         servoY.write(median);
   }
}

void loop(){
   WiServer.server_task();
   wag();
   curl();
}


Thanks!

--
Rock
rock
 
Posts: 13
Joined: Tue May 25, 2010 11:33 pm

Re: WiServer.print() and webpage limits

Postby shard7 » Sat May 29, 2010 5:34 am

Rock,

There's no inherent limit to the size of the data you can send. WiServer will call your page serving function several times to send the data in smaller chunks; it'll wait for an ACK for the current chunk before sending the next since a resend may be needed. So, depending on how fast the link is, there may several callbacks during a single call to server_task, or it may be spread out over several calls to it.

QPID reliably sends a favicon file that's over 1500 bytes, and I've run tests with even larger files and had no issues.

Looking at your code, I think RAM is the issue here. Because it uses a Harvard architecture, a literal string has to be loaded and stored in RAM, and as you know Arduinos have very little of that. For a large block of static data such as yours, stored it in flash memory and then use the special WiServer print functions to send the data directly from flash.

For example, here's a string that's stored in flash:

Code: Select all
const prog_char notification[] PROGMEM = {"<head><title>QPID</title><meta HTTP-EQUIV=\"REFRESH\" content=\"3; url=/\"></head>"};


And here's the code to send that string from a page serving function (note the use of the special print_P function):

Code: Select all
WiServer.print_P(notification);


I store strings like this in flash memory as much as possible because all those strings add up, and I'll even split up html so that I have 'static' parts that can be stored in flash. Save your RAM for things that need to change.

Shard7
shard7
 
Posts: 64
Joined: Wed May 06, 2009 11:30 am

Re: WiServer.print() and webpage limits

Postby GregEigsti » Sat May 29, 2010 9:57 am

newhobby, over in this thread has been having a similar problem where he is blowing out the Arduino's memory by including too much HTML data. I played with putting it all into PROGMEM last night (with icky use of a string table, etc.) and still got the crash to occur. After reading shard7's response here I changed the code to use WiServer's print_P() and it all works nicely.

I'm guessing my use of a string table, a bazillion PROGMEM "strings", being overly tired and watching TV out of one eye led me to write the crap code that caused the new crash (using PROGMEM). Happy to report that once I got rid of my crap code and replaced it with print_P all is well.

Greg
Check out the wiki!
uIP Stack Docs
Compatible Access Point List
WiShield user contrib branch - DNS, DHCP, AP Scanning, bug fixes, etc.
SlackLab.org - My geek projects blog.
User avatar
GregEigsti
 
Posts: 1067
Joined: Sun Aug 02, 2009 5:23 pm
Location: Sammamish WA USA (near Seattle)
  • Website

Re: WiServer.print() and webpage limits

Postby rock » Sat May 29, 2010 9:28 pm

I actually found this thread: html/css That also appears to point to using print_P() rather than print, et al. Does that mean that I should be fine if I use print_P() in smaller chunks?

The idea that it's a RAM issue is interesting. Is there any way to tell how much RAM is available, vs. how much is being used? Also, is there any good information about PROGMEM and how it's put together (edit: found PROGMEM info)? I take it from shard7's reply that PROGMEM is meant to store static data for later retrieval. It also implies that data stored in flash does NOT get pulled into RAM when called by a WiServer.print_P(). Is that correct?

Thanks for the great feedback! :-)

--
Rock
rock
 
Posts: 13
Joined: Tue May 25, 2010 11:33 pm

Re: WiServer.print() and webpage limits

Postby spidermonkey04 » Sun May 30, 2010 11:50 am

There are ways to estimate how much RAM is available discussed here. http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1236343710
I remember trying it out but reverted back to the old pencil and paper, adding up my variables.
Using PROGMEM seems the best way to go, but I'm fairly sure RAM is still used. The value from PROGMEM needs to be transferred from a flash page to the stack, used by your code and then discarded. So there is a certain amount of code overhead in using PROGMEM.
---Jared
spidermonkey04
 
Posts: 66
Joined: Thu Oct 29, 2009 6:45 pm

Re: WiServer.print() and webpage limits

Postby shard7 » Sun May 30, 2010 12:39 pm

If you declare a regular literal string, it's stored in program memory, but at runtime it gets duplicated in RAM because you want to be able to reference it just like any other variable that's stored in RAM. However, by using the PROGMEM keyword, it tells the compiler to leave it in flash and the the variable just points to it's location in flash memory. So if you try to use it like a regular string (e.g. WiServer.print), the address location is for program memory (flash), but it gets used to located data in RAM, which of course will return whatever data happens to be in that location in RAM. Special 'flash memory' functions such as WiServer.print_P treat the address differently and use it to correctly locate the data in flash memory and read it straight from there without first copying it to RAM.

Bottom line, without PROGMEM your characters are stored twice, once in flash and once in RAM. With PROGMEM, they're only stored in flash.
shard7
 
Posts: 64
Joined: Wed May 06, 2009 11:30 am

Re: WiServer.print() and webpage limits

Postby rock » Thu Jun 03, 2010 5:35 pm

I just wanted to update everyone and say that switching to PROGMEM and print_P() seems to have gotten me over the memory hurdle.

Thanks a bunch. Lovin' the WiServer so far :-)

--
Rock
rock
 
Posts: 13
Joined: Tue May 25, 2010 11:33 pm

Re: WiServer.print() and webpage limits

Postby seacritter » Fri Jul 23, 2010 4:36 am

I was bumping into the same problem. Everything thing would load and looked good. You could ping the wishield. But when I tried to load the page, it would just hang.

I placed some of the page into progmem and suddenly, the page would load. It seems that the more that I placed into memory, the more stable the program got. It seems to be faster as well...
Thanks,
Capn Scott
http://smallweatherstation.com
User avatar
seacritter
 
Posts: 45
Joined: Fri May 28, 2010 10:42 am
Location: Today? Near Philly...


Return to WiServer

Who is online

Users browsing this forum: No registered users and 1 guest