embedded web server (EWS) & persistent HTTP 1.1 connection

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Hello,

I'm trying to build a minimal embedded web server, EWS (possibly using PIC  
and later slow ARMs) as a proof of concept allowing the remote monitoring  
of various automated measurements. There seems to be a wide variety of  
available (open source, commercial and hobby) systems on various platforms.
   One thing I haven't found too often is the use of HTTP 1.1's "persistent  
connections"[1] which could cut down the amount of admin work needed for  
TCP/IP connections. While I realise that it might not be significant for  
"big" cores, it is for minimal configurations such as micro-controllers  
(eg. PICs).
   Any ideas as to why that is? (maybe i should post this in TCP/IP news  
groups instead?)

thanks,
   Mischa

[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

P.S.: Sorry, JB, I only thought of using newsgroups after I wrote you the  
email.

--
Mischa Oliver Altmann

http://www.altmann.at/~mischa (in terrible state)

Re: embedded web server (EWS) & persistent HTTP 1.1 connection
Quoted text here. Click to load it
[...]
Quoted text here. Click to load it

A few reasons you might not find it widely implemented:

1) It's optional, requires lots more resources, and benefit isn't
obvious to the user.  Embedded systems operate on tight resources, and
optional features (and many non-essential "required" features) are
omitted to conserve resources.

2) A common application is an admin page, which is often a dynamic HTML
page with static (cache-able) graphics, so only the single HTML object
is being fetched in many cases.  (i.e., a persistent connection would
not be of benefit.)

3) TCBs = RAM, which is usually precious with embedded systems,
especially when each TCB easily requires 512+ bytes.  Many have
resources for only 1-2 connections at once, so making them last for
seconds instead of milliseconds can have a huge impact on memory
requirements.  (IIRC, persistent connections are often 10+ seconds,
though I exect this is tunable.)


Practically, it would seem straightforward to implement: jump back to
the TCB input handler instead of closing the session; timeout idle
connections after X msecs. (Probably can't be <1000ms, and may need to
be 2000ms for compatibility with slow connections.)

However, with the same amount of RAM you could possibly handle 50x more
requests in this same 1000ms window if each connection were terminated
and re-established instead of persistent.  (Local connections could
easily service in <20msecs.)

So, persistent connections may save some CPU, and may reduce Internet
traffic with embedded applications, but at the expense of much more RAM
in the device, which translates to cost.


I expect you've got an application where persistent connections could be
beneficial - might I ask what that is?  Perhaps an application where
many dynamic objects would be downloading in each page (i.e., seldom
cached in the browser)?

Cheers,
Richard

Re: embedded web server (EWS) & persistent HTTP 1.1 connection

Quoted text here. Click to load it
hm, interesting thought. I assumed that using persistent connections would  
make TCP/IP stack coding easier as less connections have to be built up  
and closed down.

Quoted text here. Click to load it
you definately have a point here. For my purposes it would not be needed  
(I am only doing a proof of concept EWS) but i thought it would be a  
useful "exercise".

Quoted text here. Click to load it
agreed -- see 2)

[snip]
Quoted text here. Click to load it
see 2)

thanks for your indepth explanation - it's very much appreciated.

Quoted text here. Click to load it
I'm in Leeds, are you nearby ;-) or on the other side of the globe?

thanks,
   Mischa

--
Mischa Oliver Altmann

http://www.altmann.at/~mischa (in terrible state)

Re: embedded web server (EWS) & persistent HTTP 1.1 connection
Quoted text here. Click to load it

The quantity might be lower, but the TCP behavior would necessarily work
identically.  Probably a touch more code to detect / activate / timeout
persistence for HTTP.


Quoted text here. Click to load it

Exercises are always good if you have the time - practicality need not
apply. :-)


Quoted text here. Click to load it

Happy to!


Quoted text here. Click to load it

In the Arizona desert, USA (which this week is cold and rainy).

Though I remember visiting Leeds many years back - enjoyed the castle
enough that we missed the last train back to London. :-)

Have fun!
Richard

Re: embedded web server (EWS) & persistent HTTP 1.1 connection
Also...

I highly recommend Jeremy Bentham's book, TCP/IP Lean.  It's the best
text so far about implementing TCP on a microcontroller.  And the
example is based on a PIC.

(Amazon.com product link shortened)

Even if you aren't interested in the included code, he's got some clever
techniques for dynamic HTML objects like slide bar graphics, and another
for handling the TCP checksum with HTML.  Well worth the money as a
backgrounder to the issues and some ideas for handling them.

Enjoy!

Re: embedded web server (EWS) & persistent HTTP 1.1 connection
Quoted text here. Click to load it
hehe -- that's exactly where I was coming from! It doesn't even mention  
persistent connections and I was wondering why. It's pretty clear to me  
now, but I might just write the code to show it's possible.

thanks again,
   Mischa

P.S.: I am not aware of a castle in Leeds (though there are some ruins of  
monasteries/churches). You might mean Richmond which isn't "that" far form  
Leeds and also in the "north".

--
Mischa Oliver Altmann

http://www.altmann.at/~mischa (in terrible state)

Re: embedded web server (EWS) & persistent HTTP 1.1 connection
Quoted text here. Click to load it

Aha!

I've been thinking about a different technique than his for the checksum
handling.  It makes more sense to write a null checksum, then go back
and update the NIC buffer with the right checksum value.  And it works
for protocols other than HTTP.

I suspect the technique of guessing a checksum value & stuffing bytes at
the end to make it right was all due to the original implementation that
was SLIP-based, not Ethernet-based, and there was no buffer to go back
and update - it'd already been serialized out the port.  Very clever,
considering the resources, but unnecessary using Ethernet NIC buffers, I
think.


Quoted text here. Click to load it

What?  They moved it?! ;-)

Probably a bad assumption on my part that Leeds Castle was in Leeds. :-)

Cheers,
Richard

Re: embedded web server (EWS) & persistent HTTP 1.1 connection
Hello Mischa,

Quoted text here. Click to load it

Have you checked
http://www.sics.se/~adam/uip /

best regards
Wolfgang Muees


Re: embedded web server (EWS) & persistent HTTP 1.1 connection

Quoted text here. Click to load it

no, excellent resouce. thanks!

--
Mischa Oliver Altmann

http://www.altmann.at/~mischa (in terrible state)

Site Timeline