How much RAM needed for low end Ethernet application? - Page 2

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: How much RAM needed for low end Ethernet application?

Quoted text here. Click to load it

If assembling the packet in the first place is requires a lengthy
computation (say a 10 seconds encryption), then reproducing the
packet in order to do a retransmission might become a performance
bottleneck. In such an application it would be better to buffer the data.

For a set of simple sprintf()s or a computation that takes less than a
second, it probably is not a bottleneck to recompute the data for a
retransmission. In many applications, retransmissions are not that
frequent. (But this of course depends on the network conditions.)

Regards,

/adam
--
Adam Dunkels, Swedish Institute of Computer Science


Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

Even though you have not spcified close enough what kind of "Ethernet
applications" you have in mind, I asume this is going to be some
application that needs TCP connectivity.

Basically it boils down to how many buffers you want/need to have. On
the lower end there are those implementations which mostly exclusively
use the NIC's io buffer space which are usually limitted to one single
TCP session (if at all) or UDP based comunication only. In case of TCP
such implementations also do not have a flight size > 1. That said
only one segment is sent out and processing waits up until this
segment gets ack'ed from the other end. Such implementations are fine
if you want a little simply webpage showing some data you may collect
with the device or where such a page is used to configure the device
etc.

On the other end of the rope are those implementations where you need
to process multiple TCP sessions in paralell and where you want a
reasonable througput. If you want a reasonable speed, you need to have
a minimum of two buffers per connection so as you can send out two
segments which are not yet ack'ed by the other end (in other words a
"flight size" of two buffers) and where it's likely that buffers are
available for the reciver too.

So, as a rough estimate, I sugest to calculate 1500 bytes per buffer
times number of buffers per connection times TCP sessions in paralell.
Add probably 20% to this to have something left to maintain
task/socket state and there you go. Of course this asumes that the
buffers are held in a pool and are not strictly assigned to a given
connection.

HTH

Markus

Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

This is a very hard question, I will try to illistrate why...

In general you can hand build a TCP/IP system with very little RAM. (10K or less)
But if you want to send lots of data or want to use a traditional socket
programming interface you will need
a lot more. I would put the minimum at around 64K.
As a guess aproximatly 2 to 3  times the bytes/per second you want to transfer
over the network for buffering and
an additional 512 bytes  per connection, and 1K for miusc system state.



You need to explicitly answer 2 Questions to estimate the RAM size.

1)What type and number of connections?
Raw frames?
ICMP,ARP,UDP?
DHCP?
TCP Web?
TCP Telnet?
TCP Ftp?
TCP E-Mail?
DNS?
SNMP?


2)Do you want a traditional socket like programming interface or can you deal
with
a special minimal ram interface? This is mosst true for TCP.
I'll use the examples of a single threaded webserver....

Step 1 Negotiate a connection...
    Client sends SYN packet
    Sever sends a SYN ACK
    Client Sends an ACK.
This step will be the same reguardless of the interface. All packets are small,
does not take much memory.

Step 2 Client makes a request...
    The Client asks for a web page if there is no complicated post this likely a
single packet.
    Request packet comes in.....
Traditional:
    Using the traditional "socket read interface" the TCP stack has to take the
incomming packet and store it in a received buffer,
    the transfer it to another buffer on the read request, so ram used is 2 to 3
times the size of the received packet.

LowRam interface:
    The TCP stack takes the incomming packet and then gives to the user as a
pointer, no copy takes place, but the interface is not a
    traditional "read." Memory usage is 1.5 times the payload size.


Step 3 The Server responds...
    The Server genrates the response to the web request, no matter what the
interface I will assume that the wholw thing is builtup in a
    ram buffer, it may be built in peices, or all at once, this is just a speed
tradeoff....

Traditional: Using the traditional write interface the programmer assembles his
respons(es) and calls write for the socket.
    The TCP stack then copies this data into an outgoing packet and holds on to a
copy of the data until the othet side acknowledges it.
    The TCP system can use a lot of ram here,

LowRam Interface:
    The user asks the TCP system to give him a place to put the outgoing data, Then
the programmer puts some limited amount of data into this outgoing packet.
    The packet is submitted to the TCP system that then goes to sleep waiting for
the other side to acknowledge the packet, or until it is time to try and
retransmit the packet.
    This methodology will be slow as it reduces the tCP flow control system to a
stop and go protocol 2 to 10 times slower than a traditional TCP implemtation.

Repeate step 3 until the whole thing is transfered.


Step 4 Close the connection...
    Server sneds FIN
    Client send FIN ACK
    Server sends FIN.
    All small packets using very little memory.
























    
    




When you've explictly answered the first question what connections and how many
simultanious
connections of each type you c

    


When you say Ethernet I will assume you mean TCP/IP


Some background....

Full size ethernet rames are around 1500 bytes, + a few bytes dependign on
weither your ethernet controller puts
addistional information in the frame or as seperate registers.

If you are sending receivng UDP,ICMP,ARP etc... then you probably need to buffer
a couple incomming frames and one or two outgoing frames.
At this level the buffering can all be in the ethernet chip/controller.













Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

The question is really related to the specification of the AT91SAM7X
processor which has integrated Ethernet.

There is no SRAM in the "Ethernet controller".
The Ethernet MAC is integrated and shares the memory with
the CPU, so there is no extra memory available.
Also there is no external bus, so it is really important
that enough SRAM memory is in the chip to start with,
because this is all that is available.

Thanks for comments!

--
Best Regards,
Ulf Samuelsson
We've slightly trimmed the long signature. Click to see the full one.
Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

Which part is that?  I don't see it on the web site, but maybe I'm
not looking in the right place.

Thanks,
Eric

Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it
It is not available yet.

--
Best Regards,
Ulf Samuelsson
We've slightly trimmed the long signature. Click to see the full one.
Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it



OK, but is it announced yet?

Will it have an onboard PHY?  I'd love to see an alternative to the
Freescale MC9S12NE64 with a more powerful CPU and somewhat more RAM
(32KB would be great).

Eric

Re: How much RAM needed for low end Ethernet application?
On Thu, 3 Mar 2005 00:00:03 +0100, "Ulf Samuelsson"

Quoted text here. Click to load it


Does it have an External Bus Interface?

How much RAM on board?

regards,
Johnny.

Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

I would expect the performance to be almost identical.
Both are single cycle flash access, both a risk like.
The Motorola site has some MIPS benchmarks for the 5282
I think it's about 78 Mips for the 80 Mhz 5282.


Quoted text here. Click to load it


There are a number of low cost BDM's (BDM is Freescales version f JTAG
debugging) for the coldfire.
Both Linux and Windows hosted.

Out (Netburner's) 5282 product uses the Network to upload/download  code and
a non maskible serial routine running as a GDB stub.

If you have to have the BDM interface companies that have low cost ColdFire BDMs
are

http://www.pemicro.com /
http://www.cybertec.com.au /
http://www.macraigor.com/wiggler.htm



Paul







Re: How much RAM needed for low end Ethernet application?
On Wed, 2 Mar 2005 19:55:22 +0100, "Ulf Samuelsson"

Quoted text here. Click to load it

I see no reason apart from performance why PowerNet should
not fit. Is an EVB available yet? I could be tempted.

Stephen


--
Stephen Pelc, snipped-for-privacy@INVALID.mpeltd.demon.co.uk
MicroProcessor Engineering Ltd - More Real, Less Time
We've slightly trimmed the long signature. Click to see the full one.
Re: How much RAM needed for low end Ethernet application?
Ulf Samuelsson wrote about an unannounced AT91SAM7x part with
ethernet:
Quoted text here. Click to load it

Of RAM?  Yes, that's plenty for small-to-medium size applications.  I'd
be pretty happy with 32KB.  If you want to support largish applications,
you should either include 128KB of RAM, or allow external RAM to be
added.

If you're talking Flash, I'd recommend at least 128KB, though 64KB is
enough for small systems, and 32KB is enough for tiny systems.

Quoted text here. Click to load it

If you haven't yet decided the memory sizes, it's obviously quite
a way out yet.

Or if you have already taped out a part with 64KB and are trying to
decide whether to introduce it as a product, the answer is yes.  :-)

Eric

Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

I dont make decisions like this in Atmel,
but sometimes I can make people reverse decisions by good arguments.
If anyone has anything to complain about the Watchdog in the
SAM7S series then I am the guy to talk to.

Just wanted to know if the SRAM size was worthy of another crusade.
It looks like we can live with 64KB of SRAM for a substantial part
of the market.

--
Best Regards,
Ulf Samuelsson
We've slightly trimmed the long signature. Click to see the full one.
Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

I would be carefull when it comes to the tricks mentioned by some
posters here. Don't forget that there are SYN flood attacks, tear drop
attacks and other malicious activity out there in the internet waiting
to hit your implementation. You must handle these kind of things
reasonably well in a comercial product or otherwise you will have
lot's of problems.

Another issue is how many systems out there simply do not play by the
standards. We see systems anouncing MSS but then do not accept
segments of this size etc. etc. So again, if this is for a heavily
comercial kind of application you must consider these things too.
Especially the SYN flood thing requieres implementing SYN cookies or
other kind of counter measures or else the limitted resources of an
embedded system are eaten up instantly. The customer surely will blame
you if this happens - and not the attacker.

In the TCP stack I implemented, I'm using 50 buffers of 1500 bytes in
size in a pool. There is a low water mark which if reached means that
new connection requests are no longer granted up until enough buffers
become available. In other words there is no fixed relation between
connection and buffers but instead they are dynamically assigned whre
needed. This allows to handle ~20 connections very efficiently, but
also means at the same time that peaks of 70 or more paralell
connections still work reasonably well. The per connection state
overhead is about 50 bytes more per connection. You could easily
tailor the RAM useage by defining a different buffer pool size.

The code size used to implement this is 34 KB.

The stack supports the following features / protocols:

- ARP
- ICMP
- IP (fragment reasembly supported)
- DHCP client
- UDP
- DNS resolver
- TCP/IP
- SYN cookies
- configurable window size

I hope above information gives you an idea of what's needed RAM/Flash
size wise.

HTH

Markus

Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

Markus,

What approach did you implement for SYN cookies?

I've seen a couple schemes - one crypto-based, the other a scheme for
encoding the setup vars in the ISN.  Both seem to have challenges, not
the least of which is the limited field size of the Seq# to work with
(never mind that it violates the requirement for ISN randomization).

For example, did you sacrifice the options header / certain MSS
increments in the process?  IIRC, this is the one big point of
compromise to fit in the available field space, with the being 0 RAM
used until the reply SYN-ACK arrives (i.e., fairly impervious to SYN
flooding).

Richard

Re: How much RAM needed for low end Ethernet application?

Quoted text here. Click to load it

We encode the setup vars and then create an MD5 hash out of this which
is used as the ISN. The hash key is varied randomly and only valid for
a certain duration. The SYN/ACK therefore must arrive within this time
window to be acepted.

Quoted text here. Click to load it

The compromise is in reality not that big in that almost all TCP
implementations use some popular MSS value. Then you can't trust the
anounced MSS value anyways since there are implementations which
anounce a bigger MSS than what they actually accept. Sometimes there
are also gateways along the way which drop segments above a certain
size but fail to send the propper ICMP anouncements. Often this is the
case because a sysop firewalls everything but uses a router behind the
firewall showing this behaviour. In such scenarios it's often
impossible to tell wether the TCP implementation of the oponent is
broken or if it's just a configuration issue. This is just one area
where strict RFC compliance of your impelmenation alone does not
guarnatee seamless operation in the field. I do agree that such
situatins are rare, but they DO exist. So it's wise to have a TCP
implementation which is able to adapt the anounced MSS value
acordingly or else you will have interoperabilty issues. The problem
with these kind of things are that they ocure at the point in time
your product is in the market. It's even nastier that although your
product may follows the RFC's, the customer will point at you since
it's Windows PC's TCP implementation does adapt the anounced MSS and
therefore work. It's quite difficult to discuss such issues with end
users.... So you better prepare right from the start.

Obviousely solving these kind of issues also uses code space and
eventually also RAM. That's why I thought it's a good idea to bring
this point to Ulf's attention.

Markus


Re: How much RAM needed for low end Ethernet application?

Quoted text here. Click to load it

I'm not sure what you mean with "the tricks mentioned by some posters"
(perhaps you could be a little more specific?) but I would also like to
stress the importance of doing heavy testing of a TCP/IP stack. Send as
much "bad" packets to your stack as possible - others surely will.

Quoted text here. Click to load it

Failure to adhere to the size of the MSS seems to be the most common way
to diverge from the RFC standards. This may work fine when talking to a
PC, but may result in communication failure when trying to talk to a small
embedded device. Lacking support for IP fragment reassembly also seems
quite common. Some stacks also leave out TCP's congestion control, which
may even lead to network overload. I fully agree with you: compliance with
RFCs is important.

For a more thorough discussion of RFC compliance of small TCP/IP
implementations, please refer to my MOBISYS'2003 paper "Full TCP/IP for
8-bit architectures", available at
http://www.sics.se/~adam/mobisys2003.pdf

Regards,

/adam
--
Adam Dunkels, Swedish Institute of Computer Science


Re: How much RAM needed for low end Ethernet application?
Quoted text here. Click to load it

I started my implementation about two years ago and back then all
small TCP stack's I looked at could not resist SYN flood attacks not
to mention more complicated attacks that are reality also. I do not
say that they necesairly crash the device, but there are no counter
measures at all in place. SYN flood attacks are nasty in that the
attacker can easily hide origin by randomly varying the sender IP
address. In normal day to day's live it's not feasable to track the
source of an attack since it would requiere cooperation of sysadmins
all across the way back to the attacker. In other words, such attacks
can go on for hours thereby rendering an embedded device using this
kind of unsecure TCP implementation useless. This CAN be a very
important issue if the device is used in a productive envireonement.

With "tricks" I mostly refered those implementations which try to keep
state in the NIC's buffer, useing specially crafted sequence numbers
et all. The randomness of initial sequence numbers is extremly
important to avoid other kind of attacks (i.e. man in the middle) . If
however bit's from the sequence number are used for other purposes as
sugested, the randomness of the numbers can't be guaranteed to a high
degree enough. I do see the obvious advantages that such tricks might
have to save RAM et all, but not also considering security issues in
todays times is probably only ok for hobby projects..

Quoted text here. Click to load it

I actually was refering to some Linux TCP versions which in fact
anounce an MSS of say 1460 byte, but silently drop every segment
bigger than 1420 bytes. So, the embedded TCP stack must implement a
technique to slowly increase the MSS and maybe lower it if it sees
that segments are lost. Not that this would be a problem, but it must
be implemented and it uses code space and can't be ignored. Ignoreing
these kind of issues leads to hard to explain interoperabilty
problems.

Quoted text here. Click to load it

I currently don't have the time to read this document but surely will
do so. I have no idea wether your implementation suffers in any of the
points I put my finger to. What I can say here though is that out of
my experience RFC compliance alone is often not enough. The stack also
must be able to deal with non complying implementations - at least to
some degree.

Markus

Re: How much RAM needed for low end Ethernet application?

Quoted text here. Click to load it

I haven't found a checklist, but out of my experience the timely order
of network events is something that is hard to test. In my project,
most of the bug's that ocured after lab testing were in this area.

To give you an example, one time our implementation ran into problems
if the answer to a second ARP query arrived before the first one. Was
a stupid typo, but it got undiscoverd in the lab.

What you SHOULD do under all circumstances is visit some "hacker"
sites, collect the latest attack tools and bomb your implementation
with it. If you don't do it, you can bet that someone else will.

Another thing you can do is to use a second device of yours and
programm it to generate all these ugly situations you fear most in
your implementation. I.e. make sure to generate out of order pakets,
fragmented pakets and so on. It's a matter of fact that you can
generate the most exotic pakets with your own hardware the easiest way
since even the raw socket interface does not allow to generate all
kind of ugly things. Such a test device of yours then can also be used
to verify future releases of your software.

Markus


Re: How much RAM needed for low end Ethernet application?

Quoted text here. Click to load it

I have been running uIP together with the Contiki OS, a web server (with
support for dynamic pages), a VNC server (remote networked display), and a
small GUI with a 120x56 pixels virtual desktop on an MSP430 with 60k ROM
and 2k RAM. The device was connected using RS232 and SLIP. If I remember
correctly the entire system used around 40k ROM, of which some 20k
constituted the Java VNC client and the web pages. uIP also is a general
purpose stack and not a stripped-down application specific stack.

I would suspect that part of the explanation for the differing resource
requirements between uIP and PowerNet are that throughput is likely to be
much higher with PowerNet, and that PowerNet has a full socket API that is
easier to program than the uIP API and that makes porting Unix
applications straightforward. Also, the PowerNet web server is much more
advanced than the uIP web server.

Regards,

/adam
--
Adam Dunkels, Swedish Institute of Computer Science


Re: How much RAM needed for low end Ethernet application?

Quoted text here. Click to load it

Nice.


PowerNet was designed from the outset for 32 bit CPUs and
plenty of RAM. Yes, it has a BSD style API. All servers, e.g.
Telnet and HTTP share a common architecture based on one
thread per connection. This makes it rather RAM hungry but
it is extremely easy to program. Now that single-chip
micros have 64k RAM, the RAM hunger is much less of an
issue, but a RAM reduction exercise is under way.

Stephen


--
Stephen Pelc, snipped-for-privacy@INVALID.mpeltd.demon.co.uk
MicroProcessor Engineering Ltd - More Real, Less Time
We've slightly trimmed the long signature. Click to see the full one.

Site Timeline