Throttling network traffic

D Yuniskis wrote:

For inbound TCP traffic, you can just limit the advertized window size and the rate at which you send acks. Obviously this implies control over the lower parts of the stack.

Alternatively, many firewalls, including some of the open source ones, have traffic shaping functions (and there are some stand-alone traffic shapers, but they're rare these days except as very high end products).

Yes. But does nothing about UDP traffic.

Also, doesn't do anything about limiting unsolicited connection attempts, etc. (as well as misbehaving net-neighbors)

Ah! This is more what I need! Something I can embed in a switch (though I will then need a protocol by which each client can tell the switch what it wants/"can have", etc.).

Thanks! I'll start poking around...

--don

There's only one possible thing to do with excess UPD traffic, and that's drop it.

Not that anything can do anything about those things. You have no control over what anyone outside your local network interface is doing. And having a physically separate firewall just move the problem one box further away. A firewall on your box, whether that's provided by a traditional firewall package, or your own low level IP stack, will have to deal with those things hitting your NIC.

Yes, though it still ties up the wire. (see below)

Agreed.

But, I'm looking at the design of the switch. I.e., if this functionality is *in* the switch then all traffic *through* the switch can be metered (including choked off).

The problem then becomes one of conveying the needs of each node (clients therein) *to* the switch so that it can enforce those restrictions.

When listening for TCP connections, just delay accepting the connections e.g. by 1 s and then close the socket if not appropriate. With repeated attacks, this will fill up the listening queue (default size 5 requests).

This will of course deny service also from legitime requests, but embedded systems usually have other critical tasks to do than work as a firewall :-).

e:

e l

A number of firewalls can run in "bridge mode" (sometimes called "transparent mode", although that's sometimes a bit different - lotsa funky stuff happens in firewalls). At which point the firewall appears to be just a bridge (or what we called "switches" before marketing decided a cooler name was needed). Not sure what you're building, but a standalone switch with this kind of functionality would be more a firewall (possibly in bridge mode) rather than a conventional switch. Or is this switch embedded in another device?

[much elided]

[begin lengthy explanation]

The projects I'm involved with are getting physically (geographically?) larger. More "distributed" (in the classic sense of the word). As such, "people" can be "inside" the system/device (hard to use the device for something so physically large... but, to an insect, a PC wouldn't look like a single "device" either!).

So, people are in a position to directly interact with the device/system in ways that might not be benevolent. E.g., would you let a small (sentient) creature wander around inside your PC -- even if their intentions were *not* malevolent? "Hey, where did my video go?" "How is it that I suddenly have more physical memory?" "Why is the CPU locked in a tight loop as if the bus was shorted?"

[sorry that this is obvious but sometimes the obvious needs to be stated explicitly]

If the network fabric is a key part of the performance and behavior of the product, then reliable operation of that product (system/device) depends on the maintaining the integrity of that fabric.

Network infrastructure (cabling, switches, etc.) is now (unfortunately?) commonplace and no longer treated as "the realm of wizards". E.g., a 1cm dia orange cable rarely was tampered with while a bunch of "phone" (-like) plugs that can be plugged and unplugged effortlessly see more (ab)use.

Someone (well intentioned or not) plugging a foreign device into a switch that is a key part of a system/device and causing that device to "mysteriously" malfunction ("Hey, Tom, I just got a call from Fred in Manufacturing claiming the Frajistat3000 is doing some mighty weird things. Can you go take a look at it?") would be unfortunate, to put it mildly. :>

E.g., how often, when your PC "misbehaves", do you think: "Gee, I had better open the cover and see if someone installed or removed something while I wasn't looking..."

Your PC -- as a *system* -- expects the components to behave in predictable ways. Memory should yield up its results within N clock cycles of an address being presented to it. A video adapter should draw a blue pixel at screen location (x,y) when the value is written to .

These are BASIC ASSUMPTIONS inherent in the design of that system. (bugs and anomalous behaviors happen when assumptions prove to be invalid)

So, since I can't armor all of the connections to the system/device (the equivalent of putting the PC's components into a case and putting a lid on it), I have too protect places that it is likely that someone could interfere (regardless of motive or intent) with the performance of that system.

I have to design a switch, regardless (for other requirements), so putting this functionality in the switch fortifies the entire system. And, if I expect that functionality to reside outside the "nodes", then I can rethink the obligations that the nodes, themselves, have to the system and gain some design economies, there! (e.g., if you're wearing "braces", no need to also wear a belt!)

Of course, I have absolutely no idea how I can extend this to technologies that do *not* rely on switches (e.g., "bus" technologies like older 10Base2, wireless, etc.) :<

[8> Of course, I have absolutely no idea how I can extend this *Wired* "bus" topologies are obsolescent -- but *wireless* (the "ether" can be regarded as a bus -- of sorts) are more and more common. [consider your comments in that light -- e.g., how incredibly easy it would be to disrupt/corrupt a wireless protocol]

Yes. But, that requires preshared keys or other means of "tying" devices together. I.e., this box (node) *belongs* on this network. It adds another configuration maintenance issue. E.g., imagine if you had to "pair" your DRAM DIMMs to your PC (in addition to "plugging them in") before the PC would "recognize them" ("consent to talk to them").

I am looking for a solution that lets boxes that *should* "fit together" (nodes, switch, etc.) automagically do so while still enforcing these constraints on communications. E.g., a smart switch can be designed that still allows "foreign" devices to talk to the "system" -- without giving them "free reign" to interfere (even unintentionally) with the operation of the rest of the system.

(i.e., the system can reserve certain bandwidth WITH PREDEFINED CHARACTERISTICS for interaction with the outside world KNOWING that this was "planned" in the original design -- not a parasitic relationship)

Re: authentication -- see above.

I frown on ex post factum remedies -- we don't design cars with "toggle switch ignitions" and then rely on the threat of punishment (prison) to keep folks from stealing them! :> And, some "errors" can be costly (the "head taking" doesn't compensate for these losses) -- e.g., if an error shuts down a production line for an hour or more, no amount of "punishment" is going to make up for the missing product, wasted salaries, lost opportunities, etc.

I'd prefer a solution that tries real hard not to be broken (perhaps I'm too much of a Cynic to expect folks to read the 'Do Not Touch -- under penalty of Death' signs :> )

Cost of a "real" switch isn't significant (consider the cost of designing and producing something functionally equivalent). And, I expect the SOHO kit will eventually catch up as homes get more "wired" (I think we are the only wired house in the neighborhood). The downside to the COTS approach is it *only* addresses that one problem. I.e., if you look at the switch, etc. as components in a *traditional* design ("discretes", etc.), then you realize you have more options available to provide the functionality you need -- beyond the *obvious* ones. E.g., if you need to convert an analog value to a "digital" form, the "A-to-D" can be performed in a wide variety of means besides the "obvious".

I don't want to get sucked into the sort of COTS designs of which "automated checkouts" have become the bane (lots of boxes thrown together in a piss-poor attempt at solving what could otherwise be accomplished more effectively if the designers had started with a clean sheet of paper).

Hi Don

It depends on what you want to base your connection-by-connection limitation by.

If it is only the connection number per client then it is "simple".

or

Do you want to limit:

• each individual destination port?

• a small number of groups of destination ports classified by e.g. interactive (http,telnet) non-interactive (ftp,http-filetransfer,P2P) etc.? for each client.

Do you want to control each connections bandwidth?

Do you want to control each at the WAN? Cisco ASA firewalls can track connections numbers.

Quote: "... # stateful packet filtering (IPv4 and IPv6) [

Ideally, I want to control the bandwidth allocated to each individual connection -- i.e., (from, to, port)-tuple.

However, the constraints placed on those connections may be dynamic and/or interrelated with other connections. E.g., the total bandwidth allocated to a particular *thread* -- or process -- might be constrained and the multiple connections open *within* that task/process might have to share that allocation (using criteria that I define).

Or, the total bandwidth between a particular src_ip and dest_ip might be constrained and the various connections on that particular src_ip have to share that bandwidth (using some criteria).

Most of the traffic is proprietary RPC invocations et al. So, it is unlikely that a COTS switch would be able to recognize the particular protocol, "registered ports", etc. (though I have to be able to tolerate -- and throttle -- traffic from The Outside using well-known ports/protocols).

My stack has hooks that give me reasonably fine control over outbound traffic without the client having to be involved in this issue. I.e., a "throttling task" can update individual "quotas" for each connection based on whatever criteria I impose.

My original approach was to coordinate the activities of these "throttling tasks" among the various (cooperating) nodes in the system. Since all *inbound* traffic had to originate as

*outbound* traffic on some *other* node, I can control the inbound traffic *at* at node by simply (!) telling the throttling task(s) on each of those nodes how to throttle their own local tasks (connections).

This won't work for "outside" connections.

One solution (which currently appears the most economical) is to add another node that acts as a gateway to the outside world. It's sole purpose would be to impose throttling on the "system" side of the node -- i.e., the "out-side" looks like a normal network node while the "in-side" is constrained to follow my rules.

But, this means any "foreign" connections to the system *must* go through this gateway box. Any such connections which bypass the gateway device compromise the integrity of the system -- in much the same way that adding a bus-mastering controller to a PC (see my PC analogy, elsewhere) that wants to grab the physical bus for X% of the time "steals" bandwidth from the rest of the PC.

As I mentioned elsewhere, networks are now considered somewhat ubiquitous. Folks think they can plug something into a switch just because they know how a CAT5/6 connector "works". (e.g., note how PoE takes pains to note that it is very likely that folks will plug *telco* kit into PoE-enabled connections "because the connector *seems* to fit"). I can't risk the system misbehaving because someone did something they shouldn't have: "Ooops! I'm /sorry/!"

I'd have to see if this is driven by generic algorithms or tailored to particular protocols (i.e., my RPC is "none of the above" -- would I be able to configure things to recognize my "protocol"/application?)

I should be able to avoid all of these issues if I can get the right mechanism(s) in place -- instead of being REactive, I will be PROactive.

I'll have to chase this down.

This isn't an issue as my nodes are designed to work as compatible "components" -- adhering to whatever rules I want to impose.

However, I would have to see how this impacts an external connection that isn't "IntServ-aware" (?)

Again, for the co-operating nodes in the system, this isn't a problem as they already *want* to cooperate. As I mentioned elsewhere, it's "easy" if you can put a box around the system/device and *prevent* others from connecting/interfering.

Hmmm... I wonder if a *cheap* solution might be to repackage an existing switch and replacing the connectors with something nonstandard. I.e., so a user couldn't "casually" plug a foreign device into the switch. And, package that "gateway node" inside this switch *with* a regular connection so any attempt to plug a "foreign" device in would be forced to go through that "gateway"...

[I need to think on this and see if it might be a cheaper solution...]

Try