Atomicity in network protocols

You have the case where the validity of a setting may depend on multiple other current settings.

I reckon the only really safe way to handle this is to allow requesting any number of parameters in a single request, and also to allow setting any number in a single request, as long as the combination of all values is valid at the time the request is processed. If a (compound) setting is denied, the client gets told why.

This way, a client asks for the current settings, formulate a new compound setting, and if that fails because its notion of the current values is out of date, it can repeat the request until it succeeds or gives up.

If you need to apply the change only in the case where the "current settings" on which you relied haven't changed, include those as gating conditions in the change request.

I.e. say what you relied on, and what you want, all in one request.

Clifford Heath.

Reply to
Clifford Heath
Loading thread data ...

He's talking about TCP/IP networking, where you have to allow for congestion, dropped packets, hosts silently failing, etc.

Reply to
Nobody

I'm probably missing something, but: a) multiple clients want to use the server to access device X. (There may be more devices, but lets look at the one device)

b) this device performs measurements of a single type, such as room temperature or radiation levels or something.

c) the server can configure the device to measure the parameter at the best range/resolution available to it.

d) the server can then alter the scale / offset to suite the clients' reuqirements on an individual basis.

i.e. If the device is measuring 1 thing then why can't it do this at the best resolution compatible with its range and let the server do the presentation as needed?

As I said - I'm probably missing something.

Reply to
Rocky

From the OP, I did not get the impression that his system had to be working over the internet with it's possibly long delays, but I might be wrong. A TCP/IP connection from one device to another is just a pipe. On top of that pipe he's free to implement any protocol including tight timing constraints and the ability to deal with that. I still haven't seen any valid reason not to have a time-out on every transaction.

Meindert

Reply to
Meindert Sprang

Yes. A server may even "handle" multiple devices (of the same or different "types")

Yes. If a device can provide multiple (types of) measurements, those are treated as two different "devices".

Not quite (this may be what you're "missing" :> )

The server configures the "device" -- based on the competing and overlapping requirements of the *current* set of clients using that device.

Many devices have different "operating ranges". And, their resolution, accuracy, reporting rate, etc. may differ across each of these ranges.

E.g., a device might be able to give you "readings" in a certain measurement range at one "resolution" (let's avoid resolution vs. precision discussion, here) or "accuracy" (etc.) at a certain

*rate*...

*OR*

over the same range at a *different* resolution (etc.)-- but at a different *rate*.

For a simple mechanical analogy, imagine a shaft encoder that outputs one pulse per revolution. In a given operating range (e.g., 5 - 10 RPS... easier to deal with than RPM :> ) you can get a measurement in ~200ms (anything faster than that can't account for the shaft's speed falling *outside* that range, even if only temporarily).

How precisely you can report that "speed" depends on how finely you can measure the interval between pulses -- in that 200ms "reporting window".

For the same range of operating values, if you increase the reporting window to, e.g., 1 sec, then you can gain a fivefold increase in resolution (at the expense of assuming the shaft's speed remains constant over that interval).

If you (a client) are only interested in verifying that the shaft is rotating at "about 6 RPS", then you can get this information quickly. OTOH, if you want to know the precise speed of the shaft (to some number of decimals), you might opt to trade reporting rate (window size) for that increased resolution.

[Extending this example, if you are interested in the 50 - 60 RPS operating range, the same sorts of data may be available an order of magnitude quicker, etc.]

See above. Any particular client will typically trade between the resolution, precision, accuracy/error, latency, reporting rate, operating range, etc. constraints as best fits its needs. Further, those needs might change dynamically.

Add to this the fact that multiple clients are imposing their

*competing* requirements on the device concurrently. [And, finally, that the clients don't *know* what the capabilities of the device are a priori]
Reply to
Don Y

Note my comment re: "*network* protocols". We're not talking about a network designed for *deterministic* communications.

Can you tell me how long it takes for a packet to *leave* your workstation? Note, I am not even addressing how long it takes to get to it's destination. Just tell me, from the time you "write()" the message (assume it fits in a *single* TCP/IP segment), how long before it appears "on the wire".

And, how does this vary based on what *else* your workstation might be doing at the time?

Following that packet that just "hit the wire", how long before it makes its way through the various switches and routers between its source and destination *hosts*?

Then, howlong before it works its way back *up* through the receiving stack to the application layer on its destination? And, how long before the scheduler passes control to the task that is actually doing the read()? Again, how do other activities on that host interfere with this transport/processing delay?

Very few (traditional) network stacks offer any sort of guarantees regarding their performance. The internetwork itself likewise offers only loose "guarantees" ("best effort") about packet delivery. Hence the retries and *long* timeouts built into the underlying protocols, themselves.

Because I can't force a customer to add a completely separate network infrastructure *just* to provide more determinism in message delivery. Likewise, telling him that he can only connect particular "approved" devices to this fabric... that all of the clients must be specially designed to run on OS's with more deterministic performance characteristics, etc.

(Well, I *can* tell them to do these things but then the TCO goes up by an order of magnitude!)

This ends up being a kludge and needs to be "tuned" to a specific installation -- *and* the traffic in place at the time of the tuning! Otherwise, the system can start to oscillate as clients ask for things, get *partial* successes, eventually encounter dead-ends ("No, you can't get the resource that you want in the flavor that you need"),

*release* or *revise* their current holdings, try again, etc.
Reply to
Don Y

Yes. And, those settings can be changing while I'm "making up my mind" (i.e., the value that the LOCK provides is that it "gives me time to think")

Except the client needs to query the server to know what the current "possibilities" are.

That's what I do, presently -- except for a single setting at a time. The problem doesn't (typically) creep in between the setting of

*different* parameters but, rather, the window of vulnerability between querying the "current possibilities" and *acting* upon them.

But, there's no need to restate that. If I say "I am counting on the fact that I can specify a HIGH limit of 27.8 and that is why I am SETting a HIGH liimit of 27.8", this is no different than simply saying "SET HIGH 27.8" -- and waiting for PASS/FAIL.

I.e., the server doesn't *care* what I am relying on -- since it can't guarantee that it will be available.

E.g., the very first message I send to the server can be a "SET" even before I issue any queries! The server doesn't care that I don't know what the current "possibilities" (capabilities) happen to be; if it can satisfy my request, it will. If not, it won't.

Reply to
Don Y

"The Internet"? Only rarely. *An* internet? All the time!

As I said, elsewhere, network stacks typically don't provide *any* timeliness guarantees. There's nothing that guarantees your packets will hit the wire in *any* timeframe! (show me *a* number published for your commercial OS on your iron with some particular set of competing applications, etc.)

"Double" this uncertainty to account for the host on the other end of the pipe.

*Then*, think about the switches, hubs, bridges, routers, etc. that could lie between here and there.

Relying on timing is almost always A Bad Thing.

Reply to
Don Y

Is your workstation embedded, or is this conversation off topic?

--
Fredrik Östman
Reply to
Fredrik Östman

Hi Don,

I don't agree with you here. A time out is the only way to bail out of an unknown lockup situation simply because the time out event will *always* happen as opposed to a response from "the other side".

I agree that there are no guaranteed fixed numbers regarding delays in a TCP/IP connection. But you could do some testing on an "average" network where your application should run to determine worst case delays and design your time-out mechanism based on those figures. Which pops up the question: how reliable is a ping turnaround time to get an indication? TCP/IP is supposed to be a reliable connection. Does any of the RFC's specify a guaranteed delivery time of data? By the way: setting up a TCP/IP connection is also guarded by a time out.

Meindert

Reply to
Meindert Sprang

The "workstations" in this design are "embedded". Most aren't really even "workstations" but, rather, simply nodes that are calling upon resources served by other nodes to achieve some goal.

E.g., a "controller" could be querying different "devices" (served up by the mechanism outlined here) and, in turn, driving "actuators" served up with another (not mentioned) mechanism.

The point of my "workstation" comment was to draw attention that few network stacks have hard and fast guarantees on their services. It's the very nature of the network to be non-deterministic...

Reply to
Don Y

^^^^^^^^^^^^^^^^^^^^^^^^

Yes, but this isn't an "unknown" lockup situation! This is a deliberate LOCK with the intent of a followup UNLOCK. Timeouts to detect ERROR conditions are a different thing and are comparatively *long*... long enough to account for the worst case packet delays, collisions, retries, etc. that a *legitimate* "UNLOCK packet" might encounter.

I.e., you (I) could use a timeout to prevent a LOCK from

*erroneously* being held (too long). But, to impose a timeout to cause the LOCK to deliberately be broken in a (relatively) short time just to ensure "fairness" leaves the protocol exposed to all the timing vagaries associated with network stacks, etc.

E.g., you (I) could develop a LOCK protocol that allows the lock to be held *indefinitely*, provided the client uses keep-alives to keep informing the server "Yes, I still want to hold the lock". This allows a server that needs "a long time" to come to a decision about its subsequent actions to exceed some "nominal" time to hold a lock. If the client stops sending keep-alives or drops the connection, then the server can automatically break the lock.

But, these are just mechanisms that allow a greedy/lazy client to grab a resource selfishly rather than cooperating with other clients. As long as he follows the rules for

*holding* that lock (indefinitely!), his abuse is tolerated.

I want protocols to reward/encourage *good* behavior rather than reward *bad*.

E.g., let him hold the lock indefinitely -- but, never be able to leave the "configuration" aspect of the connection. I.e., he can set/query as much as he wants -- but can never receive any *data* from the connection (which, presumably, is his ultimate intent!). *Then*, after he releases the lock, require him to reacquire it and RESTATE his requirements. I.e., his first pass was just a "dry run". Now, if some other client (that had been waiting while he was monopolizing the service) comes in and *changes* the configuration, the previous client's abuses gain him nothing -- he's back to square one (though the other clients are, as well!)

[sorry, I'm not sure if my suggested intent is clear, here. It's too early in the morning for cogent thought! :< ]

The "normal" timeouts tend to be very long in the underlying protocols. E.g., I may be trying to contact a host across the world via some odd combination of transport media. So the protocol has to take into account those inherent transport and connection delays along the way.

Reply to
Don Y
[much elided -- hopefully the essence remains]

I've been stewing on this trying to identify and *attack* the unspoken assumptions I made in its original formulation (since I believe damn near all problems boil down to the wrong assumptions! :> )

I think the problem with this original protocol is it violates the "make everything as simple as possible -- but no simpler" adage. I.e., I have pruned it down to the point where it tries to address *every* requirement of the interface with a single set of verbs, etc.

There are a couple of different activities happening over this interface and the protocol makes no provisions for differentiating among them.

So, stepping back and looking at them individually opens the door for a better/different understanding.

The first responsibility of the interface is to advertise a device's "capabilities". Clients don't (can't!) know what to expect from devices so a means of garnering that information from the server(s) is required. I.e., A "What are your capabilities?" query (low/high range, rate, latency, accuracy, error, resolution, precision, history, etc.).

But, many devices don't have a single/simple means of expressing their capabilities. I.e., their capabilities vary based on how they are being *used*. This allows "instruments" to address a greater range of applications by trading off one "parameter" for another, etc.

This is what led to the (unfortunate?) design of the single verb, single parameter grammar for interacting with the server. It allows the client to pick *a* parameter of interest to it, query the possible value(s) for that parameter (to determine what is *legal*), *pick* a particular value for that parameter that is in line with its requirements and *then* move on to each of the remaining parameters, in order of relative importance.

Informally, the dialog is something like: "What's the greatest you can measure/report?" "That would be 27.8" "OK, so 25 won't exceed that. If I pick 25 for the HIGH, then what is the smallest you will be able to report?" "That would be 12" "OK, so if I pick 15 then you should be able to handle it. So, with those LOW/HIGH settings, what level of accuracy can you give me?" ...

Note that, in this interchange, the "SETting" of each parameter is purely hypothetical. The actual device need not reflect these choices AT THIS TIME.

So, if I treat the DISCOVERY aspect of the interface as a separate activity (with its own set of verbs or whatever), then the actions that a particular client undertakes during this phase need not have any impact on the actual operating configuration of the device/server.

I.e., a client can spend an eternity UNCONSTRAINED BY TIME exploring the various *theoretical* options available to it regardless of what the other clients are doing at the time. The need for a LOCK goes away (in this aspect of the interface) as there are no competitors. "We're just tossing some ideas around, here..."

Aside from the number of octets pushed back and forth across the interface (and the time required to do so), there is no cost to the client nor the "system" for the client to enumerate every conceivable "configuration" of interest to it. The server need only "remember" the "currently being explored" configuration (i.e., the service scales no worse than linearly with number of clients).

So, this takes care of "discovery". And, since "discovery" has nothing to do with actual device *operation*, it can be invoked at any time! A client could conceivably create multiple connections and explore multiple configurations concurrently (why? :< )

[I have reread the posts in this thread and don't *think* anyone suggested this. Or, if they did, I apologize for being too stuck in my mindset to not see the subtlety of treating this "discovery" phase as a distinct aspect of the interface.]

Now, armed with a (list of?) compatible configuration(s) that the client *knows* the device *could* accommodate, the actual "configuration" phase can be addressed. I.e., creating the contract for this particular "data channel" connection to follow.

Since the client knows all of the values of the parameters for the specific configuration desired, it can specify ALL of them in a single CONFIGURE command. This can be an atomic operation as far as the client/server are concerned.

The key issue that remains is resolving contention among competing client CONFIGURE commands.

It *seems* like the easiest approach is "first come, first served". And, since there are no timing guarantees anywhere in the protocol, the determination of who is first is left entirely to the server implementation. I.e., if it wants to look at relative timestamps of incoming packets, it can. Or, it can simply be "whichever thread grabbed the "configured" mutex first, etc.

So, CONFIGURE can return FAIL even if all of the parameters within it represent a potentially valid configuration. This is no different than a SET failing because of a conflict with some other parameter in the previous protocol definition.

However, the client can then try the *next* potential configuration (that it had previously determined as a potential alternative) without having to go through the QUERY/SET dialog given this "configuration currently adopted by the server".

I'm presently stewing over the idea of passing a *list* of potential configurations to the server as an efficiency hack. I.e., if the first configuration is not available, then try the second, etc. Return FAIL iff none are accceptable.

While this seems like it is *just* an efficiency hack, it may also open the door to performance enhancements after the connection has been established.

E.g., imagine if some client disconnects and the removal of those contractual obligations makes available a "first choice" configuration that had previously been "unavailable" for some other client. The server could *upgrade* the connection's configuration for that client.

[this opens another can of worms, though, since the server would have to ensure all other existing connections remain viable, etc. It would also necessitate an upcall to the client informing it of the possibility of upgrading the connection: "Hey! I can give you that better configuration, now, if you would like it... well??"]
Reply to
Don Y

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.