Embedded Linux: share data among different processes

I'm new to embedded Linux so this question could be very simple for many of you. Most probably, it isn't directly related to embedded world, but to Linux OS generally. Anyway I think it is a common scenario in embedded applications.

I'm going to develop a local/remote control of an electronic device. It communicates through a RS485 link. The local control will be a touch-screen display (I'm going to use QT graphic libraries). The remote control will be HTTP (web server).

I think a good approach will be to develop a simple application, the poller, that communicates with the electronic device and implements the RS485 protocol. The poller continuously acquires the current status/settings of the device and store them in some "shared" way.

The graphic application (QT-based) and the web server (CGI) should access to the data retrieved by the poller.

What is the best method to share the data generated by an application (the poller) among two or more applications (QT and CGI)? In this scenario, I think it's important to lock the "shared data" before accessing them (reading or writing), in order to avoid reading incoerent data. Indeed, if the poller writes the data at the same time (Linux OS is multi-tasking) the web server reads them, they could be incoerent.

I'm thinking to use SQLite database to store the data. The poller writes the database, HTTP and QT reads from it. It seems SQLite will take care the multi-thread/multi-process scenario.

Any suggestions?

Reply to
pozz
Loading thread data ...

do you need to keep the data from the RS485 or not?

If not a shared memory or (named) pipe or something like this accessed thro ugh a semaphore or a mutex is IMHO better. Some reading:

formatting link
formatting link
ng.com/alp-folder/

On the other hand, if you need to keep the data then a DB is the way to go.

Bye Jack

Reply to
Jack

formatting link
folder/alp-ch05-ipc.pdf

formatting link
folder/

Since you already using Qt, I can propose a solution which I use quite often. It is even flying in more than 100 helicopters. Yes, penguins can fly :-)

The poller fetches that data from the remote device via RS485 and keeps a for it local copy. If permanent storage is a requirement it can also take care of this. In addition it open a QLocalServer which listens for connections from other processes. The other (multiple) processes connect to this server using a QLocalSocket. They can then poll are are sent the data directly when the poller receives new data. This way no locking is required since the poller does its sending in a single thread. You just have to define a simple protocol. It could be line oriented ASCII.

You can also use QTcpServer/QTcpSocket instead. Then you could even have some processes running on different machines (or virtual ones).

The other - more complicated - option would be to use DBUS.

This Server/Socket combination has the advantage that it easily integrates into the signal/slot mechanism of Qt.

--
Reinhardt
Reply to
Reinhardt Behm

I did that with System V IPC, using message passing for control and shared memory blocks for info common to all the processes. I was able to avoid any locking using the one-writer/many-readers trick. The code was more low-level than many people, perhaps, would like, but it worked well.

Mel.

Reply to
Mel Wilson

It depends on how "lightweight" you want the mechanism to be. And, how "elegant".

If you can implement the "Shared Memory" extensions (a kernel option in the *BSD's -- not sure re: Linux) then this is the easiest way forward. Look for shmget(2), shmat(2), shmctl(2), et al.

Use a semaphore to control access (and discipline for thereaders/writers to follow that convention) and keep atomic regions short and sweet.

If the data isn't changing all that often, you can push it to each consumer through pipes and count on them to keep their own copies intact. (here, there be dragons)

The DBMS approach is how I am currently addressing all "persistent data". It's a fair bit heavier-handed (I've got lots of MIPS to spare) but is more elegant; a producer can't inject data into the store unless the data is appropriate for the tables/fields involved (you can't store text somewhere that expects numerics!). Likewise, the consumers need not *check* the data because the checks can be applied on admission to the DB. I.e., if the data is *in* the DB, then you know it satisfied the admission criteria!

However, I don't think SQLite has all of these hooks.

GNeuner is your go-to guy for the DB approach...

Reply to
Don Y

This is the classic and probably easiest approach, though not the most economical in terms of machine use. There are other databases besides sqlite that you can also consider. How much data are you talking about? Do you need stuff like persistence across reboots?

Reply to
Paul Rubin

1) Qt and HTTP are completely disjoint. Use of Qt is disruptive - it has its own pre-preprocessor and ... keywords specific to Qt. If I could, I would pick one and possibly carve the other off as a separate project.

HTTP is, simply put, a bloated, obese protocol which cannot be put on a decent timeline. It's intentionally entanlged with TCP in very unfortunate ways.

2) You mean you will poll for data over an RS485 link, then have two possible clients on a workstation/phone/tablet/whatever that exploit this data. 2a) Is the RS485 link half or full duplex?

2b) If it is half duplex, do you have an accurate model of the line discipline? For example, with MODBUS ( which is 1/2 dux 485 and address muxed ) there will be hard[1] timers to be managed. I recommend a state machine approach. The port cannot be used by more than one request/response pair at a time and response timeouts dominate latency.

[1] although they can be made to be adaptive and a select()/poll()/epoll() approach can be quite useful. Much depends on whether the responding node identifies itself in responses. But depending on your serial port architecture, this may get weird.

3) SQLite appears to have locking:

formatting link

3a) SQL is a bloated pain in the neck.

4) A minimal technology for a shared data store under Linux is shared memory. There are shared memory filesystems or you can develop a shared library/object file which enacts an access protocol for the shared data using the System V primitives. See "System V shmem" and "System V semaphores" through Google for details. This isn't difficult but it's fiddly. It's also the highest performing approach. I;d at least consider guarding the shmem with a semaphore, although I've never established that this is completely necessary

5) Consider using pipes for shared data, possibly. Dunno how you integrate pipes with an HTTP daemon.

6) Sockets are also a good thing to consider using, although again, I have never integrated an HTTP server with sockets for another protocol. Your "poller" offers say, a TCP socket ( to be used over localhost ) and does "multiple unicast" or alternatively actual multicast transmission of the entire data store every so often, or differential ( the things that have changed ) transmission as needed.

6a) Multicast constrains you to the use of UDP.

This is a nontrivial project.

All this being said, there may be a good software kit to solve this problem out there; I just don't know what it is.

and

7) I've seen cases where a COTS HTTP server was used to serve up Java applications to do this. That's kind of old school at this writing.
--
Les Cargill
Reply to
Les Cargill

I did something like this (TCP over USB between the base unit and the display) and it wasn't so bad. Yes there was a ton of bloaty system code under the surface (language interpreter, network stack etc.) but who cares? The application itself was reasonably simple. The ARM CPU and 64MB of ram were a few dollars, and the "disk" was a 2GB microSD card that was another couple bucks. The user application was written like any other client-server app. The shared data was managed in the application itself, that ran in a single process (multiple threads), but using a database works perfectly well if you don't mind the overhead. If you have the memory for it, Redis could be a good alternative to SQLite, depending on what the application needs.

Reply to
Paul Rubin

SQLite can be run completely in RAM (backed by swap if applicable).

Although not possible out-of-the-box, wIth some hacking it can be made to run from a memory mapped file (for persistence).

If you have relational data, I believe SQLite is the lightest weight file based RDBMS available. There are a handful of RAM-only libraries that accept SQL-like query languages.

If you need only something simple such as key-value pairs, then there are many possible implementations to choose from.

Excellent questions.

George

Reply to
George Neuner

SQLite "takes care of" that by using a single giant lock. That might work for you, since it sounds like you probably have very easy throughput and latency demands. It would be a good solution if you weren't using Qt. Les Cargill hasn't seen how remarkably lightweight SQLite actually is.

Reinhardt, do you deal with write-blocking if the reader fails? If the write is in the poller thread, any blocking on writes will delay the polling - unless you use non-blocking writes.

I think it's good advice, though I wouldn't personally have chosen Qt.

This application is not demanding enough to need a shared memory solution, with the attendant need for locks and especially, memory barriers, of which most respondents seem ill-educated. In particular, Mel Wilson's "one writer, many reader" is no longer so simple with modern multi-core architectures. Writes do not have to happen in the order they are written, so readers can get confused.

Clifford Heath.

Reply to
Clifford Heath

That's only a problem if you "roll your own" locks. IME, the SysV shared memory operations also carry along the SysV semaphores.

And, you *want* the OS to know about any locks you've implemented as, inevitably, you'll want to add another "client" down the road and then have to worry about the various types of deadlock that can occur (that the OS can mitigate, in some instances).

E.g., the "single giant lock" that SQLite uses can similarly be implemented to control ALL accesses to the shared memory segment.

You also have the advantage of *seeing* all of the code that is accessing these shared objects -- instead of wondering what SQLite is doing and when its doing it.

Reply to
Don Y

As I said, ill-educated.

As you'll find out when the SysV semaphore grants you access to some shared memory resource that has been written, but only some of the writes have been flushed.

If you're on hardware that can do this, you need to know about it, or leave the work to someone else who does... like the authors of SQLite.

Clifford Heath.

Reply to
Clifford Heath

I'd appreciate a definitive reference backing your claim.

And, the folks authoring the semaphore functions *don't*?

Reply to
Don Y

Qt internally uses non-blocking read and write. So this is not a problem.

Because of that I prefer to avoid all these locking if it is not needed. Having the data at one place controlled by one thread and send it from there to interested clients via some IPC mechanism make live much easier. And it is also a very effective encapsulation. The other can not even know, how it handled internally in the server. I would use shared memory or similar close coupled methods only if necessary for performance or other requirements.

--
Reinhardt
Reply to
Reinhardt Behm

The semaphore functions are not guaranteed to flush CPU caches or prevent out-of-order execution of writes to the shared memory. Why would they? The semaphore semantics are not intertwined with the memory semantics.

It's common for entry to the kernel to flush the cache, but you need to know - it doesn't happen by magic. But out-of-order execution can still spoil your day.

In some architectures, even a single "increment-memory" instruction can allow another CPU to observe the memory cell to be half-incremented. It's all very troubling...

Reply to
Clifford Heath

My mutex functions do exactly that -- to ensure the synchronization primitive works regardless of instruction reordering, the presence of multiple cores, etc. The whole point is to make it possible for a piece of code to be migrated to a different platform (with different hardware capabilities) WITHOUT having to reexamine each potential problem area.

I have no idea how the Linux kernel implements the SysV semaphores (or POSIX semaphores). I will have to look at the FreeBSD and NetBSD sources to see how *they* do. I'd be surprised if a kernel capable of running on such hardware would neglect these sorts of things.

All the more reason to embed these guarantees in these library functions.

Reply to
Don Y

You're still missing the point. The synchronisation primitives can fulfil their contract perfectly - with regard to the semaphore operations - but the memory write behaviour is not part of that contract. Only the semaphore operations are in the contract, and nothing about that says that all memory accesses on both sides must be coherent across all processor cores.

Your mutex functions perfectly, I'm sure - but do they guarantee that memory operations either side of the mutex - and not referencing the mutex's memory - is coherent across multiple cores. Note that I am not talking about the memory used to implement the mutex itself, but other memory.

Reply to
Clifford Heath

How? If threadX and threadY (operating on the same processor *or* on different cores) can see different orderings, then how is the contract satisfied? It's not an "increment memory" operation...

What value is a synchronization primitive if it doesn't provide synchronization? How is it fulfilling its contract if it *doesn't*?

All synchronization operations MUST perform barriers to memory and operator reordering.

Yes -- as they include memory barriers specific to the particular processor architecture involved. The whole point is NOT to force the developer to have to wonder whether it's a single-threaded environment or multithreaded; uniprocessor or multiprocessor. The function call is a fence.

From a quick peek through some of the online Linux sources, their SysV semaphores also seem to operate in a similar manner. I've not yet looked through the *BSD sources...

Are you claiming the compiler can reorder instructions *across* an opaque system call? E.g., decide to invoke the code at main() AFTER the code at exit()?? (what's to prevent this??)

Reply to
Don Y

You're just hiding by wrapping it in an IPC mechanism.

Yes, but it is also considerably more expensive. Every consumer has to be informed of every update to the data in which it has "expressed an interest" (assuming you don't broadcast ALL data to ALL consumers).

E.g., I install triggers in my RDBMS so that any update to a particular (set of) parameter(s) results in an up-call notification of the interested parties that might want to *see* that new value (so, I don't have to broadcast all updates of all data to all consumers).

But, that upcall carries considerable cost (from trigger through upcall to client notification and eventual re-query of the RDBMS). It can effectively bottleneck the RDBMS's operation (cuz another up-call could be enqueued right behind the first one even before the first has finished!).

So, the nature of the problem changes -- to one of deciding *what* information is "genuinely of interest" to each client... AND, how timely that notification should be made (if you can defer a notification, then you might be able to eliminate it in favor of a subsequent notification)

E.g., if some client (having permission to do so!) updates the current timezone, the wall-clock display should *probably* be updated to reflect that. OTOH, it probably can wait a second or two. And, if the timezone is updated AGAIN before the notification is dispatched, the first update can safely be ignored in favor of the second.

OTOH, if *every* update must be conveyed to the client(s), then the notification effectively becomes part of the update transaction and limits how frequently the value can be updated.

IME, people are more accustomed to using shared memory solutions as this mimics the familiar interplay of foreground/background processing; people *seem* to know that some coordination must exist between an ISR and its "clients".

OTOH, sharing memory in an environment with individual, protected memory spaces can be intimidating the first time.

Reply to
Don Y

Yes, another problem to solve is the other way: QT and HTTP server could send some control commands to the electronic device (switch on, switch off, change this...).

It is exactly what I will have: one writer (the poller that will continuously fetch updated data from the device) and some readers (at least, HTTP server and QT graphic libraries). Doesn't it needed a syncronization mechanism (semaphore) in this scenario, using shared memory to share common data?

Reply to
pozz

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.