Embedded Linux: share data among different processes

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
I'm new to embedded Linux so this question could be very simple for many  
of you. Most probably, it isn't directly related to embedded world, but  
to Linux OS generally. Anyway I think it is a common scenario in  
embedded applications.

I'm going to develop a local/remote control of an electronic device. It  
communicates through a RS485 link.
The local control will be a touch-screen display (I'm going to use QT  
graphic libraries).
The remote control will be HTTP (web server).

I think a good approach will be to develop a simple application, the  
poller, that communicates with the electronic device and implements the  
RS485 protocol. The poller continuously acquires the current  
status/settings of the device and store them in some "shared" way.

The graphic application (QT-based) and the web server (CGI) should  
access to the data retrieved by the poller.

What is the best method to share the data generated by an application  
(the poller) among two or more applications (QT and CGI)?
In this scenario, I think it's important to lock the "shared data"  
before accessing them (reading or writing), in order to avoid reading  
incoerent data. Indeed, if the poller writes the data at the same time  
(Linux OS is multi-tasking) the web server reads them, they could be  
incoerent.

I'm thinking to use SQLite database to store the data. The poller writes  
the database, HTTP and QT reads from it. It seems SQLite will take care  
the multi-thread/multi-process scenario.

Any suggestions?

Re: Embedded Linux: share data among different processes

Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it

do you need to keep the data from the RS485 or not?

If not a shared memory or (named) pipe or something like this accessed thro
ugh a semaphore or a mutex is IMHO better.
Some reading:
http://web.archive.org/web/20150524005553/http://advancedlinuxprogramming.c
om/alp-folder/alp-ch05-ipc.pdf
http://web.archive.org/web/20150511104538/http://www.advancedlinuxprogrammi
ng.com/alp-folder/

On the other hand, if you need to keep the data then a DB is the way to go.

Bye Jack

Re: Embedded Linux: share data among different processes
Jack wrote:


Quoted text here. Click to load it
http://web.archive.org/web/20150524005553/http://advancedlinuxprogramming.com/alp-
folder/alp-ch05-ipc.pdf
Quoted text here. Click to load it
http://web.archive.org/web/20150511104538/http://www.advancedlinuxprogramming.com/alp-
folder/
Quoted text here. Click to load it

Since you already using Qt, I can propose a solution which I use quite  
often. It is even flying in more than 100 helicopters. Yes, penguins can fly  
:-)

The poller fetches that data from the remote device via RS485 and keeps a  
for it local copy. If permanent storage is a requirement it can also take  
care of this.
In addition it open a QLocalServer which listens for connections from other  
processes.
The other (multiple) processes connect to this server using a QLocalSocket.  
They can then poll are are sent the data directly when the poller receives  
new data. This way no locking is required since the poller does its sending  
in a single thread.
You just have to define a simple protocol. It could be line oriented ASCII.

You can also use QTcpServer/QTcpSocket instead. Then you could even have  
some processes running on different machines (or virtual ones).

The other - more complicated - option would be to use DBUS.

This Server/Socket combination has the advantage that it easily integrates  
into the signal/slot mechanism of Qt.

--  
Reinhardt


Re: Embedded Linux: share data among different processes
On 23/06/16 23:05, Reinhardt Behm wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

SQLite "takes care of" that by using a single giant lock.
That might work for you, since it sounds like you probably
have very easy throughput and latency demands. It would
be a good solution if you weren't using Qt. Les Cargill
hasn't seen how remarkably lightweight SQLite actually is.

Quoted text here. Click to load it

Reinhardt, do you deal with write-blocking if the reader fails?
If the write is in the poller thread, any blocking on writes will
delay the polling - unless you use non-blocking writes.

Quoted text here. Click to load it

I think it's good advice, though I wouldn't personally have chosen Qt.

This application is not demanding enough to need a shared memory
solution, with the attendant need for locks and especially,
memory barriers, of which most respondents seem ill-educated.
In particular, Mel Wilson's "one writer, many reader" is no
longer so simple with modern multi-core architectures. Writes
do not have to happen in the order they are written, so readers
can get confused.

Clifford Heath.


Re: Embedded Linux: share data among different processes
On 6/23/2016 4:45 PM, Clifford Heath wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

That's only a problem if you "roll your own" locks.  IME, the SysV
shared memory operations also carry along the SysV semaphores.

And, you *want* the OS to know about any locks you've implemented
as, inevitably, you'll want to add another "client" down the road
and then have to worry about the various types of deadlock that
can occur (that the OS can mitigate, in some instances).

E.g., the "single giant lock" that SQLite uses can similarly be
implemented to control ALL accesses to the shared memory segment.

You also have the advantage of *seeing* all of the code that is accessing
these shared objects -- instead of wondering what SQLite is doing
and when its doing it.

Quoted text here. Click to load it


Re: Embedded Linux: share data among different processes
On 24/06/16 10:19, Don Y wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

As I said, ill-educated.

As you'll find out when the SysV semaphore grants you access to
some shared memory resource that has been written, but only some
of the writes have been flushed.

If you're on hardware that can do this, you need to know about it,
or leave the work to someone else who does... like the authors of
SQLite.

Clifford Heath.

Quoted text here. Click to load it


Re: Embedded Linux: share data among different processes
On 6/23/2016 8:25 PM, Clifford Heath wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

I'd appreciate a definitive reference backing your claim.

Quoted text here. Click to load it

And, the folks authoring the semaphore functions *don't*?

Re: Embedded Linux: share data among different processes
On 24/06/16 14:27, Don Y wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

The semaphore functions are not guaranteed to flush CPU caches
or prevent out-of-order execution of writes to the shared memory.
Why would they? The semaphore semantics are not intertwined with
the memory semantics.

It's common for entry to the kernel to flush the cache, but you
need to know - it doesn't happen by magic. But out-of-order
execution can still spoil your day.

In some architectures, even a single "increment-memory" instruction
can allow another CPU to observe the memory cell to be half-incremented.
It's all very troubling...

Re: Embedded Linux: share data among different processes
On 6/23/2016 9:40 PM, Clifford Heath wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

My mutex functions do exactly that -- to ensure the synchronization
primitive works regardless of instruction reordering, the presence
of multiple cores, etc.  The whole point is to make it possible for
a piece of code to be migrated to a different platform (with different
hardware capabilities) WITHOUT having to reexamine each potential
problem area.

I have no idea how the Linux kernel implements the SysV semaphores
(or POSIX semaphores).  I will have to look at the FreeBSD and NetBSD
sources to see how *they* do.  I'd be surprised if a kernel capable of
running on such hardware would neglect these sorts of things.

Quoted text here. Click to load it

All the more reason to embed these guarantees in these library functions.

Re: Embedded Linux: share data among different processes
On 24/06/16 15:20, Don Y wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

You're still missing the point. The synchronisation primitives can
fulfil their contract perfectly - with regard to the semaphore
operations - but the memory write behaviour is not part of that
contract. Only the semaphore operations are in the contract, and
nothing about that says that all memory accesses on both sides
must be coherent across all processor cores.

Your mutex functions perfectly, I'm sure - but do they guarantee
that memory operations either side of the mutex - and not referencing
the mutex's memory - is coherent across multiple cores. Note that
I am not talking about the memory used to implement the mutex
itself, but other memory.

Quoted text here. Click to load it


Re: Embedded Linux: share data among different processes
On 6/24/2016 12:19 AM, Clifford Heath wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

How?  If threadX and threadY (operating on the same processor *or*
on different cores) can see different orderings, then how is the
contract satisfied?  It's not an "increment memory" operation...

Quoted text here. Click to load it

What value is a synchronization primitive if it doesn't provide
synchronization?  How is it fulfilling its contract if it *doesn't*?

All synchronization operations MUST perform barriers to memory and
operator reordering.

Quoted text here. Click to load it

Yes -- as they include memory barriers specific to the particular
processor architecture involved.  The whole point is NOT to force the
developer to have to wonder whether it's a single-threaded environment or  
multithreaded; uniprocessor or multiprocessor.  The function call
is a fence.

 From a quick peek through some of the online Linux sources, their SysV
semaphores also seem to operate in a similar manner.  I've not yet
looked through the *BSD sources...

Are you claiming the compiler can reorder instructions *across* an
opaque system call?  E.g., decide to invoke the code at main()
AFTER the code at exit()??  (what's to prevent this??)

Quoted text here. Click to load it


Re: Embedded Linux: share data among different processes
On 24.6.2016 ?. 07:40, Clifford Heath wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it


It can be troubling but if you know what you are doing it is not.
Just update(that part of) caches when needed, sync (serialization opcode
for power) etc. when needed after that, use lwarx/stwcx. as needed and
you can have all the synch you need. Comes at a cost obviously
so one must be careful. Those semaphore functions you talk about
must be some high level implementation targeted at having a semaphore
which is not too costly so it cares only about itself, nothing wrong
with that. But the world does not end there (I don't know where it
ends for the C libraries people use so of course it may be as troubling
as you suggest but only as long ass one relies on libraries someone
else has written).

Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/



Re: Embedded Linux: share data among different processes
Hi Dimiter,

[40-45+C lately -- PLEASE don't tell me you've still got icicles!!  :> ]

On 6/24/2016 6:38 AM, Dimiter_Popoff wrote:
Quoted text here. Click to load it

A "synchronization primitive" is useless if it doesn't ensure
the "process state" isn't being synchronized wrt other processes.
Every instance of:

     ...   // prepare some private object

     acquire_mutex(SHARED_RESOURCE)

     ...   // make private object visible as shared resource

     release_mutex(SHARED_RESOURCE)

would break if the state of the private object was still "in
transition" *inside* the critical region.  What does the mutex *do*
for the code in that case?

Anything -- and EVERYthing -- that must be done to ensure the
consistency of the object as visible to other "agents" must happen
before the mutex is granted.  Otherwise, any changes some OTHER
agent may have made to *their* "private object" won't be
available for *this* agent to see as it manipulates the shared
resource.

Lightweight "locks" that only work within a single process/thread are
of little value.  The OS needs to be involved else it can't
recover locks from crashed processes (it needs to know who was
holding the lock at the time), resolve priority conflicts, etc.

OTOH, if used *entirely* within a single process, you can
kill the process and reclaim its memory and, thus, the lock's
holder is of no significance -- the lock AND *all* of its
potential holders are gone!  E.g., when thread() is killed,
it matters little whether produce() or consume() was holding
the lock -- the effect is entirely contained within *this*
thread (process).  No *other* processes are at risk.

thread() {
    lock_t aLock
    ...
    while (FOREVER) {
        produce()
        consume()
    }
}

produce() {
     if (something) {
         process(something)
         acquire_lock(aLock)
             // make <something> available
         release_lock(aLock)
     }
     ...
}

consume() {
     if (whatever) {
         acquire_lock(aLock)
              // access <something> previously made available
         release_lock(aLock)
     }
     ...
}

[Note that the OP's application could potentially fit this form
*if* he has control of the HTTPd's implementation]

Regards to L.

Re: Embedded Linux: share data among different processes
Clifford Heath wrote:

Quoted text here. Click to load it

Qt internally uses non-blocking read and write. So this is not a problem.
Quoted text here. Click to load it

Because of that I prefer to avoid all these locking if it is not needed.  
Having the data at one place controlled by one thread and send it from there  
to interested clients via some IPC mechanism make live much easier. And it  
is also a very effective encapsulation. The other can not even know, how it  
handled internally in the server.
I would use shared memory or similar close coupled methods only if necessary  
for performance or other requirements.
  
--  
Reinhardt


Re: Embedded Linux: share data among different processes
On 6/23/2016 9:39 PM, Reinhardt Behm wrote:
Quoted text here. Click to load it

You're just hiding by wrapping it in an IPC mechanism.

Quoted text here. Click to load it

Yes, but it is also considerably more expensive.  Every consumer has to be
informed of every update to the data in which it has "expressed an interest"
(assuming you don't broadcast ALL data to ALL consumers).

E.g., I install triggers in my RDBMS so that any update to a particular
(set of) parameter(s) results in an up-call notification of the interested
parties that might want to *see* that new value (so, I don't have to
broadcast all updates of all data to all consumers).

But, that upcall carries considerable cost (from trigger through upcall to
client notification and eventual re-query of the RDBMS).  It can effectively
bottleneck the RDBMS's operation (cuz another up-call could be enqueued
right behind the first one even before the first has finished!).

So, the nature of the problem changes -- to one of deciding *what* information
is "genuinely of interest" to each client... AND, how timely that notification
should be made (if you can defer a notification, then you might be able to
eliminate it in favor of a subsequent notification)

E.g., if some client (having permission to do so!) updates the current
timezone, the wall-clock display should *probably* be updated to reflect
that.  OTOH, it probably can wait a second or two.  And, if the timezone is
updated AGAIN before the notification is dispatched, the first update can
safely be ignored in favor of the second.

OTOH, if *every* update must be conveyed to the client(s), then the
notification effectively becomes part of the update transaction and
limits how frequently the value can be updated.

Quoted text here. Click to load it

IME, people are more accustomed to using shared memory solutions as
this mimics the familiar interplay of foreground/background processing;
people *seem* to know that some coordination must exist between an
ISR and its "clients".

OTOH, sharing memory in an environment with individual, protected memory
spaces can be intimidating the first time.

Re: Embedded Linux: share data among different processes
Don Y wrote:

Quoted text here. Click to load it

He? What am I hiding?
I proposed that the server (poller) collects the data from the device and  
sends _copies_ of it to the clients. This can happen in a single thread. So  
there is no need for any kind of locking. So there is nothing hidden behind  
an IPC mechanism.
My proposed solution uses internally Unix domain sockets and could as well  
use TCP sockets which would give the flexibility to work over different  
machines. It is proven to work in a DO-178 certified system where several of  
these mechanisms are at work to convey data from GNSS, TCAS, AHARS, camera  
control, Iridium satcomm, each with its own server. And it works in both  
directions.

Quoted text here. Click to load it

For many application scenarios the throughput and overhead is of no concern.  
And yes, I broadcast all data to all clients.

Quoted text here. Click to load it

I was never proposing to use a database system.

--  
Reinhardt


Re: Embedded Linux: share data among different processes
On 6/25/2016 6:34 PM, Reinhardt Behm wrote:
Quoted text here. Click to load it

The fact that only a single task can access the "original data".
You've got a "lock" in the IPC.  Your *one* task can't compete
with itself WHILE it is busy in the IPC.

And, none of the consumers can see anything until the IPC completes
(the receive() is atomic).

Quoted text here. Click to load it

See above.  You could add multiple "server threads" and still hide
the contention from your clients -- assuming no two server threads tried
to update the same client with the same/different data.

Quoted text here. Click to load it

I'm not saying it *won't* work.  I'm just saying that you've implemented
the locking functionality in a different manner.  If the "poller" provided
data at a faster rate than you could "broadcast" it, you'd lose data
or have to introduce another server thread -- and then deal with contention
on the server side of the IPC fence.

Don't get me wrong; my entire OS is based on message-passing.  Every function
call is a message disguised as a function.  So, making a kernel request
uses the exact same mechanisms as sending data to another local process -- or,
a *remote* process.

But, there are (significant) costs associated with this.  Crossing a protection
boundary means kernel involvement.  And, you're not just passing "payload"
but, also, the overhead of the IPC/RPC "function", marshalling arguments,
etc.

[In my case, it's a net win because the target of an IPC can be virtually
and physically relocated.  So, a call that is an IPC *now* may be an RPC
a few moments from now!  The abstraction that passing messages (vs. sharing
memory) provides is worth its cost.  (I can also share memory across
nodes but that leads to disappointment when you think in terms of LOCALLY
shared memory -- only to discover that the memory has now migrated to
another, remote node.  And, accesses are SLOWER than passing messages
would be!)]

Quoted text here. Click to load it

Of course!  I'm moving to multicore nodes simply with the idea of dedicating
a core to RPC.  Silicon is cheap, nowadays.

Quoted text here. Click to load it

Using broadcast protocols?  Or, iterating over them?

Quoted text here. Click to load it

I was indicating how I *selectively* distribute data "to folks who have
previously REGISTERED an interest".  There are thousands of processes
active in my system at any given time.  Virtually *all* of them are
unconcerned with any *particular* "data update", yet may have keen
interest in SOME update at some time!  So, there is a huge potential
for waste if I unconditionally broadcast ALL updates to everyone.

Re: Embedded Linux: share data among different processes
Clifford Heath wrote:
Quoted text here. Click to load it
<snip>
Quoted text here. Click to load it

The API for System V semaphores and System V shmem is klunky but not  
horrible.

It's not completely sufficient on its own - each thread/process
has to poll it or otherwise have a "read shmem now" signal somewhere
to be named later, but simply having an upcounter within the shmem was
good enough when I used it.

If you can block on the semaphore then you don't need that.

If one of the threads can reinitialize the contents of the shmem then
you will need some sort of signal or value of the upcounter to indicate
that or you can have a race condition or sorts.

Quoted text here. Click to load it

My efforts at profiling this* on an Allwinner A20 show the lock overhead
to be negligible. Then again, I wasn't bothering to get too
detailed about assigning threads/processes to cores.

*runtime hardwareish counter that will be domain dependent.

I chose shmem for tactical reasons other than trying to squeeze the last  
cycle out of the thing ( the shared data was already in a 'C'
struct ).

Quoted text here. Click to load it

--  
Les Cargill

Re: Embedded Linux: share data among different processes
On Thu, 23 Jun 2016 13:52:21 +0200, pozz wrote:

Quoted text here. Click to load it

I did that with System V IPC, using message passing for control and  
shared memory blocks for info common to all the processes.  I was able to  
avoid any locking using the one-writer/many-readers trick.  The code was  
more low-level than many people, perhaps, would like, but it worked well.

    Mel.

Re: Embedded Linux: share data among different processes
Il 23/06/2016 15:47, Mel Wilson ha scritto:
Quoted text here. Click to load it

Yes, another problem to solve is the other way: QT and HTTP server could  
send some control commands to the electronic device (switch on, switch  
off, change this...).


Quoted text here. Click to load it

It is exactly what I will have: one writer (the poller that will  
continuously fetch updated data from the device) and some readers (at  
least, HTTP server and QT graphic libraries).
Doesn't it needed a syncronization mechanism (semaphore) in this  
scenario, using shared memory to share common data?

Quoted text here. Click to load it


Site Timeline