Embedded Linux: share data among different processes

  

  

  

  

  

  


do you need to keep the data from the RS485 or not?

If not a shared memory or (named) pipe or something like this accessed thro
ugh a semaphore or a mutex is IMHO better.
Some reading:
http://web.archive.org/web/20150524005553/http://advancedlinuxprogramming.c
om/alp-folder/alp-ch05-ipc.pdf
http://web.archive.org/web/20150511104538/http://www.advancedlinuxprogrammi
ng.com/alp-folder/

On the other hand, if you need to keep the data then a DB is the way to go.

Bye Jack

Jack wrote:



http://web.archive.org/web/20150524005553/http://advancedlinuxprogramming.com/alp-
folder/alp-ch05-ipc.pdf

http://web.archive.org/web/20150511104538/http://www.advancedlinuxprogramming.com/alp-
folder/


Since you already using Qt, I can propose a solution which I use quite  
often. It is even flying in more than 100 helicopters. Yes, penguins can fly  
:-)

The poller fetches that data from the remote device via RS485 and keeps a  
for it local copy. If permanent storage is a requirement it can also take  
care of this.
In addition it open a QLocalServer which listens for connections from other  
processes.
The other (multiple) processes connect to this server using a QLocalSocket.  
They can then poll are are sent the data directly when the poller receives  
new data. This way no locking is required since the poller does its sending  
in a single thread.
You just have to define a simple protocol. It could be line oriented ASCII.

You can also use QTcpServer/QTcpSocket instead. Then you could even have  
some processes running on different machines (or virtual ones).

The other - more complicated - option would be to use DBUS.

This Server/Socket combination has the advantage that it easily integrates  
into the signal/slot mechanism of Qt.

On 23/06/16 23:05, Reinhardt Behm wrote:




SQLite "takes care of" that by using a single giant lock.
That might work for you, since it sounds like you probably
have very easy throughput and latency demands. It would
be a good solution if you weren't using Qt. Les Cargill
hasn't seen how remarkably lightweight SQLite actually is.



Reinhardt, do you deal with write-blocking if the reader fails?
If the write is in the poller thread, any blocking on writes will
delay the polling - unless you use non-blocking writes.



I think it's good advice, though I wouldn't personally have chosen Qt.

This application is not demanding enough to need a shared memory
solution, with the attendant need for locks and especially,
memory barriers, of which most respondents seem ill-educated.
In particular, Mel Wilson's "one writer, many reader" is no
longer so simple with modern multi-core architectures. Writes
do not have to happen in the order they are written, so readers
can get confused.

Clifford Heath.

On 6/23/2016 4:45 PM, Clifford Heath wrote:




That's only a problem if you "roll your own" locks.  IME, the SysV
shared memory operations also carry along the SysV semaphores.

And, you *want* the OS to know about any locks you've implemented
as, inevitably, you'll want to add another "client" down the road
and then have to worry about the various types of deadlock that
can occur (that the OS can mitigate, in some instances).

E.g., the "single giant lock" that SQLite uses can similarly be
implemented to control ALL accesses to the shared memory segment.

You also have the advantage of *seeing* all of the code that is accessing
these shared objects -- instead of wondering what SQLite is doing
and when its doing it.

On 24/06/16 10:19, Don Y wrote:




As I said, ill-educated.

As you'll find out when the SysV semaphore grants you access to
some shared memory resource that has been written, but only some
of the writes have been flushed.

If you're on hardware that can do this, you need to know about it,
or leave the work to someone else who does... like the authors of
SQLite.

Clifford Heath.

On 6/23/2016 8:25 PM, Clifford Heath wrote:




I'd appreciate a definitive reference backing your claim.



And, the folks authoring the semaphore functions *don't*?

On 24/06/16 14:27, Don Y wrote:




The semaphore functions are not guaranteed to flush CPU caches
or prevent out-of-order execution of writes to the shared memory.
Why would they? The semaphore semantics are not intertwined with
the memory semantics.

It's common for entry to the kernel to flush the cache, but you
need to know - it doesn't happen by magic. But out-of-order
execution can still spoil your day.

In some architectures, even a single "increment-memory" instruction
can allow another CPU to observe the memory cell to be half-incremented.
It's all very troubling...

On 6/23/2016 9:40 PM, Clifford Heath wrote:




My mutex functions do exactly that -- to ensure the synchronization
primitive works regardless of instruction reordering, the presence
of multiple cores, etc.  The whole point is to make it possible for
a piece of code to be migrated to a different platform (with different
hardware capabilities) WITHOUT having to reexamine each potential
problem area.

I have no idea how the Linux kernel implements the SysV semaphores
(or POSIX semaphores).  I will have to look at the FreeBSD and NetBSD
sources to see how *they* do.  I'd be surprised if a kernel capable of
running on such hardware would neglect these sorts of things.



All the more reason to embed these guarantees in these library functions.

On 24/06/16 15:20, Don Y wrote:




You're still missing the point. The synchronisation primitives can
fulfil their contract perfectly - with regard to the semaphore
operations - but the memory write behaviour is not part of that
contract. Only the semaphore operations are in the contract, and
nothing about that says that all memory accesses on both sides
must be coherent across all processor cores.

Your mutex functions perfectly, I'm sure - but do they guarantee
that memory operations either side of the mutex - and not referencing
the mutex's memory - is coherent across multiple cores. Note that
I am not talking about the memory used to implement the mutex
itself, but other memory.

On 6/24/2016 12:19 AM, Clifford Heath wrote:




How?  If threadX and threadY (operating on the same processor *or*
on different cores) can see different orderings, then how is the
contract satisfied?  It's not an "increment memory" operation...



What value is a synchronization primitive if it doesn't provide
synchronization?  How is it fulfilling its contract if it *doesn't*?

All synchronization operations MUST perform barriers to memory and
operator reordering.



Yes -- as they include memory barriers specific to the particular
processor architecture involved.  The whole point is NOT to force the
developer to have to wonder whether it's a single-threaded environment or  
multithreaded; uniprocessor or multiprocessor.  The function call
is a fence.

 From a quick peek through some of the online Linux sources, their SysV
semaphores also seem to operate in a similar manner.  I've not yet
looked through the *BSD sources...

Are you claiming the compiler can reorder instructions *across* an
opaque system call?  E.g., decide to invoke the code at main()
AFTER the code at exit()??  (what's to prevent this??)

On 24.6.2016 ?. 07:40, Clifford Heath wrote:





It can be troubling but if you know what you are doing it is not.
Just update(that part of) caches when needed, sync (serialization opcode
for power) etc. when needed after that, use lwarx/stwcx. as needed and
you can have all the synch you need. Comes at a cost obviously
so one must be careful. Those semaphore functions you talk about
must be some high level implementation targeted at having a semaphore
which is not too costly so it cares only about itself, nothing wrong
with that. But the world does not end there (I don't know where it
ends for the C libraries people use so of course it may be as troubling
as you suggest but only as long ass one relies on libraries someone
else has written).

Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/

Hi Dimiter,

[40-45+C lately -- PLEASE don't tell me you've still got icicles!!  :> ]

On 6/24/2016 6:38 AM, Dimiter_Popoff wrote:


A "synchronization primitive" is useless if it doesn't ensure
the "process state" isn't being synchronized wrt other processes.
Every instance of:

     ...   // prepare some private object

     acquire_mutex(SHARED_RESOURCE)

     ...   // make private object visible as shared resource

     release_mutex(SHARED_RESOURCE)

would break if the state of the private object was still "in
transition" *inside* the critical region.  What does the mutex *do*
for the code in that case?

Anything -- and EVERYthing -- that must be done to ensure the
consistency of the object as visible to other "agents" must happen
before the mutex is granted.  Otherwise, any changes some OTHER
agent may have made to *their* "private object" won't be
available for *this* agent to see as it manipulates the shared
resource.

Lightweight "locks" that only work within a single process/thread are
of little value.  The OS needs to be involved else it can't
recover locks from crashed processes (it needs to know who was
holding the lock at the time), resolve priority conflicts, etc.

OTOH, if used *entirely* within a single process, you can
kill the process and reclaim its memory and, thus, the lock's
holder is of no significance -- the lock AND *all* of its
potential holders are gone!  E.g., when thread() is killed,
it matters little whether produce() or consume() was holding
the lock -- the effect is entirely contained within *this*
thread (process).  No *other* processes are at risk.

thread() {
    lock_t aLock
    ...
    while (FOREVER) {
        produce()
        consume()
    }
}

produce() {
     if (something) {
         process(something)
         acquire_lock(aLock)
             // make <something> available
         release_lock(aLock)
     }
     ...
}

consume() {
     if (whatever) {
         acquire_lock(aLock)
              // access <something> previously made available
         release_lock(aLock)
     }
     ...
}

[Note that the OP's application could potentially fit this form
*if* he has control of the HTTPd's implementation]

Regards to L.

Clifford Heath wrote:



Qt internally uses non-blocking read and write. So this is not a problem.


Because of that I prefer to avoid all these locking if it is not needed.  
Having the data at one place controlled by one thread and send it from there  
to interested clients via some IPC mechanism make live much easier. And it  
is also a very effective encapsulation. The other can not even know, how it  
handled internally in the server.
I would use shared memory or similar close coupled methods only if necessary  
for performance or other requirements.
  

On 6/23/2016 9:39 PM, Reinhardt Behm wrote:


You're just hiding by wrapping it in an IPC mechanism.



Yes, but it is also considerably more expensive.  Every consumer has to be
informed of every update to the data in which it has "expressed an interest"
(assuming you don't broadcast ALL data to ALL consumers).

E.g., I install triggers in my RDBMS so that any update to a particular
(set of) parameter(s) results in an up-call notification of the interested
parties that might want to *see* that new value (so, I don't have to
broadcast all updates of all data to all consumers).

But, that upcall carries considerable cost (from trigger through upcall to
client notification and eventual re-query of the RDBMS).  It can effectively
bottleneck the RDBMS's operation (cuz another up-call could be enqueued
right behind the first one even before the first has finished!).

So, the nature of the problem changes -- to one of deciding *what* information
is "genuinely of interest" to each client... AND, how timely that notification
should be made (if you can defer a notification, then you might be able to
eliminate it in favor of a subsequent notification)

E.g., if some client (having permission to do so!) updates the current
timezone, the wall-clock display should *probably* be updated to reflect
that.  OTOH, it probably can wait a second or two.  And, if the timezone is
updated AGAIN before the notification is dispatched, the first update can
safely be ignored in favor of the second.

OTOH, if *every* update must be conveyed to the client(s), then the
notification effectively becomes part of the update transaction and
limits how frequently the value can be updated.



IME, people are more accustomed to using shared memory solutions as
this mimics the familiar interplay of foreground/background processing;
people *seem* to know that some coordination must exist between an
ISR and its "clients".

OTOH, sharing memory in an environment with individual, protected memory
spaces can be intimidating the first time.

Don Y wrote:



He? What am I hiding?
I proposed that the server (poller) collects the data from the device and  
sends _copies_ of it to the clients. This can happen in a single thread. So  
there is no need for any kind of locking. So there is nothing hidden behind  
an IPC mechanism.
My proposed solution uses internally Unix domain sockets and could as well  
use TCP sockets which would give the flexibility to work over different  
machines. It is proven to work in a DO-178 certified system where several of  
these mechanisms are at work to convey data from GNSS, TCAS, AHARS, camera  
control, Iridium satcomm, each with its own server. And it works in both  
directions.



For many application scenarios the throughput and overhead is of no concern.  
And yes, I broadcast all data to all clients.



I was never proposing to use a database system.

On 6/25/2016 6:34 PM, Reinhardt Behm wrote:


The fact that only a single task can access the "original data".
You've got a "lock" in the IPC.  Your *one* task can't compete
with itself WHILE it is busy in the IPC.

And, none of the consumers can see anything until the IPC completes
(the receive() is atomic).



See above.  You could add multiple "server threads" and still hide
the contention from your clients -- assuming no two server threads tried
to update the same client with the same/different data.



I'm not saying it *won't* work.  I'm just saying that you've implemented
the locking functionality in a different manner.  If the "poller" provided
data at a faster rate than you could "broadcast" it, you'd lose data
or have to introduce another server thread -- and then deal with contention
on the server side of the IPC fence.

Don't get me wrong; my entire OS is based on message-passing.  Every function
call is a message disguised as a function.  So, making a kernel request
uses the exact same mechanisms as sending data to another local process -- or,
a *remote* process.

But, there are (significant) costs associated with this.  Crossing a protection
boundary means kernel involvement.  And, you're not just passing "payload"
but, also, the overhead of the IPC/RPC "function", marshalling arguments,
etc.

[In my case, it's a net win because the target of an IPC can be virtually
and physically relocated.  So, a call that is an IPC *now* may be an RPC
a few moments from now!  The abstraction that passing messages (vs. sharing
memory) provides is worth its cost.  (I can also share memory across
nodes but that leads to disappointment when you think in terms of LOCALLY
shared memory -- only to discover that the memory has now migrated to
another, remote node.  And, accesses are SLOWER than passing messages
would be!)]



Of course!  I'm moving to multicore nodes simply with the idea of dedicating
a core to RPC.  Silicon is cheap, nowadays.



Using broadcast protocols?  Or, iterating over them?



I was indicating how I *selectively* distribute data "to folks who have
previously REGISTERED an interest".  There are thousands of processes
active in my system at any given time.  Virtually *all* of them are
unconcerned with any *particular* "data update", yet may have keen
interest in SOME update at some time!  So, there is a huge potential
for waste if I unconditionally broadcast ALL updates to everyone.

Clifford Heath wrote:

<snip>


The API for System V semaphores and System V shmem is klunky but not  
horrible.

It's not completely sufficient on its own - each thread/process
has to poll it or otherwise have a "read shmem now" signal somewhere
to be named later, but simply having an upcounter within the shmem was
good enough when I used it.

If you can block on the semaphore then you don't need that.

If one of the threads can reinitialize the contents of the shmem then
you will need some sort of signal or value of the upcounter to indicate
that or you can have a race condition or sorts.



My efforts at profiling this* on an Allwinner A20 show the lock overhead
to be negligible. Then again, I wasn't bothering to get too
detailed about assigning threads/processes to cores.

*runtime hardwareish counter that will be domain dependent.

I chose shmem for tactical reasons other than trying to squeeze the last  
cycle out of the thing ( the shared data was already in a 'C'
struct ).

On Thu, 23 Jun 2016 13:52:21 +0200, pozz wrote:



I did that with System V IPC, using message passing for control and  
shared memory blocks for info common to all the processes.  I was able to  
avoid any locking using the one-writer/many-readers trick.  The code was  
more low-level than many people, perhaps, would like, but it worked well.

    Mel.

Il 23/06/2016 15:47, Mel Wilson ha scritto:


Yes, another problem to solve is the other way: QT and HTTP server could  
send some control commands to the electronic device (switch on, switch  
off, change this...).




It is exactly what I will have: one writer (the poller that will  
continuously fetch updated data from the device) and some readers (at  
least, HTTP server and QT graphic libraries).
Doesn't it needed a syncronization mechanism (semaphore) in this  
scenario, using shared memory to share common data?