There are some analog signals that could change every second. The poller will fetch new updated data about every one second. I would avoid pushing new data to consumers (HTTP server and QT libraries) every second.
I will have a lot of spare MIPS too.
I don't need to save data in a persistent way (after a reboot, the poller will start to retrieve new data). I thought to create the SQLite database file in a RAM/temporary filesystem. This should speed up the access and should avoid Flash memory corruption.
The overall configuration of the device could be maximum 1kB (I think it will be around 100 bytes). However it doesn't change frequently, so it can be retrieved only at startup and only when it really changes.
The status data that changes frequently will be maximum 100-200 bytes (I think it will be less than 100 bytes).
QT choice could be changed, but the embedded Linux platform I'm going to use is shipped with many ready-to-use examples of simple QT applications that use the TFT touch display. This is the main reason of QT choice.
HTTP server is a requirement: the users will be able to connect to the Linux box through a simple web browser. Of course I will use a ready-to-use HTTP server (I'm thinking of lighttpd) with client-side javascripts to retrieve data from a server-side script (a simple cgi).
Anyway they are really *two* different projects/processes running on the same Linux box. I couldn't understand why you compare QT and HTTP.
There is only one process that will access the RS485 link, the "poller" process running on the embedded Linux board. The data will be shared at the Linux board level, not RS485 link.
Half duplex.
I don't think I will have problems with protocol at the RS485 wire level.
So it should be ok for my application.
Really? I thought exactly the opposite thing. Why?
At the moment I think it's the best solution for me. I started studying the shared memory under Linux. I never used it before.
It seems to me, shared memory fits better on my application.
Yes, sockets are another approach for inter-process communication. Maybe I will start with shared memory.
Yes, I know.
I hope it really exists and someone will point me to it.
It can be troubling but if you know what you are doing it is not. Just update(that part of) caches when needed, sync (serialization opcode for power) etc. when needed after that, use lwarx/stwcx. as needed and you can have all the synch you need. Comes at a cost obviously so one must be careful. Those semaphore functions you talk about must be some high level implementation targeted at having a semaphore which is not too costly so it cares only about itself, nothing wrong with that. But the world does not end there (I don't know where it ends for the C libraries people use so of course it may be as troubling as you suggest but only as long ass one relies on libraries someone else has written).
Ask yourself what the finest granularity you need/want in the data. Is it one big block of data that only makes sense as a whole? Or, lots of smaller blocks that can stand on their own?
E.g., if you were reporting meteorological data (wind speed, air temp, humidity, barometric pressure, etc.), allowing to see *one* of these updated without the others being simultaneously updated could lead to a faulty forecast (because forecasts rely on the interpretation of ALL of the results, in concert).
"A lot" can have different meanings to different people :>
You typically would NOT want to be updating those in FLASH as you'll wear out the FLASH in short order.
Note that data in a DBMS typically is "larger" than you would achieve if storing in a single, typed variable (before adding the overhead of indexes, etc.). And, you will also require resources for the WAL, if supported.
I.e., before you make this DBMS decision, you might want to throw together a simple schema and store your dataset to see just how big it ends up.
You should also think about how you will be interacting with the data. E.g., returning to the weather report analogy, if you are planning on cherry picking one value at a time from the *set* of values, then you risk reporting data that "doesn't make sense" -- because the "temperature" you reported a moment ago doesn't correspond to the "barometric pressure" you are reporting now.
You may need to approach the data transactionally and, if you don't want to lock up the DBMS while your client process chugs along at *its* pace, this might require you to introduce some transaction buffering
*between* the two entities -- which might not have been in your original design plans.
Lastly, if you settle on that approach, "unconstrain your thinking" in considering what you *could* do with data thusly "organized". I found that lots of code that I'd have normally placed in the application (i.e., client) can be pushed back into the RDBMS and the way I *think* about the data that it supplies.
[40-45+C lately -- PLEASE d> It can be troubling but if you know what you are doing it is not.
A "synchronization primitive" is useless if it doesn't ensure the "process state" isn't being synchronized wrt other processes. Every instance of:
... // prepare some private object
acquire_mutex(SHARED_RESOURCE)
... // make private object visible as shared resource
release_mutex(SHARED_RESOURCE)
would break if the state of the private object was still "in transition" *inside* the critical region. What does the mutex *do* for the code in that case?
Anything -- and EVERYthing -- that must be done to ensure the consistency of the object as visible to other "agents" must happen before the mutex is granted. Otherwise, any changes some OTHER agent may have made to *their* "private object" won't be available for *this* agent to see as it manipulates the shared resource.
Lightweight "locks" that only work within a single process/thread are of little value. The OS needs to be involved else it can't recover locks from crashed processes (it needs to know who was holding the lock at the time), resolve priority conflicts, etc.
OTOH, if used *entirely* within a single process, you can kill the process and reclaim its memory and, thus, the lock's holder is of no significance -- the lock AND *all* of its potential holders are gone! E.g., when thread() is killed, it matters little whether produce() or consume() was holding the lock -- the effect is entirely contained within *this* thread (process). No *other* processes are at risk.
Possibly because one could be done away with - I don't know the customer requirements necessarily.
As long as you've played that game before, you should be fine.
That's my opinion, but it's based on (biased) direct observation. This is mainly true because it's completely a string interface and constructing queries dynamically requires marshalling and unmarshalling a pseudo-natural language. Plus, the infrastructure can be somewhat daunting - although I've seen mainly MS SQL Server rather than SQLite.
The API for System V semaphores and System V shmem is klunky but not horrible.
It's not completely sufficient on its own - each thread/process has to poll it or otherwise have a "read shmem now" signal somewhere to be named later, but simply having an upcounter within the shmem was good enough when I used it.
If you can block on the semaphore then you don't need that.
If one of the threads can reinitialize the contents of the shmem then you will need some sort of signal or value of the upcounter to indicate that or you can have a race condition or sorts.
My efforts at profiling this* on an Allwinner A20 show the lock overhead to be negligible. Then again, I wasn't bothering to get too detailed about assigning threads/processes to cores.
*runtime hardwareish counter that will be domain dependent.
I chose shmem for tactical reasons other than trying to squeeze the last cycle out of the thing ( the shared data was already in a 'C' struct ).
If "frequently" means less than a few times per second and you have enough hardware resources, you'll probably have an easier time writing this as if it were a desktop or web app than using a traditional embedded approach. That can even including writing the host part in a server-side scripting language instead of something like C.
Inferno can easily do this (esp for applications that aren't very "demanding"). There's a nominal HTTPd included in the (free) distribution. Inferno (through its Limbo language) also supports communication channels as first-class objects. E.g., the OP's "poller" can just pass specific typed objects down a channel (like a pipe -- except the endpoints can be on different CPUs, etc.) connecting it to the "user interface/web server" task.
For endpoints on the same host (i.e., you don't have to worry about handling "remote host not available" conditions), the code is trivial.
One delightful advantage of the distributed capabilities of Inferno is that you can run hosted inferno on a PC and interact with the application "naturally" without any artificial IDE's. I.e., you could simulate the poller by writing a Limbo app that accepts keyboard input and passes it to the "user interface" task -- running on your target hardware -- without making any significant changes to the system as a whole.
If you can pick hardware for which a *native* Inferno port is available (e.g., rpi), then your total footprint can be under ~1MB for the OS
*and* your application. Limbo apps are amazingly small; and, its close enough to C that it isn't terribly intimidating though lack of support for pointers can be distressing; and, getting used to tuples takes a few
*seconds* (at which point, you'll wonder how you ever lived without them!)
He? What am I hiding? I proposed that the server (poller) collects the data from the device and sends _copies_ of it to the clients. This can happen in a single thread. So there is no need for any kind of locking. So there is nothing hidden behind an IPC mechanism. My proposed solution uses internally Unix domain sockets and could as well use TCP sockets which would give the flexibility to work over different machines. It is proven to work in a DO-178 certified system where several of these mechanisms are at work to convey data from GNSS, TCAS, AHARS, camera control, Iridium satcomm, each with its own server. And it works in both directions.
For many application scenarios the throughput and overhead is of no concern. And yes, I broadcast all data to all clients.
The fact that only a single task can access the "original data". You've got a "lock" in the IPC. Your *one* task can't compete with itself WHILE it is busy in the IPC.
And, none of the consumers can see anything until the IPC completes (the receive() is atomic).
See above. You could add multiple "server threads" and still hide the contention from your clients -- assuming no two server threads tried to update the same client with the same/different data.
I'm not saying it *won't* work. I'm just saying that you've implemented the locking functionality in a different manner. If the "poller" provided data at a faster rate than you could "broadcast" it, you'd lose data or have to introduce another server thread -- and then deal with contention on the server side of the IPC fence.
Don't get me wrong; my entire OS is based on message-passing. Every function call is a message disguised as a function. So, making a kernel request uses the exact same mechanisms as sending data to another local process -- or, a *remote* process.
But, there are (significant) costs associated with this. Crossing a protection boundary means kernel involvement. And, you're not just passing "payload" but, also, the overhead of the IPC/RPC "function", marshalling arguments, etc.
[In my case, it's a net win because the target of an IPC can be virtually and physically relocated. So, a call that is an IPC *now* may be an RPC a few moments from now! The abstraction that passing messages (vs. sharing memory) provides is worth its cost. (I can also share memory across nodes but that leads to disappointment when you think in terms of LOCALLY shared memory -- only to discover that the memory has now migrated to another, remote node. And, accesses are SLOWER than passing messages would be!)]
Of course! I'm moving to multicore nodes simply with the idea of dedicating a core to RPC. Silicon is cheap, nowadays.
Using broadcast protocols? Or, iterating over them?
I was indicating how I *selectively* distribute data "to folks who have previously REGISTERED an interest". There are thousands of processes active in my system at any given time. Virtually *all* of them are unconcerned with any *particular* "data update", yet may have keen interest in SOME update at some time! So, there is a huge potential for waste if I unconditionally broadcast ALL updates to everyone.
Just in case I should mention -- my system didn't contain just one Big Writer. Rather, different processes were responsible for different sections of the application, and each of those processes was the sole writer for the shared data concerned with its section.
In this system, the Linux did the "cerebral" operations, and it communicated over a serial link with a "hind-brain" or "spinal column" run by a PIC32. There was a polling process that communicated with the PIC32 and dispatched input via IPC messages to the responsible processes. Some processes did quasi-real-time operations and, along with the polling process, ran at negative niceness for fastest response. Others could be given a twentieth of a second or so to respond and ran as normal processes. But each task was the sole writer of shared data that it "owned" and ordered any other changes through IPC messages.
There were a bunch of Apache/CGI processes concerned with displaying diagnostic info remotely, and they never wrote shared data. Some other CGI that implemented remote testing and control did their thing by sending IPC messages to the responsible processes.
Basically I mean: write in a scripting language, use high-level libraries or services, and (within reason) don't worry about resource consumption. What hardware are you using?
Oh you're fine, that's far more powerful than any embedded thing I've ever used. It would been a fairly powerful server back in the day. You can run a standard web server stack on something like that, with nginx, python, some kind of web client talking to the sensors, etc. In fact do you really need the Qt interface instead of just using a browser?
I'm going to use lighttpd. Any reason to use nginx instead?
The web browser will be used from a remote location.
Do you mean to use the local display to show the windows of a web browser running on the embedded Linux? This could be fine, because the interface will be the same (for the user... and for the developer).
Some questions:
- do I need a windows desktop environment (X.org, ...)?
- could I force the web browser window to be full-screen? I would avoid to see the Close Window button (the X button), the minimize button, the back button and so on...
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.