Raspberry Pi node red flow editor very slow when using Modbus

Good description. Thanks for posting the link.

FWIW, when my T440 laptop (8GB RAM, 16GB swapspace, 64 bit Fedora 27) is doing the sort of stuff it usually does, its memory usage must be nearly the opposite of an RPi in terms od RAM occupied by programs vs cache space. If I'm compiling C programs the system seems to stabilise with 1GB of code in memory (compiler, linker, make and editor) and the remaining

7GB used as cache space. Its noticeable that the first compile of the day is quite slow, the second is somewhat faster, and the 3rd and following compiles are faster still and all about the same speed.

Assuming that RPi binaries for the same set of programs are about the same size as as the X-64 ones, that implies that very much less file caching is available in the RPi, especially in mine, an oldish 512MB model B and compilation performance tends to support that: the second and subsequent compiles of the day are a bit faster than the first, but the significant speed-up shown by the T440 is never seen on the RPi.

--
Martin    | martin at 
Gregorie  | gregorie dot org

On Mon, 15 Jan 2018 17:56:14 +0000 (UTC), Kiwi User declaimed the following:

With only 0.5G to share, there is likely very little actual cache.

My Win10 box with 12GB of RAM currently shows

5.3GB in use, 6.7GB available 5.5 (of 20)GB* "committed", 6.4GB cached 1GB paged pool, 446MB non-paged pool

• (the 20, as I recall, is the total of 12GB RAM and my fixed 8GB swap file)

Should the machine have a need for more active RAM, the cache size will go down; basically actual swapping doesn't occur until the cache is near 0. The cache, on Windows, is NOT memory on swapped to disk, but just formally used RAM held in reserve should the same content be needed again.

--
	Wulfraed                 Dennis Lee Bieber         AF6VN 
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

ARM binaries are quite a bit more compact than X86-64 binaries which is why ARM hardware manages with less memory than PC hardware.

But 512MB is a *lot* less than 8GB and won't have any cache space available to speak of.

You wouldn't 512MB is marginal for compiles really and probably needs swap for some compiles - if you were using a 4GB Rock64 (or perhaps even a 2GB Odroid C2) you'd expect to see similar speed up behaviour to your

8GB laptop.(

--
Steve O'Hara-Smith                          |   Directable Mirror Arrays 
C:\>WIN                                     | A better way to focus the sun 
The computer obeys and wins.                |    licences available see 
You lose and Bill collects.                 |    http://www.sohara.org/

Try this:

top -b -n 5 > $somefile

HTH

--
Robert Riches 
spamtrap42@jacob21819.net 
(Yes, that is one of my email addresses.)

I always have a hard time accepting that. ARM is RISC, x86 CISC. I would assume one needs more RISC instructions to achieve the same CISC instructions. So RISC binaries would be bigger.

But I guess single x86 instructions are bigger then ARM instructions. And large parts of code are not even using the specialized subset of CISC...

always assuming same pointer size (32 bit vs 64 bit)

I've just done a few quick comparisons of programs I've compiled on Intel box running X-64 Fedora and my 512KB model B under Stretch. In all cases the programs are compiled with gcc and statically linked. Here's what I found:

Program X_64 RPi ======= ==== === hello.c 7376 8216 spd 87520 89884 me 159573 182208

hello.c is a slightly brighter version of the classic that can be told a name to say hullo to $ hello Hello, C world $ hello Zaphod Beeblebrox Hello, Zaphod Beeblebrox

spd is a server I wrote some time ago to allow Java programs to access serial connections and recently ported to the RPi so a C program can more easily talk to a PICAXE chip via a USB-serial converter. BTW 'porting' it to the RPi was as simple as extracting it from the CVS repository on my house server and running 'make' - a zero effort port!

me is my favourite text editor, aka microEmacs 4.0

I'm pleased to see that my guestimate that the executables would be more or less the same size was pretty good.

The only surprise is that the ARM executables are all a little bigger than the X_86 binaries by a factor of 3 to 14 percent.

--
Martin    | martin at 
Gregorie  | gregorie dot org

PS: forgot to mention that, of course a 512MB Model B doesn't have 512KB of RAM available for OS + programs - more like 445MB after some has been grabbed by the GPU and firmware for use as the built-in screen.

--
Martin    | martin at 
Gregorie  | gregorie dot org

Yes ARM instructions are all 32 bit, apart from the 16 bit thumb instruction set supported on most ARMs. Most ARM instructions are conditional which eliminates some branching and there's a barrel shifter which can be applied as part of just about any instruction (so something like a shift and add is a single instruction executed in one cycle).

The combination is very effective at producing compact efficient code (especially when thumb is used).

That was always the weak point of CISC, do the extra instructions get enough use to justify the silicon.

Indeed.

--
Steve O'Hara-Smith                          |   Directable Mirror Arrays 
C:\>WIN                                     | A better way to focus the sun 
The computer obeys and wins.                |    licences available see 
You lose and Bill collects.                 |    http://www.sohara.org/

Do a free -h

---druck

don't tell this my raspi3:

% dpkg-architecture DEB_BUILD_ARCH=arm64 DEB_BUILD_ARCH_ABI=base DEB_BUILD_ARCH_BITS=64 DEB_BUILD_ARCH_CPU=arm64 DEB_BUILD_ARCH_ENDIAN=little DEB_BUILD_ARCH_LIBC=gnu DEB_BUILD_ARCH_OS=linux DEB_BUILD_GNU_CPU=aarch64 DEB_BUILD_GNU_SYSTEM=linux-gnu DEB_BUILD_GNU_TYPE=aarch64-linux-gnu DEB_BUILD_MULTIARCH=aarch64-linux-gnu [?]

gregor

--
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org 
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06 
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe 
   `-   NP: Grupo Raison: Quitate El Chaqueton

Guys.

As much as it is good to chat 70% of the above threads have nothing to do w ith my problem. We seem to have steered away from my original topical quest ion

appreciate if you can please help.

Kind regards

n.org

AA06

urope

Well, what you've posted (free output) shows that, at the time you grabbed those snapshots, memory was not full and swap was unused, which raises two new questions:

- what was running on the RPi then, i.e. was it Node-Red + Chromium or was it something else? If something else, what was running?

- how much CPU was being used at that time?

Run TOP as normal and look at the %CPU column to see what process(es) are eating the most CPU time. If its obvious and a fairly stable set of figures, try to take a screen shot and post that. Otherwise, use the command "top -bn5 >toplog.txt" to capture five snapshots in the "toplog.txt" file and post that - it might be quite large. Note that, with 4 CPU cores in an RPi3, by default a process that is shown as using as 25% of CPU is actually using 100% of one of the four cores.

--
Martin    | martin at 
Gregorie  | gregorie dot org

thanks for getting back on track mate.

Here is a post of the latest screen shot.

FYI. the Pi is only ever running Chromium and node-red. I have no other apps media programs and/or things running. I have team viewer installed but never actually fully opened.\

Look forward to your advise

Kind regards

Thats one of the joys of newsgroups: it pays to read what others have to say. Even if it doesn't answer exactly what you asked, you may well see something that turns out to be useful or throws a different light on your problem.

Simple: get rid of Chromium and use a different browser. One that doesn't hog CPU cycles the way Chromium is doing. Or, or run it on a separate host.

This is obvious from the 'top' display: Chromium is using more than one core all by itself: 29% of the power of the four cores, showing up as two processes, one eating 110% of a core and the other thread eating 7% of another core and both consuming 27% of the available RAM. This makes it look as though you are running two copies of Chromium (1296 and 1999). If so, WHY? If you're not running two copies, what are you doing to Chromium to make it run as two processes?

Meanwhile node-red is quietly sitting in a corner, behaving itself and doing its own thing while using just 5.3% of one core and 6.8% of available RAM.

So the bottom line: Chromium is unsuited to monitoring node-red on an RPi because of its excessive CPU usage and its apparent inability to spread this load across all available cores.

Subsidiary points:

- By now you should know about the 'man' program, which is used to display manpages. There is a manpage for every program and library function installed on your RPi: a manpage described what the program/ function does and how to use it. Manpages are a standard feature in every version of Linux.

- Do you know about the 'apropos' program? It compares a search term with the summary line of every manpage and outputs a list of matching manpages. It is very useful for finding programs and library functions that are related to something you're trying to do. For instance, if you're interested in handling times or time intervals measured in seconds, you'd run "apropos seconds":

$ apropos seconds DateTime::LeapSecond (3pm) - leap seconds table and utilities sleep (3) - sleep for a specified number of seconds time (2) - get time in seconds ualarm (3) - schedule signal after given number of microseconds usleep (1) - sleep some number of microseconds $

--
Martin    | martin at 
Gregorie  | gregorie dot org

Chromium is inherently multi-process.

--
https://www.greenend.org.uk/rjk/

Understood.

architecture

That's very useful for understanding what top shows. Thanks for posting.

--
Martin    | martin at 
Gregorie  | gregorie dot org

AArch64 features New instruction set, A64 [...] Instructions are still 32 bits long and mostly the same as A32 (with LDM/STM instructions and most conditional execution dropped).

On Mon, 15 Jan 2018 22:03:19 -0800 (PST), Dannaz Perth declaimed the following:

Dump the browser?

Your browser is eating >100% (being a four-core processor, 100% means all of one core). Uh, and you seem to have two browser processes running and between them they are eating a quarter of your memory (though that may just mean Chromium runs a master process and an additional process per open tab; I don't have the experience to tell).

Using portions of multiple cores could be having an impact as each core needs to flush/reload internal cache. Possibly locking the browser to single core would help

{Lock the browser process with the large %CPU value}

Node-Red itself is not using much (though it is using a touch more than the X-window system task). In truth -- at the time of that screen grab, node-red is in a blocked/sleep state, doing nothing.

On the summary (top part) you have no defined swap file (which is the default configuration on both my RPI3 and BBB -- since if you really need swap, you don't want it on an SD card [the HINT benchmark did kill my original RPI3 SD card; I bought a USB-powered hard-drive to run a swap file on]) -- but you have a quarter of RAM still free. Total CPU usage seems to be about one third (remember -- four cores, so 25% is one core equivalent).

The killer -- as I started -- is that you have ONE browser process that is using more CPU than any single core can provide, hence resulting in processor swapping, cache misses, and whatever locks are needed to ensure one core isn't trying to read data that hasn't been written by another... Note that the fourth active process is RCU_SCHED

which appears to be involved in maintaining data structures shared between cores. {This is my first exposure to RCU}

For comparison purposes (since you stated remoting in with the browser running on another computer doesn't show the problem): set up the same run as captured here, but do not run the browser locally. Use the remote browser. Then capture the TOP output on the RPI again. Without the browser hog on the RPI, I expect you'll see total CPU% down around 5-10%, or even lower.

--
	Wulfraed                 Dennis Lee Bieber         AF6VN 
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

The URL posted by Richard says that Chromium has a master process handling the main screen and supervisory functions plus one process per tab.

That had me scratching my head for exactly the same reasons as you when I saw two processes, both called Chromium, until he posted that link: I know that the Java jvm is a multi-threaded single task and always shows up as a single line in the 'top' process list.

Yesterday I found description of Node-red internals:

The gory details are the the "Technical details" section.

Summarising, this says Node-red is written in Node.js Javascript. This runs as compiled machine code generated by Google's V8 runtime package. The control and management code runs as a single thread using non- blocking i/o. I/O requests are added to an event queue which is serviced by a thread pool and uses a call-back mechanism to activate the modules defined via the Node-red flow editor. The non-blocking i/o, event queue and thread pool components are all part of the libuv library, which also supports a mechanism for increasing the size of the thread pool from its default value of one.

Yes: good advice.

--
Martin    | martin at 
Gregorie  | gregorie dot org