Raspberry Pi node red flow editor very slow when using Modbus

On Fri, 12 Jan 2018 02:06:18 -0800 (PST), snipped-for-privacy@gmail.com declaimed the following:

First question: what do you see if running "top" (with whatever options are needed to get all processes shown)?

My suspicion would be that your monitor process is sucking up most of the CPU(s), and not leaving enough for the /browser/ itself to operate.

When remoting in, the browser is running on the host device and only small packets are being sent/received. In this condition, display changes can be rapid, and the only delay is in the recognition/response of the RPi to the update request.

Since the RPi-3 is a quad-core, you might be able to set your monitoring process(es) to only use two or three of the cores, leaving the fourth available for the OS and browser (at least, I'd hope the RPi-3 can make use of the four cores -- bad enough they haven't released a 64-bit Raspbian for it yet)

--
	Wulfraed                 Dennis Lee Bieber         AF6VN 
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Have you tried setting up some hardware swap on a usb flash drive? Chromium is a slug on my Pi3 also, but adding a fast(ish) usb flash drive configured with a swap partition speeds things up hugely.

The write speed of the flash device is critical; I've been using SanDisk Extreme with good success, there are probably faster devices out there now.

hth,

bob prohaska

On Sat, 13 Jan 2018 02:53:28 +0000 (UTC), bob prohaska declaimed the following:

If it is a matter of swap space, I'd recommend using a USB powered hard-drive. The rewriting of heavy swap usage will rapidly wear out flash memory.

Though the last time I ran a test that used up memory, the Raspbian variant didn't have a swap file and generated a kernel panic crash when it was unable to satisfy the request for more heap memory (even though the benchmark application was coded to exit cleanly if the malloc() call returned saying no memory was available).

--
	Wulfraed                 Dennis Lee Bieber         AF6VN 
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

That seems to be less of an issue than feared. I've been using USB flash swap on a Pi2 for the last two years, working much harder than a typical desktop machine, though by no means 100% 24-7. So far, no problems at all. It may yet develop problems, but I'm starting to suspect obsolescence will ends its usefulness first.

bob prohaska

But does this workload swap much? 'top' shows whether swap is being used or, more drastically, turning swap off and seeing what happens will give a rather positive indication.

Same question to the OP: when you're running the combo of red flow editor and Chromium browser does 'top' show that swap space is being used?

Swap usage details are on the 5th line (last line of the header).

--
Martin    | martin at 
Gregorie  | gregorie dot org

Sorry for the late reply.

In terms of SSD cards. i'm using the new Samsung EVO Platinum. No other apps installed. I have tried loading Raspibian Bare bones and only putting pixel and node red back, nothing else. Didn't make a difference. And as per above, if i edit the flow over the network from the anoterh computer there is no limit what i wish to do.

Ill Check "Top" in the command and copy the report and send to you ? Hope you can help guys.

Kind regards

You'll find that hard to do because it doesn't produce a static report: the screen refreshes every second or two.

The swap space line doesn't change that frequently, so you may be able to get it via screen capture, but its generally good enough to note approximately what you see and post that. The important number is the 'used' figure, which will be zero if everything that is running fits into RAM.

The watch points for system performance are:

(1) if the buff/cache figure is small almost all data i/o is going to require physical disk/hdd access which is always noticeably slower than if the required files are in the cache.

Cache size = total - (used + free) - from the KiB Mem line

(2) if a program is swapping, this adds to the physical i/o load because its continually having the transfer chunks of its memory between RAM and the swap space.

You can work out which program is swapping because its SHR value tends to exceed its RES value.

--
Martin    | martin at 
Gregorie  | gregorie dot org

On Sun, 14 Jan 2018 13:52:31 +0000 (UTC), Kiwi User declaimed the following:

I'm still of the opinion that the problem is in the CPU usage of the monitoring program -- leaving very little CPU for a browser running on the same R-Pi to be able to /render/ updates, whereas a remote connection browser will be unencumbered and only limited by the HTTP data transfer (which will be about the same for remote and local browsers).

So -- I'd consider noting the %CPU column and the associated process/command... A partial grab (from a Debian install running in VirtualBox -- USE FIXED PITCH FONT TO VIEW):

top - 10:28:51 up 37 min, 1 user, load average: 0.02, 0.07, 0.04 Tasks: 121 total, 2 running, 119 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 0.3 sy, 0.0 ni, 97.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 2052640 total, 1033268 free, 316980 used, 702392 buff/cache KiB Swap: 2095100 total, 2095100 free, 0 used. 1544840 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

552 root 20 0 392016 85964 31832 R 1.7 4.2 0:34.99 Xorg 2242 wulfraed 20 0 502160 36136 28340 S 0.7 1.8 0:01.40 xfce4-termi+ 1 root 20 0 139092 6784 5168 S 0.3 0.3 0:00.64 systemd 385 root 20 0 46500 4740 4164 S 0.3 0.2 0:00.02 systemd-log+ 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 0:00.12 ksoftirqd/0 5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H 7 root 20 0 0 0 0 S 0.0 0.0 0:00.11 rcu_sched

--
	Wulfraed                 Dennis Lee Bieber         AF6VN 
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

You may well be right. I know nothing about Node-red except for a quick look at its Wikipedia entry, which says that it is written in node.js, is single threaded and uses async i/o.

I wasn't trying to solve the OPs problem directly. He doesn't seem all that familiar with the 'top' utility, so I thought a micro-tutorial on it might be in order, especially since its manpage is quite long.

Good point, though it seems as though the node.js runtime should be at least as efficient as the Java jvm, provided that V8 generates code of similar efficiency to javac + the jvm's JIT compiler.

--
Martin    | martin at 
Gregorie  | gregorie dot org

If it's single threaded, it can't use more than one of the four CPU cores. It may be that only being able to use 25% of a Pi is just not enough CPU for acceptible response.

To see if you are swapping because system needs more memory than the Pi has, use: vmstat 1 and look at the si column. This is the rate of bringing stuff back in from swap, and indicates the system needs stuff it's previously swapped out. If this rate is high and little progress is being made by applications, this would indicate the system is thrashing because the working set size of the kernel and processes is larger than system memory. If this is the case, and since you can't add more memory to the Pi, you would have to find a way to reduce memory usage.

(Looking at the amount swapped out by itself isn't useful, because swapping out stuff that isn't needed again is harmless.)

--
Andrew Gabriel 
[email address is not usable -- followup in the newsgroup]

Indeed, that's why it is important to have a swap partition or a swap file, to free up physical memory for use as cache and buffers.

Programs that are loaded for a longer time, use swap to get rid of pages that are only seldomly used from physical memory. The OS decides on that, the program doesn't have to worry about it.

Swap I/O activity can be observed with vmstat 5 , which takes a snapshot every 5 seconds. Observe the si (swap-in)) and so (swap-out). Stop the report with ^C.

On a well-configured server that is not overloaded, si and so are zero most of the time, and not a high burden for an SD card or USB memory stick.

vmstat has a lot of useful options, it is a nice tool to pinpoint performance bottlenecks.

--
Regards, 
Kees Nuyt

FreeBSD generally doesn't swap much, a few tens of megs active most of the time. However, while recompiling the OS swap usage peaks around 1 GB, which in my case happens a few times per week. That makes _having_ swap mandatory, though it's little used.

On Raspbian, right now, Chromium is open with three tabs, one of which is a live graphic animation, the swap line from top is KiB Swap: 61374632 total, 31308 used, 61343324 free. 314760 cached Mem

With YouTube running a 720p video, the swap usage is slightly higher: KiB Swap: 61374632 total, 42736 used, 61331896 free. 354428 cached Mem and creeping up. A few minutes in it's up to KiB Swap: 61374632 total, 62212 used, 61312420 free. 305156 cached Mem

Without a swap partition Chromium is essentially useless. With swap, the browswer works quite decently.

bob prohaska

Hi People.

I have disabled the swap on the PI. I have attached a screen shot if this is any help.

Thanks for your efforts to help me.

Dan

I cannot actually attached a screen shot ? ? Please bare with me. Ill have to figure this out

Cheers

I cannot attached a picture ???

I cannot actually attach a picture for some reason im not familiar with ?

Screen Shot

Not on usenet text only gropups, no.

Post it somewhere and link to it.

-- The biggest threat to humanity comes from socialism, which has utterly diverted our attention away from what really matters to our existential survival, to indulging in navel gazing and faux moral investigations into what the world ought to be, whilst we fail utterly to deal with what it actually is.

A much better idea is to use zram.

Bye Jack