Question about ever growing swap space on raspi 4 4 GB

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View

Question about ever growing swap space on raspi 4 4 GB  

I run buster on a raspberry pi4 with 4 GB RAM.
I notice swap space is getting exhausted slowly.

Uptime:
 07:37:54 up 12 days, 17:46, 17 users,  load average: 0.35, 0.62, 0.67
  
Top shows this now:
 MiB Swap:    100.0 total,     20.7 free,     79.3 used.   3375.6 avail Mem

Here I found a way to see swap space per running process:
 https://www.cyberciti.biz/faq/linux-which-process-is-using-swap/

Running their script
 for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | sort -k 2 -n -r | less

shows:
Xorg 9136 kB
rxvt 5648 kB
fvwm 4256 kB
rxvt 2548 kB
rxvt 2388 kB
rxvt 2196 kB
rxvt 2080 kB
rxvt 1796 kB
rxvt 1764 kB
rxvt 1764 kB
rxvt 1636 kB
(sd-pam) 1580 kB
zsh 1144 kB
zsh 1124 kB
bash 892 kB
(sd-pam) 752 kB
systemd-udevd 652 kB
systemd-journal 624 kB
systemd 584 kB
systemd 548 kB
su 508 kB
xterm 504 kB
xfm 504 kB
wpa_supplicant 496 kB
ssh 492 kB
rxvt 488 kB
rxvt 484 kB
rxvt 464 kB
rxvt 464 kB
rxvt 464 kB
etc etc sorted to ever less

So it seems the system is using more and more little bits of swap space for each program!

The same on my lapop with even less memory shows zero swap for all programs!

The consequence would be that if I leave this raspi running (security camera recordings)
it will soon run out of swap space.

It this an OS (buster) problem or something else?
All that runs is some terminals and ffmpeg instances.
The same on my laptop works without swap usage
  
Any ideas what causes this swapping and / or how to stop it?




Re: Question about ever growing swap space on raspi 4 4 GB
On a sunny day (Fri, 24 Jan 2020 06:52:42 GMT) it happened Jan Panteltje

Follow up
think I found the problem, top shows:

MiB Mem :   3906.0 total,     71.6 free,    375.9 used,   3458.6 buff/cache
MiB Swap:    100.0 total,     19.9 free,     80.1 used.   3364.1 avail Mem  


kernel claims all memory space as cache,  that leaves virtually nothing (here only 71.6 MB) for programs
those programs then start swapping!


BAD BAD BAD
Linux error!!!!

Big thread about it here with possible solutions:
 https://unix.stackexchange.com/questions/253816/restrict-size-of-buffer-cache-in-linux

I have an old PC running this software with old Slackware kernel with maybe 1G of memory for 6 month 24/7 no problem..
That is where the old setup came from.
But it had a much bigger swap partition, maybe never noticed this.

What is the point of using all memory for caching?
Makes no sense to me.
Is Linux DEAD?








Re: Question about ever growing swap space on raspi 4 4 GB
  Re: Re: Question about ever growing swap space on raspi 4 4 GB
  By: Jan Panteltje to All on Fri Jan 24 2020 08:00:58


 JP> What is the point of using all memory for caching?

because it is faster... much much much faster than polling the drive all the  
time... drives, spinning rust or similar, are slow...


)\/(ark

Re: Question about ever growing swap space on raspi 4 4 GB
Jan Panteltje wrote:

Quoted text here. Click to load it

You have that backwards.

 From your 3906 MB, 375 MB is used, the kernel is keeping 71 MB free for  
immediate needs, that leaves 3458 MB that would be unused, so it  
temporarily gets used for caching, if the 71 MB is used up, the cache  
will be lowered and that memory become available for use ... no point in  
letting memory be unused when they system can benefit from using it as  
cache.

Re: Question about ever growing swap space on raspi 4 4 GB
On 24/01/2020 08:18, Andy Burns wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

Quoted text here. Click to load it

Exactly. Cache memory is essntially mirroring the disk(s). If it gets  
full the disk is activated and any writable data written to it. Read  
data is simply discarded.

Why the Pi is using swap is another matter entirely.

If it bothers you stop it, or configure a small ramdisk as swap.


--  
I would rather have questions that cannot be answered...
...than to have answers that cannot be questioned

We've slightly trimmed the long signature. Click to see the full one.
Re: Question about ever growing swap space on raspi 4 4 GB
On Fri, 24 Jan 2020 08:55:25 +0000, The Natural Philosopher wrote:

Quoted text here. Click to load it


Quoted text here. Click to load it
Putting swap space in a ramdisk doesn't help because that is (obviously!)  
mapped into RAM, and so will merely reduce the RAM available for other  
purposes. If you want to use fixed swap space, rather than the default  
swap file, create a dedicated swap partition on spinning rust or an SSD,  
BUT I would not put one on an SD card if its at all frequently used. A  
reasonable size for fixed swap space is twice RAM size.

Use gparted or one of the older command-line tools (cfdisk, fdisk etc.)  
to manage device partitioning. Running "apropos partition" will show you  
which partition managers are installed and, as always, 'man' describes  
what each can do and how to use it. Swap partitions don't need to be  
formatted after creation.


--  
Martin    | martin at
Gregorie  | gregorie dot org


Re: Question about ever growing swap space on raspi 4 4 GB
On a sunny day (Fri, 24 Jan 2020 11:39:33 -0000 (UTC)) it happened Martin

Quoted text here. Click to load it

OK, thank everybody for the replies.
So it is a swap file... did not even know that.
Anyways it does not explain why all programs now seem to use swap space.
I did the following as test / solution perhaps:
Added  
# clear cache every 15 minutes to prevent swap space usage and swap partition overflow due to too little free memory left
0,15,30,45  *   *   *   *       /bin/echo 3 > /proc/sys/vm/drop_caches
in /root/crtab
so every 15 minutes the cache is emptied and more memory stays available:
MiB Mem :   3906.0 total,   2688.8 free,    377.1 used,    840.1 buff/cache
MiB Swap:    100.0 total,     19.7 free,     80.3 used.   3379.0 avail Mem  
  
Seem to have absolutely no effect on recording cameras,
but at least 2 GB free memory is always available.
Now the 19.7 free swap space has been constant for at least an hour.

Then I started first firefox, swap space stayed the same.
Then also started chrome browser, swapspace now increased to  
MiB Mem :   3906.0 total,   2890.2 free,    378.1 used,    637.7 buff/cache
MiB Swap:    100.0 total,     20.2 free,     79.8 used.   3380.0 avail Mem  

closing both firefox and chrome does NOT release the new swap space,
it stays at 20.2 free,  
so it seems other apps were forced to use swap, 2 GB not enough ??? LOL
  
Indeed it is then probably safer to add a swap partition with huge size.

But IMO this is a basic problem in Linux.
An ever growing swap space usage is a killer in the long run,
be it in a file or on a disk.
 ?

Re: Question about ever growing swap space on raspi 4 4 GB
Quoted text here. Click to load it

I don't see ever growing swap.
You have 3.38GB of swap you could be using.
You have 100MB of swap actually enabled, of which ~80MB is used and 20MB is free.
That's approximately nothing.

If your swap is a partition, it's a fixed size and can't grow.  So if you
exceed swap the kernel will just kill a process to free up space.  The same
thing will happen if you don't enable swap.

If you want a system with more consistent performance, you can disable swap.  
Then everything either fits in RAM or the system kills it.  Then you don't
end up in a disc thrash death spiral.

Theo

Re: Question about ever growing swap space on raspi 4 4 GB
On Fri, 24 Jan 2020 12:59:41 +0000, Jan Panteltje wrote:

Quoted text here. Click to load it

You can run without swap. The command swapoff disables swapping and swapon  
(re)enables it. It can also be permanently disabled by editing the boot  
parameters / scripts/ services (depending on which Linux you're using.

Quoted text here. Click to load it
I think you may be misreading the 'top' display. There are two lines that  
show this type of information:

MiB Mem :  7647.2 total,   4483.4 free,  1206.3 used,   1957.5 buff/cache
MiB Swap: 16500.0 total,  16500.0 free,     0.0 used.   6325.5 avail Mem

The above is from this Lenovo T440, which has 8GB RAM and a 16GB swap  
partition on its hard drive.

You can also see just these two lines by running 'free -h'.

The first of these lines describes how RAM is being used: 'buff/cache' is  
the amount of RAM occupied by file buffering and other data that isn't  
owned by any specific process.

The second line describes swap space: its used space is non-zero only  
when a process has caused RAM to be oversubscribed. The kernel will first  
try to free up RAM by discarding any file blocks that have only been  
read. If this doesn't release enough RAM then disk blocks that are full  
but not yer written out and written out and released. If more RAM is stil  
needed, then excess code and heap data in the requesting process is  
written to swapspace and is subsequently paged in/written out as needed.
Swap space will be released when that process terminates.

Note the order: Fast operations are always done first, followed by slower  
ones of the previous operations didn't make enough RAM available to the  
requesting process.

Note also, that small files (both code images and data) will tend to end  
up entirely on buffer space and will stay there even after the requesting  
process has ended - until something needs to use the RAM they're  
occupying. You can see this happening on a system with enough memory: its  
why, if you're developing a reasonably large C (or Java) application,
the first compile of the day is several times slower than the rest: for  
subsequent compilations your editor, 'make', the Makefile, the C compiler  
and the source files you're working on are already buffered in memory and  
don't need to be re-read from disk.

Quoted text here. Click to load it
Nope it isn't because what you thought was swap space isn't, and  
I've never seen properly configured swapspace (i.e. RAM * 2) get filled.

The nearest I've seen to this is when a process exceeds its configured  
maximum virtual memory size. If this happens it will be killed by the  
kernel. The virtual memory size of a process is the total of RAM used  
plus swap space used. This is a limit enforced by the kernel - it can  
happen to any program that can grow to accommodate arbitrary amounts of  
data: I've only seen this happen to a Java 1.4 application of mine  
running on a 32 bit system. The cause was the garbage collector failing  
to clear released heap-space as fast as the program was grabbing new heap-
space and loading more data into it. That has never been an issue since  
Java 1.6 and its improved garbage collector were released.  


--  
Martin    | martin at
Gregorie  | gregorie dot org


Re: Question about ever growing swap space on raspi 4 4 GB
On a sunny day (Fri, 24 Jan 2020 14:48:41 -0000 (UTC)) it happened Martin

Quoted text here. Click to load it

OK, understood,
I will keep monitoring the system
free -h is nice too.


Re: Question about ever growing swap space on raspi 4 4 GB
On 24/01/2020 12:59, Jan Panteltje wrote:
Quoted text here. Click to load it

Don't do this, it is pointless interfering with the cache system, which  
will reduce system performance and wont affect swap usage in any way.  
The kernel will always reduce the cache as far as possible and give the  
memory to applications, before it is forced to do any swapping.

Quoted text here. Click to load it

Any pages of other applications which were forced in to swap will stay  
there until those pages are needed again, and which point they will be  
loaded back in to RAM and the swap space freed. They kernel wont flush  
out the swap unless you turn if off, as there is no benefit.

Quoted text here. Click to load it

Over time unused parts of long running applications will end up in swap,  
and stay there. It's not a problem unless the system is under intense  
memory demand, and it has used both all the memory, and completely  
filled the swap file.

As you keep talking about how much cache is present, the system is  
obviously not under much memory demand, and you can safely ignore some  
minor usage of the swap file.

---druck


Re: Question about ever growing swap space on raspi 4 4 GB
On Fri, 24 Jan 2020 12:59:41 +0000, Jan Panteltje wrote:

Quoted text here. Click to load it
I don't recall how big the swap file is when its initially created,  
mainly because I use swap partitions on all my systems except the RPi.
You set the swap partition size when you create it and its size doesn't  
change unless you replace it with a different sized swap partition.  

However, I think you'll find that the swap file is expanded if the kernel  
runs out of space in it but it is never automatically reduced in size,  
because the kernel can't know whether the workload that caused the  
increase was a one-off or just the first instance of the system's normal  
workload.


--  
Martin    | martin at
Gregorie  | gregorie dot org


Re: Question about ever growing swap space on raspi 4 4 GB
On Mon, 27 Jan 2020 20:04:51 -0000 (UTC), Martin Gregorie

Quoted text here. Click to load it

    Everything I've found in Google implies that one first allocates a file
of known size (cf:
https://help.ubuntu.com/community/SwapFaq#How_do_I_add_a_swap_file.3F )
which sort of indicates that it does not grow dynamically.

Quoted text here. Click to load it

    That sounds more like Windows behavior, where swap files are given an
initial size, and a maximum size (I think I normally make Windows swap a
fixed size since the OS can allocate a contiguous chunk of disk, rather
than risk a "split" swap).

    And that brings up the biggest problem with a Linux swap file -- it is
likely not going to be contiguous even on disk AND if on an SD card (or
other flash media -- even an SSD can be killed by a system that is swapping
heavily)... Flash memory behavior means that it is likely every page swap
to SD card will trigger a new allocation unit erase/write cycle and wear
leveling means nothing is contiguous.


--  
    Wulfraed                 Dennis Lee Bieber         AF6VN
     snipped-for-privacy@ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

Re: Question about ever growing swap space on raspi 4 4 GB
On Mon, 27 Jan 2020 16:22:21 -0500, Dennis Lee Bieber wrote:

Quoted text here. Click to load it
I did say that I don't use swapfiles apart from the default one set up by  
noobs on my RPi !

man 8 swapon   has just about all you need to know, including how to  
create a swapfile (using dd and /dev/zero - use dd to copy from /dev/zero,
specifying the size with something like

dd if=/dev/zero of=/swapfile  bs51%2 count39%06250

should create a 2GB swapfile called 'swapfile' in the root directory.  
'count' is the required file size in bytes divided by the block size  
specified with the 'bs' parameter 512 bytes is the default blocksize  
given on the 'dd manpage.
  

Quoted text here. Click to load it
Could well be: as I said, I've never had to set up a swapfile because all  
my non-RPIs boxes use swap partitions.

Quoted text here. Click to load it

man 8 swapon

is your friend.

And a reminder to those newer to Linux: running "apropos xxxxxx"
gives you a list with single line descriptions of every manpage with  
"xxxxxx" in the manpage's initial NAME section. It has the format:

name(s) - one line summary description

Find one that looks interesting and then use the command "man progname"  
to see the manpage for that program or subject.

The command "man man" shows you the manpage describing 'man'. Read it to  
understand the significance of the '8' in my example of reading the  
swapon manpage.


--  
Martin    | martin at
Gregorie  | gregorie dot org


Re: Question about ever growing swap space on raspi 4 4 GB
On 27/01/2020 22:38, Martin Gregorie wrote:
Quoted text here. Click to load it

I think you'll find Raspbian (Debian?) uses systemd  
dphys-swapfile.service (/etc/init.d/dphys-swapfile & /etc/dhpys-swapfile  
+ /lib/systemd/systemd-sysv-install enable/disable dphys-swapfile) to  
compute size and load a swapfile under /var/
No, it doesn't seem to change size in use. File size is (re)calculated  
at system start.


--  

Chris Elvidge, England

Re: Question about ever growing swap space on raspi 4 4 GB
On Mon, 27 Jan 2020 23:30:07 +0000, Chris Elvidge wrote:

Quoted text here. Click to load it

Does that service recreate the swapfile every time it boots? If so, can  
the service definition be edited to change swapfile size if this should  
need doing? My RPi isn't running ATM (and isn't in this room) so I can't  
easily look at it.


--  
Martin    | martin at
Gregorie  | gregorie dot org


Re: Question about ever growing swap space on raspi 4 4 GB
On 27/01/2020 23:35, Martin Gregorie wrote:
Quoted text here. Click to load it

Look at /etc/dphys-swapfile and man dphys-swapfile
AFAIK it checks the swap file on startup. Not there => create; already  
there => check if size as requested.


--  

Chris Elvidge, England

Re: Question about ever growing swap space on raspi 4 4 GB
On Tue, 28 Jan 2020 11:23:37 +0000, Chris Elvidge wrote:

Quoted text here. Click to load it

Not on all systems: neither the manpage nor /etc/dphys-swapfile exist on  
this Fedora system - but maybe thats because it has a swap partition.


--  
Martin    | martin at
Gregorie  | gregorie dot org


Re: Question about ever growing swap space on raspi 4 4 GB
On 28/01/2020 12:13, Martin Gregorie wrote:
Quoted text here. Click to load it

dphys-swapfile (as I said earlier) is a Raspbian (Debian/Ubuntu)  
artifact. I can't see why it should exist on Fedora (RedHat?).


--  

Chris Elvidge, England

Re: Question about ever growing swap space on raspi 4 4 GB
On Tue, 28 Jan 2020 13:36:17 +0000, Chris Elvidge wrote:

Quoted text here. Click to load it

I missed the 'Debian-only' notice - sorry 'bout that.
  

--  
Martin    | martin at
Gregorie  | gregorie dot org


Site Timeline