Re: Backup Nightmare - Please help!

On a sunny day (Mon, 16 May 2022 18:19:11 -0700 (PDT)) it happened Kruse Ludington snipped-for-privacy@gmail.com wrote in snipped-for-privacy@googlegroups.com:

....

Yes, I recently found internet is getting a bit stressed with so many things blocked, sucking MB web sites, etc. Without internet would life be better?

The backups: I do not have SSDs but large 3.5 TB harddisks.

[I] Never backup from a running system

as you pointed out I simply copy what changed and was important to the other disk on the other raspi. For backups of Pi SDcards I power down the Pi, take the card out, put it in the laptop, and link to the other Pi with the other 3.5TB disk (but I also have a USB card reader I could use).

I tried scp -p /dev/raspi_card IP_OTHER_PI:/mnt/sda2/backups/my_pi4_card.img scp then complained /dev/raspi_card is not a file.... (all in UNIX is a file so whats that about??) (rapi_card stands for sdc or sdd depending whatever else you have hanging from the laptop) and make sure it is NOT MOUNTED, some system like to mount things automatically I think.

OK, so then netcat to the rescue! on the destination Pi I ran: netcat -l -p 1029 > /mnt/sda2/backups/my_pi4_card.img here netcat listens on port 1029 and sends output /mnt/sda2/backups/my_pi4_card.img

And on the sending site (laptop) did: cat /dev/raspi_card | netcat -a IP_OTHER_PI -p 1029 sends whole card image to IP IP_OTHER_PI took a while via the LAN.. There are different versions of netcat around with different arguments to stop transfer on EOF, so check man netcat.

Anyways, if you want easier access to files later, then zip the partitions mount /dev/raspi_partition1 /mnt/raspi_partition1 mount /dev/raspi_partition2 /mnt/raspi_partition2

tar -zcvf raspi_partition1.tgz /mnt/raspi_partition1 tar -zcvf raspi_partition2.tgz /mnt/raspi_partition2 then copy the compressed partitions in the normal way (scp should work :-) ) That allows you to unzip it later and get a specific file.

Having the backup thing saved me recently from disaster,

I also backup Raspi SDcards to other SDcards (all Samsung here) so I can just swap cards if something strange happens. There is a bit more to it, but OK.

As to data transfer via internet, maybe using a drone and MicroSDcards is faster and cheaper, couple of TB ... over miles.. :-)

posting pigeons with SDcards..

Anyways after WW3 nukin likely none of the electronics s*it will still work. I do keep some optical backups too though, among those M-Discs, say for the archaeologists..

The way I do this oustide of a pi environment is simply to not attempt to clone the whole drive of a machine running off that drive. the boot environment is not normally accessible to the running machines OS. Fortunately the boot environment is stable, and does not vary much - if at all. So my suggestion would be to install a bare OS/boot partition on the backup disk, and then rsync the data to it from the working one.

In my desktop/laptop setups I simply now have realised that upgrading to a major new operating system is a chance to rid the PC of the cruft that accumulates. So a fresh install on a new hard drive and then replace the programs I find I still need :-)

For the rPi just use the SDcard for boot device. I'm not convinced booting off an SSD is normally justified.

Leaving the OS reasonably untouched on an SDcard and working using Docker Containers on the SSD works well. Just back up the Docker Data containers, save Docker config in git.

Reinstalling the OS + Docker containers is then trivial, so doesn't need backing up.

OK, reinstall is not trivial, but can be done in 11 easy steps :o). (11 is a joke, not literal.) It takes an hour or two, but I like to do it occasionally to refresh my memory.

I understand Ubuntu is now using snaps. They sound cool, and I may look at using them the next time my rPi home server goes t*ts up.

Here's what I'd do (assuming I wanted to preserve the partitioning of the master SSD:

1) Get a cycle of at least two good quality 8GB SSD (cards or SSD - doesn't matter which, but SD cards are cheaper: I like Sandisk SDs . Give each a distinct name and some sort of physical storage so you can easily know which was the most recent backup - a block of wood with saw cuts would be fine if you're backing up to SD cards. In the following I assume you're using SD cards. Have a safe place to keep them (personal firesafe, etc). 2) Extend your reboot script to: a - run 'fsck' on the whole ssd: this gives advance warning of media failures b - make a full copy of your master SSD to the backup SD using 'dd' to make the backup c - now run the remainder of your reboot script.

Using this sequence makes sure you backup is clean and that you won't ry to reboot from a corrupted filing system.

3) If you're not already running 'logwatch', enable it. It defaults to running once a day and defaults to sending a system state report to 'root', but its trivial to install a mail server (Postfix is great) and configure logwatch so its daily report gets sent to your usual mail stream. Logwatch runs an extendable list of subsystem reports, so you might want to add one to report exceptions thrown by your weather service. "man logwatch" containd pointers for configuring and customising logwatch.

Now all you need to do is once a day read the logwatch report to check that all is well, remove the last overnight backup card from the Pi, put in the offline SD card rack and replace it with the oldest backup, which will get overwritten early tomorrow.

4) You could use rsnapshot to do the backup, but, configuring it to do exactly what you want, assuming you want to keep the partition contents separate will be a bit harder than using 'dd' - IIRC rsnapshot tends to build a single directory structure while 'dd' understands partitioning and will preserve your partition structure on the backup device.

It's a Pi 4B, it's capable of doing a lot, what you want wouldn't even tax a Pi 3B, so don't worry.

That's why I do with all my Pi's.

That's unnecessary, you just increase the risk failure by unnecessary rebooting. It better to enable the hardware watchdog which will make sure that it automatically reboots if the OS hangs or (optionally) the network is unreachable.

If it's not in an environment where the noise is an issue, just leave the fan on all the time, less to go wrong.

They are tough, my Pi original 256MB is still refusing to die after 190 years of use.

The easiest way to do it is you have another Linux machine, an old PC or another Pi will do. First using the other machine clone the Pi's SDD to the backup SSD using the dd command. Share the backup SSD over NFS so the Pi can mount it. Set up a cron job on the Pi to do a differential backup to the other SSD every night.

If those terms are not familiar, there are lots of tutorials on how to use dd, NFS, cron and rsync.

What that will do is give you a backup SSD which is at most 24 hours old, and ready plug straight in to the Pi should the primary fail.

I assume you are booting direct from the SSD, as if not the SD card will probably fail long before the SSD, bit that can be backed up in exactly the same way.

---druck