Backup method

Backup methods?
I tend to reconfigure and test the capabilities of Raspbian with things
like installing servers and installing drivers from source etc. But then
I want to revert to a previous good configuration, especially since I
don't want too many processes on a lightweight CPU.
At the moment, I've got a bunch of Samsung 4G class 6 micro-SDs
from e.g.:
formatting link

...which are useful in that I don't want to fill the OS partition with
more that would fit in 4G, and I can "dd" from one to the other quite
quickly via a couple of USB adaptors on my desktop PC. And the 4G files
gzip reasonably efficiently to file backups. This is OK as a backup
system but there are probably better ways.
What backup method do you use?
Reply to
Dave Farrance
Loading thread data ...
Actually I think DD'ing & Zipping your known good system is probably as good a way as any (at least on a PI when you are talking about a 4gb ) . more traditional backups are better suited to data & config files.
--
Happiness isn't something you experience; it's something you remember. 
		-- Oscar Levant
Reply to
alister
I hold backups for all computers on my LAN on a set of USB hard drives. These are kept offline in a small fire safe so mains spikes, etc can't affect them. Each drive in turn is attached to my house server to make a weekly backup.
All machines on my LAN are backed up to the hard drive with rsync on a weekly basis immediately before I do a software upgrade on that machine. This covers my RPi as well as three Fedora-based systems. I don't run Windows apart from one dual boot legacy system, which is backed up after booting into Fedora.
The benefit of using rsync is speed, because for it does the minimum work needed to make the backup file set identical to that on the system being backed up.
--
martin@   | Martin Gregorie 
gregorie. | Essex, UK 
 Click to see the full signature
Reply to
Martin Gregorie
rsync -av run daily by root cron from my RasPis to my main desktop computer. Then CrashPlan.com backs up files from my desktop and other computers in the house. CrashPlan saves multiple versions of files based on timestamp.
I could do the rsync more often, but I don't change files on them often, and the data files that they collect are disposable.
JimR
Reply to
JimR
OK, thanks. I guess that I was thinking that there might be suitable backup apps out there that were relevant, but that's not happening. The Pi SD-Card is specific enough to require a custom solution.
I've been meaning to look at the minimal OSs like Arch and PiCore, and those would probably be best saved with a filesystem-based backup script, rather than saving 4G each time, and I'd need to format the card. That would also allow me to use other SD-card sizes. I'll adapt my image-burning bash script.
I see that the formatting utilities fdisk and sfdisk (the latter for scripts) report block numbers that differ by one block, presumably due to either starting at one or zero, which is tiresome. I'll need to get that right to align the partition correctly.
And I guess that the word that I want might be "archive" rather that "backup" since I am more concerned with making an archive of separate images rather than updating the same one with rsync, i.e. the script should decline to overwrite images (but thanks to those making suggestions). I'll pick and choose which old images I want to delete.
Reply to
Dave Farrance
I mount /home off a NFS server for the most part, however what I also do is, once I have it setup is to run
dpkg --get-selections > ~/thispi.selections
periodically.
then if I ever did have to restore it, I can start with a fresh raspbian and just clear and set the selections from that file.
(See pages on cloning a Debian installation for more details).
I have used the rsync method, and I use that on my workstation, servers, etc.
The dd/gzip method is fine, but will only restore to an SD card of exactly the same size or bigger than the old one, and it will take more time,
Gordon
Reply to
Gordon Henderson
Pull card from Pi, insert into Linux server or Linux laptop card slot and DD entire card image to server/laptop disk.
Reply to
mm0fmf
So. I've now written a script to save the Pi filesystem from an SD-Card to a tarfile and another script to partition the SD-card, format it, and write the tarfile filesystem into it. They seem to work OK on Kubuntu, and the second script also works with the Arch Linux tarfile which is the only way that Arch is supplied now.
They proved to be difficult to write with an endless stream of "gotchas" to work around. They also require a fair number of utilities to be installed.
And they only work with KDE at the moment which has its own specific way of handling udev mounting.
KDE does not automount by default but Gnome and Unity do. Not automounting means that the user must be prompted to mount from the taskbar icon, but automounting by default caused problems for partitioning and formatting. So I'd have to add umount commands in there and do another round of testing if I want it to work on other desktops. I might do that sometime, but not now.
Anyway, here's the scripts, and all I can say is that they work for me on my system as it's configured at the moment:
#!/bin/bash # Save Raspberry Pi filesystem from SD-Card to tarfile fail() { echo Error: $*; exit 1; } bak="pi-$(date +%d%b%y | tr '[A-Z]' '[a-z]')" [[ -f "$bak" ]] && fail archive already exists [[ -z $(groups $USER | grep '\bdisk\b') ]] && fail $USER not in disk group read -p "Insert SD-card then press [enter]" sleep 1 dv=/dev/$(dmesg | grep -o '\[sd.\]' | tail -1 | tr -d '\[\]') [[ "$(sfdisk -g $dv 2>&1)" != *cylinders*heads* ]] && fail re-insert SD-card [[ -z $(udevadm info $dv 2>&1 | grep ID_BUS=usb) ]] && fail not USB device labs="$(blkid ${dv}{1,2} -s LABEL -s TYPE | sed 's:/dev/sd.::')" echo "At ${dv}:" echo "$labs" if [[ "$labs" == *vfat*ext4* && "$labs" != *FAT-BOOT*ext4-main* ]]; then read -p 'Ready to set SD-card partition labels. Press [enter]' e2label ${dv}2 ext4-main || fail label setting failed fatlabel ${dv}1 FAT-BOOT || fail need to install dosfstools read -p "Remove and re-insert SD-Card. Then press [enter]" sleep 1 fi labs="$(blkid ${dv}{1,2} -s LABEL -s TYPE | sed 's:/dev/sd.::')" [[ "$labs" != *FAT-BOOT*vfat*ext4-main*ext4* ]] && fail re-insert device ras="/media/$USER/ext4-main" boot="/media/$USER/FAT-BOOT" read -p "Mount SD-Card filesytems via USB icon or when prompted. [enter]" [[ ! -d $ras || ! -d $boot ]] && fail should mount at $ras and $boot read -p "Ready to fetch filesystem [ent]" echo Copying files from boot partition to boot directory sudo cp -fa $boot/* $ras/boot/ sudo chown root:root $ras/boot/* echo "Tarring to $bak.tar" # c=create-archive z=gzip f=file numeric-owner=suppress user:group names sudo tar -C $ras --numeric-owner --one-file-system -czf $bak.tar.gz "." sudo chown $USER:$(id -ng $USER) "$bak".tar.gz # user ownership echo Erasing temporary files in /boot directory sudo rm -f $ras/boot/* echo "Done" ls -lh $bak.tar.gz
# ------------------------------------------------------------------
#!/bin/bash # Burn tarfile filesystem to SD-Card for Raspberry Pi fail() { echo Error: $*; exit 1; } PATH=$PATH:/sbin:/usr/sbin # for Debian users [[ "$1" != *.tar.gz ]] && fail tar.gz file required as parameter [[ -z $(groups $USER | grep '\bdisk\b') ]] && fail $USER not in disk group read -p "Insert SD-card then press [enter]" sleep 1 dv=/dev/$(dmesg | grep -o '\[sd.\]' | tail -1 | tr -d '\[\]') [[ "$(sfdisk -g $dv 2>&1)" != *cylinders*heads* ]] && fail re-insert SD-card [[ -z $(udevadm info $dv 2>&1 | grep ID_BUS=usb) ]] && fail not USB device mod=$(udevadm info $dv | grep 'ID_MODEL=' | sed 's/.*ID_MODEL=//') labs="$(blkid ${dv}{1,2} -s LABEL -s TYPE | sed 's:/dev/sd.::')" if [[ "$labs" != *FAT-BOOT*vfat*ext4-main*ext4* ]]; then read -p "Ready to partition SD-card in $mod at $dv. Press [enter]" echo -e "1,59,0C,*\n60\n0,0\n0,0\n" |sudo sfdisk -uM -L $dv read -p $'\n'"Remove and re-insert SD-card. Then press [enter]" sleep 1 read -p "Ready to format SD-card. [enter]" mkfs.vfat ${dv}1 -n "FAT-BOOT" || fail format failed mkfs.ext4 ${dv}2 -F -L "ext4-main" || fail format failed read -p "Done. Remove and re-insert SD-card. [enter]" fi ras="/media/$USER/ext4-main" boot="/media/$USER/FAT-BOOT" read -p "Mount SD-Card filesytems via USB icon or when prompted. [enter]" [[ ! -d $ras || ! -d $boot ]] && fail should mount at $ras and $boot read -p "Ready to write tarfile to SD-card. [enter]" echo Untarring to sd-card... sudo rm -rf /media/$USER/ext4-main/* /media/$USER/FAT-BOOT/* sudo tar --numeric-owner -xzpf "$1" -C "$ras" sync echo Copying files from boot directory to boot partition sudo cp -d --preserve=mode,timestamps $ras/boot/* $boot/ if [[ $? = 0 ]]; then echo Erasing temporary files in /boot directory sudo rm -f $ras/boot/* fi sync echo "Done."
Reply to
Dave Farrance
Is that wise? Especially after the vulnerabilities exposed in recent days.
Reply to
mm0fmf
It *is* wise, provided it?s you who knows the script you are running.
Gregor
Reply to
Gregor Szaktilla
Indeed. Not sure if that guy was trolling or just confused by the way some of the net tabloids had covered the shellshock vulnerability. To be vulnerable, you'd have to be running a non-Debian/Ubuntu derived Linux distro, that hadn't been patched recently, that was running an Internet-facing web-server that you'd set up with SHTML pages that contained CGI code that was written in a language that called via the shell such as SSI (Server Side Includes). SSI is a commonly used language for websites, hence the widespread vulnerability, so I guess that the real worry would have been for those website owners that didn't know what code they were running because they had commissioned a website from an author that they could no longer contact.
Reply to
Dave Farrance
I clone the SD/mmc cards/ssd drives to other, compatible media. I then often exchange it and store the original to assure that I can, indeed, run from the backup.
Other than that, all source and important documents are checked into cvs at a hosting center.
-- mrr
Reply to
Morten Reistad
I have a ZFS fileserver which snapshots daily at 1 minute to midnight. When I am developing on Pi's at home, I have them rsync to a directory on the ZFS fileserver about half an hour before the snapshot, using the --inplace option and --block-size= ZFS record size.
This way, I can retrieve a file as it was at that time on any previous day, using minimal ZFS file overhead (ZFS will only keep copies of blocks which change from one day to the next).
When a Pi goes off-site to its final destination (which I usually can't access remotely), my backup is by popping out the sdcard, and making a sector-by-sector copy of it on my laptop. This only happens sporadically, but generally, there's no data which needs backing up kept on the sdcard at this point. Keep meaning to do something better here though.
--
Andrew Gabriel 
[email address is not usable -- followup in the newsgroup]
Reply to
Andrew Gabriel
No not trolling. It's not just web servers either.
I didn't bother looking into your scripts so I didn't see whether you where using any bash specific features. I thought I'd prod you to consider dash or sh. I know I get into a habit of writing #!/bin/bash when other shells would do and would be probably better. The recent Shellshock issues caused me to stop and think a bit after having checked the many systems I have facing the raw net were patched.
Reply to
mm0fmf
It?s all about responsibility :-)
If you?re motivated to really know what you?re doing, you have the best motivation to dive deeper into things.
Have a nice day there,
Gregor
Reply to
Gregor Szaktilla

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.