Shrinking the SD card occupancy

A good disk imager program (I don't know the program you mention) has the ability to skip or specially treat blocks that are all 00.

That should reduce the space (and speed things up) at least until you have used the space once. Putting a large file on the card and then deleting it will not put the blocks back to 00.

You're backing it up wrong - IMO.

Although there are many strategies - go google for Linux backup solutions. If you have another Linux PC, or a VPS you can SSH into, then it becomes utterly trivial with rsync.

Another strategy is to consider the SD card expendable, and just re-image it from scratch should you ever need to - then you need to store your files elsewhere - using NFS/CIFS for example.

Gordon

This is the approach I've been taking since I have a fair bit of experience with digital cameras, where leaving photos on the SD card is just asking for trouble.

My main use of the Pi at the moment is as a GPS track capture device; the track logs are written to the card, and moved off it as soon as I get it home. There's a small perl script to handle the capturing, which I wrote elsewhere and deployed to the pi's card as part of the setup procedure.

I have no other Linux systems. Re-imaging from scratch means I have to spend time re-installing all the software, and compiling some of it can take ages. So accepting that I'm not backing it up in the way you would choose, is there a way of shrinking the SD card occupancy so that it will image to a smaller .IMG file?

Yes. Boot from a suitable Linux live CD and backup the partition using Clonezilla. SystemResueCD, for example, includes Clonezilla, and IIRC Clonzilla can do on the fly compression of backups. The trick is not to shrink the SD card "occupancy" but to use a partition/disk backup tool which is aware of file system formats and able to use some intelligence to avoid wasting time and space backing up/restoring areas of partitions which contain no useful data. The image compression is an added bonus on top of that. There are similar tools for Windows if you prefer to be parted from your money.

No. Not for WinDiskImager (or whatever it is called). It backs up the whole card to an image, no matter what is on it.

I have a Linux script that runs in safe-mode on the Pi and will backup the card contents to another card in a usb card reader on the Pi. I also use it to back cards up to smaller cards - if the data will fit. One advantage is that it creates a full working card that can be booted from immediately.

I backup my SD cards weekly. First I shrink to 3GB using gparted then copy to HDD with dd. (Then use gparted to restore to original size.)

There are Windows solutions too:

Simplest: get yourself an SD card reader and periodically use it to copy the contents of both the partitions onto the hard drive on your PC. When I plug my RPi's SD card into my Linux laptop I see the two partitions as separate disk drive icons: I'm guessing, but I assume Windows is also capable of doing that.

Best: Install PuTTY on your Windows box and connect both machines to your home network. You can use the PuTTY ssh client to login on the RPi, run tar to make an archive of everything on your main Raspbian partition, and then use PuTTY's scp utility to copy the archive file to your PC's hard disk. You can also use scp to move individual files in either direction, but it will be tiresome: hence my suggestion to use tar archives (aka known as tarballs) for backups: moving one big file across beats the hell out of moving loads of little ones.

Alternatively, enable the FTP server on the RPi and install an FTP client on your Windows box. Now you can use FTP to transfer files and whole directories between the two. There are some nice graphical Windows FTP clients available. OTOH, PuTTY may by now include an sftp client, which can do the same things as FTP, but probably won't have the cute graphical interface.

Mine are also backed up weekly, just before I do the weekly software upgrade. However, I use rsync for the job. Much faster.

I use a CVS repository as a matter of course to store my program sources, scripts, web pages, etc. on my other Linux boxes, so I installed the client end on my RPi where it provides the same no-brainer backup service as it does on my other boxes.

This also makes life really simple if I need any of my other C code or scripts on the RPi: just checkout the module(s), run 'make all' and then 'make install'. Job done.

On Fri, 21 Jun 2013 22:03:24 +0000 (UTC), Martin Gregorie declaimed the following:

Unless things have changed, Windows won't recognize the Linux file system(s).

OK, so scratch that idea.

The OP's easiest choice is probably to use PuTTY for the file transfer, since by default the SSH daemon is already configured and running on the RPI and he doesn't need to install and configure an FTP server on the RPi until he's had a chance to find his feet with it.

Is the PI EXT2/3?

I thought it might be FAT..

backing up a running Raspberry pi to windows such that it can be restored from windows seems potentially problematic.

you caould create a tar of the linux partition - but there's no way to restore it

you could just copy the files - but you'll loose the permissions. and again, no way to restore.

You could create an empty image file on a windows share create and formt the two partitions needed mount them loopback and copy the contents in all fairly complex steps but likely to result in a usable backup.

The first partition is FAT, just for booting. The main partition is ext4.

Folks, many thanks for all the suggestions, they are much appreciated. Whilst I like the idea of continuous or automated backups, it simple isn't necessary in my environment, so it's going to be simple just to take the RPi down from time to time and run the Win32 Disk Imager program, and perhaps use 7-zip to compress the resulting file.

What I haven't yet done, and obviously should, is to try restoring one on the image files to a fresh card and checking it works!

I think it can be done well enough to cover most cases provided that the tarball only contains the /home directory structure *and* you've kept the card image you used to set up the SD card. To recover:

- reflash the SD card with the saved image,

- boot the RPi and do a full software upgrade

- use PuTTY to copy the tarball back to the RPi

- unpack the tarball with tar.

The only other stuff I'd add to this is that:

- any configuration files you've edited in /etc should also have copies kept somewhere in the /home structure.

- if you're making use of the /usr/local/* structure to hold locally developed or other nonstandard software do this as root: cd /usr mv local /home/local ln -s /home/local local which allows the transplanted /usr/local/* structure to work just as it always did, but makes sure that its contents are backed up whenever /home is backed up.

All newly installed distros I've seen have an empty /usr/local/* structure, so after reflashing the card, doing the update and untarring your /home structure, you do these as root:

- put the saved copies of all the modified /etc files back

- cd /usr

- rm -rf local

- ln -s /home/local local

and everything should work as before. I've been using the symlinked /usr/local trick for years without problems.

I've been wondering if pivot_root would allow one to switch from the SD card to a USB memory stick. If so, one could then do things to the SD card.

I gave someone a Linux-formatted USB mem stick, and they complained that it didn't work - because they used Windows, as it turned out. But there was third party Windows software which did allow access, though I never verified that.

Ext2read allows read-only access to ext2/3/4fs from Windows.