shrinking an image

Or just use gparted, the graphical partition editor. Select partition, click resize, draw down the size, and click apply.

Remember to let there be some free space available, there may be some changes of config files etc. that need some in the install phase. Besides /var/log, /tmp etc.

-- mrr

Rob wrote:

I started on a script to do this myself, then found that there were sufficient "gotchas", that an alternative method seemed better. If this is for your personal use, for backups and restoring, then consider saving a tarfile of the filesystem instead.

I've put my own scripts below, which might be useful as reference for creating your own script, if nothing else. These are KDE-oriented and assume that SD-cards are not auto-mounted, but are manually mounted via a USB icon that appears in the task bar, and that they are mounted at locations determined by partition labels that are written by the scripts. (K)ubuntu has /sbin in the user's path, and allows user access to filesystems if the user is in the "disk" group. I have passwordless sudo set up on my user account.

The scripts have been successfully used for backing up and restoring Raspbian, Raspbmc, and Arch. And for the initial setup of Arch, which is distributed as a filesystem tarfile. They work on Kubuntu, anyway.

#!/bin/bash # Save Raspberry Pi filesystem from SD-Card to tarfile fail() { echo Error: $*; exit 1; } bak="pi-$(date +%d%b%y | tr '[A-Z]' '[a-z]')" [[ -f "$bak" ]] && fail archive already exists [[ -z $(groups $USER | grep '\bdisk\b') ]] && fail $USER not in disk group read -p "Insert SD-card then press [enter]" sleep 1 dv=/dev/$(dmesg | grep -o '\[sd.\]' | tail -1 | tr -d '\[\]') [[ "$(sfdisk -g $dv 2>&1)" != *cylinders*heads* ]] && fail re-insert SD-card [[ -z $(udevadm info $dv 2>&1 | grep ID_BUS=usb) ]] && fail not USB device labs="$(blkid ${dv}{1,2} -s LABEL -s TYPE | sed 's:/dev/sd.::')" echo "At ${dv}:" echo "$labs" if [[ "$labs" == *vfat*ext4* && "$labs" != *FAT-BOOT*ext4-main* ]]; then read -p 'Ready to set SD-card partition labels. Press [enter]' e2label ${dv}2 ext4-main || fail label setting failed fatlabel ${dv}1 FAT-BOOT || fail need to install dosfstools read -p "Remove and re-insert SD-Card. Then press [enter]" sleep 1 fi labs="$(blkid ${dv}{1,2} -s LABEL -s TYPE | sed 's:/dev/sd.::')" [[ "$labs" != *FAT-BOOT*vfat*ext4-main*ext4* ]] && fail re-insert device ras="/media/$USER/ext4-main" boot="/media/$USER/FAT-BOOT" read -p "Mount SD-Card filesytems via USB icon or when prompted. [enter]" [[ ! -d $ras || ! -d $boot ]] && fail should mount at $ras and $boot read -p "Ready to fetch filesystem [ent]" echo Copying files from boot partition to boot directory sudo cp -fa $boot/* $ras/boot/ sudo chown root:root $ras/boot/* echo "Tarring to $bak.tar" # c=create-archive z=gzip f=file numeric-owner= suppress user:group ascii names sudo tar -C $ras --numeric-owner --one-file-system -czf $bak.tar.gz "." sudo chown $USER:$(id -ng $USER) "$bak".tar.gz # user ownership echo Erasing temporary files in /boot directory sudo rm -f $ras/boot/* echo "Done" ls -lh $bak.tar.gz

#!/bin/bash # Burn tarfile filesystem to SD-Card for Raspberry Pi fail() { echo Error: $*; exit 1; } [[ "$1" != *.tar.gz ]] && fail tar.gz file required as parameter [[ -z $(groups $USER | grep '\bdisk\b') ]] && fail $USER not in disk group read -p "Insert SD-card then press [enter]" sleep 1 dv=/dev/$(dmesg | grep -o '\[sd.\]' | tail -1 | tr -d '\[\]') [[ "$(sfdisk -g $dv 2>&1)" != *cylinders*heads* ]] && fail re-insert SD-card [[ -z $(udevadm info $dv 2>&1 | grep ID_BUS=usb) ]] && fail not USB device mod=$(udevadm info $dv | grep 'ID_MODEL=' | sed 's/.*ID_MODEL=//') labs="$(blkid ${dv}{1,2} -s LABEL -s TYPE | sed 's:/dev/sd.::')" if [[ "$labs" != *FAT-BOOT*vfat*ext4-main*ext4* ]]; then read -p "Ready to partition SD-card in $mod at $dv. Press [enter]" echo -e "1,59,0C,*\n60\n0,0\n0,0\n" |sudo sfdisk -uM -L $dv read -p $'\n'"Remove and re-insert SD-card. Then press [enter]" sleep 1 read -p "Ready to format SD-card. [enter]" mkfs.vfat ${dv}1 -n "FAT-BOOT" || fail format failed mkfs.ext4 ${dv}2 -F -L "ext4-main" || fail format failed read -p "Done. Remove and re-insert SD-card. [enter]" fi ras="/media/$USER/ext4-main" boot="/media/$USER/FAT-BOOT" read -p "Mount SD-Card filesytems via USB icon or when prompted. [enter]" [[ ! -d $ras || ! -d $boot ]] && fail should mount at $ras and $boot read -p "Ready to write tarfile to SD-card. [enter]" echo Deleting existing files... untarring to sd-card... sudo rm -rf /media/$USER/ext4-main/* /media/$USER/FAT-BOOT/* sudo tar --numeric-owner -xzpf "$1" -C "$ras" sync echo Copying files from boot directory to boot partition sudo cp -d --preserve=mode,timestamps $ras/boot/* $boot/ if [[ $? = 0 ]]; then echo Erasing temporary files in /boot directory sudo rm -f $ras/boot/* fi sync echo "Done."

It is not for backups and restoring, it is for maintaining a customized image (with a lot of software and configuration) that can later be saved on a server for dd to other SD cards. I.e. to convert a running Pi back into an installation image similar to the Raspbian image you start from.

Even when using a standard SD card size (4GB) there is a problem: we now have a batch of cards that although marked with 4GB like our working image, has a slightly smaller number of blocks. So the image cannot be dd'd to it. But a more generic solution would shrink the image to just below 2GB so any card of 2GB or more can be used.

I see you are using tar. I was thinking about using resize2fs, which works OK. But what keeps me back is the whole sequence (that is documented elsewhere) of:

- find a new suitable size

- resize the filesystem to that size

- resize the partition to exactly the right size for the filesystem

- resize the entire image to exactly the size of the two partitions that are on the card

And of course when a small error is made it will not be immediately apparent but there will be some mysterious filesystem corruption.

I am hoping that someone already did this job because many ready-to-go images appear on the internet.

Rob

That still leaves one step: to create a new .img file of the correct size (i.e. the sum of the partition sizes).

Of course on the first use of the card, it will be expanded to the size of the card using raspi-config.

Yep. And restore the first-boot configuration that runs raspi-config.

Well, if you do find a script, tell us where, and if you have to write your own, then post it in this group, please.

Hello Rob

"Rob" schrieb im Newsbeitrag news: snipped-for-privacy@xs8.xsall.nl...

The "dd" command lets you specify a specific count of blocks (the "count" parameter). In your case, I would use "fdisk -l /dev/sdx" to your SD card which shows a listing like

root@pcvater:~# fdisk -l /dev/sdc

Disk /dev/sdc: 8068 MB, 8068792320 bytes

249 heads, 62 sectors/track, 1020 cylinders, total 15759360 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0002c262

Device Boot Start End Blocks Id System /dev/sdc1 8192 122879 57344 c W95 FAT32 (LBA) /dev/sdc2 122880 5785599 2831360 83 Linux root@pcvater:~# _

For your understanding:

root@pcvater:~# echo $(((122879-8192+1) / 2))

57344 root@pcvater:~# echo $(((5785599-122880+1) / 2)) 2831360 root@pcvater:~# _

i.e. the "Blocks" columns shows 1024 bytes blocks where "Start" and "End" shows it of 512 byte block units.

In your case, use the highest block number in the "End" column (5785599 in that case), then add one block, that gives the correct block number for your "dd" command:

dd if=/dev/sdc of=mynew_raspberrypi_image.img bs=512 count=5785600

For better performance, use a larger block size. The maximum can be determined as follow:

root@pcvater:~# echo $(((5785599+1) % 8192))

2048 root@pcvater:~# echo $(((5785599+1) % 4096)) 2048 root@pcvater:~# echo $(((5785599+1) % 1024)) 0 root@pcvater:~# echo $(((5785599+1) % 512)) 0 root@pcvater:~# _

=> You can us a block size up to 1024*512 bytes in your "dd" command. I often use 16384 bytes, so according to

root@pcvater:~# echo $((16384 / 512))

32 root@pcvater:~# echo $(((5785599+1) / 32)) 180800 root@pcvater:~# _

you can use

dd if=/dev/sdc of=mynew_raspberrypi_image.img bs=16384 count=180800

for creating your image.

Andreas

--
"127.0.0.1 was ist das? Ich kenne nur ::1!" - www.swissipv6council.ch

Thanks Andreas. I know the general principles, but I still wonder if I need to invent this wheel or if an existing script exists that does the above (and more) to do the whole job of shrinking an SD card image.

Preferably I take an image obtained by "dd" of an entire SD card to a diskfile (on a PC), then run some script that takes an input and output image name as a parameter, and shrinks the image to such a size that maybe 100MB is left on the partition of the resulting image so it can be dd'd to another card and safely be booted and resized back to the size of the destination card using raspi-config.

As doing this is not my goal in life but merely a small step inside a large project that I am doing, I prefer a ready solution. And I would expect that creators of e.g. the Raspbian image have such a script in their toolbox, so I only need to locate it.

IMO the way to go is Dave's method:

1) save tar files as your master because they have no close dependency on the partition sizes 2) Use fdisk to create empty partitions on the new SD card. 3) Format them (use mke2fs for the ext3 partition and mkfs.fat / mkfs.vfat for the FAT partition 4) untar the tarballs into the relevant partition.

This method avoids dependencies on the exact partition sizes. All that matters is that a partition is big enough to contain the tarball content you want to put into it. It should be easy enough to script too.

Oh yeah, don't buy cheap SD cards if you want them to hold a more or less fixed number of blocks. I use Sandisk because they are one of the few branded SD cards where the factory is owned by the brand. IIRC Samsung is another. The rest buy whatever cards are available at the time, slap their stickers on and push them out the door, so there's no telling what you might get.

--
martin@   | Martin Gregorie 
gregorie. | Essex, UK 
org       |

I'll note that my SD burn-script, that I posted as well, takes care of the partitioning, formatting, and untarring. That too turned out to be easier than figuring out how to resize an image.

Hello Rob

"Rob" schrieb im Newsbeitrag news: snipped-for-privacy@xs8.xsall.nl...

You can easily create a shell script for that like this one:

#!/bin/bash # mkrpi_image.sh - Create SD card image with partition optimized size # Draft - not tested!

if (($# != 2)) then echo "Usage: $0 " exit 1 fi

blockcount=$(fdisk -l $1|awk 'BEGIN { maxblock=0; } \ { \ if ($3 == $3 + 0) { \ if($3 > maxblock) { \ maxblock = $3; \ } \ } \ } \ END { print (maxblock + 1)/32; }') dd if=$1 of=$2 bs=16384 count=$blockcount exit $? # ^_ Pass dd's return value to the calling script for error processing

Andreas

--
"127.0.0.1 was ist das? Ich kenne nur ::1!" - www.swissipv6council.ch

Update by myself

"Andreas Meile" schrieb im Newsbeitrag news: snipped-for-privacy@mid.individual.net...

Some important improvements to avoid processing the "Disk /dev/sd... Disk identifier: 0x0002c262" prolog part from the "fdisk -l" part as well as better error handling:

#!/bin/bash # mkrpi_image.sh - Create SD card image with partition optimized size # Draft - not tested!

if (($# != 2)) then echo "Usage: $0 " exit 1 fi

blockcount=$(fdisk -l $1|awk 'BEGIN { maxblock=-1; tablebegun=0; } \ { \ if (tablebegun == 1) { \ if($3 > maxblock) { \ maxblock = $3; \ } \ } \ if ($3=="End") { \ tablebegun = 1; \ } \ } \ END { print (maxblock + 1)/32; }')

# Stop script when fdisk -l failed if ((blockcount == 0)) then echo "Wrong special device file / partition table reading issue" exit 1 fi

# Copy the image dd if=$1 of=$2 bs=16384 count=$blockcount exit $? # ^_ Pass dd's return value to the calling script for error processing

Andreas

--
"127.0.0.1 was ist das? Ich kenne nur ::1!" - www.swissipv6council.ch

Ok that is for the final part of the job, shrinking the partition image to correspond to an already decreased size of the second partition.

I'll check if it works and maybe try to add the fs and partition resizing. But I'll probably try to find the Raspbian maintainer's version control system to see if something is there already.

Why does everyone doing anything on a Pi always have to lower the river rather than raise the bridge.

You have the original image such as Raspbian which is less than 4GB. When you copy that to an SDcard you can use resize2fs to make it use all the card space. Simples, as that infernal Meerkat would say.

So now you want to add files and if you add them to the SD card they are on a disk that's a different size to the original image and it gets complex resizing everything.

So take the original image and loop mount it on PC (not the Pi)

mount -o loop /mnt/rpi-image

Then the file system appears at /mnt/rpi-image and you copy the files into that. Unmount it and then dd the image to the SD card, boot in the pi and run resize2fs.

If the image file contains bootloaders and other partitions, which a Pi image will then you use fdisk to tell you where the partition starts.

fdisk -l

and this tells you the block size and the sector the image starts on. For the 2015-02-16-raspbian-wheezy.img fdisk shows:

The boot partition starts at 1 and the filesystem starts at 8 with a blocksize of 512.

8*512 = 4096

mount -o loop,offset=4096 /mnt/rpi-image

and you can now copy files to and from the image.

It's really not rocket science. Of course if you don't have a PC running Linux, only Windows of some kind then you are deliberately making life difficult for yourself. So download the free version of VMware and install a simple Linux system as a VM on your Windows machine to make your Pi life easier.

Yes, you've described the manual twiddling technique that the OP said right at the outset that he didn't want to do every time. And it involves burning an image to the card and then discarding it as an intermediate step. Obviously, it's the easiest way for a one-off, but it's desirable to find a neater automated process if you're regularly doing backups and have SD cards of varying sizes.

Really, I don't recall mentioning shrinking images or copying back to the image but always making the changes to the image before copying it to a card. i.e. you maintain the master image and write that to a card when needed. You can then resize the image to fit the card if you need or you have different sized cards. Or more simply remake the entire partition from the file system for the different sized cards available.

i.e. you mount the image filesystem, make the changes and then generate a new image for 2/4/8GB cards and copy the the best sized image. But you do all the fannying about on something with some horsepower and not the SD card and Pi.

Great, so it won't take you a moment to write a script to do that and post it here, then?

No it wouldn't.

Is that agreeing or disagreeing?

Yes.