Zeroeing out unused sectors before DD-ing ?

Do you have a question? Post it now! No Registration Necessary

Threaded View
Hello all,

I'm intending to do a raw backup (alike DD do if I'm not mistaken) of the SD  
card on a Windows machine and than zipping it.  For that reason I would like  
to clear all unused sectors before making the copy, as than the zipping will  
result in the smallest possible file.

Q: Does Stretch have a program available which can do that, and if so which  
is it ?

Regards,
Rudy Wieser



Re: Zeroeing out unused sectors before DD-ing ?
On 06/03/2019 06:50, R.Wieser wrote:
Quoted text here. Click to load it
Mmm. I suspect you will be out of luck.

The only way to make sectors that are not in use zero that I can think  
of, is to do a complete write of a blank piece of something..with zeros  
and copy only the valid data ontop it.


That means working at the file copy level, not the DD level.

i.e. you need to create a new partion system that is all zeros, except  
the partion inmformation of course, then copy all the files onto it and  
then  zip the lot up.




--  
If I had all the money I've spent on drink...
..I'd spend it on drink.

We've slightly trimmed the long signature. Click to see the full one.
Re: Zeroeing out unused sectors before DD-ing ?
TNP,

Quoted text here. Click to load it

Always a possibility.

Quoted text here. Click to load it

Alas, I'm looking for a backup method of an in-use card (updated, installed  
programs - the works).  Would also be nice if I could backup a  
LibreElec/Kodi configured card with it.

Quoted text here. Click to load it

I am not at all sure if I can copy /all/ files (to begin with), and if  
writing them back to a newly created SD card (using noobs) will result in,  
functionally, the same as the origional.

Which is why my thoughts went toward a "raw" copy - either the full SD card,  
or per partition (not sure about which one yet).

And to be honest, I just now realized that clearing out unused sectors on a  
LibreElec/Kodi configured card might be a problem ... (no user accessible  
OS).

Regards,
Rudy Wieser



Re: Zeroeing out unused sectors before DD-ing ?
On Wed, 6 Mar 2019 08:52:02 +0100

Quoted text here. Click to load it


For the home partition I've used 'dd if=/dev/zero of=blank'
Then once it reports disc full 'rm blank'

Rinse and repeat for any other partitions.


--  
W J G

Re: Zeroeing out unused sectors before DD-ing ?
Am 06.03.19 um 09:16 schrieb Folderol:
Quoted text here. Click to load it
So do I.  Don't forget to wipe the /tmp and /ver/tmp dirs before  
zeroing.  There might be a lot of garbage not needed anymore.
Regards
JJenssen


Re: Zeroeing out unused sectors before DD-ing ?
On 06/03/2019 07:52, R.Wieser wrote:
Quoted text here. Click to load it

The PI seems to work thisaway - please correct me if you know better .

1/. It needs first partition as VFAT containing some boot level stuff.
2/. Everything else is just files in an EXT4 partion
3/. There is no 'boot sector' as such.
4/. The ext4 partiton is in use mounted as root
5/. the vfat partion is mounted as /boot.
6/. The ONLY info OUTSIDE of files that is on te SD card iare te  
partition UUIDS which are used by fstab and mount to mount the  
partitions. You can get roynd that anyway by editing fstab to use te raw  
device types..e.g.
/dev/mmcblk0p2 on /
and
/dev/mmcblk0p1 on /boot

If you do that you have a bootable system irrespetive of partition UUID


Quoted text here. Click to load it
#
??? how come no users accessible OS?

You can allegedly ssh into LibreElec. It is a stripped down linux.

So most basic linux comnnads should be there somewhere.

Well in any case using - say - a live linux DVD to boot an intel class  
machine, and an SD card reader, its no big deal to mount both partitions  
on the SD card, and file copy their contents onto the host machines disk.

Then you have essentially all the file level info. Which you can zip up.

To create a bootable image in a new SD card, you use a live linux boot  
system, to create a 40Mbyte vfat partition on a new SD card and format  
the rest as ext4

Then unzip and copy the files to the partrtions, eject the card and put  
it in a Pi and boot it..



Quoted text here. Click to load it


--  
All political activity makes complete sense once the proposition that  
all government is basically a self-legalising protection racket, is  
We've slightly trimmed the long signature. Click to see the full one.
Re: Zeroeing out unused sectors before DD-ing ?
On 2019-03-06 07:50, R.Wieser wrote:
Quoted text here. Click to load it

Find where your partition ends
(some clever use of fdisk is likely needed)

use dd and 'seek' option to offset the output, use /dev/zero as input

dd if=/dev/zero of=/your/card seek=some_offset

Now - I've never done it - so I don't guarantee it will work.
And I doubt that the space saved is worth the effort.

But in theory, it should be doable



--  
--


Re: Zeroeing out unused sectors before DD-ing ?
R.Wieser wrote:

Quoted text here. Click to load it

<http://manpages.ubuntu.com/manpages/xenial/man8/zerofree.8.html

Re: Zeroeing out unused sectors before DD-ing ?
On 06/03/2019 09:12, Andy Burns wrote:
Quoted text here. Click to load it

Quoted text here. Click to load it

Thats a neat tool. I wonder if it would work on an SD card with wear  
levelling tho?

I suppose from the point of the OP it wouldnt matter...The SDcar would  
fully rersent iotse;lf as full of zeroes.

I have to say that the thought of imposing multiple writes on an SD card  
to clear it fills me with horror.

I'd far rather take an image of it and zero that.

so DD what's there off to an image FIRST, loop mount it, clear it,  
compress it..




--  
"First, find out who are the people you can not criticise. They are your  
oppressors."
We've slightly trimmed the long signature. Click to see the full one.
Re: Zeroeing out unused sectors before DD-ing ?
On Wed, 6 Mar 2019 09:12:05 +0000

Quoted text here. Click to load it

I'd have done  
cat /dev/zero > dummy.file && rm dummy.file  
easy to remember and type but apparently a lot slower than zerofree.

The secure delete package has an option to wipe empty space too,
sfill -fz
would seem to do the job, but I'm not sure how it does it.


Re: Zeroeing out unused sectors before DD-ing ?
On Wed, 06 Mar 2019 07:50:21 +0100, R.Wieser wrote:

Quoted text here. Click to load it

Not what you asked for, but I have used 'partimage' on linux systems for  
years. It will back up the used sectors of a partition. It can also  
compress the image and/or backup over the network. It is linux only, and  
seems not to be in the raspbian repository. You could use a linux live cd  
on your win machine to do it - or you could download the source and build  
it on your raspbian. I first became aware of partimage several years ago  
when I wanted to upgrade the hard drive on a laptop. Copied out the  
image, partitioned the new drive and copied back - as I recall it can  
restore an image to a larger partition with no hassle.

Re: Zeroeing out unused sectors before DD-ing ?
On Thu, 07 Mar 2019 20:30:47 +0000, ray carter wrote:

Quoted text here. Click to load it
You can also try rsync: which is a Raspbian package and will copy data to  
any other partitions and ignore cruft on the source:

- use a partitioning utility to set up partitions on the new SD-card (in
  a USB-SD card adapter or a USB-connected HDD

- use rsync to copy the partition contents across.

- boot off the new SD card.

You can also use any archiving system such as tar for the EXT4 partition  
and ZIP for the FAT partition. Don't use zip for the EXT4 partition  
because it will play merry hell with Linux permissions and file ownership.

Any of these will only copy the contents of directories and files, while  
ignoring anything that isn't part of a directory or file.


--  
Martin    | martin at
Gregorie  | gregorie dot org

Re: Zeroeing out unused sectors before DD-ing ?
Quoted text here. Click to load it

I'm not aware of a tool in Stretch (I assume that is debian? - I'm
not very up-do-date but anyway) - the following should do the same:

Create a file that uses the empty space on that disk:

,----
| dd if=/dev/zero of=file-on-the-disk.bin bs=1m
`----

dd stops when the disk is full - the empty space is written with
zeroes.

then delete the file and make your disk backup. dd backups are
usually the whole disk, but when you zip it you should see the
difference (depending on your otherwise empty space)

Quoted text here. Click to load it

Re: Zeroeing out unused sectors before DD-ing ?
Quoted text here. Click to load it

Make your Image-Backup with dd and then use "PiShrink"  
(https://github.com/Drewsif/PiShrink /). This tool shrinks your image to  
minimal.
HTH

Re: Zeroeing out unused sectors before DD-ing ?
R.Wieser wrote:

Quoted text here. Click to load it

Before I take out my SD card to let dcfldd (an enhanced version of dd)
copy an image that gets piped to 7z, I make this via ssh:

sudo ionice -c 3 fstrim -v /

This takes a while. But my 32 GB card's copy uses up 6 GB instead of 10
or so GB (packed).

B. Alabay

--  
http://www.thetrial.eu/
???????????????

Site Timeline