Create NDIF disk image from RasPi SD card

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
The 2018-06-27-raspbian-stretch-lite.img file I use to create RasPi boot
SD cards from is an NDIF disk image.

Can I create one of these from an SD boot card on which I have installed
updates and additional configuration, to make it easier to create new
boot cards from?

Daniele

Re: Create NDIF disk image from RasPi SD card
On Wed, 18 Jul 2018 15:36:41 +0200,
real-not-anti-spam-address@apple-juice.co.uk (D.M. Procida) declaimed the
following:

Quoted text here. Click to load it
    Crawling around the official site (I normally just grab the NOOBS full
image)

    Both are available as direct ZIP files, or as Torrent (which I've never
used). If you fetched via Torrent, the format may be influenced by the
client you used -- Firefox on Win10 is showing ".zip.torrent" if I click on
the torrent link.

Quoted text here. Click to load it
    A Google search for NDIF seems to imply that it is a deprecated
Apple-specific format (replaced by UDIF).

    On Windows, if one has /identical spec/batch/ SD cards, I'd probably
use Win32DiskImager to create an .IMG file from a configured SD card, then
change cards and use Win32DiskImager to write the IMG file to the new
cards. Note that minor differences between cards that claim to be the same
size/speed can cause the imager program to fail (if one card reserves a few
MB more for wear-leveling space the usable size becomes smaller than the
IMG file one started with)


--  
    Wulfraed                 Dennis Lee Bieber         AF6VN
     snipped-for-privacy@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Re: Create NDIF disk image from RasPi SD card

Quoted text here. Click to load it

To be clear, I don't care much about the format of the disk image, just
about having *a* disk image or file, created from an SD card, that I can
then re-use.

Daniele

Re: Create NDIF disk image from RasPi SD card
Quoted text here. Click to load it


If you're on a Mac as I think you might be, insert the SD card.

$ diskutil list

look at the list of devices shown, find the one that's the right size,
probably at the bottom of the list. For instance it might be shown as disk3.
If you get this wrong you might nuke your system disc, so be careful.

$ diskutil unmountDisk disk3

if it's still showing in Finder (if you click the eject button in Finder it
might disappear from the list).

$ sudo dd if=/dev/rdisk3 of=your-image.img bs=1m

Will make an image of the filesystem, the same size as your SD card (16GB,
32GB, whatever).  If you want compression, try something like:

$ sudo dd if=/dev/rdisk3 bs=1m | xz > your-image.img.xz


Theo

Re: Create NDIF disk image from RasPi SD card

Quoted text here. Click to load it

Thanks, excellent instructions.

I didn't know it wasn't necessary to create the .img file before
starting (which is what I was trying to do).

And, ejecting in the Finder does something more than just unmounting -
if you eject, then the volume won't be available for disk operations. So
it has to be an unmount, either in the Terminal or in Disk Utility.

Daniele

Create NDIF disk image from RasPi SD.
Hello Theo,

T> $ sudo dd if=/dev/rdisk3 of=your-image.img bs=1m

At the Pi 3B+ this gives the error:
dd: invalid number '1m'

Use 1M instead of 1m  ;-).
I am now writing Raspbian Stretch 2018-06-27 img from a DownLoad dir
on a 32 GB usb card which is /dev/sda to another 32 GB card as /dev/sdb.
The 16 GB SDcard booted from had too little free space as temporary storage.
About 10 minutes I know the result ;-).

sudo dd if=/media/pi/rootfs/home/pi/Downloads/Raspbian/2018-06-27-raspbian-
stretch.img of=/dev/sdb bs=1M
4600+0 records in
4600+0 records out
4823449600 bytes (4.8 GB) copied, 567.882 s, 8.5 MB
He's ready and it works well. Ofcourse many things need configuration.

Henri.


Re: Create NDIF disk image from RasPi SD.
Quoted text here. Click to load it

Daniele is on a Mac, where it's 1m.  Similarly for *BSD (from where Macs
get their dd).  On Linux it's 1M.

It's really rather annoying they have this difference.

(you can do without the 'bs' blocksize setting, but the copy will go a lot
more slowly as it'll copy one byte at a time)

Theo

Re: Create NDIF disk image from RasPi SD.
On a sunny day (21 Jul 2018 13:11:58 +0100 (BST)) it happened Theo

Quoted text here. Click to load it

Actually 512 bytes at the time,
and that is often the exact sector size of SDcards,
so normally no need to specify a bigger blocksize.
Try it, makes no difference whatsoever, except for reporting perhaps.
What does make a difference is fast cards, and a fast card reader.

From  
man dd
ibs=BYTES
              read up to BYTES bytes at a time (default: 512)
obs=BYTES
              write BYTES bytes at a time (default: 512)


Create NDIF disk image from RasPi SD.
Hello Jan,

RM>> (you can do without the 'bs' blocksize setting,
TM>> but the copy will go a lot more slowly as it'll copy one byte at a time)

JP> Actually 512 bytes at the time,
JP> and that is often the exact sector size of SDcards,
JP> so normally no need to specify a bigger blocksize.

Sorry but of course there is a difference in speed versus blocksize.

JP> Try it, makes no difference whatsoever, except for reporting perhaps.

See my results below.

JP> What does make a difference is fast cards, and a fast card reader.

Yes, that too.

JP> From
JP> man dd
JP> ibs=BYTES
JP>               read up to BYTES bytes at a time (default: 512)
JP> obs=BYTES
JP>               write BYTES bytes at a time (default: 512)

Remark that bs <> ibs and obs.
I donot know the different effects of it.
May be next round when I make new versions for the skippers.

See here my results of several days and hours of SDcard filling
with the latest Raspbian Stretch Linux, OpenCPN and inland ECDIS navigation
charts.

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb bs=1M
14784+0 records in
14784+0 records out
15502147584 bytes (16 GB) copied, 1707.88 s, 9.1 MB/s

1707.88 seconds is about 48 minutes.

Other tries with the same card reader/writers for source and targets
with all 16 GB Kingston microSDcards Class 10 rated max. 80 MB/sec.:

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb bs=4M
3696+0 records in
3696+0 records out
15502147584 bytes (16 GB) copied, 1709.89 s, 9.1 MB/s

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb bs=4M
3696+0 records in
3696+0 records out
15502147584 bytes (16 GB) copied, 1940.11 s, 8.0 MB/s

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb bs=1M
14784+0 records in
14784+0 records out
15502147584 bytes (16 GB) copied, 1682.93 s, 9.2 MB/s

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb
30277632+0 records in
30277632+0 records out
15502147584 bytes (16 GB) copied, 6219.24 s, 2.5 MB/s
pi@raspberrypi:~ $

103 minutes, that's much more than the first 48.

For the last Kingston SDcard, I had to use no blocksize, i.e. the default one,
because it was a slower 16 GB Class 10 microSDcard rated max. 45 MB/sec.
The source SDcard was a newer Kingston 16 GB Class 10 microSDcard rated
max. 80 MB/sec.
Using a bigger blocksize resulted in a not good filled slower card,
especially the EXT4 partion was not copied right and had to much free space, en  
many directories and files missing ;-(.
Even the Raspbian SDcard Copier could not handle this different source and
target speeds, only DD with default blocksize did the right copy trick.
So beware.
And always dismount the source and target media before starting a backup
with the Raspbian SDcard Copier or the DD-command.
Otherwise you will damage the card's contents when dismounting the original
blank fat image on the just copied one, i.e. small fat and large ext4 parts.

This night my 32 GB version is being copied as a backup.
Unluckily I forgot the bs parameter this time,
so it will take a lot more time I think.
But that does not matter when I sleep.
Tomorow I can report the exact result ;-).
May be I should store SDcard images at an external HDD.

Another difference I remarked was that the Raspbian SDcard Copier left less
free space on the destination card after copying the same sized source card.
The 16 GB source card had 2.0 GB free,
and the identical make and type target version only 1.9 GB, strange.
When making the backup with the DD-command, they were exactly the same.
But be very carefull when using DD, as it can be very dangerous when making
mistakes in source and destination parameters, or inserting the cardreaders in  
a different order. That insertion order influences the device name,
i.e. which one is /dev/sda and wich one /dev/sdb?
So good luck in copying.

Geetings van Henri.


Create NDIF disk image from RasPi SD.
Hello Jan,

HD> This night my 32 GB version is being copied as a backup.
HD> Unluckily I forgot the bs parameter this time,
HD> so it will take a lot more time I think.
HD> But that does not matter when I sleep.
HD> Tomorow I can report the exact result ;-).
HD> May be I should store SDcard images at an external HDD.

So here are the results as promised:

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb
60456960+0 records in
60456960+0 records out
30953963520 bytes (31 GB) copied, 10383.6 s, 3.0 MB/s
pi@raspberrypi:~ $

10384 seconds is 173 minutes or 2 hours and 53 minutes.
pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb bs=1M
29520+0 records in
29520+0 records out
30953963520 bytes (31 GB) copied, 3811.36 s, 8.1 MB/s

64 minutes for a 32 GB Class 10 card.
A little bit slower as the 16 GB did took about 28 minutes.
May be I should use 2M buffers for a 32 GB card the next time?
You see here that higher up the buffer from default 512 KB block size
up to 1 MB has a big effect in speed!
173 minutes versus 64 for the same amount is almost a 3 times increase!
That's the proof that setting a right blocksize does matters really.

HD> Another difference I remarked was that the Raspbian SDcard Copier left
HD> less free space on the destination card after copying the same sized
HD> source card.
HD> The 16 GB source card had 2.0 GB free,
HD> and the identical make and type target version only 1.9 GB, strange.
HD> When making the backup with the DD-command, they were exactly the same.
HD> But be very carefull when using DD, as it can be very dangerous when
HD> making mistakes in source and destination parameters, or inserting the
HD> cardreaders in a different order. That insertion order influences the
HD> device name,
HD> i.e. which one is /dev/sda and wich one /dev/sdb?

You unluckily did not answer this point.

A new quirk from Rasbian Stretch (necessarry for a Pi 3B+)
is that you cannot read a 100 % backup of a SDCard of the same size.
At older versions of Raspbian (Jessie/Wheezy) this was no problem.
First I thought my SDcards or reader/writer were defective.
Then I remarked I could read a 16 GB backup SDcard when booted from
a 32 GB Raspbian Stretch OS. The other way round also worked.
But I can not read a 100 % 16 GB backup when booted at a 16 GB card.
The same happens to a 100 % 32 GB backup when booted from a 32 GB card.

In the SDcard copier util I saw a new option to tick: New Partition UUIDs.
I think that will make them different from the original booted from.
Yes, I found the answer today in the help text from the SDcard Copier util:

Under Raspbian Stretch and later versions, you cannot mount two partitions with
the same UUID, so you will not be able to mount a cloned SD card when booted
from the disk from which it was cloned. If you need to do this, check the "New
Partition UUIDs" box before copying.

Ok, that's a change we have to live with since 14 March 2018 at the Pi 3B+.
I luckily now know a solution around this new quirk.
I helps a lot if you have at least two USB2 (micro-)SDcard readers/writers.

Even if you donot have a Pi 3B+ but older ones, it is still a good idea to
start using the last raspbian Linux, in this case Stretch over Jessie or
Wheezy.

But I still do not understand why the resulting copy has a lower free space?
That's why I am using now the dd Linux command as showed above,
which results of course in the same UUIDs, but with the same free space.
So good luck in copying.

Greetings van Henri.


Re: Create NDIF disk image from RasPi SD.
On 24/07/2018 02:59, Henri Derksen wrote:
Quoted text here. Click to load it

What do you mean by 'read'? Do you mean using dd to /dev/null, or using  
dd to copy it on the card the machine has booted from? I'd be very  
surprised if the former failed, and you should not be doing the latter,  
as writing to a running image will cause it to be corrupted.

When copying between cards, it is common to find that no two of the same  
stated capacity have exactly the same number of sectors, and you can't  
perform an exact copy from a larger card to a smaller one.

To get around this, after setting up an Pi image and allowing it to  
expand to the size of the card, I then remove it and use gparted to  
shrink the last partition by 1M. This means it can safely be copied to a  
slightly smaller card without losing anything vital. Note this may not  
be possible with the NOOBs card layout, which is why I avoid it.

---druck

Create NDIF disk image from RasPi SDcard.
Hello David,

Quoted text here. Click to load it

I mean mounting them.

Quoted text here. Click to load it

DR> What do you mean by 'read'?

I wanted to mount the new card after making a backup on it,
and that failed because it has the same UUID as the one booted from ;-(.

Quoted text here. Click to load it

No, not at all.

Quoted text here. Click to load it

No, I never did that.

Quoted text here. Click to load it

Yes, but i did not even tried that.
The new card is ok, and you can read it with dd,
but you can NOT mount it any more if it is a 100 % backup from a Raspbian  
Stretch Linux you booted from.
Does not matter you made that backup with the Raspbian util SDCard copier,
or using dd with one or two card reader/writers from the local /dev/mmcblk0.
With the SDcard copier you now have the option at Stretch to change the UUID  
during copying from the internal microSDcard you booted from, i.e. /dev/mmcblk0
to a card (same make, type and capacity) via a sdcard reader/writer at
the destination: /dev/sda or /dev/sdb etc.

Quoted text here. Click to load it

Yes I knew and never did.
With the Raspbian SDCard Copier from the Accessoires menu that is impossable.

Quoted text here. Click to load it

I know for many years.
That often happened when using different makes and/or types,
but I seldom do that.

Quoted text here. Click to load it

I know this tric and I sometimes did.
But nowadays I allways use the same make, type and capacity of microSDcards.

Quoted text here. Click to load it

I know. It has a different layout at the end of the card.

Quoted text here. Click to load it

Sorry, but I hate NOOBS for more reasons ;-(.
I never use it anymore, I only did:

pi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/sdb bs=1M

at a Pi3B booted from a Raspbian Jessie Linux,
but the card on /dev/sda is a Raspbian Stretch Linux.
When booting from that new Stretch Linux,
i.e. in the internal card slot: /dev/mmcblk0
I can NOT mount the backup in a cardreader made from that same Raspbian
Stretch Linux, because it has the same UUID as the booted one ;-(.
That's now the new problem there never was before Stretch, i.e. with Jessie,
and I donot understand why the Stretch developers made that differend UUID
decision?
So now my solution is to made two Stretch versions;
one at a 16 GB card, and one at a 32 GB card, so they are different.
Or mount the Stretch crad from a Pi booted from Jessie.

From the help of de SDcard copier util:

"
Under Raspbian Stretch and later versions, you cannot mount two partition,
with the same UUID, so you will not be able to mount a cloned SD card when
booted from the disk from which it was cloned. If you need to do this, check
the "New Partition UUIDs" box before copying.
"

When backing up using the Raspbian SDcard copier I remarked that the copy had  
less free space than the original card from the same make, type and capacity.
I donot understand why? I only use Kingston microSDcards.
When I made the backup using dd with two card readers/writers,
the free space is the same at both cards,
but then you have also the same UUID's,
wich is a conflict when you want to mount them.

Is there a utility to change the UUID on a backed up card afterwards?
Or is that only possible when making a backup with the Raspbian SDCard Copier?

Henri.


Re: Create NDIF disk image from RasPi SDcard.
Henri Derksen wrote:

Quoted text here. Click to load it

$apropos UUID
dbus-uuidgen (1)     - Utility to generate UUIDs
findfs (8)           - find a filesystem by label or UUID
swaplabel (8)        - print or change the label or UUID of a swap area
UUID (3pm)           - Perl extension for using UUID interfaces as defined
in e2fsprogs.
uuidd (8)            - UUID generation daemon
uuidgen (1)          - create a new UUID value








Create NDIF disk image from RasPi SDcard.
Hello David,

DR>>> When copying between cards, it is common to find that no two of the same
DR>>> stated capacity have exactly the same number of sectors, and you can't
DR>>> perform an exact copy from a larger card to a smaller one.

HD>> I know for many years.
HD>> That often happened when using different makes and/or types,
HD>> but I seldom do that.

DR>>> To get around this, after setting up an Pi image and allowing it to
DR>>> expand to the size of the card, I then remove it and use gparted to
DR>>> shrink the last partition by 1M. This means it can safely be copied to a
DR>>> slightly smaller card without losing anything vital.

HD>> I know this tric and I sometimes did.
HD>> But nowadays I allways use the same make, type and capacity of
HD>> microSDcards.

In my archives I found this:

7.969.177.600 Bytes at Kingston 8 GB C10
1st Fat partition is 56 MB
2nd EXT partition is rest
According  RISC OS !CloneDisc
7.421 GiB
15.564.800 sectors

Kingston 8 GB C4
Accordimg  RISC OS !CloneDisc
7.318 GiB
15.347.712 sectors

Intenso 8 GB C4
7.746.879.488 Bytes write
According  RISC OS !CloneDisc
7.214 GiB
15.130.624 sectors

So the differences are very much more then your mentioned 1 MB ;-(.
That's why I always buy the same make, type and capacity Kingston microSDcards
from the same supplier, a shop 10 km from here, and mostly four at a time. I
often need them to create an AIS+ECDIS system with Rasbian Linux + OpenCPN
ECDIS-viewer and most European governmental S57 inland waterway charts and
also the OpenSeaMap charts from .NL and .BE for the smaller canals, rivers
and lakes to support the skippers of old recreational ships, former cargo
ships and old tugs etc. See <http://www.lvbhb.nl .
The Netherlands has the largest collection of historical ships in the world.
At that website there is also a chapter about AIS+ECDIS.
I am one of the members of that club, and made OpenCPN ECDIS viewer work under
Raspbian Wheezy on the Pi 1B in december 2014 for our skippers.
I am now with version 5 of the 16 GB SDcard using Raspbian Stretch, and just
compiled OpenCPN 4.99.0 to it, together wich many inland ECDIS charts of
Western Europe. That S57 + OSM charts alone take 3.4 GB !
It consists of maps from The Netherlands, Belgium, France, Germany,
Switserland, Austria and Poland. But there are more if you wisch to install,
i.e. south east Europ countries.
I did not research if there are inland water charts for England, Norway and  
Sweden?
So there is very much to sail to and on.
Good luck in copying (micro-)SDcards.

Henri.


Re: Create NDIF disk image from RasPi SD.
On a sunny day (Tue, 24 Jul 2018 13:59:00 +1200) it happened
snipped-for-privacy@f1208.n80.z2.binkp.net (Henri Derksen) wrote in

Quoted text here. Click to load it
mm
You are copying on a raspi???
Using external card reader with 2 slots?  
What is your hardware setup?


I copy as folows.
power off raspi
take card out
put it in laptop
copy to /dev/zero (for demo card speed)
that is always about the same as what hdparm -t /dev/card says,
no matter what blocksize.
At least on my 8GB card.


If you copy to a file on harddisk, say some.img
then you also need to look at how fast that harddisk goes sector by sector.
USB card readers may do things to speed, especially old ones,
if you saturate the USB2...?

Here some test on my laptop with ever increasing filesize,
card in /dev/sdc  hardisk partition is /dev/sda5  destination file q1
card is latest 32 GB Samsung cllass 10 or sometging that I mentioned in this thread before.
do not have more free space ATM, but no real differences:


~ # dd if=/dev/sdc of=/mnt/sda5/q1 count=1 bs=1M
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0687257 s, 15.3 MB/s

~ # dd if=/dev/sdc of=/mnt/sda5/q1 count20%48    
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.080926 s, 13.0 MB/s


~ # dd if=/dev/sdc of=/mnt/sda5/q1 count10% bs=1M
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.622834 s, 16.8 MB/s

~ # dd if=/dev/sdc of=/mnt/sda5/q1 count20%480    
20480+0 records in
20480+0 records out
10485760 bytes (10 MB) copied, 0.6317 s, 16.6 MB/s


~ # dd if=/dev/sdc of=/mnt/sda5/q1 count10%0 bs=1M
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.85355 s, 15.3 MB/s

~ # dd if=/dev/sdc of=/mnt/sda5/q1 count20%4800    
204800+0 records in
204800+0 records out
104857600 bytes (105 MB) copied, 6.79278 s, 15.4 MB/s
dd if=/dev/sdc of=/mnt/sda5/q1 count10%00 bs=1M  0.01s user 5.44s system 6% cpu 1:18.03 total


~ # dd if=/dev/sdc of=/mnt/sda5/q1 count20%48000    
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB) copied, 68.2988 s, 15.4 MB/s
dd if=/dev/sdc of=/mnt/sda5/q1 count20%48000  0.58s user 9.93s system 15% cpu 1:08.57 total

~ # dd if=/dev/sdc of=/mnt/sda5/q1 count10%00 bs=1M
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 69.051 s, 15.2 MB/s


The card read speed:
# hdparm -t /dev/sdc

/dev/sdc:
 Timing buffered disk reads:  46 MB in  3.10 seconds =  14.82 MB/sec
panteltje10: /mnt/sda5 # hdparm -t /dev/sdc

/dev/sdc:
 Timing buffered disk reads:  46 MB in  3.10 seconds =  14.83 MB/sec


The harddisk read speed:
# hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 338 MB in  3.00 seconds = 112.64 MB/sec
panteltje10: /mnt/sda5 # hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 370 MB in  3.01 seconds = 123.02 MB/sec

So, I do not know what all your problems are, but this is get back to basics...

Of course you should not do the copy on a rapi itself, it is not a real computah ;-) ;-) ;-)

And it is just data, sector by sector, does not matter what is on it.






Create NDIF disk image from RasPi SDcard.
Hello Jan,

JP> You are copying on a raspi???

Yes, why not, I do not have another Linux machine.

JP> Using external card reader with 2 slots?

External card reader yes, but NOT using the 2 slots at the same time.
The source is the internal SDcard slot: /dev/mmcblk0
The destination is /dev/sda

JP> What is your hardware setup?

Raspberry Pi, 1B, 1B+, 2B, 3B and 3B+, Logitech wireless keyboard and mouse
using a 2.4 GHz USB2A dongle, DVI-D monitor, and one or two  
SDcardReaders/writers booted from the latest Raspbian Stretch 2018-06-27
and of course updated and upgraded at 2018-07-22.

JP> I copy as folows.
JP> power off raspi
JP> take card out
JP> put it in laptop

I donot have a laptop with Unix/Linux. but only a Win2K one,
or a desktop with Win XP I hardly use.
But I can boot from a Lubuntu 14.04 CD.

JP> copy to /dev/zero (for demo card speed)

Why should I do that?
It only takes my valuable time ;-(.

JP> that is always about the same as what hdparm -t /dev/card says,

Raspbian does not know that command: hdparm.

JP> no matter what blocksize.
JP> At least on my 8GB card.

You have Raspbian Stretch Linux from 2018?

JP> If you copy to a file on harddisk, say some.img
JP> then you also need to look at how fast that harddisk goes sector by
JP> sector.
JP> USB card readers may do things to speed, especially old ones,
JP> if you saturate the USB2...?

I see.

JP> Here some test on my laptop with ever increasing filesize,
JP> card in /dev/sdc  hardisk partition is /dev/sda5  destination file q1
JP> card is latest 32 GB Samsung cllass 10 or sometging that I mentioned in
JP> this thread before.
JP> do not have more free space ATM, but no real differences:

My problem is the new Raspbian Stretch UUIDs !

JP> So, I do not know what all your problems are, but this is get back to
JP> basics...

That only happens when I am backing up between two differend speeds SDCards.
I mostly use the same make, type, speed and capacity cards.

JP> Of course you should not do the copy on a rapi itself, it is not a real
JP> computah ;-) ;-) ;-)

Besides my old M$ DOS BBS PC and Acorn RiscPC at RISC OS 4.04, I do not have  
anything else than the menioned 5 Pi's, Win2K laptop and a Win XP desktop.

JP> And it is just data, sector by sector, does not matter what is on it.

For copying yes, but for mounting, it makes a huge difference.
You can not mount SDcards anymore with the same UUID as the booted one
since Raspbian Stretch Linux, witch is a quirc from March 2018.
And I still do not know why they made that impossable.
With the older Raspbian Jessie or Wheezy mounting a 100 % backup card
was no problem at all.

Henri.


Re: Create NDIF disk image from RasPi SDcard.
On a sunny day (Mon, 30 Jul 2018 20:07:00 +1200) it happened
snipped-for-privacy@f1208.n80.z2.binkp.net (Henri Derksen) wrote in

Quoted text here. Click to load it


OK, to make a long story short:

  Do Not Do That

When you make copy from a running system to an other card,
then the source changes while the copy is being made.

Links may be changed, log files may be appended, or created,  
data may be changed.

So the new copy will likely not run, and even if it does it WILL contain
errors and inconsistencies.

The right way is stop the raspi (type poweroff in a terminal perhaps) and do the copy
on an other system.





Create NDIF disk image from RasPi SDcard.
Hello Jan,

JP>>> Using external card reader with 2 slots?

HD>> External card reader yes, but NOT using the 2 slots at the same time.
HD>> The source is the internal SDcard slot: /dev/mmcblk0
HD>> The destination is /dev/sda

JP> OK, to make a long story short:
JP> Do Not Do That

JP> When you make copy from a running system to an other card,
JP> then the source changes while the copy is being made.

Yes I know, but in this case, it does not matter.

JP> Links may be changed, log files may be appended, or created,
JP> data may be changed.

If you use the machine intensively, yes.

JP> So the new copy will likely not run, and even if it does it WILL contain
JP> errors and inconsistencies.

Not if you donot use the machine during copying.

JP> The right way is stop the raspi (type poweroff in a terminal perhaps)
JP> and do the copy on an other system.

Last time I was on a ship with a Pi 2B, only one cardreader and nothing else.
There were no other machines.
He was running a corrupted backup from a card his nephew made for him.
Some people donot shut down the Pi after use ;-(.
I only had one new fresh filled card with me,
and he had another card (same type) I made for him in march 2018.
So I put my new card in the internal slot, and his card from March 2018
in the cardreader and we made the backup with the internal SDcard copier  
program. This way we always had a working card (from his nephew) if things went  
wrong. During the backup we did nothing with the machine and it took 90 minutes  
to complete the backup. A long time compared to the 28 minutes I got at my Pi  
3B+ with two cardreaders and dd with 1M blocks.
I did not expect a three times longer backup time at a Pi 2B versus a 3B+
But afterwards the new copy worked much better than the crappy backup did.

When sailing and traveling with my foldable bike,
I do not have enough room to transport a large amount of computer hardware.
So next time I have to take 2 cardreaders with me, and leave that clean  
underware out of my case ;-).
I did not use it anyway this time.
But if I take two cardreaders with me, you should see I only need one,
and because of an accident (bad meal), I do not have clean underware.
So it is always trouble to pack the right things when traveling ;-).

Henri.


Re: Create NDIF disk image from RasPi SDcard.
On a sunny day (Wed, 01 Aug 2018 09:53:00 +1200) it happened
snipped-for-privacy@f1208.n80.z2.binkp.net (Henri Derksen) wrote in

Quoted text here. Click to load it

Yes, that is complicated, you could go naked though, only wear a backpack.
You did say 'you do not use machine during copying'
Linux is very much alive, it is doing things all the time, type
ps avx
to see some processes.
or type
top

That sort of fiddling is not a chance I want to take,
but maybe for an emergency copy..
The other problem you then have is that you cannot run a verify.
diff myimage /dev/sd?
I wrote a special program for that to compare dvd and bluray images, but it will compare anything:
 http://panteltje.com/panteltje/dvd/dvdimagecmp-0.3.tgz
dvdimagecmp -a /dev/sd? -b myimage.iso
should work
Every disk burn here is compared with it, it gives relative speed compared to CDs and reports sectors that differ too.

Anyways, for 'on the road' (or in a train) I have a small Asus eeepc that runs Linux
with a Huawei 3G USB stick to connect to the net, takes hardly any space, and has a real keyboard.
Usually I just ssh -Y to my home LAN with it, and nobody could tell I was not there.

Greetings
Jan


Create NDIF disk image from RasPi SDcard.
Hello Jan,

Skipped a lot of quotes.

HD>> When sailing and traveling with my foldable bike,
HD>> I do not have enough room to transport a large amount of computer  
HD> hardware.
HD>> So next time I have to take 2 cardreaders with me, and leave that clean
HD>> underware out of my case ;-).
HD>> I did not use it anyway this time.
HD>> But if I take two cardreaders with me, you should see I only need one,
HD>> and because of an accident (bad meal), I do not have clean underware.
HD>> So it is always trouble to pack the right things when traveling ;-).

JP> Yes, that is complicated, you could go naked though, only wear a
JP> backpack.

No do it the other way round, wear the long and tick heavy clothes you need
local, and pack just the small parts to keep it compact on bike/train.
Ok, it's hot when traveling, but then you can carry as much as possible.

JP> You did say 'you do not use machine during copying'
JP> Linux is very much alive, it is doing things all the time, type
JP> ps avx
JP> to see some processes.
JP> or type
JP> top

I knew there were, but not the amount of it.

JP> That sort of fiddling is not a chance I want to take,
JP> but maybe for an emergency copy..

Yes, that was the case here.

JP> The other problem you then have is that you cannot run a verify.
JP> diff myimage /dev/sd?

Until now I never did.
I just tried if the copy works well.
But indeed shut down the source card, and making a backup of it when it not  
runs is the best procedure.

JP> I wrote a special program for that to compare dvd and bluray images, but
JP> it will compare anything:
JP> http://panteltje.com/panteltje/dvd/dvdimagecmp-0.3.tgz
JP> dvdimagecmp -a /dev/sd? -b myimage.iso
JP> should work

Some time I will have a look at it, thanks.

JP> Every disk burn here is compared with it, it gives relative speed
JP> compared to CDs and reports sectors that differ too.

For CD's that comparision is really necessarry,
as CD's are not as reliable as magnetic storage is.

JP> Anyways, for 'on the road' (or in a train) I have a small Asus eeepc
JP> that runs Linux with a Huawei 3G USB stick to connect to the net,
JP> takes hardly any space, and has a real keyboard.

That real keyboard is a must I think.
But the EEEPC has little memory and small storage.

JP> Usually I just ssh -Y to my home LAN with it,

Some Dutch friend in FidoNet used a Raspberry Pi 3B in his backpack running on  
a local powerbank using WiFi and 3G.
His very small portable machine was connected to it via his own WiFi.
Nice to see at the HCC computerclub in De Bilt last year.
He has internet connection to his home and nobody sees a 3G dongle ;-).

JP> and nobody could tell I was not there.

Ofcourse one can say that. Your mobile connection was via a GSM mast.
You are watched by camera's everywhere today, and the police can find you via  
your mobile phone and/or cash dispencers you used, or digital payments with  
your PIN card.
So donot tell the phrase you did above, it simply is not true ;-).
And second your connection has two weak links, the 3G provider,
and your local ISP at home, i.e. ADSL or Fibre.

When the Boiler at home has a leakage, the RCD could shut down and your home  
router and internetconnection also.
Luckily I do not have such a boiler, but many years ago I had a timerswitch
which activated my RCD ones a day, until I found out the source of the problem.
So modern technology is not allways working well, donot rely to much on it.

Henri.


Site Timeline