1. Home
  2. Raspberry Pi Group
  3. remotely loading a new SD image

remotely loading a new SD image

I have a Raspberry Pi located in a datacenter. I can reach it only via the network.

When I would want to change the entire SD card image, I could of course send a new card by snailmail and ask the datacenter people to put it in my Pi, but I was thinking about the following:

Would it not be possible to send a new image via the network and have the Pi put it on the card and reboot from it?

There could be a program that you first upload to the Pi, under the existing (and hopefully functional) installation, then run it and it will receive the (compressed) SD card image over a TCP connection. If not too big it could store it in memory first, to be extra secure against network loss during the upload.

Then it would have to unmount the disk partitions, uncompress and write the SD card, and reboot.

Of course the critical part is: would it be possible on a Pi with a single program running to forcibly unmount the root fs without the program crashing (e.g. because it uses demand paging)

I envision it would first stop all "unnecessary" programs to make room in memory and to close open files, turn off the swap, receive the compressed image and put it in RAM, do a check, then stop all remaining programs except itself, unmount the disks and then write the image (uncompressing as it goes) to /dev/mmcblk0.

After that, it should just do a forced reset e.g. using the watchdog.

That should work, shouldn't it?

A slightly more daring approach would be sending the image over the network and having the receiving program uncompress and write it on-the-fly. This of course means end of story when the network goes down during the upload, but that risk is tolerable. Of course it also means the network must remain functional during the phase the disk has been unmounted and the system is in a minimal mode. Not sure if that is possible. But it would mean a larger image can be loaded (which even when compressed is larger than 400MB or so that would be the limit when using RAM).

Is anyone aware of such a program already having been written?

Reply to
Rob
Loading thread data ...

...

RaspBMC does something like this. The initial download contains a cut-down kernel and a script which, on first boot, downloads and installs the full distribution.

--
Dave
Reply to
dave

You might be able to get out this hole but consider instead not getting into it in the first place.

The classic approach would be to have two partitions, a ?live? partition and a ?recovery? partition. Normally you boot to the former, if you boot to the recovery partition it provides an interface to overwrite the live partition. This doesn?t quite meet your spec as stated but might still solve the underlying problem.

Another possibility would be to do the whole job from an initramfs. / would be a ramdisk and nothing would have any dependency on the storage you?re trying to overwrite.

--
http://www.greenend.org.uk/rjk/
Reply to
Richard Kettlewell

Ok, I installed Raspbmc this weekend using the installer download but was not aware that it works that way. I'll look into the image to see how it works. I'm especially interested in how they flash the downloaded image over the installer (if that is happening at all).

Reply to
Rob

I got the Pi delivered with raspbian already on it and taking up the full SD card. To resize it, it needs to be unmounted. Not practical in this case.

The initramfs idea is interesting, it would be possible to reboot into an alternative system that gets the image from the network. A lot trickier than I envisioned, when something goes wrong there has to be some recovery possible. Maybe the first thing it could do is reset the cmdline.txt to a normal boot.

Reply to
Rob

There must be a backstory to your Raspberry Pi being in a datacentre.

Pray tell.

--
Graham. 

%Profound_observation%
Reply to
Graham.

What you describe is not far off how the new NOOBS software works. Basically you have an additional recovery partition with your SD card images on it in a compressed form, and then a boot process that can expand one of those onto the normal partition, and make it ready to use. It sounds like with some changes, you could prepare your new image, compress it and upload to the recovery partition. The trigger the recovery boot loader to do the setup for you on next reboot.

The NOOBS software is interactive to let the user select one of several OSs and also choose default settings. You would need something scripted that just recovered one selected image.

--
Cheers, 

John. 

/=================================================================\ 
|          Internode Ltd -  http://www.internode.co.uk            | 
|-----------------------------------------------------------------| 
|        John Rumm - john(at)internode(dot)co(dot)uk              | 
\=================================================================/
Reply to
John Rumm

formatting link

For early (first 2000) applicants it was free.

Reply to
Rob

Ok but what I need is something that can be remotely installed on an already running Raspbian system on a 16GB SD card. There is lots of space but it has all been consumed by the root partition. I read that it is possible to shrink the ext4 partition, but only when it is not mounted.

So it is a bit tricky to convert the system to such a setup. Maybe it is possible to make a tmpfs, copy some critical stuff there and mount it as the root fs so then the fs on the card can be unmounted.

All this is more than what I am looking for: just a way to change the image of the entire card in one or two steps.

When the Pi was delivered, of course the first thing I checked was the disk layout (one could order either 8GB or 16GB cards). They should have left the rootfs at 2 or 8 GB, in that case I would certainly have created another partition on the card.

Reply to
Rob

Yup its obviously going to be more complicated to get to the new arrangement from where you currently are. I think I would be inclined to work out a system that would do what you want going forward, and go for the snail mail update the first time!

I was under the impression that is what most of the current installation procedures do anyway... normally the option to expand the FS to the full size of the card is user selected after first boot?

--
Cheers, 

John. 

/=================================================================\ 
|          Internode Ltd -  http://www.internode.co.uk            | 
|-----------------------------------------------------------------| 
|        John Rumm - john(at)internode(dot)co(dot)uk              | 
\=================================================================/
Reply to
John Rumm

How much space is there in the FAT partition? You could use that as your root fs, or deploy a secondary initrd in there that you could reboot into.

For the record, RISC OS treats the SD as just a big floppy disc - it'll quite happily run once booted without any SD mounted at all. But that's probably not helpful given the position you're currently in.

What I'd be most worried about is things going wrong - is there a get-out-of-jail procedure if something breaks?

I suppose NFS root is out of the question? (Even, temporarily, NFS over the internet?)

Theo

Reply to
Theo Markettos

Well, it is mostly a "thought experiment", I have no immediate need to do it soon. I am a longtime (open)SuSE user and I am a bit disappointed by the level of the default configuration and the character-mode system management tools offered by Debian/Raspbian. So an option would be to switch from Raspbian to openSUSE, but only when this is doable.

I have seen that on other local experiments. I presume that they did it for me when they first booted the Pi to assign its fixed IP address, enable IPv6 and set a secure password. Probably they thought they would help the new user, and avoid service tickets with "why do I only have 2GB while I ordered 16GB".

Reply to
Rob

Well, in fact there are multiple Raspberries on the local network, two of them I could temporarily use (e.g. to park an image).

The get-out-of-jail procedure would be to send a newly loaded and configured SD card to the datacenter and ask them to replace it. The time to recovery would be unknown, maybe a couple of days.

It is possible to remount the rootfs readonly in Linux, but I am not completely certain if that is enough when I subsequently overwrite it and attempt to reboot. Probably it is, when my last action is to enable the watchdog and wait for it to trigger. (that does a hard-reset, not some reboot sequence that may try to look in files)

Reply to
Rob

In a mad moment - Rob mumbled : [snip]

Why not send a 'spare' card now, then only a 'phone call or email is needed to get you back on line.

--
|)    [ 
|)ryn [vans            mail to - BrynEvans@bryork.freeuk.com
Reply to
Bryn Evans

I can already do that when I foul things up and I want to fall back to the bare Raspbian install. Just call and they will juggle the card for a working one. I think it will cost me 10 euro, though.

Reply to
Rob

What exactly are you trying to do, i.e. are you wanting to replace every byte of content on the SD card or merely to upgrade some (arbitrarily complex) application or dataset?

If the former, I have no suggestions, but if the latter, have you considered setting up an APT code database somewhere that's under your control and that can be accessed by the RPi? Once that's done you can easily extend the RPi's configuration so that running apt-get pulls down any updates you've staged by adding them to your local codebase.

Alternatively, can you use sftp to do the job?

Why install an entire new image if you can use apt-get to upgrade almost any part of a running system? After all, it is designed to do exactly that: it can and will replace the kernel if its asked to.

--
martin@   | Martin Gregorie 
gregorie. | Essex, UK 
org       |
Reply to
Martin Gregorie

See other remarks: the objective is to replace Raspbian by openSUSE.

Because I don't like apt-get. YaST is so much more convenient.

When I want to install a new package, I first have to search the internet to find what that package is called. In YaST I can search the package database and tick packages I want to install in a fullscreen application, seeing what dependent packages will be loaded immediately. It will warn me when conflicts occur and allow me to resolve them by deleting other packages or selecting between options.

I found a program that probably will do sort of the same thing in debian (I forgot the name), but when I start it I get an initial screen but I have no idea what keys to press, and it does not act on obvious keys. Sometimes it starts doing something but not what I want it to do. It may be obvious to the regular user, but for me it is as unusable as emacs or gnu-info.

Reply to
Rob

OK. I thought that was part of a future wish-list of yours. Given that's what you want, IIWY I'd set up and test a new openSUSE card and apply any needed tweaks, i.e. make sure sshd is enabled and tested if you need remote access, before posting it to them with a nice note asking them to swap cards and reboot.

--
martin@   | Martin Gregorie 
gregorie. | Essex, UK 
org       |
Reply to
Martin Gregorie

In Debian-land, synaptic will do that. But it's an X program.

At the console there's aptitude (without any extra arguments, otherwise it operates like apt-get) or dselect. I agree the keystrokes are potentially confusing, but I think they're mostly usable with arrow keys.

There's also 'aptitude search ' which is a command line package search.

But TBH I find in-app searching to be quite limiting anyway, and Google combined with packages.debian.org to be more flexible (eg discover that a package is not available for your system because it used to be available but is no longer maintained)

Theo

Reply to
Theo Markettos

or apt-cache search

etc :-)

--
Ineptocracy 

(in-ep-toc?-ra-cy) ? a system of government where the least capable to  
lead are elected by the least capable of producing, and where the  
members of society least likely to sustain themselves or succeed, are  
rewarded with goods and services paid for by the confiscated wealth of a  
diminishing number of producers.
Reply to
The Natural Philosopher

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.