Pi2 bootcode.bin puzzle

Have tried another known good USB cable?

No idea, but one problem I've had in the past is that USB devices can take a while to initialise, and it varies quite a bit between devices. This could be a reason why boots will sometimes fail - especially with eg spinning HDD.

Often it was necessary to add an explicit delay so that system didn't try to read them before they were ready. I think on Linux the kernel cmdline options were 'rootwait' and 'rootdelay' but I don't know if bootcode.bin has any kind of setting like that.

Theo

I've tried with and without the presence of a timeout file on the fat32 partition of the microSD, far as I can tell it makes no difference once things start going wrong. I do use it out of habit.

Does bootcode.bin pay any attention to config.txt? I thought it did, but am not sure it has any effect on USB mass storage boot. The docs say don't use any extra files on the fat32 partition.

Thanks for writing,

bob prohaska

I've even tried different usb-sata bridges. One in particular with an ASMT chipset that seemed to work everywhere else I tried it fails to boot. Plugged into a running Pi, or any other host, the disk seems to work fine with whatever bridge I try.

Probably this is just a limitation of the Pi2 v1.1 USB system. I've got another Pi2v1.1 with an old PATA disk roughly as old as the Pi2, that combo seems to work fine.

It just crossed my mind that I have another USB hard disk nearly as old as the Pi2. Maybe that could serve as a sanity check. Oops, guess not. It's a Western Digital MyBook, a NAS device. Rats!

Thanks for writing!

bob prohaska

I believe that config.txt is consumed by the GPU-side firmware, which is start*.elf. bootcode.bin is just enough to be able to load start.elf and config.txt from storage. I'd guess that if that storage is USB, it's too early to have read config.txt.

Theo

I didn't try booting from USB drives until the Pi 3B, but it wasn't until the Pi 4B with it's USB3 ports did that become really worthwhile to use SSDs.

It may still work on the Pi 2B, but being slower you may need to increase the value of rootwait in cmdline.txt - particularly if it is a spinning disc rather than an SSD - it could take 60 seconds before the drive is readable.

---druck

I think the problem here is that start.elf, config.txt, the kernel and cmdline.txt are all on the USB device - you can't put a timeout in cmdline.txt because by that point you already need the USB device up to read that file.

The alternative approach would be to have a regular bootcode.bin, start.elf, kernel, etc on the SD card and then tell the kernel to find its rootfs on a USB drive, at which point rootwait may help. (I think Bob is using FreeBSD but there is probably an equivalent option there). Or to interpose u-boot for the 'kernel' on the SD card, and then tell u-boot to find the kernel on USB.

Theo

That's the only way it works on older devices.

---druck

It looks like the path of least resistance is to let FreeBSD boot from microSD, mount that as root and then mount the USB disk as /usr, with /var/, /tmp/ and swap mounted from the USB device, either as links to /usr or maybe as hard partitions.

That worked fine in the very early days of FreeBSD on the Pi2. There isn't much write activity to the root filesystem in a "production" environment, only during OS upgrades. If the microSD is sufficiently oversized it'll last long enough.

I'd hoped bootcode.bin could be upgraded/tuned to boot from a USB3 disk on a Pi2, but apparently not.

Thanks to everyone for writing!

bob prohaska

Perhaps I'm coming here too late, but the way I did it was to keep the whole of the /boot directory on the SD card and put the whole of / (minus /boot contents) onto the USB SSD. Change the PARTUUID parameter in cmdline.txt to the new SSD partuuid and of course update the SSD /etc/fstab to the new values. Worked fine for Pi2B and Pi2B+.

Supposedly you can make them USB boot by just providing a bootcode.bin on SD, then they pick up start.elf, config.txt etc from USB. I've not tried this, but it appears to work for some people.

(previously bootcode.bin didn't know anything about USB, but since Pi 3/4/5 can USB boot I believe that code has now made itself into bootcode.bin)

Theo

I've yet to figure out what the FreeBSD equivalent to cmdline.txt is. That might be the key to making your approach work.

Is this what fstab would look like?

/dev/da0s2a mounted on / /dev/mmcsd0s2a mounted on /boot /dev/mmcsd0s1 mounted on /boot/efi /dev/da0s2d mounted on /usr

That doesn't look impossible, but ..... Somehow it seems wrong to me, as it requires access to /boot "out of order". Maybe the loader can do it.

In the end, mounting the microsd as root and just hanging /usr off of it will relieve most of the limitations of flash storage. /var, /tmp and /home can be linked to /usr, swap is hardware anyway. This much I know how to do.

Early on there was much concern about "wearing out" flash storage. In practice, it's turned out to be a minor issue. Write delays can be a problem, but it really bites hard only on network traffic. Putting the most-written filesystems on a USB disk solves most of the problems. It would be nice to not need microSD at all once booted, but in practical terms that isn't hugely beneficial.

Thanks for writing!

bob prohaska

Fbsd 13.x on a pi4, booting off a usb disk:

# cat /boot/msdos//config.txt [all] arm_64bit=1 dtparam=audio=on,i2c_arm=on,spi=on dtoverlay=mmc dtoverlay=disable-bt device_tree_address=0x4000 kernel=u-boot.bin

At least, I assume that's the equivalent file. :-)

AIUI, config.txt sets up the Pi, cmdline.txt furnishes runtime arguments to the kernel when that kernel is linux. If this isn't true somebody please correct me!

Thanks for writing,

bob prohaska

It is

Yes. and the important bit is that cmdline.txt determines what file system is used to boot it *initially* e.g. console=serial0,115200 console=tty1 root=PARTUUID=778a9e44-02 \ rootfstype=ext4 fsck.repair=yes rootwait noswap=1

Note the root=command. On the USB drive you must also have the same ID in the /etc/fstab file...

PARTUUID=778a9e44-02 / ext4 defaults,noatime 0 1

So an SD card boot on PIOS can use a USB drive to finish the boot and mount the root filesystem.

As for BSD, there must be an equivalent file somewhere

Does it show the FreeBSD beastie text-mode screen?

Theo

Might it be possible to let the machine boot from microSD to the loader prompt, next re-load the kernel from the USB drive, then start the new kernel and finally mountroot from USB?

That seems too easy. I must be missing something.

Thanks for writing!

bob prohaska

Well, it's less easy than I thought, but it does seem to work.

The method is to interrupt boot just before the kernel starts. Next, unload the kernel, set the currdev to disk0s2a, set the root device to da0s2a, load the kernel again (from the new root device and boot.

Setting currdev and rootdev can be done in /boot/loader.conf on the microSD's FreeBSD, I've asked on the freebsd-arm mailing list where the unload and load commands should go.

There are still some problems with usb disk discovery failing, but this is a big improvement.

Thank you Theo!

bob prohaska

I've just looked at what's on mine:

root@kirk:/ # df /boot/msdos/ Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/msdosfs/MSDOSBOOT 51096 25084 26012 49% /boot/msdos

root@kirk:/ # ls /boot/msdos/ EFI bcm2710-rpi-3-b-plus.dtb fixup.dat fixup_db.dat start4db.elf LICENCE.broadcom bcm2710-rpi-3-b.dtb fixup4.dat fixup_x.dat start4x.elf README bcm2711-rpi-4-b.dtb fixup4cd.dat overlays start_cd.elf armstub8-gic.bin bootcode.bin fixup4db.dat start.elf start_db.elf armstub8.bin config.txt fixup4x.dat start4.elf start_x.elf bcm2710-rpi-2-b.dtb dtb fixup_cd.dat start4cd.elf u-boot.bin

So I seem to be running with no cmdline.txt. Boots happily off the attached usb drive, no sdcard.

But I stand corrected.