Pi5 M.2 HAT

Oct 28, 2024 Last reply: 1 year ago 12 Replies

official RPi NVMe HAT available



I notice Jeff Geerling damned it with faint praise by saying the rubber bumper is the best official RPi product of the year ...


256GB NVMe – 40k IOPS (4kB random reads)
> 70k IOPS (4kB random writes)
> 512GB NVMe – 50k IOPS (4kB random reads)
> 90k IOPS (4kB random writes)

read and write numbers transposed?


The official NVMe Pi Hat has been out for months, I guess I should get one, or maybe an alternative. I just bought a NVMe USB enclosure which has appalling performance, way worse the USB/SATA.

Does the rPi5 use U-Boot for NVMe. I've had a lot of problems with U-Boot not recognising NVMe drives on a Orange Pi. The latest Armbian Boot loader is working, I need to have a look to see if they are using U-Boot, I'm pretty sure they were in the past.

Oh, I don't have a Pi5, and though I kept hearing about 3rd party NVMe HATs and lack of official one

Anyway, is it likely the write speeds are faster than the read speeds? I know some enterprise SSDs come in "read mostly" or "write mostly" flavours, but for a Pi?

I can *envisage* a way in which writes are cached in DRAM pending a big enough chunk to actually erase and write a whole block of data, whilst reads are happening more or less without caching.

I prefer the NVMe base boards rather than the hats to keep the top side clear for fans and GPIO cards.

I've got the Pimoroni NVNe base on one (with the pibow coupe 5 case and official cooler - make sure you get the long PCie cable), the other is in the Argon NVMe case (from PiHat) which has an NVMe base board and cooler built in. The performance of both boards is identical with the same drive.

Note that the performance of NVMe drives can vary considerably though.

I had no problems with either of mine recognising the NVMe. I was using USB SSDs with each of them, so I booted from a temporary SD card, cloned the SSD on to the NVMe, and then rebooted with just the NVMe present.

---druck

OK, I see there is a story about rPi launching actual NVMe M.2 SSDs. As opposed to a hat. I've no idea why they would do that. The obvious suspicion is cashing in on a brand name.

formatting link

It's hard to know what is going on with the Raspberry Pi guys, the RK3588 devices are clearly faster, lower energy, albeit with s*it software support. Who knows what will happen with the next generation Arm SoCs. I guess maybe Raspberry Pi have a clue, and hence decided to monetise the brand now, before a new product wipes the floor with them.

Dunno, IOPS doesn't mean a lot to me. As TNP says, maybe a write operation is to cache, and a read is from main memory.

On many solid state persistence devices you see a very fast initial write (presumably to cache) before quickly settling down to a much lower rate for big files.

It may be they are doing it because supply of small-capacity 2230 NVMe is a bit of a minefield. eg I checked scan.co.uk and the smallest 2230 they have is 512GB. There are some 256GB sold by Amazon.co.uk which are more expensive (and a few more dispatched by other sellers, of variable trustworthiness). At least with the RPi brand you know they're compatible, and they seem to be decent value.

It'll depend on what fab slots they can get. Not everyone can fab on the latest process, especially with a budget. Also how much cache they can afford to put on the die.

RK3588: (64+64+512)*4+(32+32+128)*4+3072 = 6400KiB RPi5 : 512*4+2048 = 4096KiB

RK3588 also has 4 extra A55 cores which RPi doesn't have, but is more expensive ($100+ for the Banana Pi).

Any decent benchmarking tool should get past the cache to exercise the real storage.

I think they've got them around the wrong way. Their ODM Biwin's 2230 has more read than write IOPS:

formatting link
Theo

That is a useful factoid. Thanks

On 30/10/2024 22:07, Pancho wrote: peration is to cache, and a read is from main memory.

And on any linux system with plenty of RAM in cache.

I only have a 100MBPS LAN currently, and file transfers appear to be blazingly fast for the first few gigabytes and then appear to stall :-)

I think that alone is why they are selling SD cards and power supplies.

My local PI hut also sells non RPI USB to SATA cables, because they are tested and 'known to work'.

I think for a hobbyist, having dependable 'known to work' peripherals is worth a little extra cash.

I have three 2242 NVMe, they work fine, apart from some versions of U-Boot boot loader (They actually worked in older versions, then stopped working). A couple of those are 256GB from a couple of years ago, due to the price low differential I would buy 512GB now.

I'm thinking of getting a M.2 NVMe adapter for my rPI5, I'll probably get a Pimoroni one, because it take standard 2280 drives. Best to go with the flow.

It's only costs a little more, but the RK3588 is now two years old. For me it it was the first PC quality Pi type device. I think the next generation of Pi type devices may be genuine mass consumer products. The rPi niche is disappearing, rPi will either become much bigger or much smaller. Saying others have a manufacturing advantage is supporting the idea rPi might have problems in future.

FWIW I'm not a traditional Linux/Unix person. When MS Win NT provided a proper OS, I dropped Unix and was a MS Fanboi for 25 years. I looked at Linux occasionally, but not seriously. Now I think Arm/Linux is about to become a serious mass market desktop PC.

MS seem to be a little nervous too.

These are marketing benchmarks, about as reliable as VW diesel vehicle emission tests. i.e. They could do it properly, but they don't want to.

Yeahbut...

formatting link
650K IOPS Max. Random Read 4K 800K IOPS Max. Random Write 4K

But as I said, I don't really understand what IOPS means. The same device quotes a faster Max Read than Max Write (presumably sustained read/write).

I'm not saying the Raspberry Pi NVMe spec is right, just that it is good manners to have a bit more evidence before accusing it of being wrong.

Neither was I. I was micro only, except cross compiling on DEC unix on two jobs. I then get exposed to unix making it talk over modems. It made a great server. Especially on PCs - SCO unix was massive value for money, but a desktop? Dont make me laugh. You needed a sun SPARC at the cheapest to do that. But a PC with windows was a quarter the price or less. Staff kept saying that linux was as good as unix, and it was, so it went into our servers, but there was no real GUI desktop environment to match windows till the early noughties. I was never a Fanboi - having to support it showed what a POS it was compared with Unix/Linux but what else was there?

So when I retired I tried the desktop - around 2005 I think. It was OK but I still needed a winPC for some apps, Then I messed with virtual box and installed my old windows PC in that and havent looked back.

I think my first PC was around 1985. So twenty years on DOS.Windows desktops

And now getting on for twenty on Linux. Wow!

Agreed, if you don't need the small size then I'd go 2280 - plenty more to choose from.

4K random read test: If the disc contains N blocks of size 4K Repeat:

- roll a dice between 0 and N to give you D

- read 4K block number D from the disc

- throw away the data that came back IOPS = how many times you can do that in a second

IOPS is a function of how well the flash and controller can manage an unpredictable workload. It's also a function of transfer speed to some extent - you still need to move the data. That 800K IOPS is 3.2GB/s or

26.2Gbps. The Pi's single PCIe lane is only officially rated at Gen2 or 5Gbps, which puts a hard limit of 4K IOPS of 150K (and PCIe transfers have additional overhead on top of that)

A more interesting graph in reviews is 'performance consistency': if you give it a sustained write workload like this, after many minutes eventually the speed falls as all the on-drive buffers fill up. How well engineered it is will show whether it can hold its performance, degrade gradually, hop around erratically, or fall off a cliff.

Theo

Join the Discussion

Have something to add? Share your thoughts — no account required.

Didn't find your answer?

Ask the community — no account required