Verifying SD Cards

On the Pi, have a look around /sys/class/mmc_host/ or /sys/block/mmcblk or a similar location. This gives you parameters from the card itself - here's a list:

On a fake, these details are often bogus - for example a serial number of '0' or '99' is suspect, while a serial of '24592839DFSDFJSD' is more plausible (google it. If someone else has the same serial number, something is wrong)

This only works if you have a directly-connected MMC/SDHCI controller - not if the card is attached by USB.

is the canonical reference for this kind of fishy-ness.

Theo

On Wed, 04 May 2016 02:24:33 -0700, mike Gave us:

There are no McCoy brand memory sticks.

So if you see one you know it's been doctored.

--
Steve O'Hara-Smith                          |   Directable Mirror Arrays 
C:>WIN                                      | A better way to focus the sun 
The computer obeys and wins.                |    licences available see 
You lose and Bill collects.                 |    http://www.sohara.org/

Seems worse than that, the Raspberry Pi is not SDXC/EXFAT compatible. SDXC means sizes larger than 32 GB. Those devices are formatted as exFAT and so won't boot from that partition. But they can be booted by creating more than one partition with the first being FAT32. I think the other partition can be exFAT.

I don't see anything about a speed limitation. In fact, UHS-1 has a speed of 13 MB/s and speeds higher than this have already been measured on the rPi. So what about the rPi would limit UHS-1 performance?

"UHS-I has a bus interface speed of up to 104 MB/s."

I don't get why they keep increasing the size capabilities of the SD cards so incrementally. SDHC only covered 4 GB to 32 GB expanding the size 16 fold. SDXC expands the range another 64 fold. Why so little? Memory capacity will continue to ramp up doubling every two years, so why not give the standard some legs? SDXC will need to be replaced in another 8 to 10 years.

--

Rick C

Thanks. When I get my 64 GB card booted on the pi, I'll check the internal info.

--

Rick C

That last sentence is the point. I don't know if any of these cards is the "real McCoy". In researching this I found reports of getting fake cards from mainstream brick and mortar retailers. So who *can* you trust?

I'm more concerned with day to day operations than backups. Backups will likely be done over the network anyway and so will be limited by those speeds.

--

Rick C

Could be it's dead, Jim.

--
Bah, and indeed, Humbug

I'm not up on the pi with this card yet. Turns out 64 GB has crossed a line and I'll have to do special formatting to use it.

--

Rick C

On Wed, 4 May 2016 14:40:05 +0100, Ahem A Rivet's Shot Gave us:

Bob's yer uncle.

On Wed, 04 May 2016 15:14:39 +0100, "Kerr Mudd-John" Gave us:

I'm a doctor, Jim, not a memory card maker!

"Brain, brain.... what is brain?"

SDXC is a card type. You have a different initialisation procedure for SD or SDHC or SDXC. The Pi supports SDXC.

exFAT is a disc format. If you're using Debian you need to install exfat-fuse. Or you can reformat to something more sensible.

The Pi boot partition has to be FAT. exFAT is not FAT. Supporting exFAT would require a) a more complex format to be implemented in the relatively small mask ROM on the SoC, b) making that implementation bomb-proof in the face of all the strange cards out there and c) paying a patent licence to Microsoft for every Pi shipped. The simpler and cheaper option is not to use exFAT.

You can set the emmc_clock in config.txt - it's usually 100MHz. I think that will be the upper limit on speed, but I don't remember all the modes that are listed in the spec (there is a 208 MHz mode in there I think, which is where 104MB/s comes from). Since I haven't seen the Arasan IP core documentation I don't know what the hardware actually supports.

This irritates me too. Lack of support can reduce the longevity of existing devices (it's now getting trickier to buy 2GB cards to work on old non-SDHC devices, for instance).

The initialisation process is 'just software' so it's possible to upgrade an existing device to support SDHC/SDXC/etc, but in many things (cameras, USB readers etc) the software doesn't get upgraded.

Theo (who has a vendor-supplied SD IP core which does initialisation in hardware, which makes supporting SDHC much more painful)

Hmmm... you've said the Pi boot partition has to be "FAT" which is what I said. You've said twice to not use exFAT, but you don't say what

*can* be used. If FAT32 were to be used for all partitions on an SDXC card, it would require four partitions for a 128 GB card. That pretty well sucks. Even needing two partitions sucks. Would NTFS work?

I misread the quote I provided. I thought it was bits rather than bytes. So that's not a limitation I think.

I assume the SDHC cards topped out at 32 GB because they used FAT32 which also tops out at 32 GB. Rather than switch the format used, they just worked within that limit and postponed the bigger change for SDXC cards until later.

--

Rick C

I'm a bit confused. I just looked it up and FAT32 has an upper limit of

2 TB! So why can't FAT32 be used for one partition holding all of a 64 or 128 GB SD card?

--

Rick C

I did a little digging and there is some sort of functional limit to FAT32 at 32 GB but it seems to mostly be Microsoft's laziness by not supporting the creation of larger partitions with their built in tools, or maybe just some of them. I found a utility that let me format the 64 GB SDXC card to FAT32.

So will I be able to boot a raspberry Pi from this SDXC card this way?

--

Rick C

Isn't the FAT32 limit due to address space limitations? If you want bigger partitions, you need to have bigger allocation units. Wastes a lot of space if you have many small files.

You need to format the boot partition with FAT (16 or 32). You can make additional partition(s) in whatever format you like. The standard Raspbian install has a small boot partition in FAT and the rest of the SD card is ext3 or ext4. This model is fairly common in PC Linux installs - a basic ext2/ext3 partition for booting, and then whatever crazy RAID/LVM/ZFS/encrypted partition setup you want for the rest of the disc. It means you don't have to teach the bootloader how to understand your encrypted RAID and prompt for your smartcard or whatever it might need, you can do that once the kernel is running. If you want to make additional data, rather than OS, partitions, NTFS or exFAT would be fine. You'd have to install the necessary packages to read them, though (ntfs-3g and exfat-fuse).

I think I may have got it slightly wrong.

Between SD and SDHC is a command-level difference. It isn't just a question of formatting, the firmware responds differently. This is the flowchart:

- the left branch is for SD, the right branch for SDHC (where you find out capacity at the end) Between SDHC and SDXC I can't see any protocol differences (not that I've actually interacted with an SDXC at the command level, hence my confusion). So I think it's marketing that out of the box SDHC is formatted FAT32 and SDXC is formatted exFAT. Underneath they speak the same protocol and there's nothing to stop you reformatting as one or the other.

(One slight wrinkle is that the wear levelling may be optimised for the FAT32 or exFAT format - but that goes out the window when you're trying to use the card for an OS anyway).

You could be right there. Windows not supporting formatting FAT32 > 32GB might be the kind of marketing-inspired reason for the name change, rather than any difference in substance of the actual hardware.

Theo

Use the Windows 98 format utility. XP did not support more than 32G on FAT32.

That's not a limit. That's a tradeoff. I suppose if you had millions of small files it would waste a lot of space. Up to that point it's a macht nichts or mox nix, your choice.

The point is there is nothing to prevent the use of a single bootable partition on SDXC cards.

I really don't like partitioning mass storage into multiple smaller partitions.

--

Rick C

FAT32 has a 2**32 sector limitation, and a 2**28 cluster limitation. At 2TB, you'd need 8KB clusters.

The 2**32 sector limitation is trivial, and exists in a single 32-bit field in the boot sector (and it's backup), should someone patch* around that, you could then have 2**28 clusters of 32 or 64KB (most stuff will support the former, a lot of stuff will break with the latter), or 8/16 TB.

The 28-bit cluster count field is fairly ingrained in FAT32, but probably would not be hard to tweak to about 30** bits (the reserved bits are not well used). Combined with the sector count fix, you could have 32 or 64TB volumes.

Of course, either change would require everyone to agree to use it if the portability of FAT32 volumes is actually of interest. The other question is whether you'd actually want to. FAT is a pretty bad format for modern sized storage devices - random accesses are painful, you're limited to 4GB files, the directories are a mess if you use long filenames, and it's not particularly robust. Its one saving grace is universal support.

*Presumably you could use the same technique that's use to support the *32* bit sector count - if the 16-bit sector count field is zero, use the (newer) 32-bit field. If that's zero, use a yet bigger field. **More is possible, 31 should also be fairly easy, going to (most) of a 32-bit number should be possible too, but would require a more serious change to FAT management logic.