Been using a base installation of Debian on a 2GB IDE flash drive plugged into one of these:
formatting link
My question is about the swap partition on the flash drive. Flash eventually wears out. Surely a flash based swap partition will wear out that part of the flash a lot quicker than the flash used for the rest of the Linux installation. Is this considered a problem or just accepted as a "feature" of using a flash drive.
Can I run Linux without a swap partition or base the swap partition in RAM?
You can run Linux without swap (assuming you have enough ram, of course!). I've seen a tutorial on the web for setting up a swap partition on a ram disk, but I cannot fathom why that might be a useful idea (unless you have a system with some fast ram, and some very slow ram).
It is also quite possible to put swap on a flash drive. It will wear out eventually, but how much of an issue that is depends on the type of flash, your usage patterns, and what sort of wear levelling you are using (a swap file on a jffs2 or yaffs disk will get wear levelling over the whole disk, for example, while swap on a partition will have much less levelling).
If you system normally as enough memory for its needs without using swap, then it's likely that /tmp will cause more wear than the swap. One idea might be to include a swap file (there won't be many writes to it if it is not used), and mount /tmp on tmpfs. That way, your swap file will be used if needed for space for temporary files, but under normal usage small tmp files will not cause any writes at all.
It absolutely doesn't make sense to use a IDE flashdrive or a CF-flashdrive with a jffs2 filesystem. The internal electronics of such devices already *do* use the whole flash memory equally. You don't win anything when using another mechanism for 'equalized' writings on top of that. Jffs is intended for direct use, such as a microcontroller with direct attached flashram where the microcontroller is responible for the complete access.
For the rest, I totally agree: even with 512 megs you don't really need swap-space any more, at least if you are running the same software and tasks again and again.
I would love to actually get a REAL and AUTHORITATIVE answer about the CF wear leveling issue. I haven't been able to get the semiconductor manufacturers to release details about internal wear leveling, and the information on the net is typically sparse, often incorrect, and usually from suspect sources.
I work in embedded systems and have been trying for two years to get CF card manufacturers to clarify (on the record) whether they do wear leveling, and if so, does their guarantee of 100,000 writes apply to a single hardware bit or are they taking wear leveling into account when they make those guarantees.
As for flash options, the CF card is still the best to use simply because it has a full IDE compatible parallel interface rather than the serial interface of other type chips.
Reading through the thread I noticed a lot of guesswork and a total lack of understanding about how the pendrive actually works. Everyone treats it is a black box that has "wear leveling". So let me crack the box open to some degree.
Nearly two years ago I have spoken to a person that reverse engineered the behaviour of several chips used in pendrives. At the time that reverse engineering apparently covered most of the market. The details were quite lengthy but can be condensed to two words: Smartmedia format.
Smartmedia is a very simple format and can easily be studied. Afaik access to the specs only requires registration. Alternatively one can study drivers/usb/storage/alauda.c, which implements the format. I will summarize the important bits below, assuming the reader has a good understanding about what flash is.
All flash memory is split into zones of usually 1024(1) erase blocks. Within each zone, 1000 logical blocks are mapped to the 1024 physical blocks. That leaves 24 spare one. Normally the spare blocks are erased and contain no data. The 1000 mapped physical blocks contain the associated logical number somewhere in the OOB area.
Reading from smartmedia first requires reading all logical numbers from all physical blocks. With this information a simple map, essentially a
1024-entry array is created. Once the map is set up, logical number get translated to physical ones with a simple lookup.
Writing is slightly more complicated. First one of the spare blocks is selected. New data is written to the spare block. If only a partial block is written, the remainder has to be copied from the old block. After the new block is written, the old block is erased and becomes a spare.
If the chip loses power during a write, several physical block may contain the same logical number. In that case the ECC information is used to verify each of the blocks. If one of the block was not written completely, the other is used. If one block has ECC errors, the other is used. If both are correct, a random one is used.
That's all there is to smartmedia format. It is _very_ simple. It is also surprisingly efficient, although it does have some shortcomings. So let us look at the problems and how they interact with filesystems.
Write overhead
If a filesystem only writes a small amount of data, typically 512 or
4096 bytes, smartmedia has to erase and write a full block. Most flashes used in embedded systems has block sizes of 128KiB or so. Most flashes used for smartmedia have 16KiB. Writing 16KiB when the filesystem only requests writing 4KiB increases the wear 4x and reduces performance 4x.
Wear leveling
Wear leveling happens implicitly by picking a different physical block from the spares on each write. However, some blocks are never used. If a physical block is mapped to a logical block that never gets written, it is out of the rotation. Two seperate 1024-block areas have their internal wear leveling each, but nothing is spreading high wear from one area to another.
So if a theoretical filesystem would only ever write to the same logical block, smartmedia can spead the wear over 25 blocks or less. Less, if any physical blocks are bad and cannot get used (2).
More realistically, if Ext3 is having a very hot area - the journal - that area is not getting any wear leveling worth mentioning. Journaling filesystems on smartmedia are a bad idea. Most journals are bigger than a smartmedia area, which is usually 1000 * 16KiB. The round-robin access pattern of the journal already provides perfect wear leveling _within that area_. Smartmedia does not add anything.
Hidden caching
Some chip designers seem to have noticed the 4x write overhead and try to outsmart the filesystem. Their chips start writing the new block, but don't copy data from the old block just yet. Instead they wait for further write requests, hoping to write adjacent data to the same block.
If the power fails while the chip is waiting for further data, smarmedia format requires the unfinished block to get erased and its content to get discarded. What happens if the user just typed "sync" and yanks the pendrive as soon as the command returns is anyone's guess. How many people ever considered their pendrives to perform caching and require proper barriers?
FAT requirement
When I claimed there was nothing more to smartmedia, I was actually lying. Smartmedia has the odd requirement that only FAT is supported as a filesystem. In fact, the specifications describe FAT in great detail. I have already seen FTLs(3) that deliberately look into the FAT and pre-erase blocks if the corresponding files have been deleted from the FAT. Let's just hope they always detect non-FAT filesystems.
After all this, my recommendation for filesystems is to do several things:
a) Do wear leveling!
Smartmedia wear leveling is limited to within areas. Any cross-device wear leveling must be done by the filesystem. FAT does that fairly well. The Ext family doesn't.
b) Write aligned 16KiB blocks
Anything else will cause write overhead and kill both performance and later the complete device. Most hard disk filesystem do this anyway. Any effort to reduce head seek time and rotational delay will also help your pendrive.
(1) Some smaller chips have a single zone of 256 or 512 blocks instead. Also, the first block in the first zone gets special treatment, so the first zone effectively only has 1023 blocks. (2) Most manufacturers specify that their chips has the behaviour of several chips used in pendrives. At the time that
Maybe I should add that things have allegedly improved. Many devices today support both static and dynamic wear leveling. Dynamic wear leveling is what smartmedia does. It depends on the filesystem writing somewhere to move those blocks. Static wear leveling will also move blocks that are not written.
However, the exact nature of wear leveling is not disclosed. And I see no reason to trust an undisclosed "static and dynamic wear leveling" any more than I trust smartmedia.
I will even go further and claim that nothing short of a filesystem can do proper wear leveling across the complete device. The reason smartmedia introduced "areas" was to bound the time until it's map is created and the device can get accessed. If the map spanned a large
64GB device, access times would go sky-high.
Any method I can imagine to offer good wear leveling will result in either a filesystem or at least a simplified one-file-system with the only file being the "block device" exported outward. So naturally my answer to the problem is called LogFS.
Yes you can. Why not? If you have enough usable memory for your application there is no reason for swap. You can disable CONFIG_SWAP in your kernel to save a few bytes of kernel code. Or keep it enabled then you will be able to enable swap at runtime on demand (run swapon/swapoff commands at any time).
Sorry - I didn't notice that he was using a drive with wear levelling. In that case, a swap partition (if you decide to use swap at all) will be levelled just as well as a swap file would be.
The IDE flash drive is a 2GB version of one of these (Embeddisk Module).
formatting link
It plugs directly into an IDE socket and can be treated just like an ordinary hard disk.
My installation procedure was to simply pop a cd with an iso image of the Debian business card installation into a temporarilty connected USB CDROM and perform a base installation. I then installed xserver and opera for the kiosk application that I am developing.
Oh, due to the small size of the drive I only have two partions, one for the swap and the other for everything else.
Yes it does. For this reason Flash memory chips have a so called wear balancer, that scatters write operations to the same logical address evenly on all physical flash cells. The happens transparently and over the whole memory. So this is no problem at all.
Of course you can. Swap is purely optional for Linux. If you've got enough RAM there's no reason for a swap partition at all.
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.