Installing linux on an IDE flash drive. Swap partition question

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Been using a base installation of Debian on a 2GB IDE flash drive
plugged into one of these:

http://www.ipoxtech.com/ADE-2100.php

My question is about the swap partition on the flash drive. Flash
eventually wears out. Surely a flash based swap partition will wear out
that part of the flash a lot quicker than the flash used for the rest of
the Linux installation. Is this considered a problem or just accepted as
a "feature" of using a flash drive.

Can I run Linux without a swap partition or base the swap partition in RAM?

Thanks.

Re: Installing linux on an IDE flash drive. Swap partition question
Quoted text here. Click to load it

You can run Linux without swap (assuming you have enough ram, of
course!).  I've seen a tutorial on the web for setting up a swap
partition on a ram disk, but I cannot fathom why that might be a useful
idea (unless you have a system with some fast ram, and some very slow ram).

It is also quite possible to put swap on a flash drive.  It will wear
out eventually, but how much of an issue that is depends on the type of
flash, your usage patterns, and what sort of wear levelling you are
using (a swap file on a jffs2 or yaffs disk will get wear levelling over
the whole disk, for example, while swap on a partition will have much
less levelling).

If you system normally as enough memory for its needs without using
swap, then it's likely that /tmp will cause more wear than the swap.
One idea might be to include a swap file (there won't be many writes to
it if it is not used), and mount /tmp on tmpfs.  That way, your swap
file will be used if needed for space for temporary files, but under
normal usage small tmp files will not cause any writes at all.

mvh.,

David

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it


It absolutely doesn't make sense to use a IDE flashdrive or a CF-flashdrive
with a jffs2 filesystem.
The internal electronics of such devices already *do* use the whole flash
memory equally. You don't win anything when using another mechanism
for 'equalized' writings on top of that.
Jffs is intended for direct use, such as a microcontroller with direct
attached flashram where the microcontroller is responible for the complete
access.

For the rest, I totally agree: even with 512 megs you don't really need
swap-space any more, at least if you are running the same software and
tasks again and again.


Cheers

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

I would love to actually get a REAL and AUTHORITATIVE answer about the CF
wear leveling issue.  I haven't been able to get the semiconductor
manufacturers to release details about internal wear leveling, and the
information on the net is typically sparse, often incorrect, and usually
from suspect sources.

I work in embedded systems and have been trying for two years to get CF
card manufacturers to clarify (on the record) whether they do wear
leveling, and if so, does their guarantee of 100,000 writes apply to a
single hardware bit or are they taking wear leveling into account when
they make those guarantees.

As for flash options, the CF card is still the best to use simply because
it has a full IDE compatible parallel interface rather than the serial
interface of other type chips.

-Wiseguy


Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

Maybe this could help. It was one of the posts on LKML in a longer thread
about the same question:

On Mon, 11 June 2007 12:13:19 +0200, DervishD wrote:
Quoted text here. Click to load it

Reading through the thread I noticed a lot of guesswork and a total lack
of understanding about how the pendrive actually works.  Everyone treats
it is a black box that has "wear leveling".  So let me crack the box
open to some degree.

Nearly two years ago I have spoken to a person that reverse engineered
the behaviour of several chips used in pendrives.  At the time that
reverse engineering apparently covered most of the market.  The details
were quite lengthy but can be condensed to two words: Smartmedia format.

Smartmedia is a very simple format and can easily be studied.  Afaik
access to the specs only requires registration.  Alternatively one can
study drivers/usb/storage/alauda.c, which implements the format.  I will
summarize the important bits below, assuming the reader has a good
understanding about what flash is.


All flash memory is split into zones of usually 1024(1) erase blocks.
Within each zone, 1000 logical blocks are mapped to the 1024 physical
blocks.  That leaves 24 spare one.  Normally the spare blocks are erased
and contain no data.  The 1000 mapped physical blocks contain the
associated logical number somewhere in the OOB area.

Reading from smartmedia first requires reading all logical numbers from
all physical blocks.  With this information a simple map, essentially a
1024-entry array is created.  Once the map is set up, logical number get
translated to physical ones with a simple lookup.

Writing is slightly more complicated.  First one of the spare blocks is
selected.  New data is written to the spare block.  If only a partial
block is written, the remainder has to be copied from the old block.
After the new block is written, the old block is erased and becomes a
spare.

If the chip loses power during a write, several physical block may
contain the same logical number.  In that case the ECC information is
used to verify each of the blocks.  If one of the block was not written
completely, the other is used.  If one block has ECC errors, the other
is used.  If both are correct, a random one is used.

That's all there is to smartmedia format.  It is _very_ simple.  It is
also surprisingly efficient, although it does have some shortcomings.
So let us look at the problems and how they interact with filesystems.

1. Write overhead

If a filesystem only writes a small amount of data, typically 512 or
4096 bytes, smartmedia has to erase and write a full block.  Most
flashes used in embedded systems has block sizes of 128KiB or so.  Most
flashes used for smartmedia have 16KiB.  Writing 16KiB when the
filesystem only requests writing 4KiB increases the wear 4x and reduces
performance 4x.

2. Wear leveling

Wear leveling happens implicitly by picking a different physical block
from the spares on each write.  However, some blocks are never used.  If
a physical block is mapped to a logical block that never gets written,
it is out of the rotation.  Two seperate 1024-block areas have their
internal wear leveling each, but nothing is spreading high wear from one
area to another.

So if a theoretical filesystem would only ever write to the same logical
block, smartmedia can spead the wear over 25 blocks or less.  Less, if
any physical blocks are bad and cannot get used (2).

More realistically, if Ext3 is having a very hot area - the journal -
that area is not getting any wear leveling worth mentioning.  Journaling
filesystems on smartmedia are a bad idea.  Most journals are bigger than
a smartmedia area, which is usually 1000 * 16KiB.  The round-robin
access pattern of the journal already provides perfect wear leveling
_within that area_.  Smartmedia does not add anything.

3. Hidden caching

Some chip designers seem to have noticed the 4x write overhead and try
to outsmart the filesystem.  Their chips start writing the new block,
but don't copy data from the old block just yet.  Instead they wait for
further write requests, hoping to write adjacent data to the same block.

If the power fails while the chip is waiting for further data, smarmedia
format requires the unfinished block to get erased and its content to
get discarded.  What happens if the user just typed "sync" and yanks the
pendrive as soon as the command returns is anyone's guess.  How many
people ever considered their pendrives to perform caching and require
proper barriers?

4. FAT requirement

When I claimed there was nothing more to smartmedia, I was actually
lying.  Smartmedia has the odd requirement that only FAT is supported as
a filesystem.  In fact, the specifications describe FAT in great detail.
I have already seen FTLs(3) that deliberately look into the FAT and
pre-erase blocks if the corresponding files have been deleted from the
FAT.  Let's just hope they always detect non-FAT filesystems.


After all this, my recommendation for filesystems is to do several
things:

a) Do wear leveling!

Smartmedia wear leveling is limited to within areas.  Any cross-device
wear leveling must be done by the filesystem.  FAT does that fairly
well.  The Ext family doesn't.

b) Write aligned 16KiB blocks

Anything else will cause write overhead and kill both performance and
later the complete device.  Most hard disk filesystem do this anyway.
Any effort to reduce head seek time and rotational delay will also help
your pendrive.


(1) Some smaller chips have a single zone of 256 or 512 blocks instead.
    Also, the first block in the first zone gets special treatment, so
    the first zone effectively only has 1023 blocks.
(2) Most manufacturers specify that their chips has <3% bad blocks.  24
    spare blocks for 1000 logical ones makes 2.4%.  It is conceivable
    that the often-quoted <3% number is designed to work with
    smartmedia.
(3) Flash Translation Layers - any format that implements block device
    behaviour on flash memory.  Smartmedia is but one example, although
    a very popular one.

Jörn


Quoted text here. Click to load it

Maybe I should add that things have allegedly improved.  Many devices
today support both static and dynamic wear leveling.  Dynamic wear
leveling is what smartmedia does.  It depends on the filesystem writing
somewhere to move those blocks.  Static wear leveling will also move
blocks that are not written.

However, the exact nature of wear leveling is not disclosed.  And I see
no reason to trust an undisclosed "static and dynamic wear leveling" any
more than I trust smartmedia.

I will even go further and claim that nothing short of a filesystem can
do proper wear leveling across the complete device.  The reason
smartmedia introduced "areas" was to bound the time until it's map is
created and the device can get accessed.  If the map spanned a large
64GB device, access times would go sky-high.

Any method I can imagine to offer good wear leveling will result in
either a filesystem or at least a simplified one-file-system with the
only file being the "block device" exported outward.  So naturally my
answer to the problem is called LogFS.

Re: Installing linux on an IDE flash drive. Swap partition question
Quoted text here. Click to load it

Sorry - I didn't notice that he was using a drive with wear levelling.
In that case, a swap partition (if you decide to use swap at all) will
be levelled just as well as a swap file would be.

Quoted text here. Click to load it

Re: Installing linux on an IDE flash drive. Swap partition question
AFAIK, you can make Linux not use a swap partition. Maybe it's defined
by a Kernel Parameter when booting.

-Michael

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

AFAIK not. Its done by the swapon/swapoff commands at any time you like.

JB

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

It's even simpler...: If there *IS* *NO* swap partition available at all,
the kernel runs and doesn't use it....




Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

Hmmm, the kernel does *not* detect and use any swap partitions by its own!
Activating swap is the job of userspace!

JB


Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

Robert,

Can I urge you to post your linux install procedure on the flash card? It
would help me and probably others too.

Thank you.


--

Jerry McBride (jmcbride@mail-on.us)

Re: Installing linux on an IDE flash drive. Swap partition question
Quoted text here. Click to load it
The IDE flash drive is a 2GB version of one of these (Embeddisk Module).

http://www.embeddisk.com/products.htm

It plugs directly into an IDE socket and can be treated just like an
ordinary hard disk.

My installation procedure was to simply pop a cd with an iso image of
the Debian business card installation into a temporarilty connected USB
CDROM and perform a base installation. I then installed xserver and
opera for the kiosk application that I am developing.

Oh, due to the small size of the drive I only have two partions, one for
the swap and the other for everything else.


Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

Yes you can. Why not? If you have enough usable memory for your application
there is no reason for swap. You can disable CONFIG_SWAP in your kernel to
save a few bytes of kernel code. Or keep it enabled then you will be able
to enable swap at runtime on demand (run swapon/swapoff commands at any
time).

JB

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it
Where did you purchase the board? I dont see it actually for sale at the link
you posted.
Eric

Re: Installing linux on an IDE flash drive. Swap partition question
Quoted text here. Click to load it
We got the board from a UK supplier, http://www.bvm-store.com
Here is a direct link to the board
http://www.bvm-store.com/ProductDetail.asp?fdProductId43%6

Hope that helps.

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it
sort of - how much was it for a single board?
Eric

Re: Installing linux on an IDE flash drive. Swap partition question
Quoted text here. Click to load it
I think the price for an indiviadual board is £150 excluding any tax.

Re: Installing linux on an IDE flash drive. Swap partition question

Quoted text here. Click to load it

Yes it does. For this reason Flash memory chips have a so called
wear balancer, that scatters write operations to the same
logical address evenly on all physical flash cells. The happens
transparently and over the whole memory. So this is no problem
at all.

Quoted text here. Click to load it

Of course you can. Swap is purely optional for Linux. If you've
got enough RAM there's no reason for a swap partition at all.

Wolfgang Draxinger
--
E-Mail address works, Jabber: snipped-for-privacy@jabber.org, ICQ: 134682867


Site Timeline