Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
On 10 Jul 2003 15:25:07 -0700

Quoted text here. Click to load it

This post was probably refused because it *looks* like you're asking
for free technical support on a commercial product -- they probably
felt that you should hire a consultant. Just a guess.

Quoted text here. Click to load it
<snip>
Quoted text here. Click to load it

How about feeding it a stream of known data and then just shutting
down the power at various points? There's nothing like a pile of
empirical data for answering questions . . . :-)

--
Kevin Nathan (Montana, USA)
Open standards. Open source. Open minds.
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
...
Quoted text here. Click to load it

Is it important for the device not to self-destructed in the field?

Quoted text here. Click to load it
...

CF are close to hard drives, but not identical.

...
Quoted text here. Click to load it

You might not have a choice!
Quoted text here. Click to load it
...


If the Sandisk controller does not handle the necessary command
function,
no other compact flash will.

Quoted text here. Click to load it

It does not matter.  You will destroy the CF after a few hundred
thousand cycles, probably within days with a Journalized file system.

Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
sales@ide-cf.info-for.us (Sales for IDE-CF flash drive) wrote in message
Quoted text here. Click to load it

Yes, preventing the system from self-destructing is even more
important than not corrupting our data.  This is why our root
filesystem is in an initrd ramdisk that lives in the /boot partition,
which we don't have mounted when we're booted.

Quoted text here. Click to load it

The hardware is pretty much fixed at this point (we've already gone
through initial "qual-testing", for instance).  We definitely won't be
able to change the design to use a hard drive rather than a CF disk,
and even if we were able to, I can't imagine a HD being more reliable
in this application than a solid-state disk.

Quoted text here. Click to load it

I wonder why they wouldn't allow disabling write caching.  Naturally
performance suffers with it turned off, but if it's required to turn
it off to use a journalled filesystem, you'd think they'd support
it...

Quoted text here. Click to load it

Well, that's definitely not true.  We've gone for much more than a
matter of days and the CF has not been destroyed.  We did some
calculations (admittedly doing a lot of hand-waving, since, for
instance, we can't get the details of SanDisk's wear-leveling) and it
was our belief that the disks would far outlast the service lifetime
of the unit...

--
Dan Harkless
snipped-for-privacy@harkless.org
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

A harddisk has no limitation on the count of rewrites of any sector. The
CF cards has (though its complicated and well hidden). So this is not a
case of reliability but of specification. If you want to use a CF card
for some kind of application that permanently rewrites some data you
will quickly destroy it. There is a file system for linux that is
written exactly for the purpose of using Flash in the best possible way,
but it only works correctly with naked flash chips and can't be updated
for CF cards, as there is no specification how exactly the CF card
handles the "spreading" of data, and supposedly it's different for
different brands.

-Michael

Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

Our understanding is that the SanDisk CF's built-in wear-leveling will
prevent the disk from wearing out for many, many years.  Do you know
this not to be the case?

Quoted text here. Click to load it

Yes, that filesystem is JFFS2, and was discussed elsewhere in the
thread.

--
Dan Harkless
snipped-for-privacy@harkless.org
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

Of course it tries to do so.

I don't know the Method used by Sandisk, but from another brand I
learned that they do the built-in wear-leveling in a way that they
measure the time used to program the words in a page and if that time
gets too long they use one of the spare pages. (BTW.: JFFS (on a "naked"
flash chip) uses all pages in a cyclic way.)

In any case changing a single byte forces a complete Flash sector (some
512 K Byte, depending on the internals of the device) to be rewritten.
So you can roughly calculate the count of possible write requests when
you know the page size the count of spare pages and the number of
allowed (re-)write accesses to a page.

Using a journalling file system will make things much worse as each
write will result in additional updates of the Journal.

-Michael

Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Hello Michael,


Quoted text here. Click to load it

A NAND Flash Sector is 16 KBytes and is erased in 2 ms.

My advice to the original poster is: use CF Cards from Sandisk (the
"industrial" Grade(?)) and have some sort of Accumulator and a power
fail line to do a sync on the CF before shut down. And don't use a
journaling filesystem!

JFFS2 may be a better file system for FLASH, but it is mainly for
NOR Flashes.

best regards
Wolfgang Mes

(who uses JFFS2 on Strata Flash)




Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

When you say reliable is this mostly a concern because of the effects of
vibration?

I know that ruggedized hard drives are available for use in aeroplane
applications where the hard drive is mounted on vibration proof mounts
increasing the life of the hard drive against vibrations.  However they
are big + expensive.  

Have you checked out microdrives? - I would imagine they would perform
much better than standard hard drive with regards to vibration.  If you
have to use a hard drive maybe it would be worth checking out their
vibration specs.  

Regards,

Paul..

Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
...
Quoted text here. Click to load it

Just to have fun with it.  We took a brand-new 4M CF, write random
data to it continuously.  It crashed (drive not ready) after approx.
25,000 write cycles in three days.  We will run a few more tests to
confirm it and post the results later.

Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs

Quoted text here. Click to load it

Maybe a reformat helps?

There are some differences between CF cards from different vendors, so
please post vendor and exact part information too.

Regards,

--
Bernhard Roessmann
Don't Fear The Penguins!


Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

Nop, can't access it anymore (drive not ready).

Quoted text here. Click to load it

Controller: Feiya 1106ET-C314
Flash Memory: SEC KOREA 943 KM29W32000ATS

Quoted text here. Click to load it

Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs

Quoted text here. Click to load it



Probably not.


Try to get information from your vendor _when_ the CF  perfroms  it's
wera levelling. Chances are that it does so only when power cycling.
Running the CF continouosly == ruining it.


Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88   Web: www.denx.de
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
     snipped-for-privacy@harkless.org (Dan Harkless) writes:
Quoted text here. Click to load it

I'm not sure if *any* filesystem can help to make this reliable:
AFAIK, the behavior of the disk device itself in case of a power
loss (e.g. during a write operation) is undefined. It may well
happen that the disk's sector formatting gets corrupted, rendering
the disk unusable. IMHO, no filesystem software can compensate for
that.


Rob

--
Robert Kaiser                     email: rkaiser AT sysgo DOT de
SYSGO AG                          http://www.elinos.com
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it


The sector (if it *really* is only a single one) gets *damaged* from the
drive electronics' point of view. Surely you can find the garbled
sector: the drive does this for you by returning an I/O error if you
attempt to access it (usually after performing prolonged retries and
recalibrates). The only way to cope with such a defect is to add the
bad sector to the drive's defect list, so the drive will avoid it in
the future and access a spare one instead. If the drive is capable of
doing this transparently (I wouldn't bet on it), then a journalling
filesystem may be able to recover, but only so many times (because
sooner or later, the drive will run out of spare sectors).

The behavior of a disk drive in case of power loss during active write
is completely unspecified in general. You may be able to find specific
models designed to cope with such situations, but in general, there is
no way to predict what will happen.

Btw, Compact Flashes appear to be particularly ill-suited for this
situation: There have been several reports from IMO trustworthy people
on this ML that a sudden power loss during write operation can render
a CF *completely* unaccessible.

Quoted text here. Click to load it

.. only if the *media* is 100% predictable ...

Quoted text here. Click to load it

Assuming a reliable media for storage (e.g. Flash memory (*not*
CompactFlash!) or DiskOnChip devices), JFFS2 is said to be up to
the challenge.

Rob

--
Robert Kaiser                     email: rkaiser AT sysgo DOT de
SYSGO AG                          http://www.elinos.com
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
bitbucket@invalid-domain-see-sig.nil (Robert Kaiser) wrote in message
Quoted text here. Click to load it

Yes, we have experienced this.  Due to the inflexibility of SuSE's
init.d startup/shutdown ordering scheme, they try to shut down PCMCIA
prior to taking down PCMCIA network interfaces, which caused our boxes
(until I kludged a fix) to hang at the "Shutting down PCMCIA" step.

One time I powered my system off during this hang (at which time I
guess writes were still being done to Flash), and when I booted back
up, the CF disk had I/O errors that were not fixed by re-making the
partitions and filesystems.

Did some research and found a post saying that some CF disk reader
peripherals have low-level format capability, but haven't looked
further into that yet.

Luckily, our device has a large capacitor which will keep us running
for up to a second if we lose AC power.  We plan to have a device
driver which will detect the power loss and lock out all other tasks
besides the driver to prevent there from being any active writes to
Flash as we lose capacitor power.

Quoted text here. Click to load it

The hardware was already decided before I joined the company, so I'm
not sure what the decision-making process was that settled on
CompactFlash, but we do need to have read/write speeds not massively
slower than hard disks, and relatively large capacity (250 MB).  Would
Flash memory or DiskOnChip fulfill those constraints?

--
Dan Harkless
snipped-for-privacy@harkless.org
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

Several years ago i do some tests in with low level HD disk writes an
NEVER got really damaged sectors, but some with checksum errors after
power down. But if I overwrote them with new data, everything was ok.
I can't beleive that sectors of a hard disk can

Quoted text here. Click to load it

Some of this rumors I heard, too.


Regards,


--
Bernhard Roessmann
Don't Fear The Penguins!


Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs

Quoted text here. Click to load it


You may find that the _whole_device_ does not respond any more to the
normal command set. Bingo!

[Yes, I have seen such behaviour with my own eyes. Twice.]


Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88   Web: www.denx.de
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

Can you give me some information about the vendor and type of the CF you
used?

Regards,

--
Bernhard Roessmann
Don't Fear The Penguins!


Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs

Quoted text here. Click to load it

The first one was a SimpleTech 32 MB CF (don't remember  exact  type,
this was 1 ... 1.5 years ago). When it died, I repeated the test with
a  no-name  CF  card  which  died  within  the  first  100  cycles of
write/power-off. I was not in the mood of ruining more cards at  that
time.

Best regards,

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88   Web: www.denx.de
We've slightly trimmed the long signature. Click to see the full one.
Re: Have ext3 on SanDisk CF but can't disable write-back caching as kernel instructs
Quoted text here. Click to load it

I understand ;-)

I will try this with two Sandisk CF cards (one consumer, one industrial)
in the next weeks. And if I don't forget it, I will post the results
here :-)



Regards,

--
Bernhard Roessmann
Don't Fear The Penguins!


Site Timeline