RAM-based filesystems metadata

The metadata for the file /var/log/messages is stored in the file (yes, a directory is just another type of file) /var/log

With your configuration, no.

I can't comment on whether or not atime is useful in this case, but I can tell you that it wont hurt to leave atime alone.

exec()ing programs would cause disk read activity, but then again, that counts as "reading files"

Directory traversal, and directory operations like file creation and deletion would cause disk activity, for directories stored on that disk. If your flash driver were formatted as a journalling file system, the journal updates would also cause disk I/O.

One self-correction: the metadata is stored in special blocks (called "inodes") within the device. The directory just contains pointers to these blocks of metadata. In any case, the metadata for your /var/log/ messages file would be contained within the tmpfs that is the /var filesystem, and not on your hard disk.

......

I would recommend setting noatime on any rw mounted flash-drive partition. I even use noatime on hardrive partitions - speeds things up quite a bit. There is a thread at the moment in the linux-kernel-mail-list on ditching atime, and introducing a sort-of-atime but with reduced overhead. But - in general atime is braindead for flash file systems - unless you have a specific reason to record access time of course :-) My own view is that any sort of journaled file system is also bad for flash, in that it increases the number of writes, compared with a non-journaled fs.

One clarification: tmpfs file systems are not always in RAM. They use VIRTUAL memory, which means that they may be paged out to the swap partition if necessary. They aren't really "RAM disks".

So I think the main performance benefit of tmpfs is not that the data is guaranteed to be in memory, but that they don't require that data (normal or "meta") be flushed to disk frequently. Normal filesystems generally force metadata to permanent storage as soon as possible, and regular data fairly often. Tmpfs will only flush data to disk when the RAM pages are needed for something else.

Lew, please correct me if I'm wrong.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

First off, /var is used for many things other than just temporary files and log files. You might find you have trouble if it is empty on a reboot (for example, some processes may expect /var/log/ to exist). tmpfs filesystems are cheap - there is no problem making several of them for /var/tmp, /var/log and /var/locks for example.

Second, as others have mentioned, using noatime could be useful for your flash drive. For tmpfs, storing the access time is just a write to ram (since you have no swap, so the tmpfs could not be on disk), and therefore pretty fast.

Third, what makes you think wear on the flash disk is a problem? Assuming the flash disk is of decent quality, then you can write to

*each* sector hundreds of thousands or millions of times. Combine that with wear levelling (modern flash disks have wear levelling built in - if you are rolling your own with standard flash devices, you can use a file system like jffs2), and you have a disk that is several orders of magnitude more reliable than a hard disk. You should have no qualms about putting swap files on the flash disk or any other heavily used files - the advantage of tmpfs is in the speed (since the data never needs to be flushed to disk). You should also be using a journalled file system - the extra writes for the journal are not a problem, while long check times for an unclean ext2 mount could well be.

Of course, you'll want to read the specs for your flash drive and do some calculations yourself.

mvh.,

David

Several directories are created at boot-time:

mkdir -p /var/{lib,log,run,tmp}

OK.

/dev/hda1 / ext2 defaults,noatime 0 1

No need to set noatime on tmpfs filesystems.

My main concern is not so much wear from the writes, but system latency induced by the writes. This flash drive does not seem to support DMA, so the CPU has to spoon feed the data. Since I'm trying to do "real-time" work, I figured it would be a good idea to try and minimize "disk" activity.

I wish I had the datasheet. I'll look for it.

# ./hdparm -I /dev/hda

/dev/hda:

ATA device, with non-removable media Model Number: PQI IDE DiskOnModule Serial Number: DOM5F00017230 Firmware Revision: ra03.00e Standards: Likely used: 1 Configuration: hard sectored not MFM encoded head switch time > 15us fixed drive disk xfer rate > 5Mbs Logical max current cylinders 500 500 heads 8 8 sectors/track 32 32 -- bytes/track: 0 bytes/sector: 528 CHS current addressable sectors: 128000 LBA user addressable sectors: 128000 device size with M = 1024*1024: 62 MBytes device size with M = 1000*1000: 65 MBytes Capabilities: LBA, IORDY not likely Buffer type: 0002: dual port, multi-sector Buffer size: 1.0kB bytes avail on r/w long: 4 Cannot perform double-word IO R/W multiple sector transfer: Max = 1 Current = 0 DMA: not supported PIO: pio0 pio1 pio2

# ./hdparm -v /dev/hda

/dev/hda: multcount = 0 (off) IO_support = 1 (32-bit) unmaskirq = 1 (on) using_dma = 0 (off) keepsettings = 0 (off) readonly = 0 (off) readahead = 256 (on) geometry = 500/8/32, sectors = 128000, start = 0

Do you consider a Pentium 3 at 1267 MHz to be a fast processor?

;-)

OK, that makes sense if the disk writes are slow and hog your processor. Of course, it depends on the rest of the system - sometimes with a fast processor, direct IO writes are significantly faster than DMA (assuming the drive can accept data at a high enough rate).

mvh.,

David

... snip ...

It's at least twice as fast as a 500 MHz one. :-)

--
 Chuck F (cbfalconer at maineline dot net)
   Available for consulting/temporary embedded and systems.

For an embedded system, I guess it's OK (anything over 10 MHz is fast in my book...). It is certainly possible that direct writes will be faster than DMA with such a cpu, albeit at higher cpu usage costs (obviously!). As long as you have kernel preemption of some sort, that should not hinder the rest of the system.

Unless the direct writes occur in a critical section which cannot be preempted (obviously).

Indeed. The major players seem to agree (??) on ditching atime.

Huh? Ingo Molnar appears to agree with himself that he would rather change the kernel than use the existing interface.