Hi all, in a previous thread we discussed about MMC timings and write speed. These memory do not seem to guarantee a minimum sustained write bandwidth.
I was thinking of migrating to CompactFlash (more pins, but seem faster). Producers claim a very high sustained write speed (>1MB/sec), but I'm not so confident to obtain such values.
Can someone, according to its experience, comment on the write performance that can be obtained when a microcontroller controls a compact flash card for data logging? (i.e. how much time do I have to wait for the flash to complete the sector write after I send all the 512 bytes?)
In my application I need to store about 120KB/sec, so if the whole operations lasts some ms, I should be just fine!
As usual any comment, suggestion or discussion is welcome!
I have written an image capture system that writes to compact flash. The system uses FAT formatted CF cards. I found that the write speed is partly determined by the compact flash card itself: on one card I wrote at approximately 120 kB/s whereas another card was 30kB/s. You can get CF cards with different speeds: 4x, 8x, etc. This write time was on a fairly slow processor.
So, do you think that it would be enough to buy a fast compact-flash? (i heard about 66x!) Which brand/model/size allowed you to achieve the 120KB/sec?
Tnx 4 your answer! Aldo
Andrew Jacks>>in a previous thread we discussed about MMC timings and write speed.
With those CF cards that you can buy these days 120KB/sec ist absolutely no problem - not even with a slow 8-bitter.
I.e. on a Rabbit 2000 I achived ~450KB/sec "netto" write speed towards a FAT 16 filesystem.
The key issue is the sector IO routine. Here it's well worth the effort to use assembly if your controller is slow. I.e. formentioned R2000 only did ~100KB/sec useing 'C' code and the Zworld compiler.
Wow.. that's great! Considering that you also update the fat, the "gross" (raw sectors) write rate is probably even higher! Did you perform "single" sector writes or "multiple"?
Do you think that assembly or C could make the difference? And what about the impact of:
- memory buffers
- single/multiple sectors write
- mode of CF: ATA mode, Memory Mode, etc.
Did you use some publicly available code, or did you just write your own routines?
Single sector writes are just fine. Thats especially true if you consider the fact that memory is costly on small systems. Todays CF cards will anyways be magnitudes faster than your embedded controller (asuming your are not playing with a 400Mhz XScale or such :-) ) so multiple sector writes would only place more burden on the part of the host system.
Ohh yesss definately. Again, especially the code writing and reading a sector is VERY important. I got the best results by also using A10 that said the so called but totally missleading "memory mapped mode". There you can use special block move operations your controller might support (as it's the case with the R2000 i.e.). Note thought that the difference is not all that big.
The C versus assembly story is of course different when it comes to implementing FAT. I never ever would consider implementing FAT in assembly these days. That's an area where 'C' is much more apropriate.
You must buffer wisely. My FAT library uses two FAT cache sectors. A "regular" one which is used to follow cluster chains during IO whereever it ocures, and - most important for the write speed - one that "hovers" over the area from where new clusters are allocated. like this allocating a new cluster does not require seeking all of the FAT everytime you need a new one. Then there are also two general purpose IO buffers used and one per open file. This is however only the case because my implementation must and does support multiple "processes" in paralell which must be able to modify a single file concurently.
Another trick is to only buffer sector data where needed. I.e. small writes will go into the cache secotr where bigger amonts got into the cache up until it's full and flushed, but the the remaing part of the user buffer space is directly written up to the reminder which then goes again into the cache etc. With tricks like this you can increase write speed for "bigger" amounts of data siginficantly - where bigger is everything > 512 KB. Needless to say that you also can have lot's of influence on the application layer. Chooseing filelayouts etc. so as the file IO wher it have to be fast is sector alligned can have a significant impact etc. etc.
As mentioned, single. IMHO most CF cards not even really support multiple in that the number of sectors returned is always 1. I seem to remember that the specs even mention this somewhere but might be that this has changed over time or that I remember incorrectly.
The mode does not matter that much except for the trick with the block move operations which if memory serves only is available in memory mapped mode. Note though that while there is a difference in the achvieable speed it is not THAT big (i.e. probably 10% gain or such).
I'm one of those crazy people who are faster in writing their own library than understanding someone elses code :-) Seriousely, I had to write all of it on my own for various reasons.
If you need complete FAT compliance this is going to use a week or two of your time if you can limit yourself to FAT 16 and no long name suport (the latter would be a real nightmare btw.)
However, if you only are after logging data, you can prealocate the file in a first simple step, then simply write consecutive sectors during the real logging. If you do this cleverly, you really don't have to have much really FAT specific code hence you might save lots of time. Don't expect to be able to port existing say Linux FAT Code to an 8 bitter though.
Code that deals with sector IO should be readily available. If you really are in the dare need of this, I can help you out.
Depends on the CF. One major CF card controller is a re-programmed ARM controller. Until flash can be programmed faster than 100x, there is no point speeding up the controller anyway.
So a good idea may be implementing low-level sector access routines is asm, and using them in C for handling FAT16...
I just need one file, and as you suggest I might pre-map it with fixed size, for example filling it with 0, and then changing only the data sectors...(with no overhead updating the fat) Shall the buffering unit be one sector or one cluster?
But definitely, as I have only to append new data to existing one in the flash.. what happens when I issue to the flash a WRITE SECTOR command?
After writing the command register and the other ones, the flash expects
512 bytes to be written (with A2 A1 A0 set to 0 0 0 , right?)
But, what happens if I write for example only 20 bytes, then I wait some time by letting my cpu doing other things, and then write other 60 bytes, then I wait some time doing some calculations and finally write the remaining 512-20-60 bytes?
I mean... can I use the CF as a single sector buffer? Or just have I to write all the 512B of a sector in a single shot with no interruptions?
yeah! I'm one of those guys too! (...sometimes :-) )
Good hint, tnx.
Didn't even think about this! :-)
Thank you, I already have found something, I'll first check this out.
Markus, thanks for the wise advices and the good hints, it was a really helpful discussion. Bye! Aldo
Obviousely just one sector since with your aproach there are no clusters. I mean if you format the media and then create the file in advance as the one and only file (asuming you do that external in a PC or such) you can count on the fact that you have subsequent sectors. That said you don't even have to read the fat at all to follow clusters. You would have to guarantee though that never ever a user inserts a CF into your device that was not prepared this way. Otherwise you still need to read along the cluster chain and in doing so you may want to spend a buffer for caching the FAT.
It expects the data and you transfer it. Thereafter you check the status and if it's ok all is well. Otherwise you get an error in return. However such errors are very rare.
yes
I never had the need for such an aproach. The CF specs may state somewhere a general timeout or such. Apart of this I don't see any obvious reason why this should not work. I actually would expect a CF card to act this way and as such this might make your live much easier.
That's as fast as the inner loop can possibly run, doing PIO with that microcontroller (PIC18F6680). Happens to be a bit slower than the slowest specified ATA Mode 0 timing.
--
> I'll have a look at the url, thank you very much.
> Aldo
According to CompactFlash Revision 3.0 the maximum transfer speed can be up to 66MB/sec and with a DMA transfer mode. This is great news for the future of Compactflash. With 16 parallel data pins and enough physical space to hold tera bytes of information, almost secures it to be the leader in high-end memory and OEM devices into the near future. Although data transfer speeds haven't reached this level yet, it is only because of the hardware design costs. Manufactures of the PCMCIA hardware interface need to update their hardware to support the current revision 3.0. Then, this will spawn the manufactures of memory cards to take advantage of these new speeds. A lot of people think that the read/write speed of Flash memory is a limitation of data transfer, however in theory, almost any speed can be achieved with different design layouts which include data buffers and massive parallel architectures.
It's not inconceivable that future revisions of the CompactFlash specifications will include data transfer speeds a magnitude better than the current standard.
No, the write speed of flash memory is not the limit of the write speed of CompactFlash cards. Yes, each bit/byte of Flash has a write cycle limit, however it is possible to store incoming data to a FIFO buffer then mux it out to parallel channels of FIFO's that feed the multiple sections of Flash. Then there is no write cycle limit, and the limit then falls back to the port speed.
Currently, the real limits are the design costs vs. the available market.
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.