Guarantee Critical Regions in Portable code. Portable Semaphores?

Hi,

I am creating an Open-Source thread-safe FAT file-system implementation called FullFAT.

formatting link
(fullfat.googlecode.com)

The library is ultimately designed to be portable across anything that can compile C code.

I am providing 2 functions FF_EnterCriticalRegion() and FF_ExitCriticalRegion().

The developer needs to modify these 2 functions so that they work appropriately on their platform.

  • Disable System interrupts - thus preventing Kernel context swtiching also.

However, can I use this to safely implement my own semaphore system?

What if the developer uses a dual-core embedded platform? How can I guarantee a safe atomic operation.

James

Reply to
james.walmsley
Loading thread data ...

If these functions disable interrupts, then yes you can.

Sounds like an interesting project. How large do you anticipate the full system to be, say on an ARM7 using GCC for an example?

Also, from your WEB site. "Others incorrectly refuse to add support for LFN's due to Microsoft patent issues. I have researched this, and we can implement LFN's without the risk of a lawsuit from MS. Read up about their statement on the FAT license agreement."

I would be interested in how you have come to this conclusion. I have asked lots of people about this issue and the conclusion is always "um dunno, but Microsoft do not have a history of suing for breaches, and are very unlikely to start now". Not exactly water tight..

--
Regards,
Richard.

+ http://www.FreeRTOS.org
Designed for microcontrollers.  More than 7000 downloads per month.
New FreeRTOS eBook!  Learn how to use an RTOS.
Reply to
FreeRTOS.org

Microsoft white-paper on FAT32:

formatting link

The license here is what I initially read, but reading it again makes me not so sure. I'm going to do some more research into this, and try to find a more definitive answer. Why does legal speak have to be so confusing!!

Anyway I don't want to get into that argument here. I'll publish my findings along with the FullFAT library when its finished.

It should be very small, the major memory footprint comes from the caching. Which is user configurable in sizes of multiples of 512 bytes. Minimum 1 single buffer of 512 bytes.

The code is still being written, so I cannot give any final estimates of code size. I'm going to write an email to microsoft to try and get a plain english explanation of the licensing.

James

patent

risk

asked

but

unlikely

Reply to
james.walmsley

d

This whole issue of patents sprang up around the time the original XBox was released.

Most embedded applications do not require LFN support, so it is always going to be a low priority feature for me. However if someone explicitly asks me to add it to DOSFS, I'll do it only (a) in a private release for that person, and (b) with the written understanding that they assume all IP implications.

Reply to
larwe

... snip ...

Please don't strip attributions for any material you quote. Their is no intimation about who 'your' represents in your message. This denies credit, etc.

I don't remember exactly what the lifetime of a patent is in the USA, but Microsofts patents on the LFN mechanism must be near to running out. They appeared with W95, thus in 1995, and are at least 14 years old now.

--
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: 
            Try the download section.
Reply to
CBFalconer

Is your file system API compliant to POSIX or any other standard?

:))) Naive idea.

Disabling the interrupts is a bad idea anyway.

No.

The only way to guarantee the atomic access is by the use of services provided by OS. Your FAT driver should provide the OS porting layer. You will need many other system functions besides semaphores. Such as time related functions, memory management, locale, share, thread local storage, etc. etc.

Vladimir Vassilevsky DSP and Mixed Signal Consultant

formatting link

Reply to
Vladimir Vassilevsky

can

Hello, look at the lwIP project, they use an OS abstraction layer that works nicely, if you use the same abstraction layer your file system will work on any OS that already ported lwIP without extra work.

regards, Giovanni

--
ChibiOS/RT http://chibios.sourceforge.net
Reply to
gdisirio

If your embedded application intends to read memory cards / sticks with music on them, users will expect to see long file names. At least, my MP3 archive was the first on my FAT partitions where I left 8.3 land.

However, as far as I understand, MS' patents mostly apply to allow coexistence of 8.3 and long file names. Most embedded applications don't need *that*. My FAT implementation never even looks at the 8.3 names if there is a LFN, and it only generates them to not leave the 8.3 field blank. In contrast, Windows uses 8.3 names all the time, leading to major nuisances such as "dir c:\*1*" showing "c:\Program Files" and "c:\Documents and Settings", which have a "1" in their short name.

Stefan

Reply to
Stefan Reuther

Well, (a) the world DOES NOT need yet another MP3 player project, and (b) the important information is in the ID3 tags, not in the filename. Parsing the filename is the last resort if your MP3 file is missing ID3s.

Correct... the patent raised the most often in this discussion talks about the idea of maintaining two parallel databases of filenames with links associating one with the other. But FAT with 8.3+LFNs is not the only place MS uses this technique; they also use it for example when serving MacOS volumes from an NT-family server.

That's a bug in the way CMD.EXE parses wildcards, though - not an inherent FS bug. Observe that an Explorer window does not show the 8.3 equivalent for a file for which an LFN exists, and an explorer search on a FAT volume also suppresses these 8.3 orphans.

Reply to
larwe

You mean, we should ignore all those millions of people wanting to buy devices with MP3 ability and leave them to the Chinese? I'm glad my boss disagrees.

Then how do you parse playlists? How do you display meaningful folder names? And how do you explain your users why you display "01_-_A~1.MP3" as a file name, gibberish they've last seen in an obsolete program ten years ago? 99% of the MP3 files I produce don't have tags. Simply because I'm too lazy to fill them in, and because file names work very well.

That bug, if you want to call it that, is in the FindFirstFile function, and is documented in the API description (e.g. page "Searching Filenames" in win32.hlp).

Stefan

Reply to
Stefan Reuther

Apple isn't Chinese :)

Hmm, I forgot about that. It has been so many years that I've been in the iPod economy (along with the majority of other consumers) that I haven't found an issue with it.

Still, on the topic of embedded FS'es, I stand by my original comment that the majority of embedded applications have no need of LFNs. Thank God, the vast majority of embedded projects are not Yet Another MP3 Player.

ll.

Ah, but they don't for me - I want to be able to sort on album, artist, year at the very minimum.

It's an API limitation which is worked around practically everywhere else in Windows, and the absence of a workaround in CMD.EXE is I would say a bug (or rather a lack of priority, since MS deprecates use of the command-line).

Reply to
larwe

It's not a naive idea, its just challenging. Of course there are issues when porting anything. However you can design code to be portable, and make any porting easier for the end developer.

Can you explain why on a single core system why you think that disabling interrupts won't guarantee atomic operations, and also explain why you think it is bad.

Working on BlackFIN, disabling interrupts means that the VDK Os scheduler simply cannot switch threads/processes because as with all OS's this requires a timer interrupt.

Obviously disabling interrupts is bad if done over to long a period of time. It is sometimes necessary I believe, and if you fully understand when to do it and how then its ok. Essentially it should only be done over a very short period of time.

On a single-core system, if disabling interrupts prevents the OS from task switching, this should mean that all operations are atomic. If that is the case, a simple mutex lock system should implementable?

I don't know the answers, I just want your opinions, and your explanations.

James

implementation

can

functions,

Reply to
james.walmsley

Op Sat, 31 Jan 2009 19:09:27 +0100 schreef james.walmsley :

Allow only one OS process to execute file system operations. No need for semaphores or disabling of interrupts, just a suitable message passing system. Replace the FS functions by stubs that communicate with the FS process.

--
Gemaakt met Opera's revolutionaire e-mailprogramma:  
http://www.opera.com/mail/
Reply to
Boudewijn Dijkstra

.....yet.

-- Regards, Richard.

  • formatting link
    Designed for microcontrollers. More than 7000 downloads per month.

  • formatting link
    New FreeRTOS eBook - learn how to use an RTOS.

Reply to
FreeRTOS.org

As far as I know, Apple doesn't make music centres, nor DVD players or TV sets (which often come with USB these days). And even if they did, I'm sure, and would hope, that one or another company would find a little competition interesting :)

If you consider the vast amount of industry electronics, then yes. A PBX doesn't need long file names to be able to accept configuration updates from a memory card. But consumer electronics?

Actually, LG Electronics even has (or wants?) a patent for a washing machine with MP3 player. German news article:

It was *not* published on April 1st.

Stefan

Reply to
Stefan Reuther

It is not the business of the application layer to interfere with the system activities. Disabling interrupts is the kernel level operation.

Because there is cache, DMA, speculative execution and deferred reads/writes.

You should not make any assumptions about the atomicity of any operation unless you know the target hardware and software in the very detail. There are the OS services which you should rely upon.

VDK is junk. Being dissatisfied with VDK and ucLinux, I rolled my own OS for BlackFin.

How can the FAT driver know when it is safe to disable the interrupts and for how long it is safe to disable the interrupts? That is very application and platform dependent, and it is not the business of the driver to make any assumptions.

Disabling interrupts doesn't imply atomicity, because there is cache, DMA, speculative execution and deferred reads/writes.

Only if it based on the services provided by target OS.

Drop the project.

Vladimir Vassilevsky DSP and Mixed Signal Design Consultant

formatting link

Reply to
Vladimir Vassilevsky

The whole point is that it gets implemented inside the OS level. Obviously for standalone single-threaded applications there's no problem. FullFAT is below the application level, and provides an application level interface.

I'm not making assumptions about how to ensure an atomic operation on another platform. (Because I cannot!) I provide 2 functions, and inside them I ask the developer to do whatever is necessary to enter a critical region, to ensure atomic operations. Then to leave that state when the second function is called. By calling the appropriate OS services.

It's true that DMA and other peripherals on the core could be running.

Operating systems provide sync functions, to remove the cache problem. So if a developer places sync functions inside my critical region functions, then all should work.

We also wrote an operating system for BlackFIN, (well the firm I work for).

You cannot know it's safe for sure, but a single increment or decrement instruction shouldn't take a dangerous amount of time. It's unlikely you'll be using a FAT driver in a critical system.

I guess the best thing to do is protect critical memory attributes with an OS provided semaphore.

I just need to find the best way to make a semaphore wrapper, that can be easily ported to other operating systems.

I won't take your advice, but thanks for helping me to think about the problems from a different angle!

I've also decided that LFN support will be built dependent on the configuration file. So LFN support will work like a module, either you use it or you don't.

From VDK it provides a function to CreateSemaphore,PendSemaphore, ReleaseSemaphore. Do most OS's essentially expose the same interfaces? If so what kind of objects do they return?

James

issues

make

disabling

scheduler

when

a

task

Reply to
james.walmsley

Thanks for the advice, I've seen this done before, and its a tidy solution, FullFAT sit's somewhere in-between the I/O layer and the application layer.

The underlying drivers provide blocks to buffers, which fullfat processes to the application layer. Of course when it can, this transfer goes directly from the driver to the application layer. FullFAT decides when and which to do.

or =A0

=A0

=A0

om/mail/

Reply to
James Walmsley

That's not always true, and in particular not true of the vast majority of PC hardware. Bus-locked atomic instructions (compare&swap, etc) allow a synchronisation style called lock-free programming. Google it and learn. These instructions are supported by hardware all the way past the caches to main memory.

I built a thread-synchronisation library that implements synchronisation primitives outside the O/S for several architectures, and there are some hundreds of thousands of computers happily running multi-threaded software built on that library.

Reply to
Clifford Heath

There is no life outside PC x86 ?

:))))))

How can you know what is supported by the particular hardware and the particular instruction set? How can you know which memory areas are shared, cached or transferred by DMA? Are you aware of the side effects?

If you have to use the number argument to prove your self importance, that means your library is worseless crap :)))))

VLV

Reply to
Vladimir Vassilevsky

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.