Simple filesystem for embedded platform

I'm using lwip that runs httpd over LPC1769 MCU. Now the filesystem for httpd can't fit the internal Flash memory anymore. So I'm going to use an external SPI Flash memory.

I need some suggestions about a filesystem to use. I need:

- a tool that creates a binary (or Intel HEX) image from a folder on my Windows development machine (the image will be downloaded to Flash);

- file management functions (open, close, read);

- simple integration with lwip httpd;

I don't need to change the filesystem, so a read-only filesystem is enough.

Reply to
pozz
Loading thread data ...

Do you really need open, close and read functions? All you actually need is to copy a block of SPI memory into ram. That might simplify things a bit.

Does your filesystem need to be independent of the main program? Your filesystem tool could read through the files in a directory, concatenate them together into a single binary file, and generate a C header with something like:

enum { start_index_html = 0, length_index_html = 4040, start_favicon_ico = 4040, length_favicon_ico = 840, start_node_js = 4880, length_node_js = 10391, ... };

If that is enough, then your "tool" could be done as a bash script if you wanted (I'd use Python myself).

It will be a little more work if you want to track filenames as well, rather than having them fixed in the code, but you could generate a table of indexes, lengths, and names. It is easier if that part can be linked directly in the C code, but it would be possible to have it at the start of the binary blob for the SPI.

Reply to
David Brown

Yes, you're right. open/close here can be avoided. read is a must (your copy from SPI to memory RAM is a *read*).

I can't understand. Suppose you need to open the URI "/index.html". How to search for this filename in your enum list?

I know I can write something from zero, but I'm sure something already written that can be reused.

Reply to
pozz

First ask yourself, /do/ you need to search for this URI in a list? An embedded web server will often have just one web page - it might not even have any other files or URI's at all. Then you don't need to parse the incoming URI in any way - you return the same "file" each time. For the web server in an application I have, I have 4 files - index.html, favicon.ico, 404.html, and a jquery.js file. The searching can be done be a set of "if (strcmp....)" checks that are fixed in the code. (My server is a little bit beyond that, but not much.)

You are not trying to re-create Apache here - there is no need to make things more complex than they have to be for the task in hand. I don't know what features you need here, and whether such a method for "searching" is enough - but it is possible.

My script for reading a bunch of files and putting them in a static data structures linked directly with my C code (I guess my microcontroller has more flash than yours!) is about 130 lines of Python. Half of that is static text copied directly to the generated header and C file. I wrote that myself, along with the httpd daemon in C, because I found it easier to get the features I wanted than to use the demo server and code from the LWIP httpd server. The "filesystem" bit was easy. Perhaps writing it from zero is not as hard as you think it might be - while finding and adapting existing solutions may be more effort. (Of course I cannot give you an answer here, I am just giving ideas for consideration.)

Reply to
David Brown

formatting link

Bye Jack

--
Yoda of Borg am I! Assimilated shall you be! Futile resistance is, hmm?
Reply to
Jack

If the file open frequency is not that often: reserve the first (or few) sectors on the SPI flash for some sort of file name list - file name, start address in SPI flash, length

In your open code read through the file list, find the file and set an internal file hanle to the address on the SPI flash. On the read side, read from SPI, increment address, etc.

If opens happen frequenly, read the file name map from SPI on boot and save in some type of memory table. You can scan that quick and then set the file address in your file handle structure

For writing have some tool that creates an image and upload it. Or maybe even upload a tar or zip image and have the web server take care of it. If your file list is not that long just pick a sector to start the files, unpack the files from the stream coming in to your web server, write to SPI and keep track of starting address and length. At the end go write the file list sector(s). You bulk earse the SPI flash at the beginning so everything is ready.

Updating a single file is a lot tricker and if that is the case you really need some type of flash file system.

--
Chisolm 
Republic of Texas
Reply to
Joe Chisolm

Chan's file system is great stuff - used it many times. But Pozz needs WAY less than even Chan's petit version. Pozz just needs a simple small set of read-only files...

Reply to
Dave Nadler

My static web files will be around 20. However I got your point: auto-generate a hardcoded file-list made by a sequence of !strcmp(). I think it would be ok.

Reply to
pozz

That's it, yes. An alternative is to generate a hash from the strings and use that in a switch statement.

This all depends somewhat on whether your C code in internal flash and your "files" in external flash need to be independent or if they can be tightly coupled. The idea is to avoid making something more complicated than necessary, while still being flexible enough to handle changes or new files.

Reply to
David Brown

Am 02.02.2018 um 13:22 schrieb pozz:

You can even let your generator generate the strcmp sequence.

I would probably go for some sort of

struct { uint32_t start; uint32_t size; char name[24]; };

and let my generator build an array of that. Last time I did that, I stored the structs in the flash as well, to get independence between the program and data images.

Stefan

Reply to
Stefan Reuther

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.