How to export linux fs using USB MassStorage device

What are you trying to accomplish ?

There is a free EXT2/3 driver for Windows that does work finw with USB sticks and so it will work with your device, too, if you export a media that is formatted as EXT2/3.

If you want to avoid any installation on the Windows PC, you need to do or find a virtualization that simulates an FAT formatted device. Regarding the simplicity of FAT, I suppose that is quite doable in a read-only mode, especially if there is only a main directory, no long file names and the all files are smaller than 512 Bytes.

-Michael

Correction:

512 Byte is nonsense. Without additional complexity, you can offer files as large as one cluster and you can define the cluster size when doing this software. IIRC the max cluster size is 64 K.

-Michael

... and assigning multiple clusters to a file would also not be a big problem (in read-only mode). I suppose assigning more clusters than the file length requires is not really forbidden.

BTW.: regarding the latest patent-claim by Microsoft, you are not allowed to deal with long file names on FAT drives in an embedded device.

-Michael

Hi Michael,

That's exactly what I want to do. Any idea where to start looking for such a virtualization - and yes, read only is all I need? As I have not found one yet I'm looking into the possibility of implementing it myself. I'm just not sure what the right approach would be. My idea would be to implement a block device driver. The so created block device I would provide to the g_file storage module. Any suggestions?

Regards, Dan

I'm not aware of such a virtualization, maybe you need to do it yourself. But FAT is not that complicated, so read-only seems quite doable.

The PC requests blocks of 512 byte sectors and you need to proved them. You can simulate a constant FAT and an appropriate root directory giving file (supposedly constant) names, sizes and (constant) start start cluster numbers. The PC now will request the appropriate sectors containing the clusters with the file content, the bytes of which you will fetch from the real files

That might be possible and would allow for using the g_file software you mentioned to hook same to the USB interface, but I feel that this is much more complex than handling the appropriate USB read block yourself (i.e. modifying the g_file software ). (But I might be wrong.)

-Michael

I see. Would have been nice if there where a solution available.

Well I looked into it and that might be the final solution, but if I write an own device driver I only have to worry of the device driver not working. When modifying g_file I would need to worry about breaking the UBS part ...

Anyway, thank you for your support, Dan

You are right, but writing a block driver is much more challenging than writing user software, especially regarding debugging.

I suppose g_file interfaces the block device it is runtime-configured to use at just a few places. You would replace "write" block-device access(es) by an error condition and the "read" the block-device access(es) by a function call to the function you will create. Same gets a sector number and count and is supposed to fill a buffer with a sequence of bytes. This can't harm the USB functionality.

This done you might even be inclined to use your function to do the block driver which of course is the more "clean" solution.

Of course I would be interested to see the source code once you get it going :)

-Michael

Yes, that's exactly how they do it - read raw blocks from a file or a device

Well, lets see if I ever get that far ^^

Well it sounds easy as you write it, but creating a fat in the background - at least some caching for the most important structures is required - and give the right block - created on the fly - back when asked for the block number seems not an easy task to me.

So I plan to first create a few simple things as user apps:

- simple static fat which can be mounted using a loop device

- simple app read a directory structure, convert it into static fat, mountable using loop

Now write a simple block device driver (module) which builds upppon simple app 2 challenges here might be:

- access directory using vfs access inside the kernel instead of user space

- get the blocks dynamicly created This block device should be mountable as fat

If that is done I only need to load g_file_storage with the device

- sounds easy ... right ^^

Dan

I don't see what you mean here. For a read-only thingy you don't "create ab fat in the background", but you simulate one. You provide the PC with a completely constant file-system definition and FAT (only one is necessary) and a directory that only is dynamic regarding the file site. Now the PC will request blocks with the files' content according to the FAT information you provided. Your program will read these blocks from the real files-

I don't see why you need a loop device - or anything additional to the modified g_file software at all.

???

To me that sounds rather complicated (compared to just modifying the g_file program to simulate an FAT device for the PC.

-Michael

Typo: ... "file size" ...

-Michael

Regarding the interface to g_file, I can't say much, but the simulation of a simple (i.e. static, sauf than the file content), I'd estimate some

500 lines of easily testable C code. A draft should be doable in a day.

-Michael

Still ranting on...

If the PC request sector 0 it gets a file system description. IIRC same defines the sector where the FAT starts, where the root dir starts and where the file area starts. I'd do an FAT 16 simulation using 32 K clusters. This information is static (always the same).

If the PC requests a sector of the root file system it gets the description of the files provided. This information is static, only the files' sizes can be set according to the current situation. The file's start sectors can be something like 0x100, 0x200, 0x300, ... I suppose a single sector (normal is 512 bytes) will be sufficient (or do you plan for a huge number of files)

If the PC request a sector from the FAT you can calculate these 512 bytes to represent sector sequences like 0x100, 0x101, 0x102, ...,

0x200, 0x201, 0x202, .... 0x300, 0x301, 0x302, .... the sequence length done according to the max file length you want to support. This information is static but I suppose it's better to calculate it on request than to store it somewhere.

If the PC requests a sector (or multiple) from the file area (sector = file area start sector + cluster * (clustersize / sectorsize) ) you calculate the cluster no, and thus the file in question and the relative location of the sector. So you can open the file and read the appropriate bytes and provide them via USB.

Sounds quite doable to me.

BTW.: Where did you fined the g_file code ? I'd like to take a look at it but googeling did not show a location.

-Michael

Keep doing it ;) - been a bit bussy.

Yep. that's the plan, but you have to create the directory entries for it. And they might change - using an arbitrary directory in the Linux file system as root of the fat.

Of course, using a directory with files only for starters would be a sensible approach.

Its part of the Linux kernel driver/usb/gadget/file_storage.c

Dan

OK if you don't want to provide just a constant set of files, things get somewhat more complicated. Nonetheless the files could start at cluster numbers like 0x100, 0x200, 0x300, ...

But what will be the file names ? Long file names are not allowed because of the Microsoft patent (TomTom just payed M$ for allowing them to use this).

I feel that a constant set of files would suffice in most application that provide monitoring an embedded device via the USB File Media interface.

I'll take a look...

I see that it's a driver, not a user land application. This could make things somewhat more complicated. (1) debugging and (2) I don't know how to access files from a Kernel driver (should be possible, though).

Ranting along: Maybe you could create a device driver that provides a block device for g_file to use and provides a simple character device that a userland daemon can hook to and wait for request from the block site which it will answer accordingly.

Or you create a modified g_file driver that provides a character device for that daemon instead of using a block device interface of another driver.

Maybe this easier than doing everything in a driver.

-Michael

Create a fat filesystem inside an image file on your linux file system. Point gadget at this storage space, and it will represent it to the USB host as a mass storage device containing a fat filesystem.

What you can't do is modify the contents of that file while it's mounted on another machine as a USB mass storage device - even if it's mounted read only, the remote machine isn't expecting the contents of the disk to go and change unexpectedly, as that could break any caching of data or metadata by the remote system.

I think if you want the embedded device to be able to modify it while it's mounted, then windows is going to have to see it is a shared file system, like a network one. To do that, you'd either need to install drivers on the windows system - for example, gadget's usb network driver, and then install samba on the embedded device and treat it as a windows share.

That of course is very true and has not been discussed yet.

If installing something propriety on the Windows system is necessary, maybe using "serial via USB" might be the easiest way to go.

-Michael

Good idea, only problem is, that the file system I want to provide is much bigger than my flash storage or my memory. So statically creating the file system is out of the question.

I'm clear on that. But if changes happen on the embedded device I still can unload g_file_storage and start it again

No, it must be simple mass storage.

Dan

Imagine a mounted SD-Card combined with data from the embedded system. Representing multiple partitions or devices are no option either.

Dan