How to export linux fs using USB MassStorage device

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Hello,

I'm using an embedded linux system and want to make a part of the file
system accessible from a Windows System using USB mass storage.

I know about the usb gadget driver g_file_store. This however only
works with devices or files.
As I want to be able to access the mass storage from Windows the
device/partiton or file must be formated in fat.

Is there a solution out there which lets me map a part of the linux fs
(tempfs, jffs2, fuse) as a fat formated block device or file?


Re: How to export linux fs using USB MassStorage device
What are you trying to accomplish ?

There is a free EXT2/3 driver for Windows that does work finw with USB
sticks and so it will work with your device, too, if you export a media
that is formatted as EXT2/3.

If you want to avoid any installation on the Windows PC, you need to do
or find a virtualization that simulates an FAT formatted device.
Regarding the simplicity of FAT, I suppose that is quite doable in a
read-only mode, especially if there is only a main directory, no long
file names and the all files are smaller than 512 Bytes.

-Michael

Re: How to export linux fs using USB MassStorage device
Correction:
512 Byte is nonsense. Without additional complexity, you can offer files
as large as one cluster and you can define the cluster size when doing
this software. IIRC the max cluster size is 64 K.

-Michael

Re: How to export linux fs using USB MassStorage device
... and assigning multiple clusters to a file would also not be a big
problem (in read-only mode). I suppose assigning more clusters than the
file length requires is not really forbidden.

BTW.: regarding the latest patent-claim by Microsoft, you are not
allowed to deal with long file names on FAT drives in an embedded device.

-Michael

Re: How to export linux fs using USB MassStorage device
Quoted text here. Click to load it

Hi Michael,

That's exactly what I want to do. Any idea where to start looking for
such a virtualization - and yes, read only is all I need?
As I have not found one yet I'm looking into the possibility of
implementing it myself.
I'm just not sure what the right approach would be. My idea would be
to implement a block device driver.
The so created block device I would provide to the g_file storage
module.
Any suggestions?

Regards,
Dan

Re: How to export linux fs using USB MassStorage device
Quoted text here. Click to load it

I'm not aware of such a virtualization, maybe you need to do it
yourself.  But FAT is not that complicated, so read-only seems quite
doable.

The PC requests blocks of 512 byte sectors and you need to proved them.
You can simulate a constant FAT and an appropriate root directory giving
file  (supposedly constant) names, sizes and (constant) start start
cluster numbers. The PC now will request the appropriate sectors
containing the clusters with the file content, the bytes of which you
will fetch from the real files



Quoted text here. Click to load it

That might be possible and would allow for using the g_file software you
mentioned to hook same to the USB interface, but I feel that this is
much more complex than handling the appropriate USB read block yourself
(i.e. modifying the g_file software ). (But I might be wrong.)

-Michael

Re: How to export linux fs using USB MassStorage device
On 14 Jul., 07:42, Michael Schnell
Quoted text here. Click to load it

I see.
Would have been nice if there where a solution available.

Quoted text here. Click to load it

Well I looked into it and that might be the final solution, but if I
write an own device driver
I only have to worry of the device driver not working.
When modifying g_file I would need to worry about breaking the UBS
part ...

Anyway, thank you for your support,
Dan

Re: How to export linux fs using USB MassStorage device

Quoted text here. Click to load it

You are right, but writing a block driver is much more challenging than
writing user software, especially regarding debugging.

I suppose g_file interfaces the block device it is runtime-configured to
use at just a few places. You would replace "write" block-device
access(es) by an error condition and the "read" the block-device
access(es) by a function call to the function you will create. Same gets
a sector number and count and is supposed to fill a buffer with a
sequence of bytes. This can't harm the USB functionality.

This done you might even be inclined to use your function to do the
block driver which of course is the more "clean" solution.

Of course I would be interested to see the source code once you get it
going :)

-Michael

Re: How to export linux fs using USB MassStorage device
On 14 Jul., 23:47, Michael Schnell
Quoted text here. Click to load it
Yes, that's exactly how they do it - read raw blocks from a file or a
device

Quoted text here. Click to load it
Well, lets see if I ever get that far ^^

Quoted text here. Click to load it

Well it sounds easy as you write it, but creating a fat in the
background - at least some caching for the most important structures
is required -  and give the right block - created on the fly - back
when asked for the block number seems not an easy task to me.

So I plan to first create a few simple things as user apps:
- simple static fat which can be mounted using a loop device
- simple app read a directory structure, convert it into static fat,
mountable using loop

Now write a simple block device driver (module) which builds upppon
simple app 2
challenges here might be:
- access directory using vfs access inside the kernel instead of user
space
- get the blocks dynamicly created
This block device should be mountable as fat

If that is done I only need to load g_file_storage with the device
- sounds easy ... right ^^

Dan

Re: How to export linux fs using USB MassStorage device
Quoted text here. Click to load it

I don't see what you mean here. For a read-only thingy you don't "create
ab fat in the background", but you simulate one. You provide the PC with
a completely constant file-system definition and FAT (only one is
necessary) and a directory that only is dynamic regarding the file site.
Now the PC will request blocks with the files' content according to the
FAT information you provided. Your program will read these blocks from
the real files-

Quoted text here. Click to load it

I don't see why you need a loop device - or anything additional to the
modified g_file software at all.

Quoted text here. Click to load it

???

Quoted text here. Click to load it

To me that sounds rather complicated (compared to just modifying the
g_file program to simulate an FAT device for the PC.

-Michael

Re: How to export linux fs using USB MassStorage device
Typo: ... "file size" ...
-Michael

Re: How to export linux fs using USB MassStorage device
Regarding the interface to g_file, I can't say much, but the simulation
of a simple (i.e. static, sauf than the file content), I'd estimate some
500 lines of easily testable C code. A draft should be doable in a day.

-Michael

Re: How to export linux fs using USB MassStorage device
Still ranting on...

If the PC request sector 0 it gets a file system description. IIRC same
defines the sector where the FAT starts, where the root dir starts and
where the file area starts. I'd do an FAT 16 simulation using 32 K
clusters. This information is static (always the same).

If the PC requests a sector of the root file system it gets the
description of the files provided. This information is static, only the
files' sizes can be set according to the current situation. The file's
start sectors can be something like 0x100, 0x200, 0x300, ... I suppose a
single sector (normal is 512 bytes) will be sufficient (or do you plan
for a huge number of files)

If the PC request a sector from the FAT you can calculate these 512
bytes to represent sector sequences like 0x100, 0x101, 0x102, ...,
0x200, 0x201, 0x202, .... 0x300, 0x301, 0x302, .... the sequence length
done according to the max file length you want to support. This
information is static but I suppose it's better to calculate it on
request than to store it somewhere.

If the PC requests a sector (or multiple) from the file area (sector =
file area start sector + cluster * (clustersize / sectorsize) ) you
calculate the cluster no, and thus the file in question and the relative
location of the sector. So you can open the file and read the
appropriate bytes and provide them via USB.

Sounds quite doable to me.

BTW.: Where did you fined the g_file code ? I'd like to take a look at
it but googeling did not show a location.

-Michael

Re: How to export linux fs using USB MassStorage device
On 16 Jul., 23:05, Michael Schnell
Quoted text here. Click to load it
Keep doing it ;) - been a bit bussy.

Quoted text here. Click to load it

Yep. that's the plan, but you have to create the directory entries for
it.
And they might change - using an arbitrary directory in the Linux file
system as root of the fat.

Of course, using a directory with files only for starters would be a
sensible approach.

Quoted text here. Click to load it

Its part of the Linux kernel
driver/usb/gadget/file_storage.c

Dan


Re: How to export linux fs using USB MassStorage device
Quoted text here. Click to load it

OK if you don't want to provide just a constant set of files, things get
somewhat more complicated. Nonetheless the files could start at cluster
numbers like 0x100, 0x200, 0x300, ...

But what will be the file names ? Long file names are not allowed
because of the Microsoft patent (TomTom just payed M$ for allowing them
to use this).

Quoted text here. Click to load it

I feel that a constant set of files would suffice in most application
that provide monitoring an embedded device via the USB File Media interface.
Quoted text here. Click to load it

I'll take a look...

I see that it's a driver, not a user land application. This could make
things somewhat more complicated. (1) debugging and (2) I don't know how
to access files from a Kernel driver (should be possible, though).

Ranting along: Maybe you could create a device driver that provides a
block device for g_file to use and provides a simple character device
that a userland daemon can hook to and wait for request from the block
site which it will answer accordingly.

Or you create a modified g_file driver that provides a character device
for that daemon instead of using a block device interface of another
driver.

Maybe this easier than doing everything in a driver.

-Michael

Re: How to export linux fs using USB MassStorage device

Quoted text here. Click to load it

Create a fat filesystem inside an image file on your linux file
system.  Point gadget at this storage space, and it will represent it
to the USB host as a mass storage device containing a fat filesystem.

What you can't do is modify the contents of that file while it's
mounted on another machine as a USB mass storage device - even if it's
mounted read only, the remote machine isn't expecting the contents of
the disk to go and change unexpectedly, as that could break any
caching of data or metadata by the remote system.

I think if you want the embedded device to be able to modify it while
it's mounted, then windows is going to have to see it is a shared file
system, like a network one.  To do that, you'd either need to install
drivers on the windows system - for example, gadget's usb network
driver, and then install samba on the embedded device and treat it as
a windows share.

Re: How to export linux fs using USB MassStorage device
Quoted text here. Click to load it

That of course is very true and has not been discussed yet.

If installing something propriety on the Windows system is necessary,
maybe using "serial via USB" might be the easiest way to go.

-Michael

Re: How to export linux fs using USB MassStorage device
On 18 Jul., 22:44, Michael Schnell
Quoted text here. Click to load it

No, it must be simple mass storage.

Dan

Re: How to export linux fs using USB MassStorage device

Quoted text here. Click to load it

o some means needs to be introduced that the files don't change while a
connection is established.

-Michael

Re: How to export linux fs using USB MassStorage device
On 20 Jul., 22:43, Michael Schnell
Quoted text here. Click to load it

Yep, thats an issue, but first let me ghet started.

Dan

Site Timeline