Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD? - Page 2

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD?

Quoted text here. Click to load it


Thank you for that lead in! :-)

My problems are these:

1. SSDs are randomly accessible with no moving parts
   They are just slower versions of RAM.
   RAM is not formatted, but still just as easy to use!

2. SSD read sectors can be 4K, erase sectors may be 512k
   This will strain every known file system approach.

3. SSDs sectors need erasing before writing.
   This doubles the work load for an MMU which at the
   moment skips and dances around the issue.

4. Damn hard to properly format and read/write SSDs in an embedded
   system which adds layers of overhead.

5. Linux is built with an MMU in mind, but with an SSD, an MMU is
   extreme waste and extreme overhead.

6. Database support overhead is next to impossible to handle
   in a small embedded system if a filing system is present.

   That won't do with advent of 32 bit cheap as chips computing
   having arrived on the scene.



Seems to me the slate needs to be wiped clean.

1. Remove MMU. Kiss it good bye now!
  
   Replace it with software managed data shuffling and more
   DMA controllers.

2. Eliminate formatted drives immediately!!

   How? Here is something I posted earlier (unfinished).

#-----#

SSDs being memory that retain data after power down,
some additional work needed to extend the RAM model to SSD.


First cut at this (deeply technical):

1. Starting from highest address available and counting down,
   the data at the higher locations are all just pointers
   pointing at data in the rest of the SSD which are
   stored at address beginning at the 0 end.

   Call this the root pointer table.

   Note 1, 2


2. Pointer data grow from one end, the data itself grows from the other
   end.

3. No need to format the drive with this scheme!!!!!!!

4. Perfect for the embedded world because there is no code overheads
   associated with formatting.
   Put a new SSD in, the drive can be used immediately
   without having to format it! Perfect for TVs with PVRs, to Linux
   desktop OSes.

5. Each pointer points to arbitrary lengths of data and is free
   of context.
   That creates a problem because each device now doesn't know what
   data to expect when it opens the pointer table and tries to
   read the data that each pointer points at.

   The best way out of this is to agree on what the first 4 bytes in
   a structure pointed to by pointers from root table
   will hold. It should be unique number to indicate specific types
   of data.

   Note 3, 4.

6. Database operations become infinitely easy and orders of magnitude
   faster for desktops, mainframes and embedded devices.
  
   The reason is that each root pointer points to data that identifies
   itself without needing a filename.

   A database program can create thousands of data types and sift through
   the root table to find all its data without reference to a single file
   name.

   Note 3b.

7. A large part of file management operations can be dropped
   for faster database, gaming and embedded applications, because
   the file names and their management is modular and easy to drop
   when not needed.

   Note 3c.

8. Parallel operations by multiple CPUs to manage different
   data structures independently of each other across multiple
   SSDs.

   A small CPU could be managing the root pointer table, file management
   tables and cache management
   on a separate battery backed RAM and write back to the SSD
   only when it needs to.

   This means the file management, doesn't interfere with caching.
   The caching could be much smarter because it is software controlled.

etc.

Note 1
   The pointer with value 0 means its a null pointer
   while a pointer with all bits set to 1 means its the pointer
   list terminator. Because 0 is null pointer, sector 0 is
   is not used for standard data.

Note 2
   The root pointer table begins at the last page, but it may be
   a broken/worn out sector. To detect this, the first 4 bytes are not used
   and contain a unique improbable number. A check sum of this is number
   in the last 4 bytes of that sector. These 8 bytes must work out for the
   data in that sector to be read by operating system and treated as valid.
   Otherwise operating system moves on to the last but 1 sector to
   seek a valid sector for the beginning of the root pointer table.

Note 3
   The first 4 bytes in
   a structure pointed to by pointers from root table gives that
   data a implied data structure.

   For example, the value 1 could mean the rest of the data
   in that block are pointers to bad sectors that should not be used.
      
   Value 2 could be used to point to a data structure that gives
   filenames to data structures pointed to by the root table.
   Each entry points to a string that is a filename. If the pointer
   is 0, then no filename exists for that object. For each
   entry in the root pointer table, there is a corresponding entry
   in the filename table with same offset.
   #
   The separation of file objects from file names is
   perfect for minimalist embedded systems, as they can
   use the objects pointed to root table directly without having to maintain
   filename overheads in many an environment!!

   Value 3 could be used to mean a unix style text file.

   Value 4 could be used to mean database data objects with another 4 bytes
   to follow that indicates specific data type such as address, stock code,
   name, etc.

   Some open source authority will have to issue the numbers on
   a reasonable and free basis to avoid collisions.

Note 3b
   Value 5 could be used mean a collection of pointers of
   database data objects. These are copies of pointers
   taken from root table.
   It effectively collects randomly scattered data objects to make
   a complete database.
   #
   For embedded this is great because they can now manipulate
   database objects without file and filename management!
   No MMU.
   It is many orders of magnitude faster for desktops and mainframes
   to use this method because root pointer operations are atomic.
   You could build a transaction, and switch a pointer at
   the last minute to complete the transaction.

Note 3c
   Value 6 could be used to mean file permissions, dates, file size etc.
   This is useful for standard file management. Again this feature
   can be dropped in embedded where it may not be needed.
  
Note 4
   To prevent running out of numbers, the 4 bytes can be optionally
   be extended to 8 bytes. The switch over happens when the MSB of the
   4 byte number is set to 1 to indicate an 8 byte number.



Re: Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD?
Quoted text here. Click to load it


And how about a lead out - maybe you'd like to try and learn something
about SSDs, flash, disks, filesystems, ram, and pretty much everything
else you've got wrong here before continuing?

Quoted text here. Click to load it

They are not randomly writeable, unlike RAM.

Quoted text here. Click to load it

RAM used for filesystems /is/ formatted.  That is how it can work as a
filesystem.

Formatting is just a matter of setting up a basic structure on the
disk/flash/ram block so that the OS can reliably store and retrieve
data.  That's all - and that's all that is done with SSD's too.

Quoted text here. Click to load it

Erase sectors are not usually so big, but they are still big.  And write
sectors are also perhaps 4K.  This means you can't write less data than
that - flash is not randomly writeable, in the way RAM is.

Reading and writing 4K blocks at a time sounds quite like current large
hard disks to me.

Quoted text here. Click to load it

It is not work of an "MMU".  The SSD controller has to handle block
erasing, fragmentation, re-mapping, wear levelling, etc.  This will
always be the case for flash (if the SSD controller doesn't do it, the
OS must do it - as it does with bare-bones flash filesystems like jffs2).

Quoted text here. Click to load it

It is not a challenge at all.  If you are using embedded Linux, you can
format, read and write SSDs just like any other media.  If you are
working on minimal sized embedded systems, you can use a simpler system
like FAT.

Quoted text here. Click to load it

MMUs are part of the system of directly accessing memory - SSDs (and
other flash) is not directly accessed, and the MMU is not involved.
Throwing around technical terms that you don't understand does not make
you look any better.

Quoted text here. Click to load it

You've pulled that claim out of thin air, without showing any knowledge
of databases, filesystems, or embedded systems.

The whole point of using embedded Linux is that you can use Linux
software - including databases (big ones like Postgresql, small ones
like sqllite, or whatever) in exactly the same way as you would on a
non-embedded Linux system.

It is fair to say that if you have an embedded system with directly
connected flash (NAND or NOR), and you have a particular simple database
requirement (say, a logging system), then you could be more efficient by
having a simpler structure on the NAND rather than a full-blown
filesystem.  But that would not be an SSD.

<snip more nonsense>

Re: Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD?

Quoted text here. Click to load it


If any of what you rant about has anything to with reality or is
in with the topic, invite yourself back.



Quoted text here. Click to load it


Re: Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD?
Quoted text here. Click to load it

If any of the nonsense that you rant about has anything to do with reality
then you should invite yourself back.

Once AGAIN - you seem know absolutely nothing about anything and aren't shy
about exposing your ignorance.


--
"I've done for example SQL optimization and the reason I could do it better
than the best high level experts can do was because I think in hex and bits
We've slightly trimmed the long signature. Click to see the full one.
Re: Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD?
Quoted text here. Click to load it
--------------------------------------------------------------------------------------
Quoted text here. Click to load it

Are you really this clueless or are you trolling?  Storage media needs to be
formatted (aka - "organized") to make data retrieval possible.




Re: Next year will see 100GB SSD cheap as chips, but why does Linux need to format an SSD?

Quoted text here. Click to load it
--------------------------------------------------------------------------------------
Quoted text here. Click to load it


Aha... Is that a rhetorical question I spy....
--
?aah, los gringos otra vez!?

Site Timeline