Il 15/06/2018 19:11, Richard Damon ha scritto: > On 6/15/18 11:20 AM, pozz wrote: >> Il 15/06/2018 16:21, David Brown ha scritto: >>> On 15/06/18 15:10, pozz wrote: >>>> Il 15/06/2018 11:25, David Brown ha scritto: >>>>> On 15/06/18 09:38, pozz wrote: >>>>>> Il 14/06/2018 12:49, David Brown ha scritto: >>>>>>> On 14/06/18 12:20, pozz wrote: >>>>>>>> I need to save some data on a non-volatile memory. They are some >>>>>>>> parameters that the user could change infrequently, for example 10 >>>>>>>> times >>>>>>>> per day at a maximum. In the range 100-2kB. >>>>>>>> >>> >>>>>>> >>>>>>> Multiple copies (at least 2) are key, along with timestamps or >>>>>>> counters >>>>>> >>>>>> Two should be sufficient to prevent corruption caused by interruption >>>>>> during the writing of a block of data. >>>>>> Maybe three (or more) are needed to face memory *physical* corruption >>>>>> (maybe for many writings). >>>>>> I think I can ignore this event in my actual project. >>>>>> >>>>> >>>>> You use more for wear levelling. Often your flash chip is /way/ bigger >>>>> than you need - perhaps by a factor of 1000 simply because that's what >>>>> you have on stock, or that's the cheapest device in the package you >>>>> want. Spread your writes over 1000 blocks instead of 2, and you have >>>>> 500 times the endurance. Use a good checksum and accept that sometimes >>>>> a block will be worn out and move onto the next one, and you have >>>>> perhaps a million times the endurance (because most blocks last a lot >>>>> longer than the guaranteed minimum). >>>> >>>> So the writing process should be: >>>> >>>> 1. write the data at block i+1 (where i is the block of the current >>>> data in RAM) >>>> 2. read back the block i+1 and check if checksum is ok >>>> 3. if ok, writing process is finished >>>> 4. if not, go to block i+2 and start again from 1. >>>> >>> >>> Yes. >>> >>> Then comes step 5 (for flash with separate erasing) : >>> >>> 5. If you have written block x, check if block x+1 (modulo the size of >>> the device) is erased. If not, then erase it ready for the next write. >>> >>> Note that it does not matter if the erase block size is bigger than the >>> program block size - if it is, then your "erase block x+1" command will >>> cover the next few program blocks. >>> >>>>>>> and checksums. >>>>>> >>>>>> This is interesting. Why do I need a checksum? My approach is to use >>>>>> only a magic number plus a counter... and two memory areas. >>>>>> At first startup magic number isn't found on any areas, so the device >>>>>> starts with default and write data on Area1 (magic/0, where 0 is the >>>>>> counter). >>>>> >>>>> You use checksums to ensure that you haven't had a power-out or >>>>> reset in >>>>> the middle of writing, >>>> >>>> Only for this thing, you can write the counter as the last byte. If the >>>> writing is interrupted in the middle, counter hasn't written yet, so the >>>> block is not valid (because considered too old or empty). >>> >>> Nope. You can't rely on that, unless you are absolutely sure that you
like >>> that, even if they provide an interface that matches it logically. >>> >>> A common structure for a modern device is to have 32-byte pages as a >>> compromise between density, cost, and flexibility. (Bigger pages are >>> more efficient in device area and cost.) When you send a command to >>> write a byte, the device reads the old 32-byte page into ram, erases the >>> old page, updates the ram with the new data, the writes the whole 32 >>> byte page back in. >>> >>> The write process is done by a loop that writes all the data, reads it >>> back at a low voltage to see if it has stuck, and writes again as needed >>> until the data looks good. Then it writes again a few times for safety >>> - either a fixed number, or a percentage of the number of writes taken. >>> >>> So it is /entirely/ possible for an interrupted write to give you a >>> valid counter, but invalid data. It is also entirely possible to get >>> some bits of the counter as valid while others are still erased (giving >>> ones on most devices). >>> >>> And that is just for simple devices that don't do any fancy wear >>> levelling, packing, garbage collection, etc. >>> >>> >>>> >>>>> and that the flash has not worn out. >>>>> >>>>>> >>>>>> When the configuration is changed, Area2 is written with magic/1, >>>>>> being >>>>>> careful to save magic/1 only at the end of area writing. >>>>>> >>>>>> At startup magics and counters from both area are loaded, and one area >>>>>> is chosen (magic should be valid and counter should be the maximum). >>>>>> >>>>>> I think this approach works, even when the area writing is interrupted >>>>>> at the middle. >>>>>> >>>>>> Why do I need checksum? The only thing that comes in mind is to >>>>>> prevent >>>>>> writing errors: for example, I want to write 0x00 but the value really >>>>>> written is 0x01, maybe for a noise on the serial bus. >>>>>> >>>>>> To solve this situation, I need checksum... but also I need to re-read >>>>>> and re-calculate the checksum at *every* area writing... and start >>>>>> a new >>>>>> writing if something was wrong. >>>>>> >>>>>> Do you have a better strategy? >>>>> >>>>> You calculate the checksum for a block before writing it, and you check >>>>> it when reading it. Simple. >>>> >>>> Do you calc the checksum of all the data block in RAM, including padding >>>> bytes? >>> >>> Yes, of course. The trick is not to have unknown padding bytes. I make >>> a point of /never/ having compiler-generated padding in my structs. >>> >>> So you have something like this: >>> >>> #define sizeOfRawBlock 32 >>> #define noOfRawBlocks 4 >>> #define magicNumber 0x9185be91 >>> #define dataStructVersionExpected 0x0001 >>> >>> typedef union { >>> uint8_t raw8[sizeOfRawBlock * noOfRawBlocks]; >>> uint16_t raw16[sizeOfRawBlock * noOfRawBlocks / 2]; >>> struct { >>> uint32_t magic; >>> uint16_t dataStructVersion; >>> uint16_t crc; >>> uint32_t count; >>> >>> // real data >>> } >>> } nvmData_t; >>> >>> static_assert(sizeof(nvmData_t) == (sizeofRawBlock * noOfRawBlocks), >>> "Check size of nvnData!"); >> >> Why raw8[]? >> >> I think you can avoid raw16[] too. If you have the function: >> >> uint16_t calc_crc(const void *data, size_t size); >> >> you can simply call: >> >> nvmData.crc = >> calc_crc( ((unsigned char *)&nvmData) + 8, sizeof(nvmData) ); >> >> > > I typically also do something like this. The data structure is a union > of the basic data structure with a preamble that includes (in very fixed > locations) a data structure version, checksum/crc, and if a versioning > store a timestamp/data generation number. A 'Magic Number' isn't often > needed unless it is a removable media as it will either be or not the > expected data, nothing else could be there (if the unit might have > different sorts of programs, then a piece of the data version would be a > program ID.) > > Often I will have TWO copies of the data packet.
TWO copies in EEPROM or in RAM?
Do you need raw_data[] array only to set a well-known size to flash_parms? Do you alloc an entire (or a multiple of) flash sector in RAM? Even if your params in flash take only 1/5 or 1/2 or 1/10 of a sector?
Is it really necessary? What happens if you don't care about unused bytes in flash_parms? I think... nothing.
It seems difficult, if the changes from one version to the other are "important".
struct { uint16_t foo; uint16_t bar; uint16_t dummy; } SystemParameters;
struct { uint8_t foo; uint8_t bar; uint8_t dummy; } SystemParametersOld;
union { uint8_t raw_data[DATA_SIZE]; struct { struct StandardHeader header; struct SystemParameters parms; struct SystemParametersOld parms_old; } flash_parms;
for (uint8_t sector = 0; sector < SECTORS_NUM; sector++) { uint16_t memory_address = OFFSET + sector * SECTOR_SIZE; sermem_read(&flash_params, memory_address, sizeof(flash_params)); uint16_t chk_read = flash_parms.header.crc; flash_parms.header.crc = 0xFFFF; if (checksum_calc(&flash_parms, sizeof(flash_parms)) == chk_read) { /* Datablock is valid */ if (flash_parms.header.counter > prev_counter) { /* Datablock is newer */ prev_counter = flash_parms.header.counter; if (flash_parms.header.version == FLASH_PARMS_OLD) { /* How to convert old version in new version in place? */ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ } else { /* Ok, new version. No convertion is needed. */ } } } }
As I wrote in the comment, how to convert old version to new version, using the same union in RAM? Do I need another union?
drivers, >>> polling, etc., so that you can mix fast and slow tasks. >> >> Yes, ok. >> >