Hi,
On 31 Jul 2018, at 0:47, Laczen JMS wrote:
Hi Marko and Will,
I have experienced the same problem while developing nvs for zephyr. I
always got stuck when I would except that there would be changes to
flash by an "external" factor. As soon as this could happen there is a
problem with the meaning of the crc. The crc fails this means:
a. Incomplete write,
b. Some tampering occurred,
c. The len is not pointing to the correct location,
In all these cases the result will be that the data is not considered
valid, the only case where a increased crc might be helping is in b.
I think (b) is more commonly failure of flash over time, and writes
and/or erases that fail. Some times bits will be “stuck” on
flashes, especially inexpensive ones in high volume consumer products
that have no correction for this. The probability that this will happen
across a population of millions of devices is probably pretty high, and
the bugs are hard to find.
In the latest version of nvs I have changed how the data is written to
flash. The data itself is written from the start of a sector, a
"allocation entry" is written from the end of a flash sector. This
allocation entry contains a sector offset and length of the data
written. The allocation entry is of fixed size. The writing of data
is done using the following method:
1. Check if there is sufficient space (difference between allocation
entry location and next data location),
2. Write the data at the specific location, this could include a data
crc,
3. Write the allocation entry, this includes a crc calculated over the
item length and offset,
When there is insufficient space a allocation entry of all zeros is
written at the end of the allocation entries, and a new sector is
used.
At startup the allocation entry write position is found by stepping
through the allocation entries until all 0xff are found. The data
write position is found by searching the position from which all data
up to the allocation entry write position is 0xff.
This is a good way of accomplishing this, another is to use a magic on
every entry. It would be interesting to allow streaming of entries
using this approach — fixed length entries that contain type, offset,
len — perhaps encoded more efficiently. We’ve been discussing
adding a flash store interface to Mynewt similar to NVS and one of the
requirements is definitely to not require the full amount of RAM to
write a larger BLOB entry to the flash.
Sterling
