On Tue, Oct 24, 2023 at 8:43 PM Chris Angelico via Python-list <python-list@python.org> wrote: > > On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list > <python-list@python.org> wrote: > > Consider an example of bit rot. I mean what if your CPU or hard disk has a > > location where you can write a byte and read it back multiple times and > > sometimes get the wrong result. To be really cautions, you might need your > > software to write something in multiple locations and when it reads it back > > in, check all of them and if most agree, ignore the one or two that don't > > while blocking that memory area off and moving your data elsewhere. Or > > consider a memory leak that happens rarely but if a program runs for years > > or decades, may end up causing an unanticipated error. > > > > True, but there are FAR more efficient ways to do error correction :) > Hamming codes give you single-bit correction and two-bit detection at > a cost of log N bits, which is incredibly cheap - even if you were to > go for a granularity of 64 bytes (one cache line in a modern Intel > CPU), you would need just 11 bits of Hamming code for every 512 bits > of data and you can guarantee to fix any single-bit error in any cache > line. The "if most agree, ignore the one or two that don't" design > implies that you're writing to an absolute minimum of three places, > and in order to be able to ignore two that disagree, you'd probably > need five copies of everything - that is to say, to store 512 bits of > data, you would need 2560 bits of storage. But with a Hamming code, > you need just 523 bits to store 512 reliably. > > Here's a great run-down on how efficiently this can be done, and how > easily. https://www.youtube.com/watch?v=X8jsijhllIA > > Side note: If we assume that random bit flips occur at a rate of one > every X storage bits, having redundant copies of data will increase > the chances of a failure happening. For example, using a naive and > horrendously wasteful "store 256 copies of everything" strategy, you > would be 256 times more likely to have a random bitflip, which is > insane :) You would also be able to guarantee detection of up to 128 > random bitflips. But as you can see, this puts a maximum on your > storage ratio. >
Hmmmmmmmmmm - - - - now how can I combine 'Hamming codes' and a raid array? TIA -- https://mail.python.org/mailman/listinfo/python-list
