On 11/1/26 09:45, Richard Bostrom wrote:
Hello Friends!
Is there a way of protecting against data-corruption? +10.000 files stored for
the long term.
* Mounting static file systems read-only, except when changes are needed
* Multiple copies on physically different media
* some kind of hash (checksum) to verify file integrity. Hash stored with all
backup copies.
* periodically verifying the hash
* periodically verifying read integrity of the disks -- ALL the disks,
including your extra copies (dd can do that).
Combined, that will pretty well cover you from all but hostile corruption risks
(and even then -- the multiple copies and multiple copies of the hash will make
hostile actions more difficult).
Sure...some fancy file systems claim to do a few of those steps, but if you do
all those things, you will be in better shape than any fancy file system by
itself.
The physical dd read test is pretty slick...
# dd if=/dev/rsdXc of=/dev/null bs=1m
(where "X" is your physical device you are checking).
If it returns with a zero return code, your entire disk surface was read successfully. If it
fails, you got problems. I'd recommend doing this on all important disks in your environment. In
OpenBSD, this seems to be non-destructive and can be run while file systems are mounted and active.
A few months ago, I tried this on a system which was showing some odd errors, and it showed me
that I had several bad spots on a disk under some not recently accessed files (but...yay backups!).
Thought, "wow, that's cool. Maybe I should write a script to automate this", did so,
and the very next machine I tossed it against ALSO showed bad spots (fortunately, in swap space, on
a machine that if it is swapping, there are bigger issues than a bad disk). I'll admit I was in a
bit of a panic there...how many disks did I have that had bad spots?? But turned out in my
"trusted production" systems, I just got lucky -- first two showed problems, none of the
rest.
Sometimes, deleting the damaged files, writing a "zero file" to the entire file
system
# dd if=/dev/zero of=/some/file/system/zerofile bs=1m
will force bad spots to be swapped out by the disk, and "repair" the problem,
maybe short-term, maybe until some unrelated failure, but the non-cheapskate solution is
to replace the disk, of course.
Nick.
