On Friday 09 November 2018 18:00:41 Nathan Stratton Treadway wrote: > On Fri, Nov 02, 2018 at 13:00:38 -0400, Nathan Stratton Treadway wrote: > > On Fri, Nov 02, 2018 at 12:35:06 -0400, Gene Heskett wrote: > > > Fri Nov 02 03:42:06.085962877 2018: pid 13139: thd-0x9824e00: > > > driver: not updating because origsize or dumpsize is 0 Fri Nov 02 > > > 03:42:06.086158866 2018: pid 13139: thd-0x9824e00: driver: > > > Building type FILE header of 32768-32768 bytes with name='coyote' > > > disk='/usr/games' dumplevel=1 and blocksize=0 > > > Fri Nov 02 03:42:06.086317001 2018: pid 13139: thd-0x9824e00: > > > driver: Building type FILE header of 32768-32768 bytes with > > > name='coyote' disk='/usr/games' dumplevel=1 and blocksize=0 > > > Fri Nov 02 03:42:06.092807051 2018: pid 13139: thd-0x9824e00: > > > driver: driver: send-cmd time 2461.017 to taper0: FILE-WRITE > > > worker0-0 00-00130 /usr/dumps/20181102030105/coyote._usr_games.1 > > > coyote /usr/games 1 20181102030105 "" "" "" 1 "" "" "" "" 10 > > > > Ah, interesting, looks like this applies to level 1 dumps as well. > > What does "amadmin CONFIG info coyote /usr/games shop > > /usr/lib/amanda" output? > > Okay, I believe I have tracked down this "info on small DLEs not > saved" bug... > > > Summary: > > The short-ish summary is that in Amanda 3.4 and 3.5, any dump which > ends but being shorter than 1024 bytes long (after compression) is > treated as not having happened at all, as far as Amanda's recording of > "info" statistics goes. > > This is most significant for full dumps (i.e. of very small > partitions), because it causes Amanda to think that DLE has never been > dumped (or, that the last time it was dumped was on the final run made > using Amanda 3.3 or older), and thus it schedules that DLE for a full > dump on every run. > > However, the bug actually also affects incremental dumps (as we > discussed in the above-quoted message) -- so DLEs that don't change > much on a particular day and thus end up with very tiny incrementals > end up recording those dumps as having taken place on 1969/12/31 > rather than the day they actually occurred. > > > Neither of the above situations is "fatal" as far as preventing Amanda > from actually backup up your data, but for people (such as Gene) who > are effected, a workaround workaround is simply to make sure that the > dump on a particular day is at least 1024 bytes. > > For full dumps, you can do this just by creating a "dummy" file on the > otherwise-very-empty partition in question, using data that's already > compressed so that the dump file is still big enough after Amanda > compresses it. (In my tests, I just used the bytes off the front of > the amanda_3.4.2.orig.tar.gz file I happened to have sitting around.) > > (For incrementals the trick is to make sure there is enough changing > on the partition that the incremental dump is over the cutoff size; > the best way to do that will depend on what data is on that partition, > etc.) > > > > Internal details and history: > > The bug happens because the messages that the chunker and driver > processes use to communicate with each other specify the size of the > files transferred in integral units of "kb" (=1024 bytes), and thus > the size given for very small files is "0" -- but the code in the > driver that handles updating the info database has a specific check > for zero-length dumps and refuses to update the database in that case. > (This check is found in driverio.c:update_info_dumper() .) > > It appears that the bug has existed since Amanda 3.4, when the old > chunker.c version of the chunker program was replaced with a new Perl > version. > > Both server-src/chunker.c as found in Amanda 3.3 and > server-src/dumper.c as it exists in 3.5 take special care to round > "kb" value passed for files which are short-but-not-empty files up to > "1" -- but that logic was not implemented in the Perl chunker code > when it was created.. > > (Interestingly, if I am reading the old chunker.c code correctly, it > used to round up not just very-small-but-not-empty files to 1 kb, but > actually rounded the kb figure up to count any trailing partial > kilobytes at the end of the file... while the new program seems to > just ignore those trailing partial kilobytes. Presumably this > difference simply doesn't matter -- except for the when the size is > rounded down to zero.) > > > Proposed patch: > > This is where having an actual Amanda developer would be very handy... > but given that planner.c and old-chunker.c both have special handling > for small-but-not-empty files, I figured that adding a similar check > to the new Chunker implementation is probably the best fix for this > situation, and that hopefully doing so would be safe from unwanted > side-effects.... > > So, I edited the perl/Amanda/Chunker/Controller.pm to implement such a > check, as shown in the attached patch. I've been running with this > patch in place for a couple days now, and so far it seems to have > resolved the issue for me.... > > > Nathan I had to do some file renaming because you were saving the original, then it patched it withgout generating a new properly named file leaving it with the name you saved, but I finally got it to apply to 3.5.1, and its building now. We'll see how it runs later tonight. Thanks Nathan for playing bulldog. Its appreciated.
And amcheck is happy. :) Copyright 2018 by Maurice E. Heskett -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page <http://geneslinuxbox.net:6309/gene>
