The following patch seems to fix bpc_attrib.c
-- libbackuppc-xs-perl-0.59.orig/bpc_attrib.c
+++ libbackuppc-xs-perl-0.59/bpc_attrib.c
@@ -567,10 +567,7 @@ static void setVarInt(uchar **bufPP, uch
{
uchar *bufP = *bufPP;
- if ( value < 0 ) {
- bpc_logErrf("setVarInt botch: got negative argument %ld;
- setting to 0\n", (long int)value);
- value = 0;
- }
+ value = (unsigned) value;
do {
uchar c = value & 0x7f;
NOTE: if you don't convert to 'unsigned', then the setVarInt routine
hangs. This is probably why you originally tested for a negative
number.
With this patch the new attrib file stores the negative mtime
similarly as an unsigned 32 bit integer.
Original v3 attrib entry
'MacWrite 5.0Scrap' => {
'uid' => 501,
'mode' => 33188,
'sizeDiv4GB' => 0,
'size' => 0,
'type' => 0,
'gid' => 501,
'sizeMod4GB' => 0,
'mtime' => 2212140496
},
v4 attrib entry after migrating v3 to v4:
'MacWrite 5.0Scrap' => {
'compress' => 3,
'digest' => 'd41d8cd98f00b204e9800998ecf8427e',
'gid' => 501,
'inode' => 12458250,
'mode' => 33188,
'mtime' => 2212140496,
'name' => 'MacWrite 5.0Scrap',
'nlinks' => 0,
'size' => 0,
'type' => 0,
'uid' => 501
}, },
Now, all that needs to be done is to change the attrib read functions
for v3 and v4 to read back a signed integer from the packed unsigned
integer that is stored.
[email protected] wrote at about 14:00:48 -0400 on Wednesday, April 10,
2019:
> Thanks Craig.
> I think the bugs affect multiple routines:
> 1. BackupPC_attribPrint print the mtime of v3 stored attribs as
> unsigned integers so that the 2's complement is displayed for
> negative mtimes (32-bit for v3, 64-bit for v4)
> i.e this bug affects both v3 and v4
>
> 2. BackupPC_migrateV3toV4, sets the mtime to 0 for negative mtime
> files
>
> 3. For V3, BackupPC_tarCreate (and presumably BackupPC_restore) sets the
> mtime
> of restored files to the unsigned value. Meaning that originally negative
> mtime's are set to the future!
>
> Interestingly, this bug is thus both a v3 and v4 bug.
> Surprising perhaps that it was never caught in v3 - i.e. that nobody ever had
> a negative mtime file (especially given that both macs and windows
> have an epoch before 1/1/1970)
>
> Jeff
>
> Craig Barratt wrote at about 23:30:34 -0700 on Tuesday, April 9, 2019:
> > Jeff,
> >
> > Thanks for the detective work and proposed fix. Yes, I assumed mtime was
> > unsigned, when in fact it might not be.
> >
> > Let me check into the various scenarios of v3/v4 read/write. Hopefully
> > I'll have time to push fixes this weekend.
> >
> > Craig
> >
> > On Mon, Apr 8, 2019 at 7:38 PM <[email protected]> wrote:
> >
> > > So, the problem seems to be that backuppc treats mtime as an unsigned
> > > integer -- meaning that any file created before Jan 1, 1970 is not
> > > representable.
> > >
> > > v3 represents treats the number as an unsigned integer and basically
> > > shows it's 2's complement (in 32 bits) - i.e. it's packed as an
> > > unsigned 32 bit integer ('N' type in Perl)
> > >
> > > v4 assumes a negative number is an error, sets it to zero, then
> > > defeating the purpose displays it as the 2's complement of 0! v4
> > > stores the number as a variable length unsigned integer.
> > >
> > > According to Wikipedia,
> > > The Unix time_t data type that represents a point in time is,
> on
> > > many
> > > platforms, a signed integer, traditionally of 32 bits (but see
> > > below),
> > > directly encoding the Unix time number as described in the
> > > preceding
> > > section. Being 32 bits means that it covers a range of about
> 136
> > > years
> > > in total. The minimum representable date is Friday 1901-12-13,
> > > and the
> > > maximum representable date is Tuesday 2038-01-19. One second
> > > after
> > > 03:14:07 UTC 2038-01-19 this representation will overflow.
> This
> > > milestone is anticipated with a mixture of amusement and
> > > dread—see
> > > year 2038 problem.
> > >
> > > In some newer operating systems, time_t has been widened to 64
> > > bits. This expands the times representable by approximately
> 293
> > > billion years in both directions, which is over twenty times
> the
> > > present age of the universe per direction.
> > >
> > > There was originally some controversy over whether the Unix
> > > time_t
> > > should be signed or unsigned. If unsigned, its range in the
> > > future
> > > would be doubled, postponing the 32-bit overflow (by 68
> > > years). However, it would then be incapable of representing
> times
> > > prior to the epoch. The consensus is for time_t to be signed,
> > > and this
> > > is the usual practice.
> > >
> > > Seems to me that would be best to continue to represent time as a 32 bit
> > > unsigned
> > > integer in v3 attrib files, represented using 2's complements, so that
> > > any mtime > 2^31 (about 68 years) would be considered as a negative
> > > time -- this seems to be the current implied convention.
> > >
> > > In v4, time would be written as 64 bit unsigned integer using
> > > SetVarInt where any mtime > 2^63 (about 292 billion years from now)
> > > would be considered a negative number written in 2's complements
> > >
> > > Specifically, rather than testing for the sign of 'value' in
> > > setVarInt, it might be better just to cast value to an unsigned 64-bit
> > > integer that is then variable-length encoded as is done currently.
> > >
> > > I am rusty on my 'c' but maybe just:
> > >
> > > static void setVarInt(uchar **bufPP, uchar *bufEnd, int64 value)
> > > {
> > > + unsigned int64 uvalue = value;
> > > uchar *bufP = *bufPP;
> > >
> > > - if ( value < 0 ) {
> > > - bpc_logErrf("setVarInt botch: got negative argument %ld;
> setting
> > > to 0\n", (long int)value);
> > > - value = 0;
> > > - }
> > > do {
> > > - uchar c = value & 0x7f;
> > > + uchar c = uvalue & 0x7f;
> > > value >>= 7;
> > > + uvalue >>= 7;
> > > - if ( value ) c |= 0x80;
> > > + if ( uvalue ) c |= 0x80;
> > > if ( bufP < bufEnd ) {
> > > *bufP++ = c;
> > > } else {
> > > bufP++;
> > > }
> > > - } while ( value );
> > > + } while ( uvalue );
> > > *bufPP = bufP;
> > > }
> > >
> > >
> > > Both v3 and V4 versions of BackupPC_attribPrint then should
> > > additionally be "fixed" to print mtime as a signed integer converting
> > > from a 32 or 64 bit unsigned integer depending on whether the attrib
> > > file was created with v3 or v4.
> > >
> > > [email protected] wrote at about 21:56:08 -0400 on Monday, April 8,
> > > 2019:
> > > > I should have said that the v3 case is 2's complement for 32 bit
> > > > integers while the v4 case is 2's complement for 64 bit integers.
> > > >
> > > > Also, the internal representation is clearly right in that both v3
> and
> > > > v4 are retrieving the right number, just displaying it incorrectly.
> > > >
> > > > - v3 is showing it as an unsigned rather than signed integer.
> > > > - v4 is realizing it is an unsigned integer but rejecting it as
> negative
> > > > and then erroneously setting it to zero (and topping on the cake
> > > > recording it as the 64 bit 2's complement of zero)
> > > >
> > > > [email protected] wrote at about 21:51:27 -0400 on Monday, April
> > > 8, 2019:
> > > > > OK, figured it out... VERY INTERESTING.
> > > > >
> > > > > The original file has an mtime of -2082826800 which corresponds to
> > > 5AM
> > > > > Friday, Jan 1, 1904.
> > > > > The the error message in question is actually *right* and not an
> > > > > error:
> > > > > setVarInt botch: got negative argument -2082826800; setting
> to 0
> > > > >
> > > > > i.e. it truly is a negative mtime.
> > > > >
> > > > > The reason for this odd time is that the file was created on a
> Mac128
> > > > > (circa 1985) and apparently, on old Macs (pre-PowerPC), time
> starts
> > > with midnight Jan
> > > > > 1, 1904 and runs until 2040. Now 5AM GMT is actually 12AM EST (my
> > > home
> > > > > timezone), so the file shows a 'default' old Mac start date of
> > > > > midnight Jan 1, 1904.
> > > > >
> > > > > Now for the interesting part.
> > > > > Both the v3 and v4 versions of BackupPC_attribPrint are actually
> > > wrong! 2 (different) bugs!!!!
> > > > >
> > > > > 1. v3 displays a positive mtime of 2212140496 which is is right
> only
> > > > > in the sense that it is the decimal equivalent of the 2's
> > > > > complement representation of -2082826800
> > > > >
> > > > > 2. v4 properly catches that the mtime is negative, but signals an
> > > > > error rather than writing it. it then proceeds however to
> record a
> > > > > wrong (and very large) mtime of 18446744071626724816, rather
> than
> > > > > 0. The number however is 2^64 which is the 2's complement of 0
> for
> > > > > 8 byte integers
> > > > >
> > > > >
> > > > > Fascinating. I learned a lot from nailing this one down.
> > > > > Now both the v3 and v4 bugs should presumably be corrected :)
> > > > >
> > > > > "" wrote at about 20:23:18 -0400 on Monday, April 8, 2019:
> > > > > > "" wrote at about 20:13:33 -0400 on Monday, April 8, 2019:
> > > > > >
> > > > > > Once again seems to be a problem with bpc_attrib.c:
> > > > > > BackupPC_attribPrint
> > > > > >
> > > > > > Using v3:
> > > > > > 'MacWrite 5.0Scrap' => {
> > > > > > 'uid' => 501,
> > > > > > 'mode' => 33188,
> > > > > > 'sizeDiv4GB' => 0,
> > > > > > 'size' => 0,
> > > > > > 'type' => 0,
> > > > > > 'gid' => 501,
> > > > > > 'sizeMod4GB' => 0,
> > > > > > 'mtime' => 2212140496
> > > > > > },
> > > > > >
> > > > > > Using v4:
> > > > > >
> > > > > > 'MacWrite 5.0Scrap' => {
> > > > > > 'compress' => 3,
> > > > > > 'digest' => '',
> > > > > > 'gid' => 501,
> > > > > > 'inode' => 0,
> > > > > > 'mode' => 33188,
> > > > > > 'mtime' => '18446744071626724816',
> > > > > > 'name' => 'MacWrite 5.0Scrap',
> > > > > > 'nlinks' => 0,
> > > > > > 'size' => 0,
> > > > > > 'type' => 0,
> > > > > > 'uid' => 501
> > > > > > },
> > > > > >
> > > > > > Clearly, v4 is messing up on reading 'mtime' which presumably
> is
> > > what
> > > > > > is causing the problem, consistent with the verbose output of
> > > > > > BackupPC_migrateV3toV4 (as more fully excertped below)
> > > > > >
> > > > > > > setVarInt botch: got negative argument -2082826800; setting
> to
> > > 0
> > > > > > > Wrote file MacWrite 5.0Scrap: type = 0, mode = 0100644,
> > > uid/gid = 501/501, size = 0, inode = 11616435, nlinks = 0, digest =
> > > 0xd41d8c..., bufUsed = 925
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Craig Barratt wrote at about 23:39:50 -0700 on Sunday, April
> > > 7, 2019:
> > > > > > > > Jeff,
> > > > > > > >
> > > > > > > > Please send me an attrib file that triggers that error.
> > > > > > > >
> > > > > > > > Craig
> > > > > > > >
> > > > > > >
> > > > > > > Here is the relevant verbose output from
> BackupPc_migrateV3toV4
> > > > > > > starting with the attrib file:
> > > > > > >
> > > > > > > bpc_poolRefDeltaUpdate(5a3631521955e47e179ea20d8a84ac5a, 1),
> > > count now 1
> > > > > > >
> > >
> bpc_attrib_dirWrite(/var/lib/backuppc/pc/consult/13.v4/./f%2f/fvar/fopt/fexecutor/fshare/fhome/fSystem
> > > Folder/attrib): dirPath =
> > >
> /var/lib/backuppc/pc/consult/13.v4/./f%2f/fvar/fopt/fexecutor/fshare/fhome/fSystem
> > > Folder, attribFileName = attrib, baseAttribFileName = attrib
> > > > > > >
> > >
> bpc_path_create(/var/lib/backuppc/pc/consult/13.v4/./f%2f/fvar/fopt/fexecutor/fshare/fhome/fSystem
> > > Folder)
> > > > > > > Wrote file Extensions: type = 5, mode = 040771, uid/gid =
> > > 0/1002, size = 4096, inode = 11616442, nlinks = 0, digest = 0x000000...,
> > > bufUsed = 4
> > > > > > > Wrote file %mac.rsrc: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 6753, inode = 11616437, nlinks = 0, digest = 0xc874b4...,
> > > bufUsed = 37
> > > > > > > Wrote file system.ard: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 0, inode = 11616455, nlinks = 0, digest = 0xd41d8c...,
> > > bufUsed = 85
> > > > > > > Wrote file %MacWrite 5.0Scrap: type = 0, mode = 0100644,
> > > uid/gid = 501/501, size = 198, inode = 11616432, nlinks = 0, digest =
> > > 0x85e52a..., bufUsed = 133
> > > > > > > Wrote file Excel Toolbars: type = 0, mode = 0100644,
> uid/gid =
> > > 501/501, size = 267, inode = 11616447, nlinks = 0, digest = 0x619077...,
> > > bufUsed = 191
> > > > > > > Wrote file %ParamRAM: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 198, inode = 11616457, nlinks = 0, digest = 0x5a3631...,
> > > bufUsed = 245
> > > > > > > Wrote file Printer: type = 0, mode = 0100640, uid/gid =
> > > 0/1002, size = 0, inode = 11616441, nlinks = 0, digest = 0xd41d8c...,
> > > bufUsed = 293
> > > > > > > Wrote file %system.ard: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 798, inode = 11616445, nlinks = 0, digest = 0xdcd6d0...,
> > > bufUsed = 338
> > > > > > > Wrote file %System: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 487415, inode = 11616434, nlinks = 0, digest =
> 0xa9c2a3...,
> > > bufUsed = 388
> > > > > > > Wrote file System: type = 0, mode = 0100660, uid/gid =
> 0/1002,
> > > size = 0, inode = 11616433, nlinks = 0, digest = 0xd41d8c..., bufUsed =
> 435
> > > > > > > Wrote file Browser: type = 0, mode = 0100640, uid/gid =
> > > 0/1002, size = 0, inode = 11616450, nlinks = 0, digest = 0xd41d8c...,
> > > bufUsed = 479
> > > > > > > Wrote file ParamRAM: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 20, inode = 11616443, nlinks = 0, digest = 0xcef2a5...,
> > > bufUsed = 524
> > > > > > > Wrote file %Printer: type = 0, mode = 0100640, uid/gid =
> > > 0/1002, size = 2279, inode = 11616438, nlinks = 0, digest = 0x07c53e...,
> > > bufUsed = 570
> > > > > > > Wrote file %Browser: type = 0, mode = 0100640, uid/gid =
> > > 0/1002, size = 119268, inode = 11616448, nlinks = 0, digest =
> 0x9fb0e6...,
> > > bufUsed = 617
> > > > > > > Wrote file %windows.rsrc: type = 0, mode = 0100660, uid/gid
> =
> > > 0/1002, size = 15917, inode = 11616451, nlinks = 0, digest =
> 0xd30945...,
> > > bufUsed = 665
> > > > > > > Wrote file %Excel Toolbars: type = 0, mode = 0100644,
> uid/gid
> > > = 501/501, size = 198, inode = 11616439, nlinks = 0, digest =
> 0x7c1511...,
> > > bufUsed = 717
> > > > > > > Wrote file windows.rsrc: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 0, inode = 11616453, nlinks = 0, digest = 0xd41d8c...,
> > > bufUsed = 772
> > > > > > > Wrote file mac.rsrc: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 0, inode = 11616440, nlinks = 0, digest = 0xd41d8c...,
> > > bufUsed = 822
> > > > > > > Wrote file Excel Settings (4): type = 0, mode = 0100644,
> > > uid/gid = 501/501, size = 0, inode = 11616436, nlinks = 0, digest =
> > > 0xd41d8c..., bufUsed = 868
> > > > > > > setVarInt botch: got negative argument -2082826800; setting
> to
> > > 0
> > > > > > > Wrote file MacWrite 5.0Scrap: type = 0, mode = 0100644,
> > > uid/gid = 501/501, size = 0, inode = 11616435, nlinks = 0, digest =
> > > 0xd41d8c..., bufUsed = 925
> > > > > > > Wrote file %godata.sav: type = 0, mode = 0100644, uid/gid =
> > > 501/501, size = 198, inode = 11616452, nlinks = 0, digest = 0x39c291...,
> > > bufUsed = 977
> > > > > > > Wrote file %Preferences: type = 0, mode = 0100660, uid/gid =
> > > 0/1002, size = 198, inode = 11616454, nlinks = 0, digest = 0x087760...,
> > > bufUsed = 1028
> > > > > > > Wrote file godata.sav: type = 0, mode = 0100644, uid/gid =
> > > 501/501, size = 1100, inode = 11616456, nlinks = 0, digest =
> 0xb247c3...,
> > > bufUsed = 1079
> > > > > > > Wrote file MacWrite 5.0Undo: type = 0, mode = 0100644,
> uid/gid
> > > = 501/501, size = 32, inode = 11616446, nlinks = 0, digest =
> 0x70bc8f...,
> > > bufUsed = 1129
> > > > > > > Wrote file %MacWrite 5.0Undo: type = 0, mode = 0100644,
> > > uid/gid = 501/501, size = 198, inode = 11616449, nlinks = 0, digest =
> > > 0xb5c763..., bufUsed = 1184
> > > > > > > Wrote file Preferences: type = 5, mode = 040771, uid/gid =
> > > 0/1002, size = 4096, inode = 11616431, nlinks = 0, digest = 0x000000...,
> > > bufUsed = 1241
> > > > > > >
> > > > > > >
> > > > > > > I am also attaching the relevant attrib file
> > > > > > > [DELETED ATTACHMENT attrib, Untyped binary data]
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > BackupPC-users mailing list
> > > > > [email protected]
> > > > > List:
> https://lists.sourceforge.net/lists/listinfo/backuppc-users
> > > > > Wiki: http://backuppc.wiki.sourceforge.net
> > > > > Project: http://backuppc.sourceforge.net/
> > > >
> > > >
> > > > _______________________________________________
> > > > BackupPC-users mailing list
> > > > [email protected]
> > > > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> > > > Wiki: http://backuppc.wiki.sourceforge.net
> > > > Project: http://backuppc.sourceforge.net/
> > >
>
>
> _______________________________________________
> BackupPC-users mailing list
> [email protected]
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
[email protected]
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
