backu...@kosowsky.org wrote at about 14:58:10 -0400 on Sunday, March 31, 2019:
>
> When I run 'BackupPC_migrateV3toV4' (v4.3.0), I get occasional errors
> of form:
> BackupPC_migrateV3toV4: can't read attribute file
> ./f%2fdata/fmedia/f0/fAndroid/fdata/fcom.ebay.redlaser/fcache/fhttp/attrib
>
> Plus an associated error:
> bpc_attrib_dirRead: got unreasonable file name length 852354
>
> Similarly if I try to read the attrib file directly using
> BackupPC_attribPrint, I get:
> BackupPC_attribPrint: cannot read attrib file
> <path-to-file>/f%2fdata/fmedia/f0/fAndroid/fdata/fcom.ebay.redlaser/fcache/fhttp/attrib
>
> HOWEVER, if I use the v3.1 version of BackupPC_attribPrint, the attrib file
> prints out just fine!!!
> SO, the issue seems to be with how v4.x reads attrib files
>
> And none of the filenames are longer than 255 characters...
> For example the longest is:
>
> fcache_http%253A%252F%252Fwww.swap.com%252Fi%252FM%252FAMIfv95PqwhrAhpZvHMI9wTeAMmp74SepSA1FcgE-HwOBvARxlhxJ2MNyBqHSNkxbLWiKa50yAM6dEAP7CA_DpByvhW4MB4wp6KPcmAzKpcUSgAliHCnZ4OTQ8NbA3xOUfehqKnpdFXq-xHdSJ3aU09RXP-0pfnFVDH9YnkSDLTv8IeTU3tbAvE-1000%254080.jpg
>
> In other cases, I get the same error even though the longest filename is
> less than 100 chars. So that is not the issue...
>
> NOTE: that the error is rare -- I migrated about 2 million pool files
> repeated across 100 backups... yet I only encountered about a dozen
> "unreadable" attrib files
> Also, the "got unreasonable file name length" error is always associated
> with the "can't read attribute file" error (and vice-versa)
>
> I tried to troubleshoot, but the problem seems to be in the
> BackupPC::XS:Attrib read C-code...
>
OK... I dug into the C-code a bit...
The problem occurs in bpc_attrib.c in the lines:
} else if ( magic == BPC_ATTRIB_TYPE_UNIX ) {
while ( bufP < buf + nRead ) {
uint fileNameLen;
char *fileName;
bpc_attrib_file *file;
int64 sizeDiv4GB;
uint type;
if ( nRead == sizeof(buf) && bufP > buf + nRead - 2 * BPC_MAXPATHLEN
&& read_more_data(&fd, buf, sizeof(buf), &nRead, &bufP,
attribPath) ) {
bpc_fileZIO_close(&fd);
printf("JJK: magic 2\n");
return -1;
}
fileNameLen = getVarInt(&bufP, buf + nRead);
if ( fileNameLen > 2 * BPC_MAXPATHLEN - 16 ) {
bpc_logErrf("bpc_attrib_dirRead: got unreasonable file name
length %d\n", fileNameLen);
bpc_fileZIO_close(&fd);
return -1;
}
For example, fileNameLen comes back as 852354 which is obviously a ridiculous
number.
I also played with the attrib file itself and found the following:
Specifically, when I break the attrib file into parts:
1. Every file is individually readable when broken into separate attrib files
with just one entry
2. Only certain combinations of files reconstructed into an attrib file are not
readable. For example, in one case, files 7-118 are not readable but files
6-117 or 8-118 are readable.
2. In v3.x is always readable.
I suspect that Perl is storing longer hashes in a way that is not readable by
the C-code in v4.x.
Happy to share the above examples if anyone is willing to help me
troubleshoot...
Jeff
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
