Re: Directory tree traversal problem
On 07/19/16 23:40, Matthew Seaman wrote: > Ever since I upgraded my home system to: > > % freebsd-version -ku > 11.0-BETA1 > 11.0-BETA1 > > my nightly tarsnap backups have been behaving oddly. I'm seeing the > backup process take much longer, and it is generating a lot more network > traffic than previously. Could you try comparing two consecutive archives? In particular I'm wondering if ZFS is presenting files in a different order when tarsnap lists directories... that would be annoying and would definitely increase the amount of traffic because it would break deduplication of tar headers. > I'm also getting a lot of error messages like this (repeated for each of > the ZFSes on this system.) > > Jul 19 19:00:00 lucid-nonsense backupsnap-zfs.sh[41680]: Created > snapshots zroot@20160719-1900 and children > Jul 19 19:00:00 lucid-nonsense backupsnap-zfs.sh[41680]: Backing up / > Jul 19 19:06:49 lucid-nonsense backupsnap-zfs.sh[41680]: tarsnap: .: > Unable to continue traversing directory tree: Not a directory > Jul 19 19:06:49 lucid-nonsense backupsnap-zfs.sh[41680]: Backing up /usr > Jul 19 19:58:26 lucid-nonsense backupsnap-zfs.sh[41680]: tarsnap: .: > Unable to continue traversing directory tree: Not a directory That's strange. Do you get this error when backing up an empty filesystem? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Re: Directory tree traversal problem
On 07/21/16 06:16, Colin Percival wrote: > On 07/19/16 23:40, Matthew Seaman wrote: >> Ever since I upgraded my home system to: >> >> % freebsd-version -ku >> 11.0-BETA1 >> 11.0-BETA1 >> >> my nightly tarsnap backups have been behaving oddly. I'm seeing the >> backup process take much longer, and it is generating a lot more network >> traffic than previously. > > Could you try comparing two consecutive archives? In particular I'm > wondering if ZFS is presenting files in a different order when tarsnap > lists directories... that would be annoying and would definitely increase > the amount of traffic because it would break deduplication of tar headers. > >> I'm also getting a lot of error messages like this (repeated for each of >> the ZFSes on this system.) >> >> Jul 19 19:00:00 lucid-nonsense backupsnap-zfs.sh[41680]: Created >> snapshots zroot@20160719-1900 and children >> Jul 19 19:00:00 lucid-nonsense backupsnap-zfs.sh[41680]: Backing up / >> Jul 19 19:06:49 lucid-nonsense backupsnap-zfs.sh[41680]: tarsnap: .: >> Unable to continue traversing directory tree: Not a directory >> Jul 19 19:06:49 lucid-nonsense backupsnap-zfs.sh[41680]: Backing up /usr >> Jul 19 19:58:26 lucid-nonsense backupsnap-zfs.sh[41680]: tarsnap: .: >> Unable to continue traversing directory tree: Not a directory > > That's strange. Do you get this error when backing up an empty filesystem? > Answering the second bit first: yes, I do see that with a brand new empty ZFS: lucid-nonsense:/home/matthew:# zfs create zroot/empty lucid-nonsense:/home/matthew:# zfs snapshot zroot/empty@20160721-1011 lucid-nonsense:/home/matthew:# tarsnap -cf /empty@20160721-1011 -C /empty/.zfs/snapshot/20160721-1011 . tarsnap: .: Unable to continue traversing directory tree: Not a directory This entirely failed to create an archive. Interestingly, ls(1) says its the parent '..' which isn't a directory: lucid-nonsense:/home/matthew:# cd /empty/.zfs/snapshot/20160721-1011/ lucid-nonsense:zfs/snapshot/20160721-1011:# ls -la ls: ..: Not a directory total 1 drwxr-xr-x 2 root wheel 2 Jul 21 10:11 ./ Hmmm lucid-nonsense:zfs/snapshot/20160721-1011:# stat -f "%N: %HT%SY" /empty /empty/.zfs /empty/.zfs/snapshot /empty/.zfs/snapshot/20160721-1011 . .. /empty: Directory /empty/.zfs: Directory /empty/.zfs/snapshot: Directory /empty/.zfs/snapshot/20160721-1011: Directory .: Directory stat: ..: stat: Not a directory That's unexpected. I'll ask the guys on freebsd-fs@ about that. Matthew signature.asc Description: OpenPGP digital signature
Re: Directory tree traversal problem
On 07/21/16 02:31, Matthew Seaman wrote: > On 07/21/16 06:16, Colin Percival wrote: >> On 07/19/16 23:40, Matthew Seaman wrote: >>> Ever since I upgraded my home system to: >>> >>> % freebsd-version -ku >>> 11.0-BETA1 >>> 11.0-BETA1 >>> [...] >>> Jul 19 19:58:26 lucid-nonsense backupsnap-zfs.sh[41680]: tarsnap: .: >>> Unable to continue traversing directory tree: Not a directory >> >> That's strange. [..]. > > lucid-nonsense:zfs/snapshot/20160721-1011:# stat -f "%N: %HT%SY" > /empty /empty/.zfs /empty/.zfs/snapshot > /empty/.zfs/snapshot/20160721-1011 . .. > /empty: Directory > /empty/.zfs: Directory > /empty/.zfs/snapshot: Directory > /empty/.zfs/snapshot/20160721-1011: Directory > .: Directory > stat: ..: stat: Not a directory > > That's unexpected. I'll ask the guys on freebsd-fs@ about that. For the benefit of the list (and google searches): It sounds like this was a short-lived bug in FreeBSD, fixed a few days ago: https://lists.freebsd.org/pipermail/freebsd-fs/2016-July/023586.html -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Re: Directory tree traversal problem
On 21/07/2016 22:24, Colin Percival wrote: > On 07/21/16 02:31, Matthew Seaman wrote: >> On 07/21/16 06:16, Colin Percival wrote: >>> On 07/19/16 23:40, Matthew Seaman wrote: Ever since I upgraded my home system to: % freebsd-version -ku 11.0-BETA1 11.0-BETA1 [...] Jul 19 19:58:26 lucid-nonsense backupsnap-zfs.sh[41680]: tarsnap: .: Unable to continue traversing directory tree: Not a directory >>> >>> That's strange. [..]. >> >> lucid-nonsense:zfs/snapshot/20160721-1011:# stat -f "%N: %HT%SY" >> /empty /empty/.zfs /empty/.zfs/snapshot >> /empty/.zfs/snapshot/20160721-1011 . .. >> /empty: Directory >> /empty/.zfs: Directory >> /empty/.zfs/snapshot: Directory >> /empty/.zfs/snapshot/20160721-1011: Directory >> .: Directory >> stat: ..: stat: Not a directory >> >> That's unexpected. I'll ask the guys on freebsd-fs@ about that. > > For the benefit of the list (and google searches): It sounds like this was > a short-lived bug in FreeBSD, fixed a few days ago: > https://lists.freebsd.org/pipermail/freebsd-fs/2016-July/023586.html Yes, I can confirm that upgrading has fixed the problem and that I managed to run a backup overnight last night perfectly normally. Cheers, Matthew -- Dr Matthew J Seaman 1 Newland St, Eynsham, Witney, OXON, 0X29 4LB signature.asc Description: OpenPGP digital signature
