Time Machine and hard links
ACK! So I just found out that time machine does not backup hard links. Nor does it backup meta-data to permit restoring hard links. Repeat by: keybounceMBP:~ michael$ mkdir test-link keybounceMBP:~ michael$ cd test-link keybounceMBP:test-link michael$ echo file a > a keybounceMBP:test-link michael$ ln a b keybounceMBP:test-link michael$ ln b c keybounceMBP:test-link michael$ ln c d keybounceMBP:test-link michael$ ln d e keybounceMBP:test-link michael$ ls -li total 20 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 a 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 b 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 c 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 d 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 e keybounceMBP:test-link michael$ tmutil startbackup -b Total copied: 760.28 MB (797214702 bytes) Avg speed:1231.99 MB/min (21530541 bytes/sec) After restoring from time machine: keybounceMBP:~ michael$ ls -li test-link* test-link: total 20 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 a 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 b 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 c 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 d 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 e 'test-link (original)': total 20 35248827 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 a 35248828 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 b 35248829 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 c 35248830 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 d 35248831 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 e keybounceMBP:~ michael$ Note that the restores are all different files. This is on 10.9.5. Does this still happen (no hard-links after restoring) on 10.12? What is the best way to backup a file system? The goal is "perfect restoration" (including meta data / file structure, not just data restoration), with historical replication (so not just disk cloning)? --- Entertaining minecraft videos http://YouTube.com/keybounce ___ MacOSX-talk mailing list MacOSX-talk@omnigroup.com http://www.omnigroup.com/mailman/listinfo/macosx-talk
Time Machine and hard links
ACK! So I just found out that time machine does not backup hard links. Nor does it backup meta-data to permit restoring hard links. Repeat by: keybounceMBP:~ michael$ mkdir test-link keybounceMBP:~ michael$ cd test-link keybounceMBP:test-link michael$ echo file a > a keybounceMBP:test-link michael$ ln a b keybounceMBP:test-link michael$ ln b c keybounceMBP:test-link michael$ ln c d keybounceMBP:test-link michael$ ln d e keybounceMBP:test-link michael$ ls -li total 20 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 a 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 b 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 c 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 d 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 e keybounceMBP:test-link michael$ tmutil startbackup -b Total copied: 760.28 MB (797214702 bytes) Avg speed:1231.99 MB/min (21530541 bytes/sec) After restoring from time machine: keybounceMBP:~ michael$ ls -li test-link* test-link: total 20 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 a 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 b 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 c 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 d 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 e 'test-link (original)': total 20 35248827 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 a 35248828 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 b 35248829 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 c 35248830 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 d 35248831 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 e keybounceMBP:~ michael$ Note that the restores are all different files. This is on 10.9.5. Does this still happen (no hard-links after restoring) on 10.12? What is the best way to backup a file system? The goal is "perfect restoration" (including meta data / file structure, not just data restoration), with historical replication (so not just disk cloning)? --- Entertaining minecraft videos http://YouTube.com/keybounce ___ MacOSX-talk mailing list MacOSX-talk@omnigroup.com http://www.omnigroup.com/mailman/listinfo/macosx-talk
Re: Time Machine and hard links
> What is the best way to backup a file system? The goal is "perfect > restoration" (including meta data / file structure, not just data > restoration), with historical replication (so not just disk cloning)? Ok, minor nit: If you have a file system inside a sparse bundle, and backup that sparse bundle, then yes, your hard links are preserved. Sparse bundles don't work well when you are talking about really big volume (I tested ... sigh ... it was over 2 TB). And while you can't restore the individual files, I just realized that you could mount the sparse bundle itself read-only and restore from that (might need a bash command to mount the sparse bundle read only, never tried asking finder to do it.) (In my case, I have restored from a time machine backup long ago. At least now I know that all my hardlinks are broken. But my really old stuff from PPC days is on a sparse bundle ... go figure for luck.) --- Entertaining minecraft videos http://YouTube.com/keybounce ___ MacOSX-talk mailing list MacOSX-talk@omnigroup.com http://www.omnigroup.com/mailman/listinfo/macosx-talk
Re: Time Machine and hard links
It's more less common knowledge that Time Machine uses hard links to represent files in the backup set that have not been modified during this time period, so they are linked to the most recent actual data in a previous backup set to save space. It's also true that the Mac GUI gives a user absolutely no capability to create hard links in any context at all. They have to be created in terminal using the UNIX underpinnings directly. So it's not a big leap of faith to assume that the design of time machine supports backup only of "Mac standard" file layouts, and that when time machine sees a hard link in its data, it assumes that it and only it could've created it... so it treats it in its own standard fashion when restoring it. Bottom line is, you could submit this to bug reporter as an enhancement, but I don't think it will be accepted. > On Aug 13, 2017, at 12:02 PM, Michael wrote: > > ACK! > > So I just found out that time machine does not backup hard links. Nor does it > backup meta-data to permit restoring hard links. > > Repeat by: > > keybounceMBP:~ michael$ mkdir test-link > keybounceMBP:~ michael$ cd test-link > keybounceMBP:test-link michael$ echo file a > a > keybounceMBP:test-link michael$ ln a b > keybounceMBP:test-link michael$ ln b c > keybounceMBP:test-link michael$ ln c d > keybounceMBP:test-link michael$ ln d e > keybounceMBP:test-link michael$ ls -li > total 20 > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 a > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 b > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 c > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 d > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 e > keybounceMBP:test-link michael$ tmutil startbackup -b > Total copied: 760.28 MB (797214702 bytes) > Avg speed:1231.99 MB/min (21530541 bytes/sec) > > After restoring from time machine: > > keybounceMBP:~ michael$ ls -li test-link* > test-link: > total 20 > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 a > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 b > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 c > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 d > 35248494 4 -rw-r--r-- 5 michael admin 7 Aug 13 11:27 e > > 'test-link (original)': > total 20 > 35248827 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 a > 35248828 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 b > 35248829 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 c > 35248830 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 d > 35248831 4 -rw-r--r-- 1 michael admin 7 Aug 13 11:27 e > keybounceMBP:~ michael$ > > Note that the restores are all different files. > > This is on 10.9.5. Does this still happen (no hard-links after restoring) on > 10.12? > > What is the best way to backup a file system? The goal is "perfect > restoration" (including meta data / file structure, not just data > restoration), with historical replication (so not just disk cloning)? > > --- > Entertaining minecraft videos > http://YouTube.com/keybounce > > ___ > MacOSX-talk mailing list > MacOSX-talk@omnigroup.com > http://www.omnigroup.com/mailman/listinfo/macosx-talk ___ MacOSX-talk mailing list MacOSX-talk@omnigroup.com http://www.omnigroup.com/mailman/listinfo/macosx-talk
Re: Time Machine and hard links
On Aug 13, 2017, at 12:02 PM, Michael wrote: > > What is the best way to backup a file system? With APFS already in public beta on macOS, it might be a little late to be asking about best practices on HFS+... ___ MacOSX-talk mailing list MacOSX-talk@omnigroup.com http://www.omnigroup.com/mailman/listinfo/macosx-talk
Re: Time Machine and hard links
On Sun, Aug 13, 2017 at 3:02 PM, Michael wrote: > > So I just found out that time machine does not backup hard links. Nor does it > backup meta-data to permit restoring hard links. > > ... > > Note that the restores are all different files. There are at least two Radars for this. http://openradar.appspot.com/7569890 http://openradar.appspot.com/8005841 They're probably marked as duplicates of some other ticket in the real Radar. I suspect that the root is that hard links can't span volumes so there's no existing mechanism to easily recreate those links. On Mon, Aug 14, 2017 at 8:53 AM, David Schwartz wrote: > > With APFS already in public beta on macOS, it might be a little late to be > asking about best practices on HFS+… I expect that relatively soon[TM] (re: Higher Sierra, maybe Higherer Sierra), TimeMachine will see "version 2" that operates similar to ZFS replication and a whole slew of bugs will be marked "won't fix". -- arno s hautala/-| a...@alum.wpi.edu pgp b2c9d448 ___ MacOSX-talk mailing list MacOSX-talk@omnigroup.com http://www.omnigroup.com/mailman/listinfo/macosx-talk