Chris Bennett wrote at about 08:35:31 +1030 on Wednesday, November 25, 2009: > Hi Jeff, > > > I'm very interested in this topic since I have been spending a lot of > > time developing my "shadowmountrsync" routines which attempt to > > automate the backup part of your approach outlined above. So far I > > have robustly implemented startup of shadows and backup of acls (using > > subinacl and/or getfacl). > > Great to hear from you - I was going to mention your script in my > first post, as it's a fantastic contribution to the problem I'm trying > to solve. The funny thing is I spent the better part of a weekend > working out how it all works _before_ I found your script, but what > you've put together has been a good reference for me nonetheless. > Thanks for the kind words... And I too am a newbie on Windows -- and frustrated by the complexity of what is required to get a full metal restore. Between the complexity of ntfs (with so many extra non-POSIX features) and the many special Windows files (like the registry hives), it is so much harder than Linux where you can do something as easy as 'cp -a /' or 'rsync' or 'tar' or any of dozens of other ways to get a full backup. Even worse, I haven't seen any good documentation of what needs to be done to back up an NTFS share and/or what needs to be done to properly back up the special Windows files that lie top of it.
As an aside, note that in addition to the things you mention, there are elements of the NTFS filesystem itself (even before you get to the Windows OS) that are not properly copied with 'rsync' (and hence also not with BackupPC). This includes things like reparse points (of which there are several varieties that differ across OS versions) which are only partially implemented in cygwin and may not be backed up and/or restored properly. Secondly, there are non-POSIX elements like NTFS alternate data streams (which are pretty cool but not consistent with POSIX so intentionally not supported by cygwin -- I had a thread over on cygwin on this). So, the bottom line is that a 100% reliable under all possible circumstances bare-metal backup using *nix-like tools may be almost impossible. That being said much of the above weirdness is not used if at all on standard Windows installations (for example junctions (a type of reparse point) were not really used prior to Vista and the version used in Vista is reportedly supported at least to some degree in cygwin). Alternate data streams are not even supported by Windows explorer and are not used in any common applications (though they are a great friend to rootkit writers since they can hide streams in files that look zero length). > > > 1. What is 'fileacl' and how is it better/worse than subinacl? > > (which seems to backup all the ACL info (vs. the limited getfacl > > capabilities) > > Good question - it's a tool I found in 2007 when I last had a look at > doing more then file-level backups with backuppc. > > http://www.gbordier.com/gbtools/fileacl.asp > > I had not heard/seen of subinacl or getfacl until I looked at your > work (hint: I'm not a Windows admin.. :) ) > > I'm willing to use whatever works, and fileacl claims save/restore > acl's comprehensively. > > I've tried both fileacl and subinacl with the same results. > I too only heard about 'subinacl' after first implementing 'getfacl' and then finding access weirdness that I couldn't address with 'getfacl' which led to me realize that there are other attributes out there not covered by 'getfacl' -- I had several posts on this on the cygwin list. getfacl only covers the POSIX acl's. This deficiency led me to 'subinacl' I would recommend 'subinacl' since it is supported by Microsoft and seems to get out all the ACL details (and also has some nice options to make batch changes for things like moving systems, changing machine/domain names, and changing SIDs). One thing to be careful about is that 'subinacl' like vshadow and dosdev can't be run directly from SSH (where USERNAME=SYSTEM). The problem is insidious in that 'subinacl' seems to run fine without error but all file ownership is replaced by: /owner =nt authority\system Interestingly, 'getfacl' was hacked by the cygwin folks to get around that limitation so it works even directly from SSH though as I noted above it misses the non-POSIX acl details. This requires a slight change from the posted version of my 'shadowmountrsync' script to make sure that 'subinacl' is run after the 'at' recursion. I hacked the change on my personal copy but have not had the time to do it right on the posted script (and there is a subtlety for the case where you ssh but don't want to use shadows) > I'm wondering what isn't restored by a file-level restore, permission > restore, and then a fixboot / fixmbr. Or whether it's something to do > with the change of 'hard disks' (in my case VMWare vmdk files). I seem to recall that changing disks requires some fiddling. > Or > whether the on-disk block location of some files is stored somewhere > and that breaks on the restore. Coming from a Linux background, I > don't know why it has to be so hard to understand how to perform a > full system recovery without forking out $$.. :) Me too -- it's amazing how hard it seems to be just to understand what needs to be done for a full backup let alone implement and test it. And in my opinion such complexity and obscurity just makes mistakes and bugs more likely even if you do finally implement it all. > > > 2. What does the "ntbackup of systemstate" add? > > If I understand that better, I would be happy to add it as an > > additional option to shadowmountrsync. > > Well, I don't know the details as it's all a little abstract, but > reading: > > http://technet.microsoft.com/en-us/library/cc785306(WS.10).aspx > > gives some hints on what it covers, and I'm just used to seeing it be > done with other backup tools. For some of it's components backed up > (e.g. AD), I think additional massaging is done on backup, and when > restored, additional unrolling/replaying of files onto the system > instead of just file-level recovery. But I could be way off there. I read about ntbackup of systemstate a while back. But I seem to recall that one of the key things it backs up are the registry hives which I do with shadow copy. So I wonder what the 'systemstate' adds to a full shadow copy (plus full subinacl acl backup). Specifically, the page you reference says: Windows XP Professional: The System State data includes only the registry, COM+ Class Registration database, files under Windows File Protection, and boot files." Note that the COM+ Class Registration database is stored under: C:\Windows\Registration which seems to be backed up correctly under my shadow script. So, I'm not sure what if anything it adds (at least in the non-Domain controller non-Cluster setting). > > Please keep me informed on your progress on getting bare-metal > > restores to work. I am very interested in getting this all working. > > My major aim is to do a backup/restore out of band without recovering > a base-os installed. Barring that, I want to work out what the > minimum amount of work is required before I can restore from my > backuppc files / permission dump and have the system functioning as it > was at the time of backup. That would be my goal too. > I hope someone out there with the right Windows knowledge can provide > some additional answers to the questions we are asking. I hope so but I wouldn't hold my breath. One alternative that I have been thinking about (but have neither the time nor knowledge to do) would be to adapt 'ntfsclone' with the '--metadata' and '--save-image' flags. The first flag just backs up the NTFS metada but not the file data and the second flag helps save image space. According to the manpage when compressed, the metada is typically 1-8MB. Customarily, it is used for forensics, but I wonder whether it could be combined with the raw file data from BackupPC to yield a complete byte-level accurate backup. One problem with ntfsclone is that as written, you need to run it from Linux and mount the drive... but conceptually the idea of having another program to back up the metada byte-by-byte while using the pooling of BackupPC to save the file data sounds like the ideal. In particular, accurate cloning of the metadata would make sure that you captured all the subtleties of the filesystem while pooling maximizes the efficiency of the data backup. ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ BackupPC-users mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
