Re: [Bacula-users] Vista 64 bit and sparse files
OK, I may have just found this. Running another 3rd party tool "windirstat" with admin privileges uncovered a bunch of files in the /tmp/bacula-restore/System Volume Information directory, about 34Gb to be exact. I have never excluded this dir on my 32-bit XP clients and have not had a problem and obviously there's something strange going on here because if I run the same tool on the production client with admin priv it reports /System Volume Information consuming practically no space. Very odd. Anyway, I will exclude these and run another backup this weekend and see how it goes. Thanks everyone. Mike On 3/29/2010 7:33 PM, James Harper wrote: >> >> Hi everyone, >> >> >> >> I am using Bacula 3.02 client and server to backup several machines on my >> >> home network. It has been working great. Recently, I added my first Vista >> >> machine which also happens to be 64-bit and now, although the backups >> >> complete, it takes hours and the statistics are nonsense. I can only >> >> guess that sparse files are at the root cause here, but cannot figure out >> >> how to track this down. >> >> > > > > A backup of a Windows computer uses the BackupRead API to generate the > > data to go to the SD. BackupRead reads a single stream of data that > > includes the file attributes, ownership, acls, data, Alternate Data > > Streams, and sparse information. Bacula shouldn't need to know or care > > if the file is sparse or not, it just backs up the data that BackupRead > > returns. > > > > I wouldn't expect that 'sparse=yes' would have any useful impact unless > > there is a bug in which case it will probably break things. > > > > Can you restore the job somewhere else (another windows machine) and see > > if the restored files match up with the backed up files? Look out for > > any restore errors that say that the file sizes don't match. > > > > James OK, what I ended up doing here was creating a new 64-bit Vista virtual machine from scratch and restored my production client to this VM in the /tmp/bacula-restores directory. Before the restore ran, I was using just a touch over 10Gb. When the restore was complete I had over half the disk showing used but only about one fourth showing allocated. The /tmp/bacula-restores directory shows about 9Gb in use which is about what I expect, but look at this output from TreeSize Free TreeSize Free Report, 4/1/2010 10:08 PM V 2.4 Drive: Local Disk (C:) Drive: C:\ Size: 107,503.0 MB Used: 59,869.3 MB Free: 47,633.7 MB 4096 Bytes per Cluster (NTFS) This Folder: Size: 22,316.2 MB Allocated: 22,594.4 MB Percent of Drive: 21 %Objects: 139,299 Wasted Space: 300.5 MB And if you don't want to believe that, here is what Windows chkdsk shows from a command prompt: Windows has checked the file system and found no problems. 110083071 KB total disk space. 61306064 KB in 124887 files. 64936 KB in 21539 indexes. 0 KB in bad sectors. 252575 KB in use by the system. 65536 KB occupied by the log file. 48459496 KB available on disk. 4096 bytes in each allocation unit. 27520767 total allocation units on disk. 12114874 allocation units available on disk. But yet, I do not see any actual file that is reporting a large size in Windows Explorer. 01-Apr 19:13 pendual-dir JobId 504: Bacula pendual-dir 3.0.2 (18Jul09): +01-Apr-2010 19:13:13 Build OS: i686-pc-linux-gnu ubuntu 8.04 JobId: 504 Job:RestoreFiles.2010-04-01_11.25.26_47 Restore Client: vtest-fd Start time: 01-Apr-2010 11:25:28 End time: 01-Apr-2010 19:13:13 Files Expected: 82,062 Files Restored: 81,207 Bytes Restored: 49,562,616,479 Rate: 1766.0 KB/s FD Errors: 0 FD termination status: OK SD termination status: OK Termination:Restore OK -- warning file count mismatch The restore completed OK other than a file count mismatch, but no errors about sizes or anything like that. And it thinks it restored 49Gb, which matches up pretty much exactly with the 59Gb I appear to be using now(49Gb restored +10Gb originally on the disk). Ideas? Thanks, Mike -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Vista 64 bit and sparse files
The mystery deepens. On 3/29/2010 7:33 PM, James Harper wrote: >> Hi everyone, >> >> I am using Bacula 3.02 client and server to backup several machines on my >> home network. It has been working great. Recently, I added my first Vista >> machine which also happens to be 64-bit and now, although the backups >> complete, it takes hours and the statistics are nonsense. I can only guess >> that sparse files are at the root cause here, but cannot figure out how to >> track this down. >> > > A backup of a Windows computer uses the BackupRead API to generate the > data to go to the SD. BackupRead reads a single stream of data that > includes the file attributes, ownership, acls, data, Alternate Data > Streams, and sparse information. Bacula shouldn't need to know or care > if the file is sparse or not, it just backs up the data that BackupRead > returns. > > I wouldn't expect that 'sparse=yes' would have any useful impact unless > there is a bug in which case it will probably break things. > > Can you restore the job somewhere else (another windows machine) and see > if the restored files match up with the backed up files? Look out for > any restore errors that say that the file sizes don't match. > > James OK, what I ended up doing here was creating a new 64-bit Vista virtual machine from scratch and restored my production client to this VM in the /tmp/bacula-restores directory. Before the restore ran, I was using just a touch over 10Gb. When the restore was complete I had over half the disk showing used but only about one fourth showing allocated. The /tmp/bacula-restores directory shows about 9Gb in use which is about what I expect, but look at this output from TreeSize Free TreeSize Free Report, 4/1/2010 10:08 PM V 2.4 Drive: Local Disk (C:) Drive: C:\ Size: 107,503.0 MB Used: 59,869.3 MB Free: 47,633.7 MB 4096 Bytes per Cluster (NTFS) This Folder: Size: 22,316.2 MB Allocated: 22,594.4 MB Percent of Drive: 21 %Objects: 139,299 Wasted Space: 300.5 MB And if you don't want to believe that, here is what Windows chkdsk shows from a command prompt: Windows has checked the file system and found no problems. 110083071 KB total disk space. 61306064 KB in 124887 files. 64936 KB in 21539 indexes. 0 KB in bad sectors. 252575 KB in use by the system. 65536 KB occupied by the log file. 48459496 KB available on disk. 4096 bytes in each allocation unit. 27520767 total allocation units on disk. 12114874 allocation units available on disk. But yet, I do not see any actual file that is reporting a large size in Windows Explorer. 01-Apr 19:13 pendual-dir JobId 504: Bacula pendual-dir 3.0.2 (18Jul09): +01-Apr-2010 19:13:13 Build OS: i686-pc-linux-gnu ubuntu 8.04 JobId: 504 Job:RestoreFiles.2010-04-01_11.25.26_47 Restore Client: vtest-fd Start time: 01-Apr-2010 11:25:28 End time: 01-Apr-2010 19:13:13 Files Expected: 82,062 Files Restored: 81,207 Bytes Restored: 49,562,616,479 Rate: 1766.0 KB/s FD Errors: 0 FD termination status: OK SD termination status: OK Termination:Restore OK -- warning file count mismatch The restore completed OK other than a file count mismatch, but no errors about sizes or anything like that. And it thinks it restored 49Gb, which matches up pretty much exactly with the 59Gb I appear to be using now(49Gb restored +10Gb originally on the disk). Ideas? Thanks, Mike -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Vista 64 bit and sparse files
> Hi everyone, > > I am using Bacula 3.02 client and server to backup several machines on my home > network. It has been working great. Recently, I added my first Vista machine > which also happens to be 64-bit and now, although the backups complete, it > takes hours and the statistics are nonsense. I can only guess that sparse > files are at the root cause here, but cannot figure out how to track this > down. > A backup of a Windows computer uses the BackupRead API to generate the data to go to the SD. BackupRead reads a single stream of data that includes the file attributes, ownership, acls, data, Alternate Data Streams, and sparse information. Bacula shouldn't need to know or care if the file is sparse or not, it just backs up the data that BackupRead returns. I wouldn't expect that 'sparse=yes' would have any useful impact unless there is a bug in which case it will probably break things. Can you restore the job somewhere else (another windows machine) and see if the restored files match up with the backed up files? Look out for any restore errors that say that the file sizes don't match. James -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Vista 64 bit and sparse files
Hi everyone, I am using Bacula 3.02 client and server to backup several machines on my home network. It has been working great. Recently, I added my first Vista machine which also happens to be 64-bit and now, although the backups complete, it takes hours and the statistics are nonsense. I can only guess that sparse files are at the root cause here, but cannot figure out how to track this down. Here is my fileset definition: FileSet { Name = fb-802-fileset Include { Options { signature = MD5 sparse = yes Exclude = yes IgnoreCase = yes checkfilechanges = yes # Exclude directories full of lots and lots of useless little files WildDir = "[A-Z]:/Users/*/Cookies" ... } File = "C:/" File = "C:/WINDOWS/system32/drivers/etc/hosts" File = "E:/" } I added the "sparse = yes" statement hoping it would help but it has not. The strange thing is that the "estimate" command does not show the excessive space consumed. And here is what a #list media pool=90m shows. Note that these are 120m DDS tapes with a native storage capacity of 4Gb so obviously this can not be correct. #list media pool=90m ... | 38 | 120m_1_2 | Full | 1 | 8,555,774,976 | 68 | 7,776,000 | 1 |2 | 1 | DDS-3 | 2010-03-01 14:59:13 | | 39 | 120m_1_3 | Full | 1 | 13,363,144,704 | 44 | 7,776,000 | 1 |3 | 1 | DDS-3 | 2010-03-10 22:02:52 | | 40 | 120m_1_4 | Full | 1 | 66,590,705,664 | 99 | 7,776,000 | 1 |4 | 1 | DDS-3 | 2010-03-19 20:08:02 | | 41 | 120m_1_5 | Full | 1 | 37,345,867,776 | 58 | 7,776,000 | 1 |5 | 1 | DDS-3 | 2010-03-26 17:05:10 | This is pretty typical of one of the backups I am talking about: #list jobid=457 +---+---+-+--+---+--++---+ | JobId | Name | StartTime | Type | Level | JobFiles | JobBytes | JobStatus | +---+---+-+--+---+--++---+ | 457 | fb-802-backup | 2010-03-23 17:30:03 | B| I |1,422 | 15,581,607,358 | T | +---+---+-+--+---+--++---+ # Yet here is the estimate which is about what I would expect. #estimate job=fb-802-backup fileset=fb-802-fileset level=Incremental storage=C1557A pool=90m Using Catalog "MyCatalog" Connecting to Client fb-802-fd at fb-802.mydomain.mrc:9102 2000 OK estimate files=1424 bytes=1,088,634,152 Doing an "estimate listing ..." does not really show anything unusual. The biggest files I can see in the list are related to my email at about 250Mb so I am not even sure how to identify the file(s) I might exclude if there is no way around the sparse file issue. Again, assuming this IS a sparse file issue which it really looks like it is to me. Any ideas??? Thanks, Mike -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Vista 64 - Bacula 2.4.2, VSS and symlink/junction issues
A little oversight on my bacula-dir.conf example: I do have the example exclusion that I included in my post, but a quick look will reveal that it won't help in the example instance. Here's my entire exclusion set from my VistaSet fileset: Options { WildDir = "c:/QuickenW/*/backu*" WildDir = "[a-zA-Z]:/tmp" WildDir = "[a-zA-Z]:/temp" WildDir = "[a-zA-Z]:/System Volume Information" WildDir = "c:/Windows" WildDir = "c:/Users/dennis/Local Settings/Microsoft/Windows/Temp*" WildDir = "c:/Users/dennis/Local Settings/Tem* WildDir = "c:/Users/All Users" WildDir = "c:/Users/*/Application Data/Application Data" WildDir = "c:/Users/*/AppData/Local/Application Data" WildFile = "[a-zA-Z]:/hiberfile.sys" WildFile = "[a-zA-Z]:/pagefile.sys" WildFile = "[a-zA-Z]:/Users/*/AppData/Local/Microsoft/Outlook/*.oab" WildFile = "[a-zA-Z]:/Users/*/Local*/Applicat*/Microsoft/Outlook/*.oab" WildDir = "c:/DRIVERS" exclude = yes } - Original Message - From: Bacula User To: bacula-users@lists.sourceforge.net Sent: Monday, October 27, 2008 3:59 PM Subject: Vista 64 - Bacula 2.4.2, VSS and symlink/junction issues Hello, Bacula Users. I apologize for adding yet another thread to this topic. I've seen a number of threads that make mention of these Vista problems, and they seem to drop off sometime around mid-2007. Yet, for me the issues persist, so I'm perplexed. If someone has a link to the definitive answer(s) to these issues, I'd appreciate your reply. For the client in question, I am using Backula version 2.4.2 on Vista Ultimate 64-bit. I've been using Bacula for a while now, and have come to enjoy its reliability and configurability. Unfortunately, I'm in an environment where I must also support Windows, including Microsoft's latest proof that they don't understand upward compatibility: Vista. Bacula seems to handle the various Linux flavors in the network, and even the XP client seems quite reliable. However, with Vista (especially Vista 64-bit) we are seeing a significant set of problems. The most costly issue (for me) is the monkey wrench that I started to see surface in this forum about a year and a half ago, where the "junctions" (provided by M$ to point directories to legacy directory names) cause the FD to bail out with messages like: 27-Oct 02:05 m1330-fd JobId 414: Could not open directory c:/Users/dennis/AppData/Local/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data: ERR=The name of the file cannot be resolved by the system. Not long after this message began to appear on internet, I saw a message (22-Jun-2007) from Kern Sibbald saying that the problem should be solved with a few lines of code. I've not found much of consequence on this topic in the sixteen months since that time, but the problem persists (at least for me). Have I missed some key version of the software, or pointer designed to help get past this error? (It is a big deal in spite of the following observation by Mr. Sibbald: "However, if I am not mistaken, it *is* backing up these junctions, and will probably restore them correctly...") For me, my backups fail consistently with the following: 27-Oct 02:17 m1330-fd JobId 414: Fatal error: Too many errors. 27-Oct 02:17 luke-sd JobId 414: Job write elapsed time = 00:11:54, Transfer rate = 7.516 M bytes/second By the way, exclusions do not seem to help resolve this (or maybe I don't know the proper trick). I have the following in my fileset, but to no avail: FileSet { Name = "VistaSet" Enable VSS = yes Include { Options { WildDir = "c:/Users/*/Application Data/Application Data" Exclude = Yes } File = "c:/Users/dennis/AppData/" File = "c:/pers-data/" Options { signature = MD5 compression = GZIP onefs = no } } } In spite of this exclusion, I still receive (thousands of) messages like the example from above. - - - - - - - - - - - - The other issue is VSS. I've seen threads that indicate this issue was solved long ago (cannot find an example at the moment though - sorry)... but the most recent mention I find of this is from August 8 this year, when Kern Sibbald indicated that a new build was ready - but then the trail runs cold. Is there any hope of regaining VSS support on 64-bit
[Bacula-users] Vista 64 - Bacula 2.4.2, VSS and symlink/junction issues
Hello, Bacula Users. I apologize for adding yet another thread to this topic. I've seen a number of threads that make mention of these Vista problems, and they seem to drop off sometime around mid-2007. Yet, for me the issues persist, so I'm perplexed. If someone has a link to the definitive answer(s) to these issues, I'd appreciate your reply. For the client in question, I am using Backula version 2.4.2 on Vista Ultimate 64-bit. I've been using Bacula for a while now, and have come to enjoy its reliability and configurability. Unfortunately, I'm in an environment where I must also support Windows, including Microsoft's latest proof that they don't understand upward compatibility: Vista. Bacula seems to handle the various Linux flavors in the network, and even the XP client seems quite reliable. However, with Vista (especially Vista 64-bit) we are seeing a significant set of problems. The most costly issue (for me) is the monkey wrench that I started to see surface in this forum about a year and a half ago, where the "junctions" (provided by M$ to point directories to legacy directory names) cause the FD to bail out with messages like: 27-Oct 02:05 m1330-fd JobId 414: Could not open directory c:/Users/dennis/AppData/Local/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data: ERR=The name of the file cannot be resolved by the system. Not long after this message began to appear on internet, I saw a message (22-Jun-2007) from Kern Sibbald saying that the problem should be solved with a few lines of code. I've not found much of consequence on this topic in the sixteen months since that time, but the problem persists (at least for me). Have I missed some key version of the software, or pointer designed to help get past this error? (It is a big deal in spite of the following observation by Mr. Sibbald: "However, if I am not mistaken, it *is* backing up these junctions, and will probably restore them correctly...") For me, my backups fail consistently with the following: 27-Oct 02:17 m1330-fd JobId 414: Fatal error: Too many errors. 27-Oct 02:17 luke-sd JobId 414: Job write elapsed time = 00:11:54, Transfer rate = 7.516 M bytes/second By the way, exclusions do not seem to help resolve this (or maybe I don't know the proper trick). I have the following in my fileset, but to no avail: FileSet { Name = "VistaSet" Enable VSS = yes Include { Options { WildDir = "c:/Users/*/Application Data/Application Data" Exclude = Yes } File = "c:/Users/dennis/AppData/" File = "c:/pers-data/" Options { signature = MD5 compression = GZIP onefs = no } } } In spite of this exclusion, I still receive (thousands of) messages like the example from above. - - - - - - - - - - - - The other issue is VSS. I've seen threads that indicate this issue was solved long ago (cannot find an example at the moment though - sorry)... but the most recent mention I find of this is from August 8 this year, when Kern Sibbald indicated that a new build was ready - but then the trail runs cold. Is there any hope of regaining VSS support on 64-bit Vista? Thank you for your time!!! Dennis- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Vista 64 and VSS
Bump and update. I have upgraded to 2.2.6 client but still having the same issues. Any assistance would be greatly appreciated. Thanks, Douglas Schmidt Mon, 08 Oct 2007 07:05:44 -0700 I'm attempting to backup a Vista 64 workstation with Bacula 2.2.4and getting errors for VSS on the machine. This is the first Vista 64 device I have attempted to use with Bacula. I am running multiple Vista 32 and XP clients with no errors. The error message: 06-Oct 23:30 Schmidt-hp8150-fd: Schmidt-HP8150.2007-10-06_23 .05.05 Warning: VSS was not initialized properly. VSS support is disabled. ERR=An attempt was made to reference a token that does not exist. Enable VSS = on is set in the fileset resource. In Windows Event Viewer/Windows Logs/Applications I get the following errors at the same time: Volume Shadow Copy Service error: The VSS Coordinator class is not registered. This may be caused due to a setup failure or as a result of an application's installer or uninstaller. Operation: Instantiating VSS server --- Volume Shadow Copy Service error: Unexpected error calling routine CoCreateInstance. hr = 0x80040154. Operation: Instantiating VSS server - I have uninstalled and reinstalled the client multiple times. I have also verified in Windows Services that all of the COM+ services are running. I Googled the Windows error messages and there is a fair bit of discussion out there regarding various backup applications and Vista 64. Most of it centers around Vista 64 not being capable of using the 32 bit requester. Is anyone else out there experiencing the same issues? Or does anyone have a workaround? For the Bacula development community: If there is a need for testers for Vista 32, Vista 64, XP or Linux, I am interested in assisting. Thanks, -- Douglas Schmidt -- Douglas Schmidt [EMAIL PROTECTED] - SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Vista 64
Does the Windows FD work on Vista 64? I'm getting VSS errors and from the postings on the internet it appears a number of backup applications have this issue. -- Douglas Schmidt [EMAIL PROTECTED] - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Vista 64 and VSS
I'm attempting to backup a Vista 64 workstation with Bacula 2.2.4and getting errors for VSS on the machine. This is the first Vista 64 device I have attempted to use with Bacula. I am running multiple Vista 32 and XP clients with no errors. The error message: 06-Oct 23:30 Schmidt-hp8150-fd: Schmidt-HP8150.2007-10-06_23 .05.05 Warning: VSS was not initialized properly. VSS support is disabled. ERR=An attempt was made to reference a token that does not exist. Enable VSS = on is set in the fileset resource. In Windows Event Viewer/Windows Logs/Applications I get the following errors at the same time: Volume Shadow Copy Service error: The VSS Coordinator class is not registered. This may be caused due to a setup failure or as a result of an application's installer or uninstaller. Operation: Instantiating VSS server --- Volume Shadow Copy Service error: Unexpected error calling routine CoCreateInstance. hr = 0x80040154. Operation: Instantiating VSS server - I have uninstalled and reinstalled the client multiple times. I have also verified in Windows Services that all of the COM+ services are running. I Googled the Windows error messages and there is a fair bit of discussion out there regarding various backup applications and Vista 64. Most of it centers around Vista 64 not being capable of using the 32 bit requester. Is anyone else out there experiencing the same issues? Or does anyone have a workaround? For the Bacula development community: If there is a need for testers for Vista 32, Vista 64, XP or Linux, I am interested in assisting. Thanks, -- Douglas Schmidt [EMAIL PROTECTED] - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users