Re: [Bacula-users] Ignore fileset changes
On Aug 23, 2011, at 8:43 AM, Adrian Reyer wrote: On Tue, Aug 23, 2011 at 01:38:23AM +0200, Olle Romo wrote: What I mean is that if I have a drive removed, run the job then attach the drive and run the job again, Bacula will do a full backup of the drive even if it previously had done an incremental backup on the same drive. Ideally I want it to just continue the incremental backup. Use 2 filesets and 2 jobs. One like you have now, but exclude the removable drive, The other only has the removable dribe and you only run it when the drive is there, e.g. checked by a pre-script. Regards, Adrian That will be the way to go. Problem is I have quite a few drives that come and go in different combinations. I still wish I could control that particular behavior. Thanks for the tip :) Best, Olle -- EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore fileset changes
On Wed, Aug 24, 2011 at 1:50 AM, Olle Romo oller...@gmail.com wrote: On Aug 23, 2011, at 8:43 AM, Adrian Reyer wrote: On Tue, Aug 23, 2011 at 01:38:23AM +0200, Olle Romo wrote: What I mean is that if I have a drive removed, run the job then attach the drive and run the job again, Bacula will do a full backup of the drive even if it previously had done an incremental backup on the same drive. Ideally I want it to just continue the incremental backup. Use 2 filesets and 2 jobs. One like you have now, but exclude the removable drive, The other only has the removable dribe and you only run it when the drive is there, e.g. checked by a pre-script. Regards, Adrian That will be the way to go. Problem is I have quite a few drives that come and go in different combinations. I still wish I could control that particular behavior. Thanks for the tip :) Best, Olle You can. As Adrian says you should be able to use a ClientRunBeforeJob directive. This will tell the client to run a script which should be a shell script that checks for the presence of a drive. If it does not detect the drive, exit with exit code 1 and the job will not run. If it does detect the drive, exit with exit code 0 and the backup job will run. I missed the previous portion of this thread, but I'm assuming this is a Linux client that is mounting a drive. If that's the case, then you're still functionally getting an incremental but because the directory was empty last time (when the drive was not mounted) but now has the entire drive's contents in it. The incremental job is dutifully backing up all changed data from the last incremental which happens to be all of the data on the drive now that Bacula can see it. Merely detecting the drive with a ClientRunBeforeJob script will solve that problem entirely. If you have many drives you interchange in this way, I would recommend making your script take an argument so you only have to maintain a single script that can check any drive you want as dictated by the job. If you do this I don't see why you would need 2 filesets and 2 jobs. It should work just fine with one of each unless that is addressing something I missed from previous emails. -- EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore fileset changes
On Tue, Aug 23, 2011 at 01:38:23AM +0200, Olle Romo wrote: What I mean is that if I have a drive removed, run the job then attach the drive and run the job again, Bacula will do a full backup of the drive even if it previously had done an incremental backup on the same drive. Ideally I want it to just continue the incremental backup. Use 2 filesets and 2 jobs. One like you have now, but exclude the removable drive, The other only has the removable dribe and you only run it when the drive is there, e.g. checked by a pre-script. Regards, Adrian -- LiHAS - Adrian Reyer - Hessenwiesenstraße 10 - D-70565 Stuttgart Fon: +49 (7 11) 78 28 50 90 - Fax: +49 (7 11) 78 28 50 91 Mail: li...@lihas.de - Web: http://lihas.de Linux, Netzwerke, Consulting Support - USt-ID: DE 227 816 626 Stuttgart -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore fileset changes
What I mean is that if I have a drive removed, run the job then attach the drive and run the job again, Bacula will do a full backup of the drive even if it previously had done an incremental backup on the same drive. Ideally I want it to just continue the incremental backup. At the moment I need to remember to remove the drive from the fileset then add it again when it's online. Sometimes I forget and.. well... I really wish I could control this behavior. It says in the manual this is deliberately so, in order to ensure the backup is correct, but I find it would be very helpful if I could switch this on or off. Thomas - the dynamic fileset looks interesting. I will try it. Thank you. All best, Olle On Aug 16, 2011, at 7:45 AM, Thomas Mueller wrote: Am Wed, 10 Aug 2011 09:43:05 +0200 schrieb Olle Romo: Thank you Uwe. I misunderstood the concept. Is there a way to have Bacula just continue incremental backup of a drive even if it was offline for one run? IMHO no, it will fail if a path is not available. you could create a sort of dynamic fileset e.g. create a fileset on the fly on the client. blog post I found (not using this feature this myself): http://machine-cycle.blogspot.com/2010/02/dynamic-bacula-filesets.html (don't forget to add Ignore FileSet Changes in the fileset and Accurate = yes in your job definition) - Thomas -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore fileset changes
Am Wed, 10 Aug 2011 09:43:05 +0200 schrieb Olle Romo: Thank you Uwe. I misunderstood the concept. Is there a way to have Bacula just continue incremental backup of a drive even if it was offline for one run? IMHO no, it will fail if a path is not available. you could create a sort of dynamic fileset e.g. create a fileset on the fly on the client. blog post I found (not using this feature this myself): http://machine-cycle.blogspot.com/2010/02/dynamic-bacula-filesets.html (don't forget to add Ignore FileSet Changes in the fileset and Accurate = yes in your job definition) - Thomas -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore fileset changes
Thank you Uwe. I misunderstood the concept. Is there a way to have Bacula just continue incremental backup of a drive even if it was offline for one run? All best, Olle On Aug 9, 2011, at 9:22 AM, Uwe Schuerkamp wrote: On Tue, Aug 09, 2011 at 06:07:41AM +0200, Olle Romo wrote: Hi All, 'Ignore Fileset Changes = yes' doesn't seem to have the effect I was hoping for... If a drive is missing on one run, the next time Bacula sees the drive it will proceed to backup the entire thing again. I'm sure there's an obvious way to avoid this but I can't seem to find it... Any suggestions warmly welcomed. Hello Olle, I think there's a slight misunderstanding in your concept of fileset changes. According to the docs, fileset changes affect the definition of filesets in the director config file as opposed to *actual* changes in the filesystem's you're backing up. If you're talking about removeable drives and such that are mounted somewhere in the directory, tree, I guess you should be looking at bacula crossing fs boundaries during backups (or not ;-) All the best, Uwe All best, Olle -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- uwe.schuerk...@nionex.net fon: [+49] 5242.91 - 4740, fax:-69 72 Hauptsitz: Avenwedder Str. 55, D-33311 Gütersloh, Germany Registergericht Gütersloh HRB 4196, Geschäftsführer: H. Gosewehr, D. Suda NIONEX --- Ein Unternehmen der Bertelsmann AG -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore fileset changes
On Tue, Aug 09, 2011 at 06:07:41AM +0200, Olle Romo wrote: Hi All, 'Ignore Fileset Changes = yes' doesn't seem to have the effect I was hoping for... If a drive is missing on one run, the next time Bacula sees the drive it will proceed to backup the entire thing again. I'm sure there's an obvious way to avoid this but I can't seem to find it... Any suggestions warmly welcomed. Hello Olle, I think there's a slight misunderstanding in your concept of fileset changes. According to the docs, fileset changes affect the definition of filesets in the director config file as opposed to *actual* changes in the filesystem's you're backing up. If you're talking about removeable drives and such that are mounted somewhere in the directory, tree, I guess you should be looking at bacula crossing fs boundaries during backups (or not ;-) All the best, Uwe All best, Olle -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- uwe.schuerk...@nionex.net fon: [+49] 5242.91 - 4740, fax:-69 72 Hauptsitz: Avenwedder Str. 55, D-33311 Gütersloh, Germany Registergericht Gütersloh HRB 4196, Geschäftsführer: H. Gosewehr, D. Suda NIONEX --- Ein Unternehmen der Bertelsmann AG -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore FileSet Changes NOT working as expected (BUG?)
Hello,
On 5/5/2007 11:41 AM, viashino wrote:
Hi,
I'm still trying to understand why my setup isn't working as expected.
I did some tests (thx Arno for pointing me) about the Ignore FileSet
Changes option.
This option should be no by default as stated on the manual.
Right.
Here what I did:
- I added the directive (forced to no) in the example configuration, and
set the File list to be on the client. [1]
- I deleted/purged every old job of my client
- I added one directory to /home/viashino/files (/etc/bacula) and did a
full backup. ( End time: 05-mag-2007 11:21:31)
- Added a second directory to /home/viashino/files (/var) and did an:
estimate level=Incremental listing (a snap is attached [2])
Well... the first, full, backup stored /etc/bacula. The incremental will
only sore what's changed.
I think I understand the problem: The Ignore Fileset Changes option
affects what is stored in the fileset resource. When inserting lines
through an external file, it does not watch the (dynamically) inserted
contents.
- I then did a backup (incremental) to be sure that it wasn't a problem
of the estimate (ok, this can sound silly, but I wanted to test
everything) but indeed the backup was wrong : it backuped only a cuple of
files (20 more or less).
Which ones? I suspect the ones from /var changed after the initial full
backup.
The Ignore FileSet Changes should compare the two include/exclude lists
(md5 hash, right?) and if the newest is different from the last, run a
full backup. Unfortunatly Bacula is not doing this.
Bacula compares the contents of the fileset resource, not the resulting
effective fileset.
When I added /var to my fileset and did the incremental, it backuped ONLY
the files that were modified after the backup, basing ONLY on the
time-stamp.
That's expected, as the time-stamp is the criterion by which Bacula
decides what to store.
I'm using Debian on both client and server, bacula version 2.0.3 (06 March
2007).
Also I'm facing the same problem on the Windows Client (that was the
starting point :D )
Now, is this a bug?
I doubt it... sounds like a feature request to me. Make Bacula consider
the effective fileset after including external data when checking for
fileset changes.
Unfortunately, I suspect that would break setups where the file list is
generated dynamically and might change formatting and order, but not the
actual contents.
Arno
Thank you for the help, If you need more information please tell me.
viashino
[1]##
FileSet {
Name = Full Set
Ignore FileSet Changes = no
Include {
Options {
signature = MD5
}
File = /home/viashino/files
}
Exclude {
File = /proc
File = /tmp
File = /.journal
File = /.fsck
}
}
[2]##
-rw-rw 1 mysqlmysql 100352 2007-05-05 11:21:30
/var/lib/mysql/bacula/Filename.MYI
-rw-rw 1 mysqlmysql59592 2007-05-05 11:21:30
/var/lib/mysql/bacula/Filename.MYD
-rw-rw 1 mysqlmysql 319488 2007-05-05 11:21:31
/var/lib/mysql/bacula/File.MYI
-rw-rw 1 mysqlmysql 439708 2007-05-05 11:21:31
/var/lib/mysql/bacula/File.MYD
-rw-rw 1 mysqlmysql 3072 2007-05-05 11:21:31
/var/lib/mysql/bacula/Job.MYI
-rw-rw 1 mysqlmysql 3040 2007-05-05 11:21:31
/var/lib/mysql/bacula/Job.MYD
-rw-rw 1 mysqlmysql 3072 2007-05-05 11:21:31
/var/lib/mysql/bacula/JobMedia.MYI
-rw-rw 1 mysqlmysql 650 2007-05-05 11:21:31
/var/lib/mysql/bacula/JobMedia.MYD
-rw-rw 1 mysqlmysql 4096 2007-05-05 11:21:31
/var/lib/mysql/bacula/Media.MYI
-rw-rw 1 mysqlmysql 576 2007-05-05 11:21:31
/var/lib/mysql/bacula/Media.MYD
-rw--- 1 bacula bacula 0 2007-05-05 11:21:55
/var/lib/bacula/OctOpus-Link-dir.conmsg
-rw-r- 1 bacula bacula 97261 2007-05-05 11:21:31
/var/lib/bacula/log
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
IT-Service Lehmann[EMAIL PROTECTED]
Arno Lehmann http://www.its-lehmann.de
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
Re: [Bacula-users] Ignore FileSet Changes NOT working as expected (BUG?)
Hi,
Thank you for answering.
So as you say the problem is that Bacula checks only the FileSet resource
and not the contents.
Good to know :D
I solved the problem making more or less what I expected, so my client is
saving md5 of the fileset content and comparing before starting the Backup.
Thanks again
On Tue, 08 May 2007 12:37:01 +0200, Arno Lehmann [EMAIL PROTECTED] wrote:
Hello,
On 5/5/2007 11:41 AM, viashino wrote:
Hi,
I'm still trying to understand why my setup isn't working as expected.
I did some tests (thx Arno for pointing me) about the Ignore FileSet
Changes option.
This option should be no by default as stated on the manual.
Right.
Here what I did:
- I added the directive (forced to no) in the example configuration, and
set the File list to be on the client. [1]
- I deleted/purged every old job of my client
- I added one directory to /home/viashino/files (/etc/bacula) and did a
full backup. ( End time: 05-mag-2007 11:21:31)
- Added a second directory to /home/viashino/files (/var) and did an:
estimate level=Incremental listing (a snap is attached [2])
Well... the first, full, backup stored /etc/bacula. The incremental will
only sore what's changed.
I think I understand the problem: The Ignore Fileset Changes option
affects what is stored in the fileset resource. When inserting lines
through an external file, it does not watch the (dynamically) inserted
contents.
- I then did a backup (incremental) to be sure that it wasn't a problem
of the estimate (ok, this can sound silly, but I wanted to test
everything) but indeed the backup was wrong : it backuped only a cuple
of
files (20 more or less).
Which ones? I suspect the ones from /var changed after the initial full
backup.
The Ignore FileSet Changes should compare the two include/exclude lists
(md5 hash, right?) and if the newest is different from the last, run a
full backup. Unfortunatly Bacula is not doing this.
Bacula compares the contents of the fileset resource, not the resulting
effective fileset.
When I added /var to my fileset and did the incremental, it backuped
ONLY
the files that were modified after the backup, basing ONLY on the
time-stamp.
That's expected, as the time-stamp is the criterion by which Bacula
decides what to store.
I'm using Debian on both client and server, bacula version 2.0.3 (06
March
2007).
Also I'm facing the same problem on the Windows Client (that was the
starting point :D )
Now, is this a bug?
I doubt it... sounds like a feature request to me. Make Bacula consider
the effective fileset after including external data when checking for
fileset changes.
Unfortunately, I suspect that would break setups where the file list is
generated dynamically and might change formatting and order, but not the
actual contents.
Arno
Thank you for the help, If you need more information please tell me.
viashino
[1]##
FileSet {
Name = Full Set
Ignore FileSet Changes = no
Include {
Options {
signature = MD5
}
File = /home/viashino/files
}
Exclude {
File = /proc
File = /tmp
File = /.journal
File = /.fsck
}
}
[2]##
-rw-rw 1 mysqlmysql 100352 2007-05-05 11:21:30
/var/lib/mysql/bacula/Filename.MYI
-rw-rw 1 mysqlmysql59592 2007-05-05 11:21:30
/var/lib/mysql/bacula/Filename.MYD
-rw-rw 1 mysqlmysql 319488 2007-05-05 11:21:31
/var/lib/mysql/bacula/File.MYI
-rw-rw 1 mysqlmysql 439708 2007-05-05 11:21:31
/var/lib/mysql/bacula/File.MYD
-rw-rw 1 mysqlmysql 3072 2007-05-05 11:21:31
/var/lib/mysql/bacula/Job.MYI
-rw-rw 1 mysqlmysql 3040 2007-05-05 11:21:31
/var/lib/mysql/bacula/Job.MYD
-rw-rw 1 mysqlmysql 3072 2007-05-05 11:21:31
/var/lib/mysql/bacula/JobMedia.MYI
-rw-rw 1 mysqlmysql 650 2007-05-05 11:21:31
/var/lib/mysql/bacula/JobMedia.MYD
-rw-rw 1 mysqlmysql 4096 2007-05-05 11:21:31
/var/lib/mysql/bacula/Media.MYI
-rw-rw 1 mysqlmysql 576 2007-05-05 11:21:31
/var/lib/mysql/bacula/Media.MYD
-rw--- 1 bacula bacula 0 2007-05-05 11:21:55
/var/lib/bacula/OctOpus-Link-dir.conmsg
-rw-r- 1 bacula bacula 97261 2007-05-05 11:21:31
/var/lib/bacula/log
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ignore FileSet Changes being ignored
Hi, On 11.11.2005 18:32, John Kodis wrote: I recently set up Bacula and did a full backup about a week ago, and a number of incremental since. I then added an additional directory to the exclude section of the file set resource, and as would be expected, this caused Bacula to upgrade the next backup from incremental to full. I don't want this to happen, so I canceled the job, shut down Bacula, added a Ignore FileSet Changes = yes directive to the fileset (which I understand is dangerous, and which will be removed shortly). After restarting Bacula and rerunning the just canceled job, Bacula still wants to upgrade this job to full. Is there anything else that I have to do to avoid this full backup, or any other way to work around the problem? No. I don't know if I learnt this from the manual, some release notes, or The Hard Way(TM) but the fisrt job after adding ignore fileset changes will be a full one. The reason is that the fingerprint of the fileset, which is stored in the catalog, is changed with adding the option, and thus this fileset is considered a new one. Arno -- John Kodis. --- SF.Net email is sponsored by: Tame your development challenges with Apache's Geronimo App Server. Download it for free - -and be entered to win a 42 plasma tv or your very own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- IT-Service Lehmann[EMAIL PROTECTED] Arno Lehmann http://www.its-lehmann.de --- SF.Net email is sponsored by: Tame your development challenges with Apache's Geronimo App Server. Download it for free - -and be entered to win a 42 plasma tv or your very own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
