Re: [Bacula-users] Any FileSet change determine a Full backup
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ralf Gross wrote: Troy Daniels schrieb: Did /tmp/bacula-test/dir2 contain files created/modified before job 234? No I created /tmp/bacula-test/dir2 after job 234. And if so, did Bacula back them up? Imagine this scenario: 1) /tmp/bacula-test/dir2 is created and files copied into it. 2) Full Job 234 runs and does a full backup of /tmp/bacula-test/dir1 3) More files are added to /tmp/bacula-test/dir2 4) Fileset is updated to include /tmp/bacula-test/dir2 5) Incr Job 235 runs. Will job 235 only backup files created/modified since the last full backup? Or is Bacula smart enough to recognise it's never backed this directory up at all before and include all of it's contents. Ok, now I did the following: 1.) * created /tmp/bacula-test/dir3/foo3 * didn't change the fileset * run an incr backup Terminated Jobs: JobId Level Files Bytes Status FinishedName 236 Incr 0 0 OK 12-Jul-07 11:11 x That's what I expected. 2.) * added /tmp/bacula-test/dir3/ to the fileset and reload the confg 237 Incr 0 0 OK 12-Jul-07 11:16 x The new files were not backed up! 3.) * created /tmp/bacula-test/dir5/foo5 * added /tmp/bacula-test/dir5/ to the fileset and reload the confg 238 Incr 2 0 OK 12-Jul-07 11:21 x It seems that files that were present at a previous backup are not backed up after adding the files to the fileset. So it looks like the final word on this is that it is OK to add new directories to a fileset (or, a more common example in my case, new filesystems) provided that they are really NEW directories. If I mkfs a filesystem and add it to my backup, ignore fileset changes ought to be OK. I suppose a fix could be that if you are newly adding a directory to a backup and do not wish to do a full backup, you should touch all of the files in that directory. The question is, what is the lightest touch one can do that will cause these files to be added. I would love to hear an explanation about this problem, really... I know that that is the way it is and that it might be a lot of work to change it, but it seems to the casual observer that if a directory is currently in the fileset (even if it was just added), and that directory contains files, and Bacula does NOT contain any version of them in the catalog, they should get backed up. Reliance upon filesystem timestamps is probably not a great idea because timestamps can be played with. Are they really necessary in a system that uses a catalog? - -- _ _ _ _ ___ _ _ _ |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Systems Programmer III |$| |__| | | |__/ | \| _| |[EMAIL PROTECTED] - 973/972.0922 (2-0922) \__/ Univ. of Med. and Dent.|IST/AST - NJMS Medical Science Bldg - C630 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGmyEDmb+gadEcsb4RAkslAKDIya2xGx+z/abYGjdiupOTo6yDxgCgkoHn 4aU4WXuwjZJYDRcodpErzns= =dMbw -END PGP SIGNATURE- begin:vcard fn:Ryan Novosielski n:Novosielski;Ryan org:UMDNJ;IST/AST adr;dom:MSB C630;;185 South Orange Avenue;Newark;NJ;07103 email;internet:[EMAIL PROTECTED] title:Systems Programmer III tel;work:(973) 972-0922 tel;fax:(973) 972-7412 tel;pager:(866) 20-UMDNJ x-mozilla-html:FALSE version:2.1 end:vcard - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
On Mon, 16 Jul 2007 03:40:51 -0400, Ryan Novosielski said: Ralf Gross wrote: Troy Daniels schrieb: Did /tmp/bacula-test/dir2 contain files created/modified before job 234? No I created /tmp/bacula-test/dir2 after job 234. And if so, did Bacula back them up? Imagine this scenario: 1) /tmp/bacula-test/dir2 is created and files copied into it. 2) Full Job 234 runs and does a full backup of /tmp/bacula-test/dir1 3) More files are added to /tmp/bacula-test/dir2 4) Fileset is updated to include /tmp/bacula-test/dir2 5) Incr Job 235 runs. Will job 235 only backup files created/modified since the last full backup? Or is Bacula smart enough to recognise it's never backed this directory up at all before and include all of it's contents. Ok, now I did the following: 1.) * created /tmp/bacula-test/dir3/foo3 * didn't change the fileset * run an incr backup Terminated Jobs: JobId Level Files Bytes Status FinishedName 236 Incr 0 0 OK 12-Jul-07 11:11 x That's what I expected. 2.) * added /tmp/bacula-test/dir3/ to the fileset and reload the confg 237 Incr 0 0 OK 12-Jul-07 11:16 x The new files were not backed up! 3.) * created /tmp/bacula-test/dir5/foo5 * added /tmp/bacula-test/dir5/ to the fileset and reload the confg 238 Incr 2 0 OK 12-Jul-07 11:21 x It seems that files that were present at a previous backup are not backed up after adding the files to the fileset. So it looks like the final word on this is that it is OK to add new directories to a fileset (or, a more common example in my case, new filesystems) provided that they are really NEW directories. If I mkfs a filesystem and add it to my backup, ignore fileset changes ought to be OK. I suppose a fix could be that if you are newly adding a directory to a backup and do not wish to do a full backup, you should touch all of the files in that directory. The question is, what is the lightest touch one can do that will cause these files to be added. I would love to hear an explanation about this problem, really... I know that that is the way it is and that it might be a lot of work to change it, but it seems to the casual observer that if a directory is currently in the fileset (even if it was just added), and that directory contains files, and Bacula does NOT contain any version of them in the catalog, they should get backed up. Reliance upon filesystem timestamps is probably not a great idea because timestamps can be played with. Are they really necessary in a system that uses a catalog? Bacula doesn't look at the files in the catalog when doing a Diff/Incr backup, only the date of the last backup compared to the mtime and ctime of each file. This is safe on unix, because (except for newly mounted filesystems) there is no API to set the ctime of a file. __Martin - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
Ryan Novosielski schrieb:
Has anyone done a test that verifies exactly what this does?
If I add a new directory to my fileset, and I have this directive turns
on, do the new files get backed up as part of the incremental?
I have personally been doing as Craig said -- adding new major fileset
changes near the full backup... but it seems to me that this is not
dangerous in all of the cases that I can think of... Here's one I am
just about to encounter:
1. Add new FS to machine
2. Add new FS to Bacula FileSet
3. Next day, IgnoreFileSetChanges = yes, does my new filesystem get
backed up or does it not?
I did this with 2 directories.
FileSet {
Name = -test
Ignore FileSet Changes = yes
Include {
Options {
signature = MD5
}
File = /tmp/bacula-test/dir1
File = /tmp/bacula-test/dir2 -- I added this after jobid 234, and did a
reload
}
}
Terminated Jobs:
JobId Level Files Bytes Status FinishedName
[...]
234 Full 2 0 OK 12-Jul-07 10:14
235 Incr 2 0 OK 12-Jul-07 10:15
Choose a query (1-17): 13
Enter JobId: 234
++--+
| path | name |
++--+
| /tmp/bacula-test/dir1/ | |
| /tmp/bacula-test/dir1/ | foo |
++--+
Choose a query (1-17): 13
Enter JobId: 235
++--+
| path | name |
++--+
| /tmp/bacula-test/dir2/ | |
| /tmp/bacula-test/dir2/ | foo2 |
++--+
Seems to be working.
Ralf
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
One question,
Did /tmp/bacula-test/dir2 contain files created/modified before job 234?
And if so, did Bacula back them up?
Imagine this scenario:
1) /tmp/bacula-test/dir2 is created and files copied into it.
2) Full Job 234 runs and does a full backup of /tmp/bacula-test/dir1
3) More files are added to /tmp/bacula-test/dir2
4) Fileset is updated to include /tmp/bacula-test/dir2
5) Incr Job 235 runs.
Will job 235 only backup files created/modified since the last full
backup? Or is Bacula smart enough to recognise it's never backed this
directory up at all before and include all of it's contents.
Based on the following snippet from the docs, I suspect the former:
We strongly recommend against
setting this directive to yes, since doing so may cause you to
have an incomplete set of backups.
Cheers,
Troy.
Ralf Gross wrote:
Ryan Novosielski schrieb:
Has anyone done a test that verifies exactly what this does?
If I add a new directory to my fileset, and I have this directive turns
on, do the new files get backed up as part of the incremental?
I have personally been doing as Craig said -- adding new major fileset
changes near the full backup... but it seems to me that this is not
dangerous in all of the cases that I can think of... Here's one I am
just about to encounter:
1. Add new FS to machine
2. Add new FS to Bacula FileSet
3. Next day, IgnoreFileSetChanges = yes, does my new filesystem get
backed up or does it not?
I did this with 2 directories.
FileSet {
Name = -test
Ignore FileSet Changes = yes
Include {
Options {
signature = MD5
}
File = /tmp/bacula-test/dir1
File = /tmp/bacula-test/dir2 -- I added this after jobid 234, and did a
reload
}
}
Terminated Jobs:
JobId Level Files Bytes Status FinishedName
[...]
234 Full 2 0 OK 12-Jul-07 10:14
235 Incr 2 0 OK 12-Jul-07 10:15
Choose a query (1-17): 13
Enter JobId: 234
++--+
| path | name |
++--+
| /tmp/bacula-test/dir1/ | |
| /tmp/bacula-test/dir1/ | foo |
++--+
Choose a query (1-17): 13
Enter JobId: 235
++--+
| path | name |
++--+
| /tmp/bacula-test/dir2/ | |
| /tmp/bacula-test/dir2/ | foo2 |
++--+
Seems to be working.
Ralf
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
CC'ing the list. Why doesn't reply reply to the list?? On Thu, July 12, 2007 10:15 am, Malcolm Hunter wrote: On Thu, July 12, 2007 10:00 am, Troy Daniels wrote: One question, Did /tmp/bacula-test/dir2 contain files created/modified before job 234? And if so, did Bacula back them up? I can't see how it could as it wasn't included. Imagine this scenario: 1) /tmp/bacula-test/dir2 is created and files copied into it. 2) Full Job 234 runs and does a full backup of /tmp/bacula-test/dir1 3) More files are added to /tmp/bacula-test/dir2 4) Fileset is updated to include /tmp/bacula-test/dir2 5) Incr Job 235 runs. Will job 235 only backup files created/modified since the last full backup? Or is Bacula smart enough to recognise it's never backed this directory up at all before and include all of it's contents. In my experience with backups generally, the best method is to include everything and specifically exclude what you don't want (i.e. pagefile.sys, temp dirs, etc). Filesets should be fairly static - you shouldn't need to keep modifying them everytime you add a directory. Based on the following snippet from the docs, I suspect the former: We strongly recommend against setting this directive to yes, since doing so may cause you to have an incomplete set of backups. Sounds like good advice. Regards, Malcolm -- Web Development, Technical Copy-Editing Proofreading KDE Proofreading Team KDE British English Translation Team http://l10n.kde.org/team-infos.php?teamcode=en_GB -- Web Development, Technical Copy-Editing Proofreading KDE Proofreading Team KDE British English Translation Team http://l10n.kde.org/team-infos.php?teamcode=en_GB - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
Troy Daniels schrieb: Did /tmp/bacula-test/dir2 contain files created/modified before job 234? No I created /tmp/bacula-test/dir2 after job 234. And if so, did Bacula back them up? Imagine this scenario: 1) /tmp/bacula-test/dir2 is created and files copied into it. 2) Full Job 234 runs and does a full backup of /tmp/bacula-test/dir1 3) More files are added to /tmp/bacula-test/dir2 4) Fileset is updated to include /tmp/bacula-test/dir2 5) Incr Job 235 runs. Will job 235 only backup files created/modified since the last full backup? Or is Bacula smart enough to recognise it's never backed this directory up at all before and include all of it's contents. Ok, now I did the following: 1.) * created /tmp/bacula-test/dir3/foo3 * didn't change the fileset * run an incr backup Terminated Jobs: JobId Level Files Bytes Status FinishedName 236 Incr 0 0 OK 12-Jul-07 11:11 x That's what I expected. 2.) * added /tmp/bacula-test/dir3/ to the fileset and reload the confg 237 Incr 0 0 OK 12-Jul-07 11:16 x The new files were not backed up! 3.) * created /tmp/bacula-test/dir5/foo5 * added /tmp/bacula-test/dir5/ to the fileset and reload the confg 238 Incr 2 0 OK 12-Jul-07 11:21 x It seems that files that were present at a previous backup are not backed up after adding the files to the fileset. RAlf - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
On Wed, 2007-07-11 at 12:59 -0500, Zeratul wrote: Hi Every time when I'm doing a change in the FileSet for a client, the next backup is upgraded to Full from Incremental. There is any posibility to avoid this behavior? Sometimes the changes are really small but the full backup takes a lot of space and time. Any idea will be appreciated. having recently experienced that same issue, I can answer that one...no Therefore, I only make changes to the FileSet on days that the 'Full' backup set is to be run and restart the bacula-dir daemon prior to the time. That behavior affects changes to the FileSet whether they are Include or Exclude selectors. -- Craig White [EMAIL PROTECTED] - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
this is one of the first lines in the chapter The FileSet Resource of the manual: Ignore FileSet Changes = yes|no Normally, if you modify the FileSet Include or Exclude lists, the next backup will be forced to a Full so that Bacula can guarantee that any additions or deletions are properly backed up. If this directive is set to yes, any changes you make to the FileSet Include or Exclude lists will be ignored and not cause Bacula to immediately perform a Full backup. The default is no, in which case, if you change the Include or Exclude, Bacula will force a Full backup to ensure that everything is properly backed up. It is not recommended to set this directive to yes. :) Craig White schrieb: On Wed, 2007-07-11 at 12:59 -0500, Zeratul wrote: Hi Every time when I'm doing a change in the FileSet for a client, the next backup is upgraded to Full from Incremental. There is any posibility to avoid this behavior? Sometimes the changes are really small but the full backup takes a lot of space and time. Any idea will be appreciated. having recently experienced that same issue, I can answer that one...no Therefore, I only make changes to the FileSet on days that the 'Full' backup set is to be run and restart the bacula-dir daemon prior to the time. That behavior affects changes to the FileSet whether they are Include or Exclude selectors. -- IC3S Information, Computer und Solartechnik AG Baekerbarg 6, 22889 Tangstedt, Germany Telefon+49.4109.555.121 Telefax+49.4109.555.57 E-Mail [EMAIL PROTECTED] Webhttp://www.ic3s.de/ Vorstand: Jan Behrmann (Vorsitzender), Stefan Fehlauer Aufsichtsrat: Sven Niklasson (Vorsitzender) Dresdner Bank (200 800 00), Kto. 0123 9582 00 Commerzbank (200 400 00), Kto. 8540 288 Amtsgericht Kiel HRB 7590 UST.IdNr. DE184532766 - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
from the docs: Ignore FileSet Changes = yes|no Normally, if you modify the FileSet Include or Exclude lists, the next backup will be forced to a Full so that Bacula can guarantee that any additions or deletions are properly saved. If this directive is set to yes, any changes you make to the FileSet Include or Exclude lists, will not force a Full during subsequent backups. The default is no, in which case, if you change the Include or Exclude, Bacula will force a Full backup to ensure that everything is properly backed up. We strongly recommend against setting this directive to yes, since doing so may cause you to have an incomplete set of backups. On Wed, 2007-07-11 at 11:04 -0700, Craig White wrote: On Wed, 2007-07-11 at 12:59 -0500, Zeratul wrote: Hi Every time when I'm doing a change in the FileSet for a client, the next backup is upgraded to Full from Incremental. There is any posibility to avoid this behavior? Sometimes the changes are really small but the full backup takes a lot of space and time. Any idea will be appreciated. having recently experienced that same issue, I can answer that one...no Therefore, I only make changes to the FileSet on days that the 'Full' backup set is to be run and restart the bacula-dir daemon prior to the time. That behavior affects changes to the FileSet whether they are Include or Exclude selectors. - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Any FileSet change determine a Full backup
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Has anyone done a test that verifies exactly what this does? If I add a new directory to my fileset, and I have this directive turns on, do the new files get backed up as part of the incremental? I have personally been doing as Craig said -- adding new major fileset changes near the full backup... but it seems to me that this is not dangerous in all of the cases that I can think of... Here's one I am just about to encounter: 1. Add new FS to machine 2. Add new FS to Bacula FileSet 3. Next day, IgnoreFileSetChanges = yes, does my new filesystem get backed up or does it not? Julien wrote: from the docs: Ignore FileSet Changes = yes|no Normally, if you modify the FileSet Include or Exclude lists, the next backup will be forced to a Full so that Bacula can guarantee that any additions or deletions are properly saved. If this directive is set to yes, any changes you make to the FileSet Include or Exclude lists, will not force a Full during subsequent backups. The default is no, in which case, if you change the Include or Exclude, Bacula will force a Full backup to ensure that everything is properly backed up. We strongly recommend against setting this directive to yes, since doing so may cause you to have an incomplete set of backups. On Wed, 2007-07-11 at 11:04 -0700, Craig White wrote: On Wed, 2007-07-11 at 12:59 -0500, Zeratul wrote: Hi Every time when I'm doing a change in the FileSet for a client, the next backup is upgraded to Full from Incremental. There is any posibility to avoid this behavior? Sometimes the changes are really small but the full backup takes a lot of space and time. Any idea will be appreciated. having recently experienced that same issue, I can answer that one...no Therefore, I only make changes to the FileSet on days that the 'Full' backup set is to be run and restart the bacula-dir daemon prior to the time. That behavior affects changes to the FileSet whether they are Include or Exclude selectors. - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - -- _ _ _ _ ___ _ _ _ |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Systems Programmer III |$| |__| | | |__/ | \| _| |[EMAIL PROTECTED] - 973/972.0922 (2-0922) \__/ Univ. of Med. and Dent.|IST/AST - NJMS Medical Science Bldg - C630 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGlaEPmb+gadEcsb4RAs6NAJ9PNAAdZR7e/w76Ggz5k/CvFoM0hACdHK5N WsNJFTsDHeY+O/sMCMQ8IJ4= =/zbJ -END PGP SIGNATURE- begin:vcard fn:Ryan Novosielski n:Novosielski;Ryan org:UMDNJ;IST/AST adr;dom:MSB C630;;185 South Orange Avenue;Newark;NJ;07103 email;internet:[EMAIL PROTECTED] title:Systems Programmer III tel;work:(973) 972-0922 tel;fax:(973) 972-7412 tel;pager:(866) 20-UMDNJ x-mozilla-html:FALSE version:2.1 end:vcard - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
