Hi Martin,
I managed to get back on my feet: there were several issues that made
things difficult to understand, but you helped me a lot.
Thank you so much !
And thanks also to everyone who spent time on my problem !
Best,
Sam
Le ven. 13 déc. 2024 à 19:05, Martin Simmons <mar...@lispworks.com> a
écrit :
> Using 'o' in the fileset accurate option has caused problems with restore
> for
> another user on the list recently. I don't know if it could also break
> estimate. Have you used the 'o' option in previous incremental backups?
>
> Accurate backups assume that the full path to each file is the same as
> before.
> Has /path/to/ARCHIVES (the root of the backup) changed since previous
> backups?
> Have you renamed/moved any files or directories within the files being
> backed
> up?
>
> You could try to find more info about some file that is being included
> unexpectedly by:
>
> 1. Add the listing option to the two estimate commands and record the
> outputs
> somewhere.
>
> 2. Find a file in the accurate=yes listing that doesn't appear in the
> accurate=no listing.
>
> 3. Run this SQL command:
>
> SELECT DISTINCT Job.JobId,StartTime AS JobStartTime,VolumeName, File.lstat
> FROM Job,Path,File,Media,JobMedia,Client
> WHERE File.JobId=Job.JobId
> AND File.FileIndex > 0
> AND Path.Path='%1'
> AND File.FileName='%2'
> AND Client.Name='%3'
> AND Path.PathId=File.PathId
> AND JobMedia.JobId=Job.JobId
> AND JobMedia.MediaId=Media.MediaId
> AND Client.ClientId=Job.ClientId
> ORDER BY Job.StartTime DESC LIMIT 1;
>
> where %1 is the full path to the directory of the file with trailing
> slash, %2
> is the filename of the file and %3 is the Bacula client name (lto8-fd).
>
> 4. Run the shell commands:
>
> stat /full/path/to/filename
>
> stat -t /full/path/to/filename
>
> __Martin
>
>
> >>>>> On Wed, 11 Dec 2024 11:37:19 +0100, Samuel Zaslavsky said:
> >
> > Hello everyone !
> >
> > Any ideas for solving this problem?
> > I'm stuck here...
> >
> > Thanks a lot for your help,
> >
> > Best,
> >
> > Sam
> >
> >
> > Le ven. 6 déc. 2024 à 12:38, Samuel Zaslavsky <s...@w4tch.tv> a écrit :
> >
> > > Hello all,
> > >
> > > Sorry for this late reply. (According to Murphy's law, our Bacula
> server
> > > failed, and it took some time to figure out that the SAS card was
> dead, and
> > > get another one ...)
> > >
> > > I did some tests, and there's something new :
> > > In fact, the fileset seems to be OK, and the Incremental "could"
> resume...
> > >
> > > If I do an estimate without accurate, I get around 18K files / 3To ,
> which
> > > looks like "OK" for incremental :
> > >
> > > *estimate accurate=no level=incremental job=BackupARCHIVES
> > > Using Catalog "MyCatalog"
> > > Connecting to Client lto8-fd at localhost:9102
> > > 2000 OK estimate files=17,949 bytes=3,465,825,854,672
> > >
> > > But with accurate=yes , I get 530K files / 23To, which is (almost ? see
> > > below) a FULL backup :
> > >
> > > *estimate accurate=yes level=incremental job=BackupARCHIVES
> > > Using Catalog "MyCatalog"
> > > Connecting to Client lto8-fd at localhost:9102
> > > 2000 OK estimate files=530,779 bytes=23,005,947,679,372
> > >
> > > So I have tried to estimate my job with acurate=true and several
> options
> > > for accurate in the fileset :
> > >
> > > FileSet {
> > > Name = "ARCHIVES"
> > > Ignore FileSet Changes=yes
> > > Include {
> > > Options {
> > > signature=MD5
> > > *#I have tried many accurate options here : mcso5,
> mso5,
> > > so5,M,m,o5,sm,s, or nothing (line commented)*
> > > accurate=s
> > > mtimeonly=yes
> > > #Verify = pin5
> > > }
> > > File = /path/to/ARCHIVES
> > > }
> > >
> > > Exclude {
> > > File = "/path/to/ARCHIVES/#recycle"
> > > }
> > > }
> > >
> > > But all my attempts with Job accurate=yes give me almost the same
> result -
> > > but slightly different sometimes ( could be 531,057 files, or 530,719
> > > or 530,779 files, depending on the accurate option chosen...)
> > >
> > > As I understand it, accurate=s in the Fileset definition should tell
> > > Bacula that a file with "same path, same size" shouldn't be backed up
> > > again...
> > > But this isn't the case and my understanding is clearly wrong...
> > > And as I understand it also, accurate=no would lead, in case of
> restore,
> > > to restore even suppressed and moved files, which is not what I want.
> > >
> > > So I am stuck : resume incremental backup without accurate option, or
> > > restart a full backup with accurate option. Or understand better
> things ...
> > > :)
> > > Can anyone help me there ?
> > >
> > > Thanks a lot !
> > >
> > > Sam
> > >
> > >
> > > Le mar. 12 nov. 2024 à 17:18, Radosław Korzeniewski <
> > > rados...@korzeniewski.net> a écrit :
> > >
> > >> Hello,
> > >>
> > >> wt., 12 lis 2024 o 12:00 Samuel Zaslavsky <s...@w4tch.tv> napisał(a):
> > >>
> > >>> So, precisely, how does Bacula "decides" a fileset is different from
> > >>> another one ?
> > >>>
> > >>
> > >> Bacula is checking the fileset changes based on a fileset content.
> > >>
> > >>
> > >>> Where is this information stored in the database ?
> > >>>
> > >>
> > >> It is a fileset table.
> > >>
> > >>
> > >>> How could I trick bacula to believe that all previous jobs have been
> > >>> made with the new fileset : This would be OK for me !
> > >>> My idea was that it would be using the fileset.md5 field (or
> createtime
> > >>> ?)
> > >>>
> > >>
> > >> From what I know Bacula is checking the md5 column for this purpose.
> So,
> > >> if your new fileset md5 will be the same of which is already saved
> then
> > >> Bacula won't have any traces it is a different fileset.
> > >>
> > >>
> > >>>
> > >>> So, a few related questions : When will bacula add another entry in
> the
> > >>> fileset database table ?
> > >>>
> > >>
> > >> If your current configuration includes a selected fileset and md5 does
> > >> not match the one saved in the database then Bacula will create a new
> entry.
> > >>
> > >>
> > >>> It seems it's not when doing an estimate...
> > >>> So, should I start ( and abort quickly) a dummy job using the "new"
> > >>> fileset, to see the "new" fileset in the database ? And then try to
> trick
> > >>> Bacula into thinking the old one is the new one ?
> > >>>
> > >>
> > >> Yes, it sounds good to me.
> > >>
> > >>
> > >>> If this is a pertinent approach, how exactly do that ?
> > >>> For example, where in the database is the information that a "file"
> > >>> belongs to some files set ?
> > >>>
> > >>
> > >> Database relation.
> > >> bacula=# \d file
> > >> Table "public.file"
> > >> Column | Type | Collation | Nullable | Default
> > >>
> > >>
> > >>
> -----------+----------+-----------+----------+--------------------------------------
> > >> fileid | bigint | | not null |
> > >> nextval('file_fileid_seq'::regclass)
> > >> jobid | integer | | not null |
> > >> ...
> > >> bacula=# \d job
> > >> Table "public.job"
> > >> Column | Type | Collation |
> Nullable |
> > >> Default
> > >>
> > >>
> -------------------+-----------------------------+-----------+----------+------------------------------------
> > >> jobid | integer | | not
> null |
> > >> nextval('job_jobid_seq'::regclass)
> > >> filesetid | integer | |
> |
> > >> 0
> > >> ...
> > >> bacula=# \d fileset
> > >> Table "public.fileset"
> > >> Column | Type | Collation | Nullable |
> > >> Default
> > >>
> > >>
> ------------+-----------------------------+-----------+----------+--------------------------------------------
> > >> filesetid | integer | | not null |
> > >> nextval('fileset_filesetid_seq'::regclass)
> > >>
> > >> Is it in the file.lstat column ? If yes, how is it "encoded" here ?
> > >>>
> > >>
> > >> This column is a "direct" serialisation of lstat(2), struct stat {};
> > >> buffer. It is (to some extent) OS dependent.
> > >>
> > >>
> > >>>
> > >>> Basically, I need to understand how the relationship between fileset
> > >>> conf <--> fileset in database <--> files in fileset is handled...
> > >>>
> > >>>
> > >> When Bacula detects a fileset change, because the md5 is different
> then
> > >> the one saved in the database then it creates a new entry. Then a job
> will
> > >> get a new filesetid as a reference for job entry in the database and
> all
> > >> files saved will get a jobid reference.
> > >>
> > >>
> > >>> Or maybe I am completely wrong here ?
> > >>>
> > >>
> > >> You are just asking questions. You can't be wrong asking questions.
> > >>
> > >>
> > >>> The filesystem to be backed up is mounted as NFS share on the bacula
> > >>> host.
> > >>>
> > >>
> > >> Any chance you can access this filesystem directly without mounting
> it on
> > >> the remote host? It would be better.
> > >>
> > >>
> > >>> Is it possible that the reboot/remount changed a key parameter for
> > >>> Bacula ?
> > >>>
> > >>
> > >> Well. Let's assume your first mount point is: /mnt/data. Then you
> reboot
> > >> and the new mount point is /mnt/data1. Then it makes a huge
> difference for
> > >> Bacula, for sure.
> > >>
> > >>
> > >>> So that Bacula recognizes well the filesystem as the old one,
> > >>>
> > >>
> > >> It doesn't matter. What matters are some metadata for the file, i.e.
> > >> modify time. You can configure what metadata should be verified in
> this
> > >> case. But you can't force Bacula to not backup a file which does not
> exist
> > >> at the previous backup jobs.
> > >>
> > >>
> > >>> but doesn't recognize any file anymore ? ( Remember that fileset
> name,
> > >>> and all paths in the fileset configuration are the same as before ! )
> > >>>
> > >>
> > >> I'm pretty sure your fileset content is different then the one used
> > >> before, so Bacula wants to create a full backup.
> > >>
> > >> best regards
> > >> --
> > >> Radosław Korzeniewski
> > >> rados...@korzeniewski.net
> > >>
> > >
> >
>
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
