Re: [Bacula-users] Detailed list/info about files excluded from the job run

2021-11-21 Thread Shaligram Bhagat, Yateen (Nokia - IN/Bangalore)
Hi Bill,

Thanks for the info, that was indeed very useful.

I used the bregex utility on my fileset, and found that my specification of the 
RegexFile pattern was wrong, hence the job was excluding some files.

Thanks once again.

-Yateen


-Original Message-
From: Bill Arlofski via Bacula-users 
Sent: Sunday, November 21, 2021 3:34 AM
To: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Detailed list/info about files excluded from the 
job run

On 11/20/21 10:06, Shaligram Bhagat, Yateen (Nokia - IN/Bangalore) wrote:
> Hello all,
>
> We are using Bacula 9.4.4 on Centos.
>
> Is there a way to run Bacula backup job with debug option to get a 
> detailed list/info about files excluded from the actual job run instance?
>
> Our fileset has a list of exclude files (through regexFile).
>
> But surprisingly few files are getting excluded from the backup 
> although their names DO NOT match with the exclude regex specification.
>
> Thanks
>
> Yateen Bhagat
>

Hello Yateen,

There is a utility called `bregex` that you can use to test your regexes 
against a list of files.

This way you don't have to run an actual backup and then list the files in it 
to see if your regexes are correct.

If you just want to have your job list the files as it backs them up, edit your 
Messages{} resource (the default one is
'Standard') and add the 'saved' option to the "Append = " and/or "Console =" 
lines.

Keep in mind that this can slow backups down a bit since all files will now be 
logged to a file, or to the console buffer (or
both) as they are backed up.


Best regards,
Bill

--
Bill Arlofski
w...@protonmail.com



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Fileset override in Schedule resource

2021-11-21 Thread Radosław Korzeniewski
Hello,

czw., 18 lis 2021 o 13:40 Shaligram Bhagat, Yateen (Nokia - IN/Bangalore) <
yateen.shaligram_bha...@nokia.com> napisał(a):

> Yateen : We have hundreds of lab Linux servers (let’s say 200), each
> getting data generated every day, organized in sub dirs like
>  //
>
> The data generated in the past years is no longer required usually, but
> may be needed just in case.  We call this data as archive data and want to
> have job that will run for each past year data and create an yearly archive
> (aone time activity). So instead of creating 200 servers X 5 years = 1000
> jobs, I want to minimize the number of jobs. I intend to do so by creating
> only 200 archive jobs, with variable fileset mechanism. I think this is
> what you have suggested too.
>

The "variable" fileset handling is a pain in the ***. Especially during
restore.


>
>
>
>
> This will create a job inventory that will run into thousand.
>
>
>
> What is a "job inventory"? I never heard of it in Bacula.
>
> Bacula has no problem handling hundreds of thousands of jobs.
>
> Yateen : As explained above, Job inventory means number of Jobs
> configured. I know that, Bacula has no limit on handling hundreds of them,
> but as an admin I intend to keep the list as minimum as possible by having
> some creativity.
>

Why do you need to keep it as minimal as possible? The first step I'll do
for this number of clients is to generate required configuration with
automation scripts, especially that you have an easy file directory pattern
to backup. No manual config file crafting.

> I wonder why there is no fileset override for  a job in the schedule
> resource, when there are so many other overrides provided.
>
> This requirement may be an enhancement candidate ??
>
>
>
> Let's assume you have a two totally distinct fileset resources, i.e.
>
>
>
> Fileset {
>
>   Name = FS1
>
>   Include {
>
> File = /home
>
>   }
>
> }
>
>
>
> Fileset {
>
>   Name = FS2
>
>   Include {
>
> Plugin = "qemu: vm=vm1"
>
>   }
>
> }
>
>
>
> and every other day in the Schedule for Incremental level backup you
> switch between them back and forth.
>
> What is your expected result in this case, when you select the most recent
> backup to restore?
>
> The above question is to understand the requirements.
>
>
>
> Yateen : I am planning to do something similar.
>

So what do you want to achieve during restore?
I think your solution is invalid to your requirements.

To generate a "dynamic" fileset you can use:

   - Any name preceded by an at-sign (@) is assumed to be the name of a
   file, which contains a list of files each preceded by a “File =”. The named
   file is read once when the configuration file is parsed during the Director
   startup.
   - Any name beginning with a vertical bar (|) is assumed to be the name
   of a program. This program will be executed on the Director's machine at
   the time the Job starts.
   - If the vertical bar (|) in front of File= is preceded by a backslash
   as in \|, the program will be executed on the Client's machine instead of
   on the Director's machine.
   - Any file-list item preceded by a less-than sign (<) will be taken to
   be a file. This file will be read on the Director's machine (see below for
   doing it on the Client machine) at the time the Job starts, and the data
   will be assumed to be a list of directories or files, one per line, to be
   included.
   - If you precede the less-than sign (<) with a backslash as in \<, the
   file-list will be read on the Client machine instead of on the Director's
   machine.

You can find examples at Bacula's manual -
https://www.bacula.org/11.0.x-manuals/en/main/Configuring_Director.html#SECTION002170

This should allow you to achieve what you want exactly with some external
scripting which will dynamically prepare what directories/files to backup
at the selected job.

But, when you change a fileset to switch from one year to another then
Bacula will mark all "missing" files as nonexistent/deleted for the next
job. If this is exactly what you want then great! Or I totally
misunderstood your requirements, then sorry.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users