Re: [Bacula-users] Use of Multiple Tape Drives at Once
Hi Bill, Lifesaver! Thanks for the help, it works great simultaneously. Kody On Mon, Jul 1, 2024 at 11:02 AM Bill Arlofski wrote: > On 7/1/24 2:27 PM, Kody Rubio wrote: > > Hi Bill, > > > > The following is the output of 'status director' when the jobs are > waiting. > > > > 597 Back Full146,973667.8 Gfileserver-active is > running > > 599 Back Incr 0 0 obsdata4 >is waiting for higher priority jobs to finish > > > > > > > > It seems that "obsdata4" it is waiting for a higher priority job. > > Although the fileserver-active job does not have a priority set in the > config. > > Also, obsdata4 has a Priority of 1 (highest), so I am currently unsure > on why this is happening. > > Hello Kody, > > The default Priority is 10 if not set. > > So, you have a priority 10 job running `fileserver-active`, and a job with > a different priority (1) waiting `obsdata4` > > The message `is waiting for higher priority jobs to finish` is a bit of a > red herring. It does not matter if the other jobs > have a higher or lower priority set, this will be the (misleading) message > logged. > > If you want more than one job to run at the same time, they will need to > be set to the same priority. Sure, you can also play > with different priorities, and then enable "AllowMixedPrioritiy" in > several places (your Job resources), but this will just > lead to more confusing results, I am afraid. It is generally recommended > to stick to the same priority for your normal backup > and Restore Jobs, and then set the small number of "special" jobs (Admin, > CopyControl, MigrationControl, Verify, etc) to > something else so they they do not run as the same time as "normal" jobs. > > Check the default "Catalog" job to see that while the default Backup jobs > are `priority = 10`, that job is 11 or 12 to make > sure it is run only when all the other normal, nightly backups jobs have > completed so that the catalog has all of the > information for all of the night's jobs - except, of course itself. :) > > You will need to check the Director's Storage resource simply called > `Autochanger` and make sure that it has > `MaximumConcurrentJobs` > 1, and that your jobs expected to run at the > same same time have the same priority and you should > be able to make some progress. > > > Best regards, > Bill > > -- > Bill Arlofski > w...@protonmail.com > > -- Subaru Computer Systems Associate Subaru Telescope, NAOJ Hilo, HI (808) 934-5050 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use of Multiple Tape Drives at Once
On 7/1/24 2:27 PM, Kody Rubio wrote: Hi Bill, The following is the output of 'status director' when the jobs are waiting. 597 Back Full 146,973 667.8 G fileserver-active is running 599 Back Incr 0 0 obsdata4 is waiting for higher priority jobs to finish It seems that "obsdata4" it is waiting for a higher priority job. Although the fileserver-active job does not have a priority set in the config. Also, obsdata4 has a Priority of 1 (highest), so I am currently unsure on why this is happening. Hello Kody, The default Priority is 10 if not set. So, you have a priority 10 job running `fileserver-active`, and a job with a different priority (1) waiting `obsdata4` The message `is waiting for higher priority jobs to finish` is a bit of a red herring. It does not matter if the other jobs have a higher or lower priority set, this will be the (misleading) message logged. If you want more than one job to run at the same time, they will need to be set to the same priority. Sure, you can also play with different priorities, and then enable "AllowMixedPrioritiy" in several places (your Job resources), but this will just lead to more confusing results, I am afraid. It is generally recommended to stick to the same priority for your normal backup and Restore Jobs, and then set the small number of "special" jobs (Admin, CopyControl, MigrationControl, Verify, etc) to something else so they they do not run as the same time as "normal" jobs. Check the default "Catalog" job to see that while the default Backup jobs are `priority = 10`, that job is 11 or 12 to make sure it is run only when all the other normal, nightly backups jobs have completed so that the catalog has all of the information for all of the night's jobs - except, of course itself. :) You will need to check the Director's Storage resource simply called `Autochanger` and make sure that it has `MaximumConcurrentJobs` > 1, and that your jobs expected to run at the same same time have the same priority and you should be able to make some progress. Best regards, Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Again LTO9 and performances...
Mandi! Josh Fisher via Bacula-users In chel di` si favelave... > Another way that might be better for your case is to leave > MaximumSpoolSize = 0 (unlimited) and specify a MaximumJobSpoolSize in > the Device resource instead. The difference is that when one job reaches > the MaximumJobSpoolSize it will begin writing its spool file to tape, > but it will not cause the other job(s) to pause until they also reach > the MaximumJobSpoolSize. Then by starting one job a few minutes after > the other it should be possible to mostly avoid both jobs pausing > (de-spooling) at the same time. Bingo! Thanks! -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use of Multiple Tape Drives at Once
Hi Bill,
The following is the output of 'status director' when the jobs are waiting.
597 Back Full146,973667.8 Gfileserver-active is
running
599 Back Incr 0 0 obsdata4
is waiting for higher priority jobs to finish
It seems that "obsdata4" it is waiting for a higher priority job.
Although the fileserver-active job does not have a priority set in the
config.
Also, obsdata4 has a Priority of 1 (highest), so I am currently unsure on
why this is happening.
Director {
Name = "bacula-dir"
Messages = "Daemon"
DirAddress = 127.0.0.1
QueryFile = "/etc/bacula/query.sql"
WorkingDirectory = "/var/spool/bacula"
PidDirectory = "/var/run"
MaximumConcurrentJobs = 20
Password = password
}
Job {
Name = "obsdata4"
Type = "Backup"
Level = "Incremental"
Storage = "Autochanger"
FullBackupPool = "LongTerm Archive"
IncrementalBackupPool = "Archive-Obsdata"
Client = "stars"
Fileset = "obsdata4"
Schedule = "STARS-DailyIncremental"
JobDefs = "DefaultJob"
Priority = 1
Accurate = yes
AllowDuplicateJobs = no
}
Job {
Name = "fileserver-active"
Description = "STN6 Fileserver Active Data Job"
Type = "Backup"
Level = "Full"
Storage = "Autochanger"
Pool = "Fileserver"
FullBackupPool = "Fileserver"
IncrementalBackupPool = "Fileserver"
Client = "fileserver"
Fileset = "fileserver-active"
Schedule = "Fileserver"
JobDefs = "DefaultJob"
Accurate = yes
}
On Thu, Jun 27, 2024 at 10:02 AM Bill Arlofski via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:
> On 6/27/24 1:33 PM, Kody Rubio wrote:
> > I am searching for advice and/or expertise in allowing Bacula to use
> multiple tape drives at once.
> >
> > I currently have two separate jobs that use separate Pools, that run
> around the same time. Although the second job is always
> > waiting for the other job to finish. While the second tape drive is
> open, I would like for it to use the second drive and not
> > have to wait for the other drive to be finished.
> > I also read that setting "MaximumConcurrentJobs = 1" for each will allow
> this but I get the same result.
> >
> > Below is my configuration for the devices:
>
> Hello Cody, what does `status director` show when these jobs are "wating"
>
> What does your Director Storage resource look like?
>
>
> Best regards,
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Subaru Computer Systems Associate
Subaru Telescope, NAOJ
Hilo, HI
(808) 934-5050
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restore Wizard, partial fileset & sub dir problem
On Mon, 1 Jul 2024 at 11:39, Lionel PLASSE wrote: > Well, I'm going to test to create an index for now, > > I don't use postgre as well, and I'd rather keep using mysql > > But if postgre is better... > > > > And this issue has not been reported to the development team for mysql > users? Is an issue open, do you know ? > Hello Lionel, Great. Good luck with trying it. Developers are aware of this problem nevertheless opening an issue for that might be a good idea anyway. In the past users reported some problems with this restore but at that time nobody knew why it happened. Best regards, Marcin Haba (gani) -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restore Wizard, partial fileset & sub dir problem
Well, I'm going to test to create an index for now, I don't use postgre as well, and I'd rather keep using mysql But if postgre is better... And this issue has not been reported to the development team for mysql users? Is an issue open, do you know ? De : Marcin Haba Envoyé : lundi 1 juillet 2024 11:02 À : Lionel PLASSE Cc : bacula-users@lists.sourceforge.net Objet : Re: [Bacula-users] Restore Wizard, partial fileset & sub dir problem On Mon, 1 Jul 2024 at 10:42, Lionel PLASSE mailto:pla...@cofiem.fr>> wrote: Hello, I want to validate my backup support. I decided to restore a subdir of a fileset for a client. So I use baculum web , restore wizard, select client and fileset, navigate into paths and select 'Add' for a path witch contains a lot of subdir and files over 10K. I validate the wizard and I remain waiting on the summary page and the work is never scheduled (I stop if after 1 hour of waiting) When I do the same with one or 2 files it starts immediately after the last step of wizard (the summarize page) I had to create a text file with full path of files to restore (by a sql query) and with bconsole the restore start normally Why is the restore wizard so slow (and even stuck) to generate the final list of file and schedule the job when there are so many files.? Hello Lionel, It is a known problem. I researched it some time ago. In this thread you can find it and a workaround as well: https://sourceforge.net/p/bacula/mailman/bacula-users/thread/GV1P194MB178567464CFD7C79B173F0B3B53D2%40GV1P194MB1785.EURP194.PROD.OUTLOOK.COM/ Best regards, Marcin Haba (gani) -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restore Wizard, partial fileset & sub dir problem
On Mon, 1 Jul 2024 at 10:42, Lionel PLASSE wrote: > Hello, > > I want to validate my backup support. I decided to restore a subdir of a > fileset for a client. > So I use baculum web , restore wizard, select client and fileset, navigate > into paths and select 'Add' for a path witch contains a lot of subdir and > files over 10K. > > I validate the wizard and I remain waiting on the summary page and the > work is never scheduled (I stop if after 1 hour of waiting) > When I do the same with one or 2 files it starts immediately after the > last step of wizard (the summarize page) > > I had to create a text file with full path of files to restore (by a sql > query) and with bconsole the restore start normally > > Why is the restore wizard so slow (and even stuck) to generate the final > list of file and schedule the job when there are so many files.? > Hello Lionel, It is a known problem. I researched it some time ago. In this thread you can find it and a workaround as well: https://sourceforge.net/p/bacula/mailman/bacula-users/thread/GV1P194MB178567464CFD7C79B173F0B3B53D2%40GV1P194MB1785.EURP194.PROD.OUTLOOK.COM/ Best regards, Marcin Haba (gani) -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Restore Wizard, partial fileset & sub dir problem
Hello, I want to validate my backup support. I decided to restore a subdir of a fileset for a client. So I use baculum web , restore wizard, select client and fileset, navigate into paths and select 'Add' for a path witch contains a lot of subdir and files over 10K. I validate the wizard and I remain waiting on the summary page and the work is never scheduled (I stop if after 1 hour of waiting) When I do the same with one or 2 files it starts immediately after the last step of wizard (the summarize page) I had to create a text file with full path of files to restore (by a sql query) and with bconsole the restore start normally Why is the restore wizard so slow (and even stuck) to generate the final list of file and schedule the job when there are so many files.? Lionel, ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
