Hi Dragan! >From what I can see in each Copy Job’s log, every job is grabbing a different Volume and using a different Device to read from:
JobId 9314 --> Backup Copy JobId 9448 grabbed JobId 7565 to copy : Using Device "mongodb01_baculaserver01_Copy_Restore_1" to read Copy JobId 9450 grabbed JobId 7752 to copy : Using Device "mongodb01_baculaserver01_Copy_Restore_2" to read Copy JobId 9452 grabbed JobId 8092 to copy : Using Device "mongodb01_baculaserver01_Copy_Restore_3" to read Copy JobId 9454 grabbed JobId 8433 to copy : Using Device "mongodb01_baculaserver01_Copy_Restore_4" to read. Copy JobId 9456 grabbed JobId 8912 to copy : Using Device "mongodb01_baculaserver01_Copy_Restore_0" to read. Here’s one of the logs as an example (9456): mainbackupserver-dir JobId 9456: Copying using JobId=8912 Job=mongodb01-Backup.2025-07-11_00.20.51_46 mainbackupserver-dir JobId 9456: Start Copying JobId 9456, Job=CopyJob_mongo.2025-07-15_19.33.05_39 mainbackupserver-dir JobId 9456: Connected to Storage "mongodb01_baculaserver02_Storage" at baculaserver02:9103 with TLS mainbackupserver-dir JobId 9456: Connected to Storage "mongodb01_baculaserver01_Storage_Copy_Restore" at baculaserver01:9103 with TLS mainbackupserver-dir JobId 9456: Using Device "mongodb01_baculaserver01_Copy_Restore_0" to read. baculaserver01-sd JobId 9456: Ready to read from volume "mongodb01-2255" on File device "mongodb01_baculaserver01_Copy_Restore_0" (/backup/bacula-storage/mongodb01/). baculaserver01-sd JobId 9456: Forward spacing Volume "mongodb01-2255" to addr=318 So, at least based on the logs, it seems like the jobs are not competing over the same Volume or Device. On Wed, Jul 16, 2025 at 10:27 AM Dragan Milivojević <gali...@pkm-inc.com> wrote: > Add limits for a volume so that it accepts only one job. > > On Wed, 16 Jul 2025 at 14:53, Leandro Saldivar via Bacula-users < > bacula-users@lists.sourceforge.net> wrote: > >> Hi Robert! Thanks for your time and input! >> >> What you suggested was exactly what came to my mind yesterday. I tested >> it and saw something very strange: >> I was able to start one Backup Job and 5 Copy Jobs in parallel, and >> everything looked fine at first. However, on the remote SD, the file sizes >> stopped growing independently, and a new file appeared that seems to be >> aggregating all 5 Copy Jobs into a single volume: >> >> -rw-r----- 1 root root 4.6T Jul 16 12:35 mongodb01-2510 ← this should >> be 1.5T max, seems to be merging copies >> -rw-r----- 1 root root 190K Jul 15 20:05 mongodb01-2499 >> -rw-r----- 1 root root 127K Jul 15 20:04 mongodb01-2495 >> -rw-r----- 1 root root 64K Jul 15 20:04 mongodb01-2452 >> -rw-r----- 1 root root 318 Jul 15 20:04 mongodb01-2441 >> -rw-r----- 1 root root 44G Jul 15 20:03 mongodb01-2439 >> -rw-r----- 1 root root 1.5T Jun 29 18:43 mongodb01-2273 ← expected >> result >> -rw-r----- 1 root root 1.5T Jun 29 00:55 mongodb01-2268 ← expected >> result >> >> So it seems the volumes from the parallel Copy Jobs are being merged >> unintentionally. >> >> I added an Autochanger with 5 Devices, declared a Storage using the >> Autochanger, and pointed my Copy Jobs to this new Storage resource. >> >> You can find my config here: >> https://gist.github.com/lean2206/54b37b8d599243f4b95e047fa4a8b47a >> >> Any idea what might be causing this behavior? Meanwhile, until I find a >> better approach, I'll run the Restore Job using a RunAfterJob from the >> Backup, and trigger the Copy Jobs via cron — both using separate Devices, >> in parallel. So far, this setup would require 2 Devices. >> >> Thanks again! >> >> On Tue, Jul 15, 2025 at 2:03 PM Rob Gerber <r...@craeon.net> wrote: >> >>> Leandro, >>> >>> The following assumes you are using disk based backup, where the >>> "drives" and "autochangers" are virtual. >>> >>> I would suggest creating multiple "drive" devices in your bacula-sd.conf >>> for each SD destination, then tie them together with a single autochanger >>> resource in those respective bacula-sd.conf files. In your bacula-dir.conf, >>> in your storage resources, reference the respective autochanger resources. >>> If your jobs are ran with the same priority, and all the various daemons >>> have enough "allow simultaneous jobs" specified (not the actual option, I >>> can't recall the exact wording used by Bacula), then you should see better >>> parallelism. I believe you will then be limited by your disk / netwoek >>> bandwidth, and single core performance on various Bacula tasks like hashing >>> files. >>> >>> At least, unless I misunderstand something. >>> >>> I would provide config examples, but I am afk right now. >>> >>> Robert Gerber >>> 402-237-8692 >>> r...@craeon.net >>> >>> On Tue, Jul 15, 2025, 9:51 AM Leandro Saldivar via Bacula-users < >>> bacula-users@lists.sourceforge.net> wrote: >>> >>>> Hi all, >>>> >>>> I have a client with ~2TB of data. A full backup job takes about 15 >>>> hours. Once the backup finishes, a Copy Job is triggered (Selection Type = >>>> PoolUncopiedJobs) to replicate the volume to a remote Storage Daemon — >>>> which takes another 15 hours per volume in the best-case scenario. Finally, >>>> for each backup, I run a Restore job on a replica server (same client >>>> setup), which adds yet another 15 hours. >>>> >>>> The trivial approach to parallelize these jobs (Backup, Copy, Restore) >>>> has been to define a separate Storage + Device pair for each Job. The only >>>> difference between them is the Name. All Devices share the same Archive >>>> Device path — which I'm unsure is a good practice. >>>> >>>> Is there a better way to parallelize Backup, Copy, and Restore jobs on >>>> the same Pool/Volumes without defining a separate Storage/Device for each >>>> Job? What would be considered a best-practice setup here? >>>> >>>> TL;DR >>>> I need to run Backup, Copy, and Restore jobs in parallel for the same >>>> client (Restore happens on a replica server). Ideally without creating >>>> separate Storage/Device/Pool for each job. >>>> >>>> Bacula Version: 15.0.2 (21 March 2024) >>>> >>>> This is the config that currently allows me to parallelize only Backup >>>> and Restore. Following this logic, I’d need to create additional Storage + >>>> Device just for the Copy jobs. >>>> >>>> Client { >>>> Name = "mongodb01-fd" >>>> Address = xx.xx.xx.xx >>>> FDPort = 9102 >>>> Catalog = MyCatalog >>>> Password = "PASSWORD" >>>> File Retention = 11 days >>>> Job Retention = 2 months >>>> AutoPrune = yes >>>> Heartbeat Interval = 1m >>>> TLS Require = yes >>>> TLS PSK Enable = yes >>>> } >>>> >>>> Job { >>>> Name = "mongodb01-Backup" >>>> JobDefs = "MongoDef" >>>> Pool = "mongodb01-baculaserver01-Pool" >>>> Level = Full >>>> Client = "mongodb01-fd" >>>> Storage = "mongodb01_baculaserver01_Storage" >>>> Schedule = "DisabledSchedule" # This is how I disable this client >>>> so it doesn't affect others using the same JobDef with valid Schedules. I >>>> launch the backup manually using a bconsole command. >>>> @"/opt/bacula/etc/bacula-dir.conf.d/jobdefs/RunCopyToServer.jd" # >>>> This is how I trigger Copy Jobs for clients with small backups. It works >>>> well within a 24-hour window. For this client, I should remove this line. >>>> } >>>> >>>> Pool { >>>> Name = "mongodb01-baculaserver01-Pool" >>>> Use Volume Once = yes >>>> Pool Type = Backup >>>> LabelFormat = "mongodb01-" >>>> AutoPrune = yes >>>> Recycle = yes >>>> VolumeRetention = 30 days >>>> Maximum Volumes = 365 >>>> Maximum Volume Jobs = 1 >>>> Recycle Oldest Volume = yes >>>> Next Pool = "mongodb01-baculaserver02-Pool" # All copies go from >>>> server01 to server02 >>>> } >>>> >>>> Pool { >>>> Name = "mongodb01-baculaserver02-Pool" >>>> Use Volume Once = yes >>>> Pool Type = Backup >>>> LabelFormat = "mongodb01-" >>>> AutoPrune = yes >>>> Recycle = yes >>>> VolumeRetention = 30 days >>>> Maximum Volumes = 365 >>>> Maximum Volume Jobs = 1 >>>> Recycle Oldest Volume = yes >>>> Storage = "mongodb01_baculaserver02_Storage" >>>> Next Pool = "mongodb01-baculaserver01-Pool" # If backup runs on >>>> server02, the copy goes to server01. Normally, all main backups are done on >>>> server01. >>>> } >>>> >>>> Storage { >>>> Name = "mongodb01_baculaserver01_Storage" >>>> Address = "baculaserver01.domain" >>>> SDPort = 9103 >>>> Password = "xxxxxxxxxxxxxxxxxxxxxxx" >>>> Device = "mongodb01_baculaserver01_Device" >>>> Media Type = File1 >>>> Maximum Concurrent Jobs = "10" >>>> Heartbeat Interval = 10 >>>> } >>>> >>>> Storage { >>>> Name = "mongodb01_baculaserver01_Storage_2" >>>> Address = "baculaserver01.domain" >>>> SDPort = 9103 >>>> Password = "xxxxxxxxxxxxxxxxxxxxxxxx" >>>> Device = "mongodb01_baculaserver01_Device_2" >>>> Media Type = File1 >>>> Maximum Concurrent Jobs = "10" >>>> Heartbeat Interval = 10 >>>> } >>>> >>>> Storage { >>>> Name = "mongodb01_baculaserver02_Storage" >>>> Address = "baculaserver02.domain" >>>> SDPort = 9103 >>>> Password = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" >>>> Device = "mongodb01_baculaserver02_Device" >>>> Media Type = File1 >>>> Maximum Concurrent Jobs = "10" >>>> Heartbeat Interval = 10 >>>> } >>>> >>>> Device { >>>> Name = "mongodb01_baculaserver01_Device" >>>> Media Type = File1 >>>> Archive Device = "/backup/bacula-storage/mongodb01/" >>>> Maximum Concurrent Jobs = 10 >>>> LabelMedia = yes >>>> Random Access = Yes >>>> AutomaticMount = yes >>>> RemovableMedia = no >>>> AlwaysOpen = yes >>>> } >>>> >>>> Device { >>>> Name = "mongodb01_baculaserver01_Device_2" >>>> Media Type = File1 >>>> Archive Device = "/backup/bacula-storage/mongodb01/" >>>> Maximum Concurrent Jobs = 10 >>>> LabelMedia = yes >>>> Random Access = Yes >>>> AutomaticMount = yes >>>> RemovableMedia = no >>>> AlwaysOpen = yes >>>> } >>>> >>>> #This Device is stored in baculaserver02 where only a storage daemon is >>>> running (not bacula-dir) >>>> Device { >>>> Name = "mongodb01_baculaserver02_Device" >>>> Media Type = File1 >>>> Archive Device = "/backup/bacula-storage/mongodb01/" >>>> LabelMedia = yes; >>>> Random Access = Yes; >>>> AutomaticMount = yes; >>>> RemovableMedia = no; >>>> AlwaysOpen = no; >>>> } >>>> >>>> >>>> Backup is launched via cron every 3 days using: >>>> echo 'run job=mongodb01-Backup >>>> storage="mongodb01_baculaserver01_Storage_2" yes' | docker exec -i >>>> bacula-dir /opt/bacula/bin/bconsole >>>> Restore is triggered by another cron using the default Storage defined >>>> in the Job. >>>> >>>> Any advice or shared experience would be greatly appreciated. >>>> >>>> Thanks, >>>> _______________________________________________ >>>> Bacula-users mailing list >>>> Bacula-users@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>>> >>> _______________________________________________ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users >> >
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
