| Correct. Because when you run your consolidate with a full it has to read the old full likely from your tape drive. So it has to write to disk.
Sent from my iPhone Brock Palen On Dec 23, 2023, at 3:43 PM, Russell Harmon <eatnumb...@gmail.com> wrote:
I do this and it works but it has some bugs that are reported but not had progress in few years.
My setup is:
Disk Pools:
AI-Incremental
AI-Consolidated
Tape Pool:
LTO
Offsite
I run the AI setup on the disk pools but I use a Migration Job to migrate AI-Consolidated jobs to Offsite.
This is where one bug shows up ’sometimes’ Bareos will correctly understand that the job it needs is on tape but if you run parallel jobs it won’t correctly wait for the tape drive to free up if already busy. It will instead fail. It’s easy to work around you just run it again when that happens.
Sometimes it also gets hungup with multiple disk ‘devices’ where it won’t swap the needed disk volume even though it’s not being used. You can avoid all of these by forcing serial jobs, and all ‘bugs’ are more inconvenient than show stoppers.
I also use the Offsite pool (which for me is a second tape drive but not part of an autoloader) to write monthly offsite copies using VirutalFulls
https://docs.bareos.org/TasksAndConcepts/AlwaysIncrementalBackupScheme.html#long-term-storage-of-always-incremental-jobs
This is not AI related, but regular copy jobs are not reocmended because the consolidate pulls in those jobs. So this is more of an ‘emergency get it back no more than a month old’ offsite copy.
The migration job runs a script that checks and truncates all pruned volumes to free disk space rather than waiting for them to expire by age. Again this setup I find requires few TB of disk (need to run full backups to disk) and requires some watching.
To make sure I understand, this requires enough disk space for a full backup? I can't write out each tape's worth of data to tape while the full backup is running?
#!/bin/bash
POOL=$1
for x in `echo "list volumes pool=${POOL}" | bconsole | grep -v "list volumes" | grep $POOL | awk -F\| '{print $3}'`
do
echo "prune volume=$x yes"
done | bconsole
# actaully free up disk space
echo "truncate volstatus=Purged pool=$POOL yes" \
| bconsole
Pool {
Name = AI-Incremental
Pool Type = Backup
Recycle = yes # Bareos can automatically recycle Volumes
Auto Prune = yes # Prune expired volumes
Volume Retention = 6 months # How long should jobs be kept?
Maximum Volume Bytes = 10G # Limit Volume size to something reasonable
Label Format = "AI-Incremental-"
Volume Use Duration = 7d
Storage = File
Next Pool = AI-Consolidated # consolidated jobs go to this pool
Action On Purge=Truncate
Migration High Bytes = 500G
Migration Low Bytes = 300G
}
Pool {
Name = AI-Consolidated
Pool Type = Backup
Recycle = yes # Bareos can automatically recycle Volumes
Auto Prune = yes # Prune expired volumes
Volume Retention = 6 months # How long should jobs be kept?
Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
Label Format = "AI-Consolidated-"
Volume Use Duration = 2 days
Storage = File
Next Pool = Longterm # copy jobs write to this pool
Action On Purge=Truncate
Migration Time = 7 days
Migration High Bytes = 600G
Migration Low Bytes = 300G
}
Job {
Name = "Migrate-To-Offsite-AI-Consolidated-size"
Client = myth-fd
Type = Migrate
Purge Migration Job = yes
Pool = AI-Consolidated
Level = Full
Next Pool = LTO
Schedule = WeeklyCycleAfterBackup
Allow Duplicate Jobs = no
Priority = 4 #before catalog dump
Messages = Standard
Selection Type = PoolOccupancy
Spool Data = "">
Selection Pattern = "."
RunAfterJob = "sudo /usr/local/bin/prune.sh AI-Consolidated"
Enabled = no
}
Brock Palen
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting
> On Dec 17, 2023, at 7:38 PM, Russell Harmon <eatnumb...@gmail.com> wrote:
>
> Hi there,
>
> I see the note in https://docs.bareos.org/TasksAndConcepts/AlwaysIncrementalBackupScheme.html#storages-and-pools about "at least two storages are needed" for Always Incremental, but is there a way to make this work with a *temporary* secondary storage?
>
> I have just one tape drive, and while I can temporarily use local disk as a secondary storage, I don't have enough for a full backup on disk... only enough for one tape's worth of data.
>
> Is there any way to spool a consolidated job to disk, then let me swap in a tape for the consolidated pool (therefore removing the incremental pool's tape), despool it, then resume consolidating?
>
> Thanks,
> Russ Harmon
>
> --
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/6addbe69-f3a4-4bb4-b10d-849d713d5723n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/CA%2BzrezSxi_eLzbQ%2BZvaikD69%3DFcF2Vtv%2BBRrLawBNvJYk5jXcw%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/2F62BBE3-FAAB-4A05-8721-60769260497A%40mlds-networks.com.
|
