Re: [Bacula-users] Problems with De-Duplication
To talk about deduplication in Bacula, you must know that there are
three types of deduplication:
1. File level dedup using Base jobs.
2. Block level SD dedup called aligned block deduplication requiring a
filesystems such as ZFS to do the dedup
3. Block level called Global End Point deduplication
Item 1 is has been available in the community version since the beginning.
Item 2 is not yet available in the community, but after my vacation, I
hope to make a plugin that will be
compatible with the community version. If I can find the time
to concentrate on it, this could be
ready in January 2016.
Item 3 will probably not be offered to the community for another 3-4 years.
Best regards,
Kern
On 15-08-19 03:34 PM, Raymond Burns Jr. wrote:
When will Bacula Enterprise deduplication be ported to community?
On Wed, Aug 19, 2015 at 8:32 AM E.L.L. Assoua ellass...@gmail.com
mailto:ellass...@gmail.com wrote:
Dear Heitor,
Could you check out my configuration?
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 (
client-fd is version 5.2 - Windows 8.1 Pro x64 ) with the
following properties:
*_A specific Pool for the base volumes:_*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
_*The specific FileSet for the base job:*_
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*_The base job:_*
*
*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
_*The JobDefs for the full backup job:*
_
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*_The full backup job:_*
*_
_*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
Base = BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*_The client_*
*_
_*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Beste regard,
Luc Assoua
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
mailto:Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
Dear Heitor,
Could you check out my configuration?
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 (
client-fd is version 5.2 - Windows 8.1 Pro x64 ) with the following
properties:
*_A specific Pool for the base volumes:_*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
_*The specific FileSet for the base job:*_
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*_The base job:_*
*
*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
_*The JobDefs for the full backup job:*
_
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*_The full backup job:_*
*_
_*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
Base = BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*_The client_*
*_
_*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Beste regard,
Luc Assoua
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
Hello guys, 1. Bacula Enterprise Global deduplication but it's more automated and efficient than the Community dedup. In fact it works beyond file granularity, but in block level at the storage: http://www.baculasystems.com/wp-content/uploads/bacula-enterprise-v8-global-endpoint-deduplication2.pdf 2. If you made a base backup jobs using Bacula Community and set the original Backup job correctly to compare with the Base job, probably you will backup the C:/Windows again cause, of course, there are files under it that are always changing. But it's only expected to copy the files that changed in relation to the ones copied by the Base Job. Regards, === Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified Administrator II Do you need Bacula training? https://www.udemy.com/bacula-backup-software/?couponCode=bacula-list I do Bacula training and deploy in any city of the world. More information: http://bacula.us/ +55 61 8268-4220 Site: http://bacula.us FB: heitor.faria === From: Raymond Burns rbur...@gmail.com To: E.L.L. Assoua ellass...@gmail.com, bacula-users@lists.sourceforge.net Sent: Wednesday, August 19, 2015 9:54:25 AM Subject: Re: [Bacula-users] Problems with De-Duplication Deduplication is not an option??? I thought it was a plugin that was available for purchase On Wed, Aug 19, 2015 at 7:48 AM E.L.L. Assoua ellass...@gmail.com wrote: Nobody uses the de-duplication option? -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users-- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
Nobody uses the de-duplication option? -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
Could you check out my configuration?
I do not normally do that since it's consultancy and you will probably wont
create more synapses. =)
Anyway it goes:
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 ( client-fd is
version 5.2 - Windows 8.1 Pro x64 ) with the following properties:
A specific Pool for the base volumes:
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
The specific FileSet for the base job:
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
You must have at the fileset options:
BaseJob = pmugcs5
Accurate = mcs
Verify = pin5
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
The base job:
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
The JobDefs for the full backup job:
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
The full backup job:
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
You must have the directive here:
Accurate = yes
Base = BaseJob_Windows8.1
To the magic happen you must compare the base job to the job itself, between
commas:
Base = FrankPC-Backup, BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
The client
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days # 60 days
Job Retention = 6 months # six months
AutoPrune = yes # Prune expired Jobs/Files
}
Source: http://www.bacula.com.br/manual/File_Deduplication_using_Ba.html
Beste regard,
Luc Assoua
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
When will Bacula Enterprise deduplication be ported to community?
On Wed, Aug 19, 2015 at 8:32 AM E.L.L. Assoua ellass...@gmail.com wrote:
Dear Heitor,
Could you check out my configuration?
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 ( client-fd
is version 5.2 - Windows 8.1 Pro x64 ) with the following properties:
*A specific Pool for the base volumes:*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
*The specific FileSet for the base job:*
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*The base job:*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
*The JobDefs for the full backup job:*
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*The full backup job:*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
Base = BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*The client*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Beste regard,
Luc Assoua
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
Dear Heitor,
Im going to change the configuration and test it agian.
Thank you for time!
Best regards,
Luc Assoua
Heitor Faria mailto:hei...@bacula.com.br
Wednesday, August 19, 2015 3:41 PMvia Postbox
https://www.postbox-inc.com/?utm_source=emailutm_medium=sumlinkutm_campaign=reach
Could you check out my configuration?
I do not normally do that since it's consultancy and you will probably
wont create more synapses. =)
Anyway it goes:
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 (
client-fd is version 5.2 - Windows 8.1 Pro x64 ) with the
following properties:
*A specific Pool for the base volumes:*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
*The specific FileSet for the base job:*
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
You must have at the fileset options:
BaseJob = pmugcs5
Accurate = mcs
Verify = pin5
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*The base job:*
*
*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
*The JobDefs for the full backup job:*
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*The full backup job:*
*
*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
You must have the directive here:
Accurate = yes
Base = BaseJob_Windows8.1
To the magic happen you must compare the base job to the job itself,
between commas:
Base =FrankPC-Backup, BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*The client*
*
*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Source: http://www.bacula.com.br/manual/File_Deduplication_using_Ba.html
Beste regard,
Luc Assoua
E.L.L. Assoua mailto:ellass...@gmail.com
Wednesday, August 19, 2015 3:24 PMvia Postbox
https://www.postbox-inc.com/?utm_source=emailutm_medium=sumlinkutm_campaign=reach
Dear Heitor,
Could you check out my configuration?
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 (
client-fd is version 5.2 - Windows 8.1 Pro x64 ) with the following
properties:
*_A specific Pool for the base volumes:_*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
_*The specific FileSet for the base job:*_
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*_The base job:_*
*
*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
_*The JobDefs for the full backup job:*
_
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*_The full backup job:_*
*_
_*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
Base = BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*_The client_*
*_
_*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Beste regard,
Luc Assoua
Heitor Faria mailto:hei...@bacula.com.br
Wednesday, August 19, 2015 3:14 PMvia Postbox
https://www.postbox-inc.com/?utm_source=emailutm_medium=sumlinkutm_campaign=reach
Hello guys,
1. Bacula Enterprise Global deduplication but it's more automated and
efficient than the Community dedup. In fact it works beyond file
granularity, but in block level at the storage:
Re: [Bacula-users] Problems with De-Duplication
Deduplication is not an option??? I thought it was a plugin that was available for purchase On Wed, Aug 19, 2015 at 7:48 AM E.L.L. Assoua ellass...@gmail.com wrote: Nobody uses the de-duplication option? -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with De-Duplication
Works!
Thanks Heitor!
Best regards,
Luc Assoua
Heitor Faria mailto:hei...@bacula.com.br
Wednesday, August 19, 2015 3:41 PMvia Postbox
https://www.postbox-inc.com/?utm_source=emailutm_medium=sumlinkutm_campaign=reach
Could you check out my configuration?
I do not normally do that since it's consultancy and you will probably
wont create more synapses. =)
Anyway it goes:
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 (
client-fd is version 5.2 - Windows 8.1 Pro x64 ) with the
following properties:
*A specific Pool for the base volumes:*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
*The specific FileSet for the base job:*
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
You must have at the fileset options:
BaseJob = pmugcs5
Accurate = mcs
Verify = pin5
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*The base job:*
*
*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
*The JobDefs for the full backup job:*
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*The full backup job:*
*
*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
You must have the directive here:
Accurate = yes
Base = BaseJob_Windows8.1
To the magic happen you must compare the base job to the job itself,
between commas:
Base =FrankPC-Backup, BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*The client*
*
*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Source: http://www.bacula.com.br/manual/File_Deduplication_using_Ba.html
Beste regard,
Luc Assoua
E.L.L. Assoua mailto:ellass...@gmail.com
Wednesday, August 19, 2015 3:24 PMvia Postbox
https://www.postbox-inc.com/?utm_source=emailutm_medium=sumlinkutm_campaign=reach
Dear Heitor,
Could you check out my configuration?
I've configured De-duplication on a Bacula 7.0.5.9 - CentOS 7 (
client-fd is version 5.2 - Windows 8.1 Pro x64 ) with the following
properties:
*_A specific Pool for the base volumes:_*
Pool {
Name = BaseJobs
Pool Type = Backup
Label Format =
${Level}-${Job}-${Day}-${Month}-${Year}-${Hour:p/2/0/r}${Minute:p/2/0/r}${Second:p/2/0/r}
Use Volume Once = yes
}
_*The specific FileSet for the base job:*_
FileSet {
Name = BaseFileSet_Windows8.1
Enable VSS = yes
Include {
Options {
signature = MD5
compression = GZIP
}
File = C:/Windows/
}
}
*_The base job:_*
*
*
Job {
Name = BaseJob_Windows8.1
Type = Backup
Level = Base
Client = tom-helpdesk-fd
FileSet = BaseFileSet_Windows8.1
Pool = BaseJobs
Storage = ns-bacula-sd
Messages = Standard
}
_*The JobDefs for the full backup job:*
_
JobDefs {
Name = Windows_Backup
Type = Backup
Level = Full
FileSet = Windows FileSet
Messages = Standard
SpoolAttributes = yes
Priority = 10
Write Bootstrap = /var/spool/bacula/%c.bsr
Storage = ns-bacula-sd
}
*_The full backup job:_*
*_
_*
Job {
Name = FrankPC-Backup
JobDefs = Windows_Backup
Base = BaseJob_Windows8.1
Client = tom-helpdesk-fd
Schedule = FrankPC_Backup
Pool = RegularBackups
}
*_The client_*
*_
_*
Client {
Name = tom-helpdesk-fd
Address = 10.13.37.103
FDPort = 9102
Catalog = MyCatalog
Password = ~XXX # password for FileDaemon
File Retention = 60 days# 60 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Beste regard,
Luc Assoua
Heitor Faria mailto:hei...@bacula.com.br
Wednesday, August 19, 2015 3:14 PMvia Postbox
https://www.postbox-inc.com/?utm_source=emailutm_medium=sumlinkutm_campaign=reach
Hello guys,
1. Bacula Enterprise Global deduplication but it's more automated and
efficient than the Community dedup. In fact it works beyond file
granularity, but in block level at the storage:
