Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-18 Thread Radosław Korzeniewski
Hello, 2013/1/4 Radosław Korzeniewski > Hello, > > > 2013/1/4 Silver Salonen > >> 3. Partial restores can only be made to the client that did the backup, >>> not to another client if necessary(redirected recover). >>> >> >> Not true. In our implementation we can restore any file to any >> ded

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Gary R. Schmidt
On 5/01/2013 1:42 AM, tonyalbers wrote: > Spooling is definetely off. > > using tar to dump the directory 3 times now. > Kern Sibbald's post makes all this redundant. Wait for the Enterprise version, then wait for it perhaps become available in the community version. Cheers,

[Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Kern Sibbald
Hello, Concerning the email thread on deduplication: As you have heard from Radoslaw, Bacula Enterprise now has two directions it is going. The first is with what I call "the dedup friendly" format or Aligned Volumes, which when enabled, causes Bacula to write the Volume in a format that is very

[Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread tonyalbers
Spooling is definetely off. using tar to dump the directory 3 times now. 1. Before first tar: [root@dkarhbus02 download]# sdfscli -volume-info Volume Capacity : 1.5 TB Volume Current Size : 681 B Volume Max Percentage Full : Unlimited Volume Duplicate Data Written : 0 B Volume Unique Data Writte

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Silver Salonen
On 04.01.2013 15:11, Radosław Korzeniewski wrote: Hello, 2013/1/4 Silver Salonen mailto:sil...@serverock.ee>> 3. Partial restores can only be made to the client that did the backup, not to another client if necessary(redirected recover). Not true. In our implement

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Silver Salonen
On 04.01.2013 12:42, Radosław Korzeniewski wrote: Hello, 2013/1/4 tonyalbers > Hi Silver, The downside of client-side dedupe is actually a combination of several things. 1. The client does the dedupe calculations etc. this means that it

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Gary R. Schmidt
On 4/01/2013 8:30 PM, tonyalbers wrote: > Ok, it seems to me that there is some confusion of which kind of dedupe I'm > referring to. > > What I'm talking about is variable-length block-based target dedupe, that is, > the DXi or DataDomain box receives all data from the client via the backup > s

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Radosław Korzeniewski
Hello, 2013/1/4 tonyalbers > Hi Silver, > > The downside of client-side dedupe is actually a combination of several > things. > > 1. The client does the dedupe calculations etc. this means that it takes > CPU resources on the client which it might not have. > Yes it could be a problem in some c

[Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread tonyalbers
Hi Silver, The downside of client-side dedupe is actually a combination of several things. 1. The client does the dedupe calculations etc. this means that it takes CPU resources on the client which it might not have. 2. Sometimes, client-side dedupe is chosen because of bandwidth limitations to

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread Silver Salonen
On 04.01.2013 11:30, tonyalbers wrote: > Ok, it seems to me that there is some confusion of which kind of dedupe I'm > referring to. > > What I'm talking about is variable-length block-based target dedupe, that is, > the DXi or DataDomain box receives all data from the client via the backup > se

[Bacula-users] Plans for support block-based dedupe?

2013-01-04 Thread tonyalbers
Ok, it seems to me that there is some confusion of which kind of dedupe I'm referring to. What I'm talking about is variable-length block-based target dedupe, that is, the DXi or DataDomain box receives all data from the client via the backup server (bacula SD) and then handles the deduplicatio

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Radosław Korzeniewski
Hello, 2013/1/3 Gary R. Schmidt > > Keep in mind that ZFS block-level deduplication is very expensive in > > terms of RAM. > I'd put it as *any* block-level de-duplication is RAM intensive... I'd > be very interested if someone had an algorithm that was not RAM > intensive - and didn't take fore

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Phil Stracchino
On 01/03/13 08:22, Gary R. Schmidt wrote: > On 3/01/2013 11:44 PM, Phil Stracchino wrote: > [SNIP] > >> Keep in mind that ZFS block-level deduplication is very expensive in >> terms of RAM. > I'd put it as *any* block-level de-duplication is RAM intensive... I'd > be very interested if someone

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Gary R. Schmidt
On 3/01/2013 11:44 PM, Phil Stracchino wrote: [SNIP] > Keep in mind that ZFS block-level deduplication is very expensive in > terms of RAM. I'd put it as *any* block-level de-duplication is RAM intensive... I'd be very interested if someone had an algorithm that was not RAM intensive - and did

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Phil Stracchino
On 01/03/13 02:19, Gary R, Schmidt wrote: >> Does anyone know whether bacula can be made to work on a block-level >> dedupe storage system? Are there any plans to support this >> technology? > > What exactly do you mean by "work"? > > If you mean "use as a target for disk-based storage volumes,"

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Radosław Korzeniewski
Hello, 2013/1/3 > > do you have any numbers you could share how good real world dedupe will be? > It depends on the data you backup. On our test lab we achieved 1:10 deduplication ratio without a problem. best regards -- Radosław Korzeniewski rados...@korzeniewski.net

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread lst_hoe02
Zitat von Radosław Korzeniewski : > Hello, > > 2013/1/3 Silver Salonen > >> Excellent news! Any ETA when the plugin will be available? > > > This year... :) Q3/Q4. > > It is not ready for production yet, but we use it for daily backups and it > works very good. > > best regards > -- > Radosław K

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Radosław Korzeniewski
Hello, 2013/1/3 Silver Salonen > Excellent news! Any ETA when the plugin will be available? This year... :) Q3/Q4. It is not ready for production yet, but we use it for daily backups and it works very good. best regards -- Radosław Korzeniewski rados...@korzeniewski.net

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Silver Salonen
On 01/03/2013 11:59 AM, Radosław Korzeniewski wrote: Hello, 2013/1/3 Silver Salonen mailto:sil...@serverock.ee>> On 01/03/2013 09:48 AM, Sven Tegethoff wrote: > On 03.01.2013 08:19, Gary R, Schmidt wrote: >>> Does anyone know whether bacula can be made to work on a block-level

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Radosław Korzeniewski
Hello, 2013/1/3 Silver Salonen > On 01/03/2013 09:48 AM, Sven Tegethoff wrote: > > On 03.01.2013 08:19, Gary R, Schmidt wrote: > >>> Does anyone know whether bacula can be made to work on a block-level > >>> dedupe storage system? Are there any plans to support this > >>> technology? > Bacula S

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Gary R, Schmidt
On 03.01.2013 19:00, Silver Salonen wrote: [SNIP] > > Cannot it be summarized just by "client-side global de-duplication"? > Ie. > if the file A from Client1 is already backed up, the same file from > Client2 is backed up as a pointer only. If this is not possible, we > don't have a "true" and the

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-03 Thread Silver Salonen
On 01/03/2013 09:48 AM, Sven Tegethoff wrote: > On 03.01.2013 08:19, Gary R, Schmidt wrote: >>> Does anyone know whether bacula can be made to work on a block-level >>> dedupe storage system? Are there any plans to support this >>> technology? >> What magic are you expecting? > If I had to guess I'

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-02 Thread Sven Tegethoff
On 03.01.2013 08:19, Gary R, Schmidt wrote: >> Does anyone know whether bacula can be made to work on a block-level >> dedupe storage system? Are there any plans to support this >> technology? > > What magic are you expecting? If I had to guess I'd say he wants to backup de-duplicated data only o

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-02 Thread Silver Salonen
On 01/03/13 08:52, tonyalbers wrote: > Hi everyone, > > Does anyone know whether bacula can be made to work on a block-level dedupe > storage system? Are there any plans to support this technology? > > The reason I ask is that I've gotten a few requests from people who have > devices like Quantum

Re: [Bacula-users] Plans for support block-based dedupe?

2013-01-02 Thread Gary R, Schmidt
> Does anyone know whether bacula can be made to work on a block-level > dedupe storage system? Are there any plans to support this > technology? What exactly do you mean by "work"? If you mean "use as a target for disk-based storage volumes," it's just another fscking file system, it will work

[Bacula-users] Plans for support block-based dedupe?

2013-01-02 Thread tonyalbers
Hi everyone, Does anyone know whether bacula can be made to work on a block-level dedupe storage system? Are there any plans to support this technology? The reason I ask is that I've gotten a few requests from people who have devices like Quantum DXi's or DataDomain, and these both do block-lev