Re: [Bacula-devel] space saving in the database

2008-02-12 Thread David Boyes
> How about Linux extended attributes (used by SELinux for example). > And I guess ACLs could be saved as such metadata, too? Great minds think alike...8-) A Unix stat structure is fundamentally just a bunch of metadata about an inode or chain of inodes. It probably makes sense to keep the name a

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Marc Schiffbauer
* Kern Sibbald schrieb am 12.02.08 um 19:23 Uhr: > On Tuesday 12 February 2008 18.46:06 David Boyes wrote: > > > Concerning your second paragraph, what I am proposing is a sort of > > > refactoring > > > of the lstat code that will be the first step in something like what > > > > you > > > > > prop

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Tuesday 12 February 2008 21.12:58 [EMAIL PROTECTED] wrote: > El mar, 12-02-2008 a las 19:23 +0100, Kern Sibbald escribió: > > Well, to get to that point, either someone will need to send in a patch > > or I will need to get access to a "system with more sophisticated > > filesystem structures" .

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread [EMAIL PROTECTED]
El mar, 12-02-2008 a las 19:23 +0100, Kern Sibbald escribió: > Well, to get to that point, either someone will need to send in a patch or I > will need to get access to a "system with more sophisticated filesystem > structures" ... > Don't worry, it's coming to you ;) http://oss.oracle.com/p

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Tuesday 12 February 2008 18.46:06 David Boyes wrote: > > Concerning your second paragraph, what I am proposing is a sort of > > refactoring > > of the lstat code that will be the first step in something like what > > you > > > propose. > > Good. For systems with more sophisticated filesystem str

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread David Boyes
> Concerning your second paragraph, what I am proposing is a sort of > refactoring > of the lstat code that will be the first step in something like what you > propose. Good. For systems with more sophisticated filesystem structures than the Unix byte stream model, it'd be very nice to have a well

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
Thanks for the first paragraph :-) Concerning your second paragraph, what I am proposing is a sort of refactoring of the lstat code that will be the first step in something like what you propose. On Tuesday 12 February 2008 16.20:41 David Boyes wrote: > > Just my opinion, of course. I'd be in

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Tuesday 12 February 2008 15.59:10 Jason A. Kates wrote: > Bill, > > I too think that we should defer to Kern and his list of priorities. Thanks for your vote of confidence :-) It is interesting because this is the first time I have not worked on the highest priority job (Item 1: Accurate rest

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Tuesday 12 February 2008 15.52:36 Bill Moran wrote: > In response to Cousin Marc <[EMAIL PROTECTED]>: > > [snip] > > > > One hash is not possible without seriously restricting user's > > > flexibility -- the MD5 field though not totally used as planned in 2.2. > > > (hopefully it will be in 2.2)

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread David Boyes
> Just my opinion, of course. I'd be interested to hear how much effect > this would have on others and whether they think it's worthwhile to even > investigate. Moved and seconded. I don't think the world needs to reinvent packed-decimal yet again and all the grief that fancy encoding schemes c

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Jason A. Kates
Bill, I too think that we should defer to Kern and his list of priorities. Kern has been doing a great job of prioritizing and running this project. I think that people that are that tight on storage should prune the files a few days earlier and let Kern work on functionality, disk space is fa

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Tuesday 12 February 2008 15.31:54 Adam Thornton wrote: > On Feb 12, 2008, at 2:26 AM, Kern Sibbald wrote: > > One hash is not possible without seriously restricting user's > > flexibility -- > > the MD5 field though not totally used as planned in 2.2. (hopefully > > it will > > be in 2.2) is a c

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Tuesday 12 February 2008 13.44:51 Cousin Marc wrote: > > Send me your Volume, Job, and File retention periods and the scheme you > > use for doing Full, Differential, and Incremental backups and your > > desired granularity of recovery, and I will tell you whether or not this > > idea will help

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Bill Moran
In response to Cousin Marc <[EMAIL PROTECTED]>: [snip] > > One hash is not possible without seriously restricting user's flexibility > > -- the MD5 field though not totally used as planned in 2.2. (hopefully it > > will be in 2.2) is a critical field for security and certain government > > legal

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Adam Thornton
On Feb 12, 2008, at 2:26 AM, Kern Sibbald wrote: > > One hash is not possible without seriously restricting user's > flexibility -- > the MD5 field though not totally used as planned in 2.2. (hopefully > it will > be in 2.2) is a critical field for security and certain government > legal > r

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Cousin Marc
> Send me your Volume, Job, and File retention periods and the scheme you use > for doing Full, Differential, and Incremental backups and your desired > granularity of recovery, and I will tell you whether or not this idea will > help you. > > I suspect that just doing strict pruning will reduce yo

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Monday 11 February 2008 21.02:23 Marc Cousin wrote: > On Monday 11 February 2008 20:46:09 Bill Moran wrote: > > In response to Marc Cousin <[EMAIL PROTECTED]>: > > > > [snip] > > > > > Still, I'd like to know if the md5 field is always a multiple of 32 > > > bits in length ? > > > > md5 is _alwa

Re: [Bacula-devel] space saving in the database

2008-02-12 Thread Kern Sibbald
On Monday 11 February 2008 20.34:40 Marc Cousin wrote: > On Monday 11 February 2008 18:11:10 Kern Sibbald wrote: > > Hello, > > > > I am really happy to see that someone besides myself is interested in > > this problem. I am quite concerned because since 2.2.0 came out, with > > its new more corre

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Bill Moran
In response to Marc Cousin <[EMAIL PROTECTED]>: > On Monday 11 February 2008 21:13:17 Bill Moran wrote: > > In response to Marc Cousin <[EMAIL PROTECTED]>: > > > On Monday 11 February 2008 20:46:09 Bill Moran wrote: > > > > In response to Marc Cousin <[EMAIL PROTECTED]>: > > > > > > > > [snip] > >

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Marc Cousin
On Monday 11 February 2008 21:13:17 Bill Moran wrote: > In response to Marc Cousin <[EMAIL PROTECTED]>: > > On Monday 11 February 2008 20:46:09 Bill Moran wrote: > > > In response to Marc Cousin <[EMAIL PROTECTED]>: > > > > > > [snip] > > > > > > > Still, I'd like to know if the md5 field is always

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Bill Moran
In response to Marc Cousin <[EMAIL PROTECTED]>: > On Monday 11 February 2008 20:46:09 Bill Moran wrote: > > In response to Marc Cousin <[EMAIL PROTECTED]>: > > > > [snip] > > > > > Still, I'd like to know if the md5 field is always a multiple of 32 bits > > > in length ? > > > > md5 is _always_ ex

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Marc Cousin
On Monday 11 February 2008 20:46:09 Bill Moran wrote: > In response to Marc Cousin <[EMAIL PROTECTED]>: > > [snip] > > > Still, I'd like to know if the md5 field is always a multiple of 32 bits > > in length ? > > md5 is _always_ exactly 128 bits. It's frequently represented as a 32 > character he

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Bill Moran
In response to Marc Cousin <[EMAIL PROTECTED]>: [snip] > > Still, I'd like to know if the md5 field is always a multiple of 32 bits in > length ? md5 is _always_ exactly 128 bits. It's frequently represented as a 32 character hex string, but that's just a friendly way to display it. If you us

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Marc Cousin
On Monday 11 February 2008 18:11:10 Kern Sibbald wrote: > Hello, > > I am really happy to see that someone besides myself is interested in this > problem. I am quite concerned because since 2.2.0 came out, with its new > more correct pruning algorithm when finding a Volume, there is a lot less > u

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Kern Sibbald
Hello, I am really happy to see that someone besides myself is interested in this problem. I am quite concerned because since 2.2.0 came out, with its new more correct pruning algorithm when finding a Volume, there is a lot less unnecessary pruning taking place, and the database for my backups

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Cousin Marc
Here are the current pieces of code. If you want to try it, here's how : - first, you have to be using pg 8.3 (i'll backport, there is very little work to do on that) - second, a word of caution : it's a postgresql/C stored procedure. It means it can crash postgresql. It doesn't do it on my compu

Re: [Bacula-devel] space saving in the database

2008-02-11 Thread Dan Langille
Cousin Marc wrote: > Hi, > > First an introduction on what I'm trying to do, so you know what we're > playing > with : > > Eric Bollengier and I have been discussing about saving space in the > database, > as ours is getting bigger and bigger (we're nearly at 300 million rows in > file table