Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread Marc Cousin
> Actually, it is worse than what you describe above. The problem is that I > did not consider in my proposal what happens in an Incremental backup. As > proposed, it simply will not work. > > > Maybe it would be easier to add a parentid in the Path table. > > This is probably a nice solution tha

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread demerphq
On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > On 26 Aug 2007 at 13:28, demerphq wrote: > > On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > > On 26 Aug 2007 at 0:52, demerphq wrote: > > [...] > > > > Select all and only children of a single node in a single query: > > > > select C.*

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread Kern Sibbald
On Thursday 30 August 2007 20:44, Marc Cousin wrote: > > > If we separate the current File data into Dirs and Files, first, we > > > have reduced the amount of data that we need to look through to find > > > the directory tree for a job by about a factor of 100 on most typical > > > Unix systems.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread Kern Sibbald
On Thursday 30 August 2007 20:44, Marc Cousin wrote: > > > If we separate the current File data into Dirs and Files, first, we > > > have reduced the amount of data that we need to look through to find > > > the directory tree for a job by about a factor of 100 on most typical > > > Unix systems.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-30 Thread Marc Cousin
> > If we separate the current File data into Dirs and Files, first, we have > > reduced the amount of data that we need to look through to find the > > directory tree for a job by about a factor of 100 on most typical Unix > > systems. That is already good. Then, in the Dirs table we can have th

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Cousin Marc
Le Tuesday 28 August 2007 16:12:56 Kern Sibbald, vous avez écrit : > On Tuesday 28 August 2007 13:30, Cousin Marc wrote: > > Le Tuesday 28 August 2007 11:57:28 Kern Sibbald, vous avez écrit : > > > On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > > > > For the visibility flag, it may n

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Kern Sibbald
On Tuesday 28 August 2007 13:30, Cousin Marc wrote: > Le Tuesday 28 August 2007 11:57:28 Kern Sibbald, vous avez écrit : > > On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > > > For the visibility flag, it may not be that easy : a directory may > > > > > be visible even if it's not in

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Cousin Marc
Le Tuesday 28 August 2007 11:57:28 Kern Sibbald, vous avez écrit : > On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > > For the visibility flag, it may not be that easy : a directory may be > > > > visible even if it's not in a backup. For instance, /home if > > > > /home/marc is backu

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Cousin Marc
Le Tuesday 28 August 2007 10:24:03 Kern Sibbald, vous avez écrit : > On Sunday 26 August 2007 09:17, Marc Cousin wrote: > > On Sunday 26 August 2007 07:43:25 Kern Sibbald wrote: > > > Hello Marc, > > > > > > I don't yet understand the details of how you use these tables, but it > > > seems to me th

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Kern Sibbald
On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > For the visibility flag, it may not be that easy : a directory may be > > > visible even if it's not in a backup. For instance, /home if /home/marc > > > is backuped should be displayed, so we add an entry from /home in > > > pathvisibil

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread BOLLENGIER Eric
> > For the visibility flag, it may not be that easy : a directory may be > > visible even if it's not in a backup. For instance, /home if /home/marc > > is backuped should be displayed, so we add an entry from /home in > > pathvisibility for the job where /home/marc is backed up. > > Isn't the vis

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Kern Sibbald
On Sunday 26 August 2007 09:17, Marc Cousin wrote: > On Sunday 26 August 2007 07:43:25 Kern Sibbald wrote: > > Hello Marc, > > > > I don't yet understand the details of how you use these tables, but it > > seems to me that we could modify the Bacula table structure as follows: > > > > - Split the F

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Marc Cousin
> If you don't need to grab a subtree then yes, PP is fine. If you do > its not ideal. I guess i misunderstood the problem, i thought you > wanted to get "all descendants of a given node". If not, well, > apologies for the noise. :-) > > Yves Yes, we just want direct children... The purpose is to

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Dan Langille
On 26 Aug 2007 at 13:28, demerphq wrote: > On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > On 26 Aug 2007 at 0:52, demerphq wrote: > [...] > > > > Can you explain the pros and cons over something like a parent-child > > > > relationship? That each, each file entry has a link to the parent

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread demerphq
On 8/26/07, Marc Cousin <[EMAIL PROTECTED]> wrote: > On Sunday 26 August 2007 00:52:33 demerphq wrote: [snip] > > The idea is fairly straight forward, each record has a Left and Right > > value, any children will have left and right values such that P.Left > > <= C.left and C.right <=P.right. An in

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread demerphq
On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > On 26 Aug 2007 at 0:52, demerphq wrote: [...] > > > Can you explain the pros and cons over something like a parent-child > > > relationship? That each, each file entry has a link to the parent > > > directory. This would be a self referential

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Marc Cousin
On Sunday 26 August 2007 00:52:33 demerphq wrote: > On 8/25/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > Hello Marc, > > > > > > > > Thanks for the information. I think this is something

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Marc Cousin
On Sunday 26 August 2007 07:43:25 Kern Sibbald wrote: > Hello Marc, > > I don't yet understand the details of how you use these tables, but it > seems to me that we could modify the Bacula table structure as follows: > > - Split the File table into two new tables: > - Files containing only file

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Hello Marc, I don't yet understand the details of how you use these tables, but it seems to me that we could modify the Bacula table structure as follows: - Split the File table into two new tables: - Files containing only file entries - Dirs containing the equivalent of a current File e

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dan Langille
On 26 Aug 2007 at 0:52, demerphq wrote: > On 8/25/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > Hello Marc, > > > > > > > > Thanks for the information. I think this is something that

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread demerphq
On 8/25/07, Dan Langille <[EMAIL PROTECTED]> wrote: > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > Hello Marc, > > > > > > Thanks for the information. I think this is something that is probably > > > well worth the effort to put in the

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 22:25:00 Dan Langille wrote: > On 25 Aug 2007 at 22:07, Marc Cousin wrote: > > On Saturday 25 August 2007 21:34:25 Dan Langille wrote: > > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > > Hello Marc, > > >

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dan Langille
On 25 Aug 2007 at 22:07, Marc Cousin wrote: > On Saturday 25 August 2007 21:34:25 Dan Langille wrote: > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > Hello Marc, > > > > > > > > Thanks for the information. I think this is something

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 21:34:25 Dan Langille wrote: > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > On Saturday 25 August 2007 10:31:04 you wrote: > > > Hello Marc, > > > > > > Thanks for the information. I think this is something that is probably > > > well worth the effort to put in th

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dan Langille
On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > On Saturday 25 August 2007 10:31:04 you wrote: > > Hello Marc, > > > > Thanks for the information. I think this is something that is probably > > well worth the effort to put in the core Bacula code. I'm going to study > > it a bit to see how it

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 17:48:22 Dirk Bartley wrote: > On Sat, 2007-08-25 at 09:43 +0200, Marc Cousin wrote: > > On Saturday 25 August 2007 09:03:49 Kern Sibbald wrote: > > > Hello Marc, > > > > > > Now that we have a number of "browsers" and users are doing a lot more > > > queries, it might be

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dirk Bartley
On Sat, 2007-08-25 at 09:43 +0200, Marc Cousin wrote: > On Saturday 25 August 2007 09:03:49 Kern Sibbald wrote: > > Hello Marc, > > > > Now that we have a number of "browsers" and users are doing a lot more > > queries, it might be a good idea to consider re-organizing a few of the > > Bacula table

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Thanks, Kern On Saturday 25 August 2007 10:51, Eric Bollengier wrote: > On Saturday 25 August 2007 10:31:04 you wrote: > > Hello Marc, > > > > Thanks for the information. I think this is something that is probably > > well worth the effort to put in the core Bacula code. I'm going to study > >

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Eric Bollengier
> For example, this is how we list files in a directory : > > SELECT File.FilenameId, listfiles.id, listfiles.Name, File.LStat, > File.JobId FROM > (SELECT Filename.Name, max(File.FileId) as id >FROM File, Filename >WHERE File.FilenameId = Filename.FilenameId > AND Fi

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Eric Bollengier
On Saturday 25 August 2007 10:31:04 you wrote: > Hello Marc, > > Thanks for the information. I think this is something that is probably > well worth the effort to put in the core Bacula code. I'm going to study > it a bit to see how it might be organized into a project. Code like this > might al

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Hello Marc, Thanks for the information. I think this is something that is probably well worth the effort to put in the core Bacula code. I'm going to study it a bit to see how it might be organized into a project. Code like this might also help a lot for the simple restores that Bacula curre

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 09:03:49 Kern Sibbald wrote: > Hello Marc, > > Now that we have a number of "browsers" and users are doing a lot more > queries, it might be a good idea to consider re-organizing a few of the > Bacula tables to improve the efficiency of the queries (and also perhaps > bac

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Hello Marc, Now that we have a number of "browsers" and users are doing a lot more queries, it might be a good idea to consider re-organizing a few of the Bacula tables to improve the efficiency of the queries (and also perhaps backups and restores). Please see more below... On Friday 24 Augu

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-24 Thread Marc Cousin
On Thursday 23 August 2007 04:02:07 Dirk Bartley wrote: > On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > > On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > > > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > There are basically three queries. One to get a list of > > > > >

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-23 Thread Dan Langille
On 22 Aug 2007 at 22:02, Dirk Bartley wrote: > On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > > If you can put your database on a website somewhere, I'll download it > > and try it out and work on the query that way. > > I will respectfully choose not to do that at this time. I fea

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-23 Thread Kern Sibbald
On Thursday 23 August 2007 04:02, Dirk Bartley wrote: > On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > > On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > > > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > There are basically three queries. One to get a list of > > > > > d

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-22 Thread Dirk Bartley
On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > > > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > > > > There are basically three queries. One to get a list of directories, > > > > another to get a list of files and one t

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-22 Thread Dan Langille
On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > > There are basically three queries. One to get a list of directories, > > > another to get a list of files and one to get a list of versions for a > > > specific file. > > > > > > 1.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-22 Thread Kern Sibbald
On Sunday 19 August 2007 18:35, Dirk Bartley wrote: > Hello > > Now that 2.2.0 is released, I've had the opportunity to use bat in my > work environment. It worked generally as hoped with one major > exception. The catalog browser. slow slow slow. Yes, in some cases, this is my experience too.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-21 Thread Dirk Bartley
On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > There are basically three queries. One to get a list of directories, > > another to get a list of files and one to get a list of versions for a > > specific file. > > > > 1. To get a list of directories given a list of jobs. Here is

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-20 Thread Cousin Marc
Hi, I'm interfering in the discussion as I've worked with Eric on brestore (doing the SQL part), doing something looking a lot like what you're doing now. My comments are inline Le Monday 20 August 2007 03:47:15 Dirk Bartley, vous avez écrit : > On Sun, 2007-08-19 at 13:06 -0400, Dan Langille w

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-20 Thread Dan Langille
On 19 Aug 2007 at 21:47, Dirk Bartley wrote: > On Sun, 2007-08-19 at 13:06 -0400, Dan Langille wrote: > > On 19 Aug 2007 at 12:35, Dirk Bartley wrote: > >=20 > > > There is one alternative index to the above. The index of the two > > > colums (FilenameId, JobId) is really only needed for the cond

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-19 Thread Dirk Bartley
On Sun, 2007-08-19 at 13:06 -0400, Dan Langille wrote: > On 19 Aug 2007 at 12:35, Dirk Bartley wrote: > > > There is one alternative index to the above. The index of the two > > colums (FilenameId, JobId) is really only needed for the condition of > > FilenameId = (SELECT FilenameId FROM Filename

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-19 Thread Dan Langille
On 19 Aug 2007 at 12:35, Dirk Bartley wrote: > There is one alternative index to the above. The index of the two > colums (FilenameId, JobId) is really only needed for the condition of > FilenameId = (SELECT FilenameId FROM Filename WHERE name=''). A > subquery can't be used when creating an ind