Re: FacetedSearch and MultiReader

2013-04-09 Thread Shai Erera
Hello Nicola, I think it would be good if you start a new thread to discuss this problem, as I don't think it's related to the issue in this thread. Also, I did not understand what's the problem you're running into. What used to work before 4.2 and doesn't work now? Shai On Tue, Apr 9, 2013 at

Re: FacetedSearch and MultiReader

2013-04-09 Thread Nicola Buso
Hi, I'm trying to use Lucene 4.2, but this merge of more taxonomy indexes seam is no more working. Do you have any idea why it has not to work in Lucene 4.2? Normal faceted search on a single index is working correctly. Nicola. On Thu, 2013-01-24 at 16:53 +, Nicola Buso wrote: > Hi Shai, >

Re: FacetedSearch and MultiReader

2013-01-24 Thread Nicola Buso
Hi Shai, I'd like just to give you a confirmation that your solution is working after the tests I did. Thanks again for the useful hints. Nicola. On Tue, 2013-01-22 at 06:20 +0200, Shai Erera wrote: > Hi Nicola, > > What I had in mind is something similar to this, which is possible starting >

Re: FacetedSearch and MultiReader

2013-01-23 Thread Nicola Buso
Thanks Shai, I'm trying your solution and it's working, I need to check some number to test it. As I said we are aware having big indexes, than I use facets only on subsets, but if it will result in performances issues too than I'll for sure take a look into facet sampling. Nicola. On Wed, 2013

Re: FacetedSearch and MultiReader

2013-01-23 Thread Shai Erera
> > I think we should open an issue to provide support for distributed > faceting? > Opened https://issues.apache.org/jira/browse/LUCENE-4710. BTW Nicola, I remember you said something about TBs of indexes. I just wanted to point out that if you have really large indexes, with many documents, the

Re: FacetedSearch and MultiReader

2013-01-22 Thread Michael McCandless
On Mon, Jan 21, 2013 at 11:20 PM, Shai Erera wrote: > (unfortunately, there's still no tool in Lucene to do that for you). I think we should open an issue to provide support for distributed faceting? For example, we already provide support for distributed searching (TopDocs.merge), and distribu

Re: FacetedSearch and MultiReader

2013-01-22 Thread Shai Erera
Yes, the release is wrapping up. I believe that an announcement message will be sent in the coming days. Shai On Tue, Jan 22, 2013 at 2:51 PM, Nicola Buso wrote: > I will try it. > > I see there is already a lucene-4.1.0 package (dated 2013/01/21) > available for download, do you know if this

Re: FacetedSearch and MultiReader

2013-01-22 Thread Nicola Buso
I will try it. I see there is already a lucene-4.1.0 package (dated 2013/01/21) available for download, do you know if this version will be released soon? Nicola. On Tue, 2013-01-22 at 06:20 +0200, Shai Erera wrote: > Hi Nicola, > > What I had in mind is something similar to this, which is pos

Re: FacetedSearch and MultiReader

2013-01-21 Thread Shai Erera
Hi Nicola, What I had in mind is something similar to this, which is possible starting with Lucene 4.1, due to changes done to facets (per-segment faceting): DirTaxoWriter master = new DirTaxoWriter(masterDir); Directory[] origTaxoDirs = new Directory[numTaxoDirs]; // open Directories and store i

Re: FacetedSearch and MultiReader

2013-01-21 Thread Denis Bazhenov
We have similar distribute search system and we have finished with the following scheme. Search replicas (machines where index resides) are build FacetResult's based on their index chunk (top N categories with document counts). Later on the results are merged "by hands" with summing relevant ca

[Fwd: Re: FacetedSearch and MultiReader]

2013-01-21 Thread Nicola Buso
--- Begin Message --- Hi, it's not clear your proposal. On Mon, 2013-01-21 at 18:21 +0200, Shai Erera wrote: > Hi > > > First, if it's a one time operation, you can merge the taxonomy > indexes into one, without merging the content indexes too (but you'll > need to re-map the ordinals in each

Re: FacetedSearch and MultiReader

2013-01-21 Thread Nicola Buso
Hi Shai, I was thinking to that too, but I'm indexing all indexes in a custom distributed environment than I can't in this moment have a single categories index for all the content indexes at indexing time. A solution should be to merge all the categories indexes in one only index and use your sol

Re: FacetedSearch and MultiReader

2013-01-21 Thread Shai Erera
Hi Nicola, I think that what you're describing corresponds to distributed faceted search. I.e., you have N content indexes, alongside N taxonomy indexes. The information that's indexed in each of those sub-indexes does not correlate with the other ones. For example, say that you index the category

Re: FacetedSearch and MultiReader

2013-01-21 Thread Nicola Buso
Thanks for the reply Uwe, we currently can search with MultiReader over all the indexes we have. Now I want to add the faceting search, than I created a categories index for every index I currently have. To accumulate the faceted results now I have a MultiReader pointing all the indexes and I can

RE: FacetedSearch and MultiReader

2013-01-21 Thread Uwe Schindler
Just use MultiReader, it extends IndexReader, so you can pass it anywhere where IndexReader can be passed. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Nicola Buso [mailto:nb...@ebi.ac.uk] > Sent: Mon