Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Thank you so much, Shai... Chitra On Wed, Nov 30, 2016 at 2:17 PM, Shai Erera wrote: > This feature is not available in Lucene currently, but it shouldn't be hard > to add it. See Mike's comment here: > http://blog.mikemccandless.com/2013/05/dynamic-faceting- >

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

This feature is not available in Lucene currently, but it shouldn't be hard to add it. See Mike's comment here: http://blog.mikemccandless.com/2013/05/dynamic-faceting-with-lucene.html?showComment=1412777154420#c363162440067733144 One more tricky (yet nicer) feature would be to have it all in one

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Thank you so much, mike... Hope, gained a lot of stuff on Doc Values faceting and also clarified all my doubts. Thanks..!! *Another use case:* After getting matching documents for the given query, Is there any way to calculate mix and max values on NumericDocValuesField ( say date field)? I

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Doc values fields are never loaded into memory; at most some small index structures are. When you use those fields, the bytes (for just the one doc values field you are using) are pulled from disk, and the OS will cache them in memory if available. Mike McCandless http://blog.mikemccandless.com

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hi, When opening SortedSetDocValuesReaderState at search time, whether the whole doc value files (.dvd & .dvm) information are loaded in memory or specified field information(say $facets field) alone load in memory? Any help is much appreciated. Regards, Chitra On Tue, Nov 22, 2016

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Kindly post your suggestions. Regards, Chitra On Sat, Nov 19, 2016 at 1:38 PM, Chitra R wrote: > Hey, I got it clearly. Thank you so much. Could you please help us to > implement it in our use case? > > > In our case, we are having dynamic

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hey, I got it clearly. Thank you so much. Could you please help us to implement it in our use case? In our case, we are having dynamic index and it is variable depth too. So flat facet is enough.No need of hierarchical facets. What I think is, 1. Index my facet field as normal doc value

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

I think you've summed up exactly the differences! And, yes, it would be possible to emulate hierarchical facets on top of flat facets, if the hierarchy is fixed depth like year/month/day. But if it's variable depth, it's trickier (but I think still possible). See e.g. the Committed Paths

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

case 1: In taxonomy, for each indexed document, examines facet label , computes their ordinals and mappings, and which will be stored in sidecar index at index time. case 2: In doc values, these(ordinals) are computed at search time, so there will be a time and memory trade-off

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Okay. I agree with you, Taxonomy maintains and supports hierarchical facets during indexing. Hope hierarchical in the sense, we might index the field Publish date : 2010/10/15 as Publish date: 2010 , Publish date: 2010/10 and Publish date: 2010/10/15 , their facet ordinals are maintained in

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

You store dimension + string (a single value path, since it's not hierarchical) into SSDVFF so that you can compute facet counts, either ordinary drill down counts or the drill sideways counts. You can see examples of drill sideways at http://jirasearch.mikemccandless.com, e.g. drill down on any

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hi, Lucene-Drill sideways jira_issue:LUCENE-4748 Is this the reason( ie Drill sideways makes a very nice faceted search UI

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

No, SSDVFF does not do hierarchical faceting today, but this is just a limitation of the current implementation, and with some changes (patches welcome!), it could do so. Mike McCandless http://blog.mikemccandless.com On Mon, Nov 14, 2016 at 1:38 AM, Chitra R wrote: > >

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hey, thank you so much for the fast response, I agree NRT refresh is somewhat costly operations and this is the major pitfall, suppose we use doc value faceting. While indexing SortedSetDocValuesFacetField , it stores path and dimension of the given field internally. So Can

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

On Fri, Nov 11, 2016 at 5:21 AM, Chitra R wrote: > i)Hope, when opening SortedSetDocValuesReaderState , we are > calculating ordinals( this will be used to calculate facet count ) for doc > values field and this only made the state instance somewhat costly. >

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hi Shai, i)Hope, when opening SortedSetDocValuesReaderState , we are calculating ordinals( this will be used to calculate facet count ) for doc values field and this only made the state instance somewhat costly. Am I right or any other reason behind that?

Re: Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hi The reason IMO is historic - ES and Solr had faceting solutions before Lucene had it. There were discussions in the past about using the Lucene faceting module in Solr (can't tell for ES) but, sadly, I can't say I see it happening at this point. Regarding your other question, IMO the Lucene

Faceting : what are the limitations of Taxonomy (Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index) ?

Hi All, We all know that Lucene supports faceting by providing Taxonomy(Separate index and hierarchical facets) and SortedSetDocValuesFacetField ( flat facets and no sidecar index). Then why did solr and elastic search go for its own implementation ? ( that is, solr uses block