[
https://issues.apache.org/jira/browse/LUCENE-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530182#comment-13530182
]
Shai Erera commented on LUCENE-4619:
------------------------------------
Let's leave the ctor for now, I'll make sure that FIP returns the right API in
LUCENE-4621. Iterable is great, but I think that we can risk a List? ;).
I thought about the FacetsField more. FacetsField is not really a field, right?
You cannot set its FieldType (not critical) and any of Field's set()/get()
methods are just in the way when you look at it. Also, it cannot actually
create two fields (won't be needed hopefully w/ DV), and adding the same
FacetsField twice w/ different categories will create multiple positions, which
is something that I'm not sure we should introduce.
So maybe we improve CDB's API to make it more approachable? I.e., for reusing
FacetsField you'd have to add setCPs to it? That's really like CDB. The only
different that remains then is the ability to Document.add(new FacetsField) vs.
CDB.addFields(Document).
You could do new FacetsField("Author"), but that doesn't simplify a lot I
think? I.e. you could still add the same field twice. And thinking about the
faceted search apps that I wrote, I don't always know which facets I'm going to
add. Sometimes I just get a list of CPs (read them from JSON, XML, programmatic
API) and I add them all in one shot. Starting to break that into fields will be
a mess.
While we could always say "let's add FacetsField for the common people and keep
CDB for the crazy ones", I prefer if there was one entry point. It's also
easier down the line to make changes / fix bugs?
And that that CDB is a Builder, lets you do iw.addDocument(cdb.addFields(new
Document())), ain't that awesome !? :)
I mean in the end of the day, CDB is not of type Field. But though FacetsField
is, it's very limited and app has to excess more logic when dealing with it.
Vs. CDB where you just call addFields(CPs, doc)?
> Create a specialized path for facets counting
> ---------------------------------------------
>
> Key: LUCENE-4619
> URL: https://issues.apache.org/jira/browse/LUCENE-4619
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/facet
> Reporter: Shai Erera
> Attachments: LUCENE-4619.patch
>
>
> Mike and I have been discussing that on several issues (LUCENE-4600,
> LUCENE-4602) and on GTalk ... it looks like the current API abstractions may
> be responsible for some of the performance loss that we see, compared to
> specialized code.
> During our discussion, we've decided to target a specific use case - facets
> counting and work on it, top-to-bottom by reusing as much code as possible.
> Specifically, we'd like to implement a FacetsCollector/Accumulator which can
> do only counting (i.e. respects only CountFacetRequest), no sampling,
> partitions and complements. The API allows us to do so very cleanly, and in
> the context of that issue, we'd like to do the following:
> * Implement a FacetsField which takes a TaxonomyWriter, FacetIndexingParams
> and CategoryPath (List, Iterable, whatever) and adds the needed information
> to both the taxonomy index as well as the search index.
> ** That API is similar in nature to CategoryDocumentBuilder, only easier to
> consume -- it's just another field that you add to the Document.
> ** We'll have two extensions for it: PayloadFacetsField and
> DocValuesFacetsField, so that we can benchmark the two approaches.
> Eventually, one of them we believe, will be eliminated, and we'll remain w/
> just one (hopefully the DV one).
> * Implement either a FacetsAccumulator/Collector which takes a bunch of
> CountFacetRequests and returns the top-counts.
> ** Aggregations are done in-collection, rather than post. Note that we have
> LUCENE-4600 open for exploring that. Either we finish this exploration here,
> or do it there. Just FYI that the issue exists.
> ** Reuses the CategoryListIterator, IntDecoder and Aggregator code. I'll open
> a separate issue to explore improving that API to be bulk, and then we can
> decide if this specialized Collector should use those abstractions, or be
> really optimized for the facet counting case.
> * At the moment, this path will assume that a document holds multiple
> dimensions, but only one value from each (i.e. no Author/Shai, Author/Mike
> for a document), and therefore use OrdPolicy.NO_PARENTS.
> ** Later, we'd like to explore how to have this specialized path handle the
> ALL_PARENTS case too, as it shouldn't be so hard to do.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
