[
https://issues.apache.org/jira/browse/LUCENE-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113117#comment-13113117
]
Chris Male commented on LUCENE-2309:
------------------------------------
bq. In Field.java do we already check that if the field is not tokenized then
it has a non-null stringValue()?
I don't think we do. Its always been implied (which could cause a bug). I'll
add the appropriate checks but we really need to revisit the constructors of
Field at some stage.
bq. I would like to for IW to not have to pass through the Analyzer here (ie
FieldType should know the Analyzer for that field), but let's save that for
another issue/time.
I totally agree. Theoretically FieldType could have Analyzer added to it now
and it could make use of it. But removing the Analyzer from IW seems
controversial, alas :)
bq. Likewise, multi-valued field should ideally be "under the hood" from IW's
standpoint, ie we should have a MultiValuedField and you append to a List
inside it, and then IW gets a single TokenStream from that, which does its own
concatenating of the separate TokenStreams, but we should tackle that under a
separate issue.
Its nearly possible. We've almost there on the reusable Analyzers. This can
already begin actually for non-tokenized fields and for NumericFields.
I'll make the non-null StringValue checks and then commit.
> Fully decouple IndexWriter from analyzers
> -----------------------------------------
>
> Key: LUCENE-2309
> URL: https://issues.apache.org/jira/browse/LUCENE-2309
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Reporter: Michael McCandless
> Labels: gsoc2011, lucene-gsoc-11, mentor
> Fix For: 4.0
>
> Attachments: LUCENE-2309-analyzer-based.patch,
> LUCENE-2309-getTSFromField.patch, LUCENE-2309.patch
>
>
> IndexWriter only needs an AttributeSource to do indexing.
> Yet, today, it interacts with Field instances, holds a private
> analyzers, invokes analyzer.reusableTokenStream, has to deal with a
> wide variety (it's not analyzed; it is analyzed but it's a Reader,
> String; it's pre-analyzed).
> I'd like to have IW only interact with attr sources that already
> arrived with the fields. This would be a powerful decoupling -- it
> means others are free to make their own attr sources.
> They need not even use any of Lucene's analysis impls; eg they can
> integrate to other things like [OpenPipeline|http://www.openpipeline.org].
> Or make something completely custom.
> LUCENE-2302 is already a big step towards this: it makes IW agnostic
> about which attr is "the term", and only requires that it provide a
> BytesRef (for flex).
> Then I think LUCENE-2308 would get us most of the remaining way -- ie, if the
> FieldType knows the analyzer to use, then we could simply create a
> getAttrSource() method (say) on it and move all the logic IW has today
> onto there. (We'd still need existing IW code for back-compat).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
