[
https://issues.apache.org/jira/browse/LUCENE-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188592#comment-13188592
]
Steven Rowe commented on LUCENE-3666:
-------------------------------------
bq. This example consumer code is incomplete:
[snip]
The fixed version:
{code:java}
<PRE class="prettyprint">
Version matchVersion = Version.LUCENE_XY; // Substitute desired Lucene
version for XY
Analyzer analyzer = new StandardAnalyzer(matchVersion); // or any other
analyzer
TokenStream ts = analyzer.tokenStream("myfield", new StringReader("some
text goes here"));
OffsetAttribute offsetAtt = addAttribute(OffsetAttribute.class);
try {
ts.reset(); // Resets this stream to the beginning. (Required)
while (ts.incrementToken()) {
// Use {@link
org.apache.lucene.util.AttributeSource#reflectAsString(boolean)}
// for token stream debugging.
System.out.println("token: " + ts.reflectAsString(true));
System.out.println("token start offset: " + offsetAtt.startOffset());
System.out.println(" token end offset: " + offsetAtt.endOffset());
}
ts.end(); // Perform end-of-stream operations, e.g. set the final
offset.
} finally {
ts.close(); // Release resources associated with this stream.
}
</PRE>
{code}
I also wrapped the other {{TokenStream}} examples with {code:java}try { ... }
finally { ts.close(); }{code}
bq. The copyTo must call set setPos() method of the attribute interface.
Here's the fixed version:
{code:java}
{@literal @Override}
public void copyTo(AttributeImpl target) {
((PartOfSpeechAttribute) target).setPartOfSpeech(pos);
}
{code}
I'll commit shortly.
> Update org.apache.lucene.analysis package summary
> -------------------------------------------------
>
> Key: LUCENE-3666
> URL: https://issues.apache.org/jira/browse/LUCENE-3666
> Project: Lucene - Java
> Issue Type: Improvement
> Components: general/javadocs
> Affects Versions: 3.5
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Priority: Minor
> Fix For: 3.6, 4.0
>
> Attachments: LUCENE-3666-branch_3x.patch,
> LUCENE-3666-branch_3x.patch, LUCENE-3666-branch_3x.patch,
> LUCENE-3666-branch_3x.patch, LUCENE-3666-trunk.patch, LUCENE-3666-trunk.patch
>
>
> {{package.html}} in {{lucene/src/java/org/apache/lucene/analysis/}} is out of
> date.
> It looks like the contents of the branch_3x version haven't changed
> substantially since the Lucene 2.9 release, e.g. it refers to
> {{TermAttribute}} instead of {{CharTermAttribute}}.
> The trunk version is more modern - it refers to {{CharTermAttribute}} - but
> it also has some issues. E.g., I can see that the {{LengthFilter}}
> discussion doesn't refer to {{FilteringTokenFilter}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
