[
https://issues.apache.org/jira/browse/LUCENENET-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Troy Howard updated LUCENENET-337:
----------------------------------
Component/s: Lucene.Net Core
Due Date: 28/Mar/11
Affects Version/s: Lucene.Net 2.9.2
Fix Version/s: Lucene.Net 2.9.4
Scheduled for 2.9.4 release.
> TokenAttribute for Selectively Including Tokens in Length Norm
> --------------------------------------------------------------
>
> Key: LUCENENET-337
> URL: https://issues.apache.org/jira/browse/LUCENENET-337
> Project: Lucene.Net
> Issue Type: Improvement
> Components: Lucene.Net Core
> Affects Versions: Lucene.Net 2.9.2
> Reporter: Michael Garski
> Priority: Minor
> Fix For: Lucene.Net 2.9.4
>
> Attachments: LengthNorm.patch
>
>
> This patch adds functionality to Lucene.Net that allow a TokenFilter to mark
> a Token as not to be included in the length norm calculation through the use
> of a new TokenAttribute interface LengthNormAttribute and a corresponding
> implementation LengthNormAttributeImpl. This functionality is useful to
> prevent the increase of the length norm during synonym injection,
> particularly in cases where there are a large number of synonyms in relation
> to the number of original tokens.
> Following is an example of how to use the new attribute.
> Within your custom TokenFilter, define a field to persist a reference to the
> attribute and set it's value in the constructor. When a the stream advances
> to a new Token within the call to IncrementToken() the value of the
> IncludeInLengthNorm property of the attribute is set to false for Tokens
> which should not be included in the length norm calculation. It defaults to
> true and is reset to true after each Token is consumed within
> DocInverterPerField.ProcessFields.
> {code:title=CustomTokenFilter.cs|borderStyle=solid}
> public class CustomTokenFilter : TokenFilter
> {
> private LengthNormAttribute lnAttribute;
>
> public CustomTokenFilter(TokenStream input) : base(input)
> {
> this.lnAttribute =
> (LengthNormAttribute)AddAttribute(typeof(LengthNormAttribute));
> }
>
> public override bool IncrementToken()
> {
> if (input.IncrementToken())
> {
> // make determination that the token is not to be
> // included in the length norm value
> // this example marks all tokens to not be
> // included in the length norm value
> this.lnAttribute.IncludeInLengthNorm = false;
> return true;
> }
> else
> {
> return false;
> }
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
