[ https://issues.apache.org/jira/browse/LUCENENET-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393447#comment-13393447 ]
Prescott Nasser commented on LUCENENET-337: ------------------------------------------- Is this still wanted in 3.0.3? > TokenAttribute for Selectively Including Tokens in Length Norm > -------------------------------------------------------------- > > Key: LUCENENET-337 > URL: https://issues.apache.org/jira/browse/LUCENENET-337 > Project: Lucene.Net > Issue Type: Improvement > Components: Lucene.Net Core > Affects Versions: Lucene.Net 2.9.2 > Reporter: Michael Garski > Priority: Minor > Fix For: Lucene.Net 3.0.3 > > Attachments: LengthNorm.patch > > > This patch adds functionality to Lucene.Net that allow a TokenFilter to mark > a Token as not to be included in the length norm calculation through the use > of a new TokenAttribute interface LengthNormAttribute and a corresponding > implementation LengthNormAttributeImpl. This functionality is useful to > prevent the increase of the length norm during synonym injection, > particularly in cases where there are a large number of synonyms in relation > to the number of original tokens. > Following is an example of how to use the new attribute. > Within your custom TokenFilter, define a field to persist a reference to the > attribute and set it's value in the constructor. When a the stream advances > to a new Token within the call to IncrementToken() the value of the > IncludeInLengthNorm property of the attribute is set to false for Tokens > which should not be included in the length norm calculation. It defaults to > true and is reset to true after each Token is consumed within > DocInverterPerField.ProcessFields. > {code:title=CustomTokenFilter.cs|borderStyle=solid} > public class CustomTokenFilter : TokenFilter > { > private LengthNormAttribute lnAttribute; > > public CustomTokenFilter(TokenStream input) : base(input) > { > this.lnAttribute = > (LengthNormAttribute)AddAttribute(typeof(LengthNormAttribute)); > } > > public override bool IncrementToken() > { > if (input.IncrementToken()) > { > // make determination that the token is not to be > // included in the length norm value > // this example marks all tokens to not be > // included in the length norm value > this.lnAttribute.IncludeInLengthNorm = false; > return true; > } > else > { > return false; > } > } > } > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira