[
https://issues.apache.org/jira/browse/LUCENE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658308#action_12658308
]
Hoss Man commented on LUCENE-1491:
----------------------------------
patch looks good ... the one question i have is whether the fix meets user
expectations: the patch as posted "skips" any input tokens that are shorter
then the minimum ngram length ... is that what most people will expect, or will
people expect shorter tokens to be passed through?
ie: should "min" be the minimum token size produced by the filters (a hard
min), or should it be the minimum ngram size produced by the filter (a soft
min)?
either way this patch is an improvement, i'm just wondering what we want to
define the semantics to be (or if we want to make an additional option for this)
> EdgeNGramTokenFilter stops on tokens smaller then minimum gram size.
> --------------------------------------------------------------------
>
> Key: LUCENE-1491
> URL: https://issues.apache.org/jira/browse/LUCENE-1491
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 2.4, 2.4.1, 2.9, 3.0
> Reporter: Todd Feak
> Attachments: LUCENE-1491.patch
>
>
> If a token is encountered in the stream that is shorter in length than the
> min gram size, the filter will stop processing the token stream.
> Working up a unit test now, but may be a few days before I can provide it.
> Wanted to get it in the system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
