[
https://issues.apache.org/jira/browse/LUCENE-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13062382#comment-13062382
]
Yonik Seeley commented on LUCENE-3233:
--------------------------------------
bq. but I'd like to rename existing syn filter to SlowSynonymFilter and rename
the new one to SynonymFilter.
But the lookup on the original is still faster, right? And if someone has
small synonym dicts (actually pretty common in my experience since
SynonymFilter isn't necessary used to inject synonyms in the traditional sense,
but for any mapping task) then build time and mem use won't be much of an issue
(esp if the input to match is mostly single words).
This looks great for large synonym maps, but perhaps instead of Slow* or Fast*
we could name them for the implementation and either name the new one
FSTSynonymFilter or rename the current one to MapSynonymFilter? Or is the plan
to actually deprecate the current SynonymFilter?
> HuperDuperSynonymsFilter™
> -------------------------
>
> Key: LUCENE-3233
> URL: https://issues.apache.org/jira/browse/LUCENE-3233
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Robert Muir
> Attachments: LUCENE-3223.patch, LUCENE-3233.patch, LUCENE-3233.patch,
> LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch,
> LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch,
> LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, synonyms.zip
>
>
> The current synonymsfilter uses a lot of ram and cpu, especially at build
> time.
> I think yesterday I heard about "huge synonyms files" three times.
> So, I think we should use an FST-based structure, sharing the inputs and
> outputs.
> And we should be more efficient with the tokenStream api, e.g. using
> save/restoreState instead of cloneAttributes()
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
