[ https://issues.apache.org/jira/browse/LUCENE-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2404: ---------------------------------- Fix Version/s: 4.0.0 > Improve speed of ThaiWordFilter by CharacterIterator, factor out LowerCasing > and also fix some bugs (empty tokens stop iteration) > --------------------------------------------------------------------------------------------------------------------------------- > > Key: LUCENE-2404 > URL: https://issues.apache.org/jira/browse/LUCENE-2404 > Project: Lucene - Java > Issue Type: Bug > Components: contrib/analyzers > Reporter: Uwe Schindler > Assignee: Uwe Schindler > Fix For: 3.1, 4.0.0 > > Attachments: LUCENE-2404-2.patch, LUCENE-2404-2.patch, > LUCENE-2404.patch, LUCENE-2404.patch > > > The ThaiWordFilter creates new Strings out of term buffer before passing to > The BreakIterator., But BreakIterator can take a CharacterIterator and > directly process on it without buffer copying. > As Java itsself does not provide a CharacterIterator implementation in > java.text, we can use the javax.swing.text.Segment class, that operates on a > char[] and is even reuseable! This class is very strange but it works and is > in JDK 1.4+ and not deprecated. > The filter also had a bug: It stopped iterating tokens when an empty token > occurred. Also the lowercasing for non-thai words was removed and put into > the Analyzer by adding LowerCaseFilter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org