[jira] [Updated] (LUCENE-3042) AttributeSource can have an invalid computed state
[ https://issues.apache.org/jira/browse/LUCENE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-3042: -- Fix Version/s: 3.1 3.0.4 2.9.5 I thought about it: This bug is so serious it should be fixed in all branches, too (even if never released anymore). This is important for e.g. 2.9 users whcih are stuck with that version. Committed 3.1 branch revision: 1096127 Committed 3.0 branch revision: 1096128 Committed 2.9 branch revision: 1096129 AttributeSource can have an invalid computed state -- Key: LUCENE-3042 URL: https://issues.apache.org/jira/browse/LUCENE-3042 Project: Lucene - Java Issue Type: Bug Components: Analysis Affects Versions: 2.9.4, 3.0.3, 3.1, 3.2, 4.0 Reporter: Robert Muir Assignee: Uwe Schindler Priority: Critical Fix For: 2.9.5, 3.0.4, 3.1, 3.2, 4.0 Attachments: LUCENE-3042.patch, LUCENE-3042.patch If you work a tokenstream, consume it, then reuse it and add an attribute to it, the computed state is wrong. thus for example, clearAttributes() will not actually clear the attribute added. So in some situations, addAttribute is not actually clearing the computed state when it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3042) AttributeSource can have an invalid computed state
[ https://issues.apache.org/jira/browse/LUCENE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-3042: -- Attachment: LUCENE-3042.patch Here a quick patch using a reference (1-ele array) to the state that is shared between all AttributeSources AttributeSource can have an invalid computed state -- Key: LUCENE-3042 URL: https://issues.apache.org/jira/browse/LUCENE-3042 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Uwe Schindler Attachments: LUCENE-3042.patch If you work a tokenstream, consume it, then reuse it and add an attribute to it, the computed state is wrong. thus for example, clearAttributes() will not actually clear the attribute added. So in some situations, addAttribute is not actually clearing the computed state when it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3042) AttributeSource can have an invalid computed state
[ https://issues.apache.org/jira/browse/LUCENE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-3042: -- Attachment: LUCENE-3042.patch New patch, that makes the code simplier to read: - I replaced the computeCurrentState by a getter method that does the check and returns the state. - This makes it easier to use and removes lots of checks even for hasAttributes() - Now, most methods using State now only consist of the for Loop. AttributeSource can have an invalid computed state -- Key: LUCENE-3042 URL: https://issues.apache.org/jira/browse/LUCENE-3042 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Uwe Schindler Attachments: LUCENE-3042.patch, LUCENE-3042.patch If you work a tokenstream, consume it, then reuse it and add an attribute to it, the computed state is wrong. thus for example, clearAttributes() will not actually clear the attribute added. So in some situations, addAttribute is not actually clearing the computed state when it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3042) AttributeSource can have an invalid computed state
[ https://issues.apache.org/jira/browse/LUCENE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-3042: -- Attachment: (was: LUCENE-3042.patch) AttributeSource can have an invalid computed state -- Key: LUCENE-3042 URL: https://issues.apache.org/jira/browse/LUCENE-3042 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Uwe Schindler Attachments: LUCENE-3042.patch If you work a tokenstream, consume it, then reuse it and add an attribute to it, the computed state is wrong. thus for example, clearAttributes() will not actually clear the attribute added. So in some situations, addAttribute is not actually clearing the computed state when it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3042) AttributeSource can have an invalid computed state
[ https://issues.apache.org/jira/browse/LUCENE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-3042: -- Attachment: LUCENE-3042.patch Repost patch, first had logic error. AttributeSource can have an invalid computed state -- Key: LUCENE-3042 URL: https://issues.apache.org/jira/browse/LUCENE-3042 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Uwe Schindler Attachments: LUCENE-3042.patch, LUCENE-3042.patch If you work a tokenstream, consume it, then reuse it and add an attribute to it, the computed state is wrong. thus for example, clearAttributes() will not actually clear the attribute added. So in some situations, addAttribute is not actually clearing the computed state when it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3042) AttributeSource can have an invalid computed state
[ https://issues.apache.org/jira/browse/LUCENE-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-3042: -- Component/s: Analysis Priority: Critical (was: Major) Affects Version/s: 4.0 3.2 2.9.4 3.0.3 3.1 Fix Version/s: 4.0 3.2 Just to conclude: This bug is not so serious as it appears (else someone would have noticed before), as it would never happen on 0-8-15 TokenStreams, when used like IndexWriter does. This bug only appears if you have TokenFilters and you add Attributes on the top level Filter later (after using the TokenStream for first time). Using the TokenStream means that you calculate the states and so every Filter/Tokenizer got his own cached state. Adding them a new Attribute on the last filter will never invalidate the cache of the Tokenizer. This bug could affect: - Analyzers that reuse TokenStreams partly and plug filters on top in the reuseableTokenStream() method, reusing the partially cached tokenstream. Like those, that always add a non-cacheable TokenFilter on top of a base TS. - TokenStreams that add attributes on the-fly in one of their filters. We should backport this patch to 3.x, 3.1.1 and maybe even 2.9.x and 3.0.x branches (if somebody wants to patch 3.0). In general this is a serious issue of the new TokenStream API since 2.9. AttributeSource can have an invalid computed state -- Key: LUCENE-3042 URL: https://issues.apache.org/jira/browse/LUCENE-3042 Project: Lucene - Java Issue Type: Bug Components: Analysis Affects Versions: 2.9.4, 3.0.3, 3.1, 3.2, 4.0 Reporter: Robert Muir Assignee: Uwe Schindler Priority: Critical Fix For: 3.2, 4.0 Attachments: LUCENE-3042.patch, LUCENE-3042.patch If you work a tokenstream, consume it, then reuse it and add an attribute to it, the computed state is wrong. thus for example, clearAttributes() will not actually clear the attribute added. So in some situations, addAttribute is not actually clearing the computed state when it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
