[jira] Commented: (LUCENE-2584) Concurrency issues in SegmentInfo.files() could lead to ConcurrentModificationException
[
https://issues.apache.org/jira/browse/LUCENE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12983142#action_12983142
]
Shai Erera commented on LUCENE-2584:
On one hand, it's good to add the files to a Set, so that we can be sure they
are added uniquely. On the other hand though, if we expect files are added
properly, then adding to the set is redundant. Since this code is executed once
per SI instance, I think explicitly adding to a Set is better.
Note that while the assert you added will work, if someone runs without
assertions he may get duplicate file names, if indeed they are added twice. I
think that it's not so crucial to know that the same files was added twice,
it's a very unlikely bug, but it is crucial that files() return unique names.
So can you please use a Set in the method instead of the assert (like it's done
on trunk). Also, while you're at it, the method doesn't have javadocs - they
appear in regular comments. Can you convert them to javadocs (there is a
warning there about not modifying the returned List, but it's not visible as
javadocs :).
Concurrency issues in SegmentInfo.files() could lead to
ConcurrentModificationException
---
Key: LUCENE-2584
URL: https://issues.apache.org/jira/browse/LUCENE-2584
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: 2.9, 2.9.1, 2.9.2, 2.9.3, 3.0, 3.0.1, 3.0.2
Reporter: Alexander Kanarsky
Priority: Minor
Fix For: 3.1, 4.0
Attachments: LUCENE-2584-branch_3x.patch,
LUCENE-2584-lucene-2_9.patch, LUCENE-2584-lucene-3_0.patch
The multi-threaded call of the files() in SegmentInfo could lead to the
ConcurrentModificationException if one thread is not finished additions to
the ArrayList (files) yet while the other thread already obtained it as
cached (see below). This is a rare exception, but it would be nice to fix. I
see the code is no longer problematic in the trunk (and others ported from
flex_1458), looks it was fixed while implementing post 3.x features. The fix
to 3.x and 2.9.x branches could be the same - create the files set first and
populate it, and then assign to the member variable at the end of the method.
This will resolve the issue. I could prepare the patch for 2.9.4 and 3.x, if
needed.
--
INFO: [19] webapp= path=/replication params={command=fetchindexwt=javabin}
status=0 QTime=1
Jul 30, 2010 9:13:05 AM org.apache.solr.core.SolrCore execute
INFO: [19] webapp= path=/replication params={command=detailswt=javabin}
status=0 QTime=24
Jul 30, 2010 9:13:05 AM org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed
java.util.ConcurrentModificationException
at
java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at java.util.AbstractCollection.addAll(AbstractCollection.java:305)
at org.apache.lucene.index.SegmentInfos.files(SegmentInfos.java:826)
at
org.apache.lucene.index.DirectoryReader$ReaderCommit.init(DirectoryReader.java:916)
at
org.apache.lucene.index.DirectoryReader.getIndexCommit(DirectoryReader.java:856)
at
org.apache.solr.search.SolrIndexReader.getIndexCommit(SolrIndexReader.java:454)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:261)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
at
org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:146)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2584) Concurrency issues in SegmentInfo.files() could lead to ConcurrentModificationException
[
https://issues.apache.org/jira/browse/LUCENE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12983183#action_12983183
]
Michael McCandless commented on LUCENE-2584:
Patch looks good Shai!
I don't think you need to backport to 2.9/3.0 immediately (unless you really
want to!)? We can backport if/when we do another release...
Concurrency issues in SegmentInfo.files() could lead to
ConcurrentModificationException
---
Key: LUCENE-2584
URL: https://issues.apache.org/jira/browse/LUCENE-2584
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: 2.9, 2.9.1, 2.9.2, 2.9.3, 3.0, 3.0.1, 3.0.2
Reporter: Alexander Kanarsky
Assignee: Shai Erera
Priority: Minor
Fix For: 3.1, 4.0
Attachments: LUCENE-2584-branch_3x.patch,
LUCENE-2584-lucene-2_9.patch, LUCENE-2584-lucene-3_0.patch, LUCENE-2584.patch
The multi-threaded call of the files() in SegmentInfo could lead to the
ConcurrentModificationException if one thread is not finished additions to
the ArrayList (files) yet while the other thread already obtained it as
cached (see below). This is a rare exception, but it would be nice to fix. I
see the code is no longer problematic in the trunk (and others ported from
flex_1458), looks it was fixed while implementing post 3.x features. The fix
to 3.x and 2.9.x branches could be the same - create the files set first and
populate it, and then assign to the member variable at the end of the method.
This will resolve the issue. I could prepare the patch for 2.9.4 and 3.x, if
needed.
--
INFO: [19] webapp= path=/replication params={command=fetchindexwt=javabin}
status=0 QTime=1
Jul 30, 2010 9:13:05 AM org.apache.solr.core.SolrCore execute
INFO: [19] webapp= path=/replication params={command=detailswt=javabin}
status=0 QTime=24
Jul 30, 2010 9:13:05 AM org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed
java.util.ConcurrentModificationException
at
java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at java.util.AbstractCollection.addAll(AbstractCollection.java:305)
at org.apache.lucene.index.SegmentInfos.files(SegmentInfos.java:826)
at
org.apache.lucene.index.DirectoryReader$ReaderCommit.init(DirectoryReader.java:916)
at
org.apache.lucene.index.DirectoryReader.getIndexCommit(DirectoryReader.java:856)
at
org.apache.solr.search.SolrIndexReader.getIndexCommit(SolrIndexReader.java:454)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:261)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
at
org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:146)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2584) Concurrency issues in SegmentInfo.files() could lead to ConcurrentModificationException
[
https://issues.apache.org/jira/browse/LUCENE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895119#action_12895119
]
Alexander Kanarsky commented on LUCENE-2584:
OK, I'll create patches tomorrow.
Concurrency issues in SegmentInfo.files() could lead to
ConcurrentModificationException
---
Key: LUCENE-2584
URL: https://issues.apache.org/jira/browse/LUCENE-2584
Project: Lucene - Java
Issue Type: Bug
Affects Versions: 2.9, 2.9.1, 2.9.2, 2.9.3, 3.0, 3.0.1, 3.0.2
Reporter: Alexander Kanarsky
Priority: Minor
Fix For: 3.1, 4.0
The multi-threaded call of the files() in SegmentInfo could lead to the
ConcurrentModificationException if one thread is not finished additions to
the ArrayList (files) yet while the other thread already obtained it as
cached (see below). This is a rare exception, but it would be nice to fix. I
see the code is no longer problematic in the trunk (and others ported from
flex_1458), looks it was fixed while implementing post 3.x features. The fix
to 3.x and 2.9.x branches could be the same - create the files set first and
populate it, and then assign to the member variable at the end of the method.
This will resolve the issue. I could prepare the patch for 2.9.4 and 3.x, if
needed.
--
INFO: [19] webapp= path=/replication params={command=fetchindexwt=javabin}
status=0 QTime=1
Jul 30, 2010 9:13:05 AM org.apache.solr.core.SolrCore execute
INFO: [19] webapp= path=/replication params={command=detailswt=javabin}
status=0 QTime=24
Jul 30, 2010 9:13:05 AM org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed
java.util.ConcurrentModificationException
at
java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at java.util.AbstractCollection.addAll(AbstractCollection.java:305)
at org.apache.lucene.index.SegmentInfos.files(SegmentInfos.java:826)
at
org.apache.lucene.index.DirectoryReader$ReaderCommit.init(DirectoryReader.java:916)
at
org.apache.lucene.index.DirectoryReader.getIndexCommit(DirectoryReader.java:856)
at
org.apache.solr.search.SolrIndexReader.getIndexCommit(SolrIndexReader.java:454)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:261)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
at
org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:146)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
