[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13730709#comment-13730709 ] Hudson commented on HDFS-5028: -- SUCCESS: Integrated in Hadoop-Hdfs-0.23-Build #691 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/691/]) svn merge -c 1509565. Merging from trunk to branch-0.23 to fix HDFS-5028. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510771) * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong >Assignee: zhaoyunjiong > Fix For: 1.1.3, 0.23.10, 2.1.1-beta > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727672#comment-13727672 ] Hudson commented on HDFS-5028: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1506 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1506/]) HDFS-5028. LeaseRenewer throws ConcurrentModificationException when timeout. Contributed by zhaoyunjiong (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509565) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong >Assignee: zhaoyunjiong > Fix For: 1.1.3, 2.1.1-beta > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727659#comment-13727659 ] Hudson commented on HDFS-5028: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1479 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1479/]) HDFS-5028. LeaseRenewer throws ConcurrentModificationException when timeout. Contributed by zhaoyunjiong (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509565) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong >Assignee: zhaoyunjiong > Fix For: 1.1.3, 2.1.1-beta > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727567#comment-13727567 ] Hudson commented on HDFS-5028: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #289 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/289/]) HDFS-5028. LeaseRenewer throws ConcurrentModificationException when timeout. Contributed by zhaoyunjiong (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509565) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong >Assignee: zhaoyunjiong > Fix For: 1.1.3, 2.1.1-beta > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727459#comment-13727459 ] Hudson commented on HDFS-5028: -- SUCCESS: Integrated in Hadoop-trunk-Commit #4205 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4205/]) HDFS-5028. LeaseRenewer throws ConcurrentModificationException when timeout. Contributed by zhaoyunjiong (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509565) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong >Assignee: zhaoyunjiong > Fix For: 1.1.3, 2.1.1-beta > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[
https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727278#comment-13727278
]
Hadoop QA commented on HDFS-5028:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595536/HDFS-5028.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/4756//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4756//console
This message is automatically generated.
> LeaseRenewer throw java.util.ConcurrentModificationException when timeout
> -
>
> Key: HDFS-5028
> URL: https://issues.apache.org/jira/browse/HDFS-5028
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.0.0-alpha
>Reporter: zhaoyunjiong
>Assignee: zhaoyunjiong
> Fix For: 1.1.3
>
> Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch
>
>
> In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will
> remove one dfsclient from dfsclients. Here will throw a
> ConcurrentModificationException because dfsclients changed after the iterator
> created by "for(DFSClient c : dfsclients)":
> Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077"
> java.util.ConcurrentModificationException
> at
> java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
> at java.util.AbstractList$Itr.next(AbstractList.java:343)
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406)
> at
> org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69)
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273)
> at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727005#comment-13727005 ] Tsz Wo (Nicholas), SZE commented on HDFS-5028: -- Yunjiong, good catch. Patch looks good. Just a very minor nit: how about using dfsclients.get(0) instead of dfsclients.get(dfsclients.size() - 1)? > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong > Fix For: 1.1.3 > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13723253#comment-13723253 ] zhaoyunjiong commented on HDFS-5028: dfsclients was syncronizing. The problem here is Iterator. You can get more information here: http://stackoverflow.com/questions/8189466/java-util-concurrentmodificationexception For short: The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. c.abort() will remove c(a dfsclient) from dfsclients, so iterator generated in "for(DFSClient c : dfsclients)" will throw ConcurrentModificationException. > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong > Fix For: 1.1.3 > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5028) LeaseRenewer throw java.util.ConcurrentModificationException when timeout
[ https://issues.apache.org/jira/browse/HDFS-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13723095#comment-13723095 ] Kousuke Saruta commented on HDFS-5028: -- Hi zahoyunjiong, I think your modification may reduce the likelihood of this issue but doesn't address essentially. Instead, how about synchronizing "dfsclients"? How do you think? > LeaseRenewer throw java.util.ConcurrentModificationException when timeout > - > > Key: HDFS-5028 > URL: https://issues.apache.org/jira/browse/HDFS-5028 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0-alpha >Reporter: zhaoyunjiong > Fix For: 1.1.3 > > Attachments: HDFS-5028-branch-1.1.patch, HDFS-5028.patch > > > In LeaseRenewer, when renew() throw SocketTimeoutException, c.abort() will > remove one dfsclient from dfsclients. Here will throw a > ConcurrentModificationException because dfsclients changed after the iterator > created by "for(DFSClient c : dfsclients)": > Exception in thread "org.apache.hadoop.hdfs.LeaseRenewer$1@75fa1077" > java.util.ConcurrentModificationException > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > at java.util.AbstractList$Itr.next(AbstractList.java:343) > at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:406) > at > org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69) > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
