[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578691#comment-14578691 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #223 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/223/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578710#comment-14578710 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-Yarn-trunk #953 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/953/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/CHANGES.txt Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578973#comment-14578973 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #212 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/212/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java * hadoop-yarn-project/CHANGES.txt Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578944#comment-14578944 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2151 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2151/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java * hadoop-yarn-project/CHANGES.txt Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579038#comment-14579038 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #221 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/221/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579087#comment-14579087 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2169 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2169/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577565#comment-14577565 ] Karthik Kambatla commented on YARN-2716: The test failures seem unrelated. TestWorkPreservingRMRestart fails on trunk as well, and the ATS tests are likely completely unrelated. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1457#comment-1457 ] Karthik Kambatla commented on YARN-2716: [~jianhe] - I believe the patch is good to go. Do you have any further comments? Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577899#comment-14577899 ] Jian He commented on YARN-2716: --- looks good , committing Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577947#comment-14577947 ] Hudson commented on YARN-2716: -- FAILURE: Integrated in Hadoop-trunk-Commit #7990 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7990/]) YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla (jianhe: rev 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStorePerf.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStoreZKClientConnections.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Fix For: 2.8.0 Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575848#comment-14575848
]
Hadoop QA commented on YARN-2716:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 54s | Findbugs (version ) appears to
be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | javac | 7m 49s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 56s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 24s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 6s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 1s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 30s | Tests passed in
hadoop-yarn-api. |
| {color:red}-1{color} | yarn tests | 49m 18s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 91m 36s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter |
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738095/yarn-2716-3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e46cb80 |
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8204/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8204/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8204/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8204/console |
This message was automatically generated.
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch,
yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574979#comment-14574979 ] Karthik Kambatla commented on YARN-2716: Thanks for the thorough review, Jian. Sorry for missing some simple things in the patch. bq. will the safeDelete throw noNodeExist exception if deleting a non-existing zone? safeDelete checks if the znode exists before attempting to delete it. So, shouldn't throw NoNodeException. bq. why in HA case, zkRetryInterval is calculated as below When HA is not enabled, we should give the store as much time as possible to connect to ZK. When HA is enabled, it is possible the other RM has better chance of connecting to ZK; so, we should give up trying by session-timeout. YARN-2054 has all the details. Posting a patch shortly to address all the review feedback. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575163#comment-14575163
]
Hadoop QA commented on YARN-2716:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 18s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 35s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 5s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 57s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 31s | Tests passed in
hadoop-yarn-api. |
| {color:red}-1{color} | yarn tests | 50m 47s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 92m 58s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738007/yarn-2716-2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 7588585 |
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8200/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8200/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8200/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8200/console |
This message was automatically generated.
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-2.patch,
yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575395#comment-14575395
]
Jian He commented on YARN-2716:
---
bq. safeDelete checks if the znode exists before attempting to delete it. So,
shouldn't throw NoNodeException.
ah, right sorry, I overlooked the implementation of the method
only comment is :
- removeApplicationStateInternal can also use the
{{curatorFramework.delete().deletingChildrenIfNeeded()}} instead of adding all
children manually ?
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-2.patch,
yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575406#comment-14575406
]
Karthik Kambatla commented on YARN-2716:
bq. removeApplicationStateInternal can also use the
curatorFramework.delete().deletingChildrenIfNeeded() instead of adding all
children manually ?
safeDelete adds the nodes to a transaction that creates and deletes the fencing
node as well. Curator transactions don't support {{deletingChildrenIfNeeded}}
yet.
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-2.patch,
yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575417#comment-14575417 ] Jian He commented on YARN-2716: --- bq. creates and deletes the fencing node as well. I see, thanks. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573911#comment-14573911
]
Jian He commented on YARN-2716:
---
Thanks Karthik for working this ! This simplifies things a lot.
Mostly good, few comments and questions:
- these two booleans not used, maybe removed.
{{private boolean create = false, delete = false;
}}
- is this going to be done in this jira?
{code} // TODO: Check deleting appIdRemovePath works recursively
safeDelete(appIdRemovePath);{code}
- will the safeDelete throw noNodeExist exception if deleting a non-existing
zone?
- {{new RetryNTimes(numRetries, zkSessionTimeout / numRetries));}}, I think
the second parameter should be zkRetryInterval; Also, I have a question why in
HA case, zkRetryInterval is calculated as below
{code}
if (HAUtil.isHAEnabled(conf)) {
zkRetryInterval = zkSessionTimeout /
numRetries;
{code}
- I found this
[thread|http://mail-archives.apache.org/mod_mbox/curator-user/201410.mbox/%3cd076bc8e.9ef1%25sreichl...@chegg.com%3E]
saying that blockUntilConnect is not needed to call; Suppose it’s needed, I
think the zkSessionTimeout value is too small, it would be
numRetries*numRetryInterval, otherwise RM will exit soon after retry 10s by
default.
{code}
if (!curatorFramework.blockUntilConnected(
zkSessionTimeout, TimeUnit.MILLISECONDS)) {
LOG.fatal(Couldn't establish connection to ZK server);
throw new YarnRuntimeException(Couldn't connect to ZK server);
}
{code}
- remove this ?
{code}
// @Override
// public ZooKeeper getNewZooKeeper() throws IOException {
//return client;
// }
{code}
- I think testZKSessionTimeout may be removed too ? it looks like a test for
curator
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-prelim.patch,
yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567024#comment-14567024
]
Hadoop QA commented on YARN-2716:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 9s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | javac | 7m 34s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated 3
new checkstyle issues (total was 42, now 8). |
| {color:green}+1{color} | whitespace | 0m 5s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 1m 30s | The patch appears to introduce 2
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 50m 23s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 88m 44s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-server-resourcemanager |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736498/yarn-2716-1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 5cc3fce |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/8147/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8147/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8147/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8147/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8147/console |
This message was automatically generated.
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-prelim.patch,
yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260427#comment-14260427 ] Karthik Kambatla commented on YARN-2716: We kind of need CURATOR-111 for this. Posting a patch for that. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Robert Kanter Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259699#comment-14259699 ] Karthik Kambatla commented on YARN-2716: [~rkanter] - is it okay for me to take this over? We have recently seen more issues with the current implementation, and this rewrite could greatly help. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Robert Kanter Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
