[
https://issues.apache.org/jira/browse/EAGLE-805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708091#comment-15708091
]
ASF GitHub Bot commented on EAGLE-805:
--------------------------------------
Github user wujinhu commented on a diff in the pull request:
https://github.com/apache/incubator-eagle/pull/700#discussion_r90199126
--- Diff:
eagle-jpm/eagle-jpm-mr-running/src/test/java/org/apache/eagle/jpm/mr/running/MRRunningJobManagerTest.java
---
@@ -117,4 +124,94 @@ public void testMRRunningJobManagerDelWithLock()
throws Exception {
}
+ @Test
+ public void testMRRunningJobManagerRecoverYarnAppWithLock() throws
Exception {
+
+ if(curator.checkExists().forPath(SHARE_RESOURCES) == null) {
+ curator.create()
+ .creatingParentsIfNeeded()
+ .withMode(CreateMode.PERSISTENT)
+ .forPath(SHARE_RESOURCES);
+ }
+
+ Assert.assertTrue(curator.checkExists().forPath(SHARE_RESOURCES)
!= null);
+
+ curator.setData().forPath(SHARE_RESOURCES, generateZkSetData());
+
+ ExecutorService service = Executors.newFixedThreadPool(QTY);
+ for (int i = 0; i < QTY; ++i) {
+ Callable<Void> task = () -> {
+ try {
+ MRRunningJobManager mrRunningJobManager = new
MRRunningJobManager(zkStateConfig);
+ for (int j = 0; j < REPETITIONS; ++j) {
+ if(j % 3 == 0) {
+ mrRunningJobManager.delete("yarnAppId",
"jobId");
+ } else {
+
mrRunningJobManager.recoverYarnApp("yarnAppId");
+ }
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
--- End diff --
Remove useless code
> sync some operation in RunningJobManager
> ----------------------------------------
>
> Key: EAGLE-805
> URL: https://issues.apache.org/jira/browse/EAGLE-805
> Project: Eagle
> Issue Type: Bug
> Reporter: JiJun Tang
> Assignee: Lingang Deng
> Priority: Minor
>
> RunningJobManager's recoverYarnApp and recover should acquire lock before
> call the logic.Otherwise,the mrRunningJobFetchSpout will throw exception as
> follow:
> {noformat}
> 2016-11-27 23:43:14 Thread-4-mrRunningJobFetchSpout
> org.apache.eagle.jpm.mr.running.storm.MRRunningJobFetchSpout [WARN] {}
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /apps/mr/running/xxxx/application_1476827590809_491686
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> ~[zookeeper-3.4.6.2.2.0.0-2041.jar:3.4.6-2041--1]
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> ~[zookeeper-3.4.6.2.2.0.0-2041.jar:3.4.6-2041--1]
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590)
> ~[zookeeper-3.4.6.2.2.0.0-2041.jar:3.4.6-2041--1]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
> ~[stormjar.jar:na]
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.util.jobrecover.RunningJobManager.recoverYarnApp(RunningJobManager.java:80)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.mr.running.recover.MRRunningJobManager.recoverYarnApp(MRRunningJobManager.java:41)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.mr.running.storm.MRRunningJobFetchSpout.nextTuple(MRRunningJobFetchSpout.java:100)
> ~[stormjar.jar:na]
> at
> backtype.storm.daemon.executor$fn__5629$fn__5644$fn__5673.invoke(executor.clj:585)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at backtype.storm.util$async_loop$fn__452.invoke(util.clj:465)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
