[
https://issues.apache.org/jira/browse/EAGLE-805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707753#comment-15707753
]
ASF GitHub Bot commented on EAGLE-805:
--------------------------------------
GitHub user chitin opened a pull request:
https://github.com/apache/incubator-eagle/pull/700
EAGLE-805 sync some operation in RunningJobManager
EAGLE-805 sync some operation in RunningJobManager
- Acquire lock when recoverYarnApp and recover methods were invoked.
- Add unit test for the two methods under high concurrency.
- Modify lockPath to avoid exception when curator.getChildren().
https://issues.apache.org/jira/browse/EAGLE-805
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chitin/incubator-eagle EAGLE-805
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-eagle/pull/700.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #700
----
commit b368c33e9f04a3ce39b7b91270748827b97405d3
Author: chitin <[email protected]>
Date: 2016-11-30T06:38:42Z
EAGLE-805 sync some operation in RunningJobManager
- Acquire lock when recoverYarnApp and recover methods were invoked.
- Add unit test for the two methods under high concurrency.
- Modify lockPath to avoid exception when curator.getChildren().
----
> sync some operation in RunningJobManager
> ----------------------------------------
>
> Key: EAGLE-805
> URL: https://issues.apache.org/jira/browse/EAGLE-805
> Project: Eagle
> Issue Type: Bug
> Reporter: JiJun Tang
> Assignee: Lingang Deng
> Priority: Minor
>
> RunningJobManager's recoverYarnApp and recover should acquire lock before
> call the logic.Otherwise,the mrRunningJobFetchSpout will throw exception as
> follow:
> {noformat}
> 2016-11-27 23:43:14 Thread-4-mrRunningJobFetchSpout
> org.apache.eagle.jpm.mr.running.storm.MRRunningJobFetchSpout [WARN] {}
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /apps/mr/running/xxxx/application_1476827590809_491686
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> ~[zookeeper-3.4.6.2.2.0.0-2041.jar:3.4.6-2041--1]
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> ~[zookeeper-3.4.6.2.2.0.0-2041.jar:3.4.6-2041--1]
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590)
> ~[zookeeper-3.4.6.2.2.0.0-2041.jar:3.4.6-2041--1]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
> ~[stormjar.jar:na]
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
> ~[stormjar.jar:na]
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.util.jobrecover.RunningJobManager.recoverYarnApp(RunningJobManager.java:80)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.mr.running.recover.MRRunningJobManager.recoverYarnApp(MRRunningJobManager.java:41)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.mr.running.storm.MRRunningJobFetchSpout.nextTuple(MRRunningJobFetchSpout.java:100)
> ~[stormjar.jar:na]
> at
> backtype.storm.daemon.executor$fn__5629$fn__5644$fn__5673.invoke(executor.clj:585)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at backtype.storm.util$async_loop$fn__452.invoke(util.clj:465)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
