[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126471#comment-15126471
]
Hudson commented on MAPREDUCE-6618:
---
FAILURE: Integrated in Hadoop-trunk-Commit #9215 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9215/])
MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient (jlowe: rev
59a212b6e1265adfa9b55c71b65a22157dfccf77)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YarnClientProtocolProvider.java
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Fix For: 2.7.3, 2.6.4
>
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch, MAPREDUCE-6618.5.patch,
> MAPREDUCE-6618.6.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126449#comment-15126449
]
Jason Lowe commented on MAPREDUCE-6618:
---
+1 lgtm. Committing this.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch, MAPREDUCE-6618.5.patch,
> MAPREDUCE-6618.6.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123613#comment-15123613
]
Jason Lowe commented on MAPREDUCE-6618:
---
Could you look into the failed unit tests? I'm not sure about the others, but
the TestYarnClientProtocolProvider failure looks related. Also would be nice
to cleanup the now unused import. Otherwise patch looks good.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch, MAPREDUCE-6618.5.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15124468#comment-15124468
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 29s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 45s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 25s
{color} | {color:red} Patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 220m 18s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapreduce.v2.TestMRJobsWithProfiler
|
| | hadoop.mapred.TestNetworkedJob |
| JDK v1.7.0_91 Failed junit tests | hadoop.mapred.TestNetworkedJob |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12785243/MAPREDUCE-6618.6.patch
|
| JIRA Issue | MAPREDUCE-6618 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a0657fc6512d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15124138#comment-15124138
]
Xuan Gong commented on MAPREDUCE-6618:
--
[~jlowe]
Fix the TestYarnClientProtocolProvider failure.
Create https://issues.apache.org/jira/browse/MAPREDUCE-6623 for other test
failures.
attached a new patch to address all the comments
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch, MAPREDUCE-6618.5.patch,
> MAPREDUCE-6618.6.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122324#comment-15122324
]
Xuan Gong commented on MAPREDUCE-6618:
--
[~jlowe]
Thanks for the clarification. The new patch addressed that.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch, MAPREDUCE-6618.5.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122897#comment-15122897
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 49s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 19s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
patch generated 1 new + 78 unchanged - 0 fixed = 79 total (was 78) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 172m 59s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 151m 59s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 37s
{color} | {color:red} Patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 350m 46s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapred.TestNetworkedJob |
| | hadoop.mapreduce.v2.TestMRJobsWithProfiler |
| | hadoop.mapreduce.v2.TestRMNMInfo |
| | hadoop.mapred.TestMRTimelineEventHandling |
| | hadoop.mapreduce.TestYarnClientProtocolProvider |
| JDK v1.8.0_66 Timed out junit tests |
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119722#comment-15119722
]
Jason Lowe commented on MAPREDUCE-6618:
---
Thanks for updating the patch! Couple of nits:
Should we be setting fields to null as we check them for non-null and close
them?
Is there a reason we use entrySet() instead of values() here? It ignores the
job IDs, so thinking we don't need the full entry set.
{code}
for (Entry delegate : cache.entrySet()) {
ClientServiceDelegate jobDelegate = delegate.getValue();
if (jobDelegate != null) {
jobDelegate.close();
}
}
{code}
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119835#comment-15119835
]
Xuan Gong commented on MAPREDUCE-6618:
--
[~jlowe]
Thanks for the comments.
I have remove one un-necessary null check for
{code}
private Map cache =
new HashMap();
{code} in ClientCache.java
And set fields to null initially for the un-final variables.
Attached a new patch for this.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120168#comment-15120168
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 1s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 9s {color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed with JDK
v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 7s {color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed with JDK
v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 26s
{color} | {color:red} Patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 221m 37s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapreduce.v2.TestMRJobsWithProfiler
|
| | hadoop.mapred.TestNetworkedJob |
| | hadoop.mapreduce.TestYarnClientProtocolProvider |
| JDK v1.7.0_91 Failed junit tests | hadoop.mapred.TestNetworkedJob |
| | hadoop.mapreduce.TestYarnClientProtocolProvider |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12784680/MAPREDUCE-6618.4.patch
|
| JIRA Issue |
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120198#comment-15120198
]
Jason Lowe commented on MAPREDUCE-6618:
---
Sorry, I should have been more clear. I don't think we need to initialize
these various fields to null because that's what Java does automatically.
(Doesn't hurt, just not sure it helps anything.) Instead what I was asking was
whether it would be prudent to set things to null as we close them. For
example, a code idiom like this template:
{code}
if (x != null) {
x.close();
x = null;
}
{code}
This helps guard against doing extra work in cases of a double-close or someone
trying to continue using the object after it has been closed. I'm simply used
to seeing that sort of pattern, so when I saw the code checking for null before
the close I wondered out loud whether we should be also setting the fields to
null as part of that close processing.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch, MAPREDUCE-6618.4.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117314#comment-15117314
]
Jason Lowe commented on MAPREDUCE-6618:
---
In this patch I'm confused on how the changes to the protocol provider
accomplish anything. There's a new variable, yarnRunner, initialized to null
and set nowhere else. In the close method it checks it for non-null before
doing anything, but how can it be anything but null? Note that a single
protocol provider can create multiple clients, so having the protocol provider
track a single YARNRunner client does not seem appropriate.
The close method is handed the object to cleanup, namely clientProtocol, but
it's ignoring it even in the proposed patch. It needs to cast that to a
YARNRunner and close that object.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117676#comment-15117676
]
Xuan Gong commented on MAPREDUCE-6618:
--
[~jlowe]
bq. The close method is handed the object to cleanup, namely clientProtocol,
but it's ignoring it even in the proposed patch. It needs to cast that to a
YARNRunner and close that object.
This sounds like the appropriate solution for this issue.
Attached a patch to address the comments.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118140#comment-15118140
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
53s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 31s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 37s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 24s
{color} | {color:red} Patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 222m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapreduce.v2.TestMRJobsWithProfiler
|
| | hadoop.mapred.TestNetworkedJob |
| | hadoop.mapreduce.TestYarnClientProtocolProvider |
| JDK v1.7.0_91 Failed junit tests | hadoop.mapred.TestNetworkedJob |
| | hadoop.mapreduce.TestYarnClientProtocolProvider |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12784452/MAPREDUCE-6618.2.patch
|
| JIRA Issue |
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118148#comment-15118148
]
Jason Lowe commented on MAPREDUCE-6618:
---
Patch looks better, thanks! What about the client cache that's also in
YARNRunner? It potentially has a history server client that I think also
should be closed in this case.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118277#comment-15118277
]
Xuan Gong commented on MAPREDUCE-6618:
--
Thanks for the comments. [~jlowe].
Attached a new patch to address the comment
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch, MAPREDUCE-6618.2.patch,
> MAPREDUCE-6618.3.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118554#comment-15118554
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 23s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 31s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 24s
{color} | {color:red} Patch generated 19 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 218m 37s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapreduce.v2.TestMRJobsWithProfiler
|
| | hadoop.mapreduce.TestYarnClientProtocolProvider |
| | hadoop.mapred.TestNetworkedJob |
| JDK v1.7.0_91 Failed junit tests |
hadoop.mapreduce.TestYarnClientProtocolProvider |
| | hadoop.mapred.TestNetworkedJob |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12784527/MAPREDUCE-6618.3.patch
|
| JIRA Issue |
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116670#comment-15116670
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
patch generated 1 new + 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 37s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 15s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 116m 34s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 28s
{color} | {color:red} Patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 241m 18s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
|
| | Field only ever set to null:null:
org.apache.hadoop.mapred.YarnClientProtocolProvider.yarnRunner In
YarnClientProtocolProvider.java |
| JDK
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116376#comment-15116376
]
Xuan Gong commented on MAPREDUCE-6618:
--
The problem would be much worse if the ats v1.5 is enabled. In ats v1.5, we
would create two timer tasks in TimelineClient. We would leak two more threads
every time when we start YarnClient with TimelineClient enabled.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
