[jira] [Commented] (MAPREDUCE-6610) JobHistoryEventHandler should not swallow timeline response
[ https://issues.apache.org/jira/browse/MAPREDUCE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116848#comment-15116848 ] Hudson commented on MAPREDUCE-6610: --- FAILURE: Integrated in Hadoop-trunk-Commit #9186 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9186/]) MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline (jianhe: rev d0d7c221682a88ac6e11e9b7c07513e369104b10) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/CHANGES.txt > JobHistoryEventHandler should not swallow timeline response > --- > > Key: MAPREDUCE-6610 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6610 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu >Priority: Trivial > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6610-trunk.001.patch, > MAPREDUCE-6610-trunk.002.patch > > > As discussed in YARN-4596, JobHistoryEventHandler should process and log > timeline put errors after the timeline put call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6610) JobHistoryEventHandler should not swallow timeline response
[ https://issues.apache.org/jira/browse/MAPREDUCE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated MAPREDUCE-6610: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Target Version/s: 2.8.0 Status: Resolved (was: Patch Available) Committed to trunk, branch-2, branch-2.8 thanks [~gtCarrera] Thanks [~Naganarasimha] for reviewing ! > JobHistoryEventHandler should not swallow timeline response > --- > > Key: MAPREDUCE-6610 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6610 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu >Priority: Trivial > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6610-trunk.001.patch, > MAPREDUCE-6610-trunk.002.patch > > > As discussed in YARN-4596, JobHistoryEventHandler should process and log > timeline put errors after the timeline put call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6619) HADOOP_CLASSPATH is overwritten in MR container
[ https://issues.apache.org/jira/browse/MAPREDUCE-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shanyu zhao updated MAPREDUCE-6619: --- Assignee: Junping Du (was: shanyu zhao) > HADOOP_CLASSPATH is overwritten in MR container > --- > > Key: MAPREDUCE-6619 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6619 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.8.0, 2.7.2 >Reporter: shanyu zhao >Assignee: Junping Du > > Previously env variable HADOOP_CLASSPAH in MR containers inherit from > defaults of the worker node. MAPREDUCE-6454 introduced change to overwrite > HADOOP_CLASSPATH completely. This caused regression. We need to add > additional entries to HADOOP_CLASSPATH instead of completely replacing it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6619) HADOOP_CLASSPATH is overwritten in MR container
shanyu zhao created MAPREDUCE-6619: -- Summary: HADOOP_CLASSPATH is overwritten in MR container Key: MAPREDUCE-6619 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6619 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.8.0, 2.7.2 Reporter: shanyu zhao Assignee: shanyu zhao Previously env variable HADOOP_CLASSPAH in MR containers inherit from defaults of the worker node. MAPREDUCE-6454 introduced change to overwrite HADOOP_CLASSPATH completely. This caused regression. We need to add additional entries to HADOOP_CLASSPATH instead of completely replacing it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116670#comment-15116670
]
Hadoop QA commented on MAPREDUCE-6618:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
patch generated 1 new + 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 37s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 15s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 116m 34s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 28s
{color} | {color:red} Patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 241m 18s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
|
| | Field only ever set to null:null:
org.apache.hadoop.mapred.YarnClientProtocolProvider.yarnRunner In
YarnClientProtocolProvide
[jira] [Commented] (MAPREDUCE-6610) JobHistoryEventHandler should not swallow timeline response
[ https://issues.apache.org/jira/browse/MAPREDUCE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116382#comment-15116382 ] Naganarasimha G R commented on MAPREDUCE-6610: -- +1 LGTM > JobHistoryEventHandler should not swallow timeline response > --- > > Key: MAPREDUCE-6610 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6610 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu >Priority: Trivial > Attachments: MAPREDUCE-6610-trunk.001.patch, > MAPREDUCE-6610-trunk.002.patch > > > As discussed in YARN-4596, JobHistoryEventHandler should process and log > timeline put errors after the timeline put call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated MAPREDUCE-6618:
-
Status: Patch Available (was: Open)
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated MAPREDUCE-6618:
-
Attachment: MAPREDUCE-6618.1.patch
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: MAPREDUCE-6618.1.patch
>
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116376#comment-15116376
]
Xuan Gong commented on MAPREDUCE-6618:
--
The problem would be much worse if the ats v1.5 is enabled. In ats v1.5, we
would create two timer tasks in TimelineClient. We would leak two more threads
every time when we start YarnClient with TimelineClient enabled.
> YarnClientProtocolProvider leaking the YarnClient thread.
> --
>
> Key: MAPREDUCE-6618
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
>
> YarnClientProtocolProvider creates YarnRunner which includes
> ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
> yarnclient. The yarnClient thread would be leaked due to
> {code}
> @Override
> public void close(ClientProtocol clientProtocol) throws IOException {
> // nothing to do
> }
> {code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.
Xuan Gong created MAPREDUCE-6618:
Summary: YarnClientProtocolProvider leaking the YarnClient thread.
Key: MAPREDUCE-6618
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6618
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Xuan Gong
Assignee: Xuan Gong
YarnClientProtocolProvider creates YarnRunner which includes
ResourceMgrDelegate. In ResourceMgrDelegate, we would initiate and start
yarnclient. The yarnClient thread would be leaked due to
{code}
@Override
public void close(ClientProtocol clientProtocol) throws IOException {
// nothing to do
}
{code} in YarnClientProtocolProvider
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6610) JobHistoryEventHandler should not swallow timeline response
[ https://issues.apache.org/jira/browse/MAPREDUCE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116339#comment-15116339 ] Jian He commented on MAPREDUCE-6610: lgtm, > JobHistoryEventHandler should not swallow timeline response > --- > > Key: MAPREDUCE-6610 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6610 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu >Priority: Trivial > Attachments: MAPREDUCE-6610-trunk.001.patch, > MAPREDUCE-6610-trunk.002.patch > > > As discussed in YARN-4596, JobHistoryEventHandler should process and log > timeline put errors after the timeline put call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6610) JobHistoryEventHandler should not swallow timeline response
[
https://issues.apache.org/jira/browse/MAPREDUCE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115858#comment-15115858
]
Hadoop QA commented on MAPREDUCE-6610:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:
patch generated 1 new + 321 unchanged - 1 fixed = 322 total (was 322) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 54s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed with
JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 36s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed with
JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 50s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12784208/MAPREDUCE-6610-trunk.002.patch
|
| JIRA Issue | MAPREDUCE-6610 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 61d6fc141632 3.13.0-36-lowla
[jira] [Updated] (MAPREDUCE-6610) JobHistoryEventHandler should not swallow timeline response
[ https://issues.apache.org/jira/browse/MAPREDUCE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu updated MAPREDUCE-6610: - Attachment: MAPREDUCE-6610-trunk.002.patch Thanks [~Naganarasimha] for the review! I addressed you comments except for the checkstyle one. For the processEventForTimelineServer method, the root cause for its length is the big switch statement. We need to clean up the whole method IMO. > JobHistoryEventHandler should not swallow timeline response > --- > > Key: MAPREDUCE-6610 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6610 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu >Priority: Trivial > Attachments: MAPREDUCE-6610-trunk.001.patch, > MAPREDUCE-6610-trunk.002.patch > > > As discussed in YARN-4596, JobHistoryEventHandler should process and log > timeline put errors after the timeline put call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6617) flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask
[
https://issues.apache.org/jira/browse/MAPREDUCE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115700#comment-15115700
]
Hadoop QA commented on MAPREDUCE-6617:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 55s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed with
JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 34s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed with
JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12784191/MAPREDUCE-6617.0.patch
|
| JIRA Issue | MAPREDUCE-6617 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 31f5369b52f8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /te
[jira] [Updated] (MAPREDUCE-6617) flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask
[ https://issues.apache.org/jira/browse/MAPREDUCE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated MAPREDUCE-6617: -- Attachment: MAPREDUCE-6617.0.patch I've attached the initial patch. > flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask > - > > Key: MAPREDUCE-6617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 3.0.0 >Reporter: Kousuke Saruta >Assignee: Kousuke Saruta > Attachments: MAPREDUCE-6617.0.patch > > > In JobHistoryEventHandler, flushTask is not purged after it's canceled so GC > never sweep flushTask. > It can cause memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6617) flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask
[ https://issues.apache.org/jira/browse/MAPREDUCE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated MAPREDUCE-6617: -- Status: Patch Available (was: Open) > flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask > - > > Key: MAPREDUCE-6617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 3.0.0 >Reporter: Kousuke Saruta >Assignee: Kousuke Saruta > Attachments: MAPREDUCE-6617.0.patch > > > In JobHistoryEventHandler, flushTask is not purged after it's canceled so GC > never sweep flushTask. > It can cause memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6617) flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask
Kousuke Saruta created MAPREDUCE-6617: - Summary: flushTimer in JobHistoryEventHandler should purge canceled flushTimerTask Key: MAPREDUCE-6617 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6617 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 3.0.0 Reporter: Kousuke Saruta Assignee: Kousuke Saruta In JobHistoryEventHandler, flushTask is not purged after it's canceled so GC never sweep flushTask. It can cause memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe
[
https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115267#comment-15115267
]
Hadoop QA commented on MAPREDUCE-6542:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 1s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 55s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 36s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 11s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
in trunk has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 29s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 57s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 52s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 1s
{color} | {color:red} root: patch generated 1 new + 290 unchanged - 3 fixed =
291 total (was 293) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 16s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 50s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 48s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 10s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} Patch does not generate AS
[jira] [Commented] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe
[
https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115184#comment-15115184
]
zhangyubiao commented on MAPREDUCE-6542:
MAPREDUCE-6542-v10.patch for review. git revision
> HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe
> -
>
> Key: MAPREDUCE-6542
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6542
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.2.0, 2.7.1
> Environment: CentOS6.5 Hadoop
>Reporter: zhangyubiao
>Assignee: zhangyubiao
> Attachments: MAPREDUCE-6542-v10.patch, MAPREDUCE-6542-v2.patch,
> MAPREDUCE-6542-v3.patch, MAPREDUCE-6542-v4.patch, MAPREDUCE-6542-v5.patch,
> MAPREDUCE-6542-v6.patch, MAPREDUCE-6542-v7.patch, MAPREDUCE-6542-v8.patch,
> MAPREDUCE-6542-v9.patch, MAPREDUCE-6542.patch
>
>
> I use SimpleDateFormat to Parse the JobHistory File before
> {code}
> private static final SimpleDateFormat dateFormat =
> new SimpleDateFormat("-MM-dd HH:mm:ss");
> public static String getJobDetail(JobInfo job) {
> StringBuffer jobDetails = new StringBuffer("");
> SummarizedJob ts = new SummarizedJob(job);
> jobDetails.append(job.getJobId().toString().trim()).append("\t");
> jobDetails.append(job.getUsername()).append("\t");
> jobDetails.append(job.getJobname().replaceAll("\\n",
> "")).append("\t");
> jobDetails.append(job.getJobQueueName()).append("\t");
> jobDetails.append(job.getPriority()).append("\t");
> jobDetails.append(job.getJobConfPath()).append("\t");
> jobDetails.append(job.getUberized()).append("\t");
>
> jobDetails.append(dateFormat.format(job.getSubmitTime())).append("\t");
>
> jobDetails.append(dateFormat.format(job.getLaunchTime())).append("\t");
>
> jobDetails.append(dateFormat.format(job.getFinishTime())).append("\t");
>return jobDetails.toString();
> }
> {code}
> But I find I query the SubmitTime and LaunchTime in hive and compare
> JobHistory File time , I find that the submitTime and launchTime was wrong.
> Finally,I change to use the FastDateFormat to parse the time format and the
> time become right
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe
[
https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhangyubiao updated MAPREDUCE-6542:
---
Attachment: MAPREDUCE-6542-v10.patch
git revision
> HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe
> -
>
> Key: MAPREDUCE-6542
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6542
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.2.0, 2.7.1
> Environment: CentOS6.5 Hadoop
>Reporter: zhangyubiao
>Assignee: zhangyubiao
> Attachments: MAPREDUCE-6542-v10.patch, MAPREDUCE-6542-v2.patch,
> MAPREDUCE-6542-v3.patch, MAPREDUCE-6542-v4.patch, MAPREDUCE-6542-v5.patch,
> MAPREDUCE-6542-v6.patch, MAPREDUCE-6542-v7.patch, MAPREDUCE-6542-v8.patch,
> MAPREDUCE-6542-v9.patch, MAPREDUCE-6542.patch
>
>
> I use SimpleDateFormat to Parse the JobHistory File before
> {code}
> private static final SimpleDateFormat dateFormat =
> new SimpleDateFormat("-MM-dd HH:mm:ss");
> public static String getJobDetail(JobInfo job) {
> StringBuffer jobDetails = new StringBuffer("");
> SummarizedJob ts = new SummarizedJob(job);
> jobDetails.append(job.getJobId().toString().trim()).append("\t");
> jobDetails.append(job.getUsername()).append("\t");
> jobDetails.append(job.getJobname().replaceAll("\\n",
> "")).append("\t");
> jobDetails.append(job.getJobQueueName()).append("\t");
> jobDetails.append(job.getPriority()).append("\t");
> jobDetails.append(job.getJobConfPath()).append("\t");
> jobDetails.append(job.getUberized()).append("\t");
>
> jobDetails.append(dateFormat.format(job.getSubmitTime())).append("\t");
>
> jobDetails.append(dateFormat.format(job.getLaunchTime())).append("\t");
>
> jobDetails.append(dateFormat.format(job.getFinishTime())).append("\t");
>return jobDetails.toString();
> }
> {code}
> But I find I query the SubmitTime and LaunchTime in hive and compare
> JobHistory File time , I find that the submitTime and launchTime was wrong.
> Finally,I change to use the FastDateFormat to parse the time format and the
> time become right
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6616) Fail to create jobhistory file if there are some multibyte characters in the job name
[ https://issues.apache.org/jira/browse/MAPREDUCE-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6616: - Attachment: MAPREDUCE-6616-test.patch Attaching a patch to reproduce this issue. > Fail to create jobhistory file if there are some multibyte characters in the > job name > - > > Key: MAPREDUCE-6616 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6616 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Akira AJISAKA > Labels: i18n > Attachments: MAPREDUCE-6616-test.patch > > > When creating jobhistory file, job name is trimmed within 50 characters by > default, and the name is URL-encoded *after* the job name is trimmed. > Therefore, if there are some multibyte characters in the job name, the > encoded job name can be longer than 50 characters. Eventually it can break > the limit of the file name (Usually 255 characters). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6616) Fail to create jobhistory file if there are some multibyte characters in the job name
Akira AJISAKA created MAPREDUCE-6616: Summary: Fail to create jobhistory file if there are some multibyte characters in the job name Key: MAPREDUCE-6616 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6616 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Reporter: Akira AJISAKA When creating jobhistory file, job name is trimmed within 50 characters by default, and the name is URL-encoded *after* the job name is trimmed. Therefore, if there are some multibyte characters in the job name, the encoded job name can be longer than 50 characters. Eventually it can break the limit of the file name (Usually 255 characters). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
