[jira] [Commented] (MAPREDUCE-5420) Remove mapreduce.task.tmp.dir from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239074#comment-14239074 ] Sandy Ryza commented on MAPREDUCE-5420: --- This looks good to me. > Remove mapreduce.task.tmp.dir from mapred-default.xml > - > > Key: MAPREDUCE-5420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5420 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.1.0-beta >Reporter: Sandy Ryza >Assignee: James Carman > Labels: newbie > Attachments: MAPREDUCE-5420.patch, MAPREDUCE-5420.patch > > > mapreduce.task.tmp.dir no longer has any effect, so it should no longer be > documented in mapred-default. (There is no YARN equivalent for the property. > It now is just always ./tmp). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5420) Remove mapreduce.task.tmp.dir from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239075#comment-14239075 ] Sandy Ryza commented on MAPREDUCE-5420: --- What's up with the findbugs warnings though? > Remove mapreduce.task.tmp.dir from mapred-default.xml > - > > Key: MAPREDUCE-5420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5420 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.1.0-beta >Reporter: Sandy Ryza >Assignee: James Carman > Labels: newbie > Attachments: MAPREDUCE-5420.patch, MAPREDUCE-5420.patch > > > mapreduce.task.tmp.dir no longer has any effect, so it should no longer be > documented in mapred-default. (There is no YARN equivalent for the property. > It now is just always ./tmp). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors
[
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239066#comment-14239066
]
Akira AJISAKA commented on MAPREDUCE-5800:
--
Looks like we hit http://sourceforge.net/p/findbugs/bugs/918/
> Use Job#getInstance instead of deprecated constructors
> --
>
> Key: MAPREDUCE-5800
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5800
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Affects Versions: 2.3.0, 2.6.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-5800.2.patch, MAPREDUCE-5800.patch
>
>
> There're some methods calling deprecated constructors such as {{new Job()}},
> which causes javac warnings.
> We should use {{Job.getInstance()}} to get an instance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6184) Fix findbugs warnings in mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6184: - Release Note: (was: Looks this issue is caused by http://sourceforge.net/p/findbugs/bugs/918/ Closing.) Looks this issue is caused by http://sourceforge.net/p/findbugs/bugs/918/ Closing. > Fix findbugs warnings in mapreduce-client-core > -- > > Key: MAPREDUCE-6184 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6184 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Akira AJISAKA >Assignee: Varun Saxena > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6184) Fix findbugs warnings in mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA resolved MAPREDUCE-6184. -- Resolution: Not a Problem Release Note: Looks this issue is caused by http://sourceforge.net/p/findbugs/bugs/918/ Closing. > Fix findbugs warnings in mapreduce-client-core > -- > > Key: MAPREDUCE-6184 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6184 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Akira AJISAKA >Assignee: Varun Saxena > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6182) Fix findbugs warnings in gridmix
[ https://issues.apache.org/jira/browse/MAPREDUCE-6182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA resolved MAPREDUCE-6182. -- Resolution: Not a Problem Thanks for the comment. Looks this issue is caused by http://sourceforge.net/p/findbugs/bugs/918/ Closing. > Fix findbugs warnings in gridmix > > > Key: MAPREDUCE-6182 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6182 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Reporter: Akira AJISAKA >Assignee: Varun Saxena > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6183) Fix findbugs warnings in mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA resolved MAPREDUCE-6183. -- Resolution: Not a Problem > Fix findbugs warnings in mapreduce-examples > --- > > Key: MAPREDUCE-6183 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6183 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: examples >Reporter: Akira AJISAKA >Assignee: Rohith > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6183) Fix findbugs warnings in mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239050#comment-14239050 ] Akira AJISAKA commented on MAPREDUCE-6183: -- Thanks for the comment. Looks this issue is caused by the bug http://sourceforge.net/p/findbugs/bugs/918/ I'll close this issue. > Fix findbugs warnings in mapreduce-examples > --- > > Key: MAPREDUCE-6183 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6183 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: examples >Reporter: Akira AJISAKA >Assignee: Rohith > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6183) Fix findbugs warnings in mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239046#comment-14239046 ] Rohith commented on MAPREDUCE-6183: --- I believe these findbug error would be due to either change is java version or change in findbug version. Previously these were not failed. > Fix findbugs warnings in mapreduce-examples > --- > > Key: MAPREDUCE-6183 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6183 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: examples >Reporter: Akira AJISAKA >Assignee: Rohith > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6182) Fix findbugs warnings in gridmix
[ https://issues.apache.org/jira/browse/MAPREDUCE-6182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238513#comment-14238513 ] Varun Saxena commented on MAPREDUCE-6182: - [~ajisakaa], the findbugs warnings are coming for System.out.println. And I dont think that System.out can be null. Looks like a false negative. Any change in Jenkins build environment ? > Fix findbugs warnings in gridmix > > > Key: MAPREDUCE-6182 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6182 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Reporter: Akira AJISAKA >Assignee: Varun Saxena > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6186) Minor: Remove multiple calls to requireJob() while displaying the conf page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238460#comment-14238460
]
Hadoop QA commented on MAPREDUCE-6186:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685842/MAPREDUCE-6186.patch
against trunk revision 6c5bbd7.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5069//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5069//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5069//console
This message is automatically generated.
> Minor: Remove multiple calls to requireJob() while displaying the conf page
> ---
>
> Key: MAPREDUCE-6186
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6186
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.6.0
>Reporter: Rohit Agarwal
>Assignee: Rohit Agarwal
>Priority: Minor
> Attachments: MAPREDUCE-6186.patch
>
>
> There are multiple calls to {{requireJob()}} in {{AppController.java#conf()}}
> The duplicate call seems to be introduced by mistake in
> https://github.com/apache/hadoop/commit/fe1cf3b0aca5f4d7a8af02a915b218f9b1de0fa6
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6186) Minor: Remove multiple calls to requireJob() while displaying the conf page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohit Agarwal updated MAPREDUCE-6186:
-
Status: Patch Available (was: Open)
> Minor: Remove multiple calls to requireJob() while displaying the conf page
> ---
>
> Key: MAPREDUCE-6186
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6186
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.6.0
>Reporter: Rohit Agarwal
>Assignee: Rohit Agarwal
>Priority: Minor
> Attachments: MAPREDUCE-6186.patch
>
>
> There are multiple calls to {{requireJob()}} in {{AppController.java#conf()}}
> The duplicate call seems to be introduced by mistake in
> https://github.com/apache/hadoop/commit/fe1cf3b0aca5f4d7a8af02a915b218f9b1de0fa6
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6186) Minor: Remove multiple calls to requireJob() while displaying the conf page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohit Agarwal updated MAPREDUCE-6186:
-
Attachment: MAPREDUCE-6186.patch
> Minor: Remove multiple calls to requireJob() while displaying the conf page
> ---
>
> Key: MAPREDUCE-6186
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6186
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.6.0
>Reporter: Rohit Agarwal
>Assignee: Rohit Agarwal
>Priority: Minor
> Attachments: MAPREDUCE-6186.patch
>
>
> There are multiple calls to {{requireJob()}} in {{AppController.java#conf()}}
> The duplicate call seems to be introduced by mistake in
> https://github.com/apache/hadoop/commit/fe1cf3b0aca5f4d7a8af02a915b218f9b1de0fa6
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6186) Minor: Remove multiple calls to requireJob() while displaying the conf page
Rohit Agarwal created MAPREDUCE-6186:
Summary: Minor: Remove multiple calls to requireJob() while
displaying the conf page
Key: MAPREDUCE-6186
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6186
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver
Affects Versions: 2.6.0
Reporter: Rohit Agarwal
Assignee: Rohit Agarwal
Priority: Minor
There are multiple calls to {{requireJob()}} in {{AppController.java#conf()}}
The duplicate call seems to be introduced by mistake in
https://github.com/apache/hadoop/commit/fe1cf3b0aca5f4d7a8af02a915b218f9b1de0fa6
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6057) Remove obsolete entries from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238360#comment-14238360 ] Ray Chiang commented on MAPREDUCE-6057: --- RE: findbugs. All of these bugs aren't in the files changed by this patch. > Remove obsolete entries from mapred-default.xml > --- > > Key: MAPREDUCE-6057 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6057 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Ray Chiang >Assignee: Ray Chiang >Priority: Minor > Labels: newbie > Attachments: MAPREDUCE-6057-01.patch, MAPREDUCE-6057-02.patch, > MAPREDUCE-6057-03.patch > > > The following properties are defined in mapred-default.xml but no longer > exist in MRJobConfig. > map.sort.class > mapred.child.env > mapred.child.java.opts > mapreduce.app-submission.cross-platform > mapreduce.client.completion.pollinterval > mapreduce.client.output.filter > mapreduce.client.progressmonitor.pollinterval > mapreduce.client.submit.file.replication > mapreduce.cluster.acls.enabled > mapreduce.cluster.local.dir > mapreduce.framework.name > mapreduce.ifile.readahead > mapreduce.ifile.readahead.bytes > mapreduce.input.fileinputformat.list-status.num-threads > mapreduce.input.fileinputformat.split.minsize > mapreduce.input.lineinputformat.linespermap > mapreduce.job.counters.limit > mapreduce.job.max.split.locations > mapreduce.job.reduce.shuffle.consumer.plugin.class > mapreduce.jobhistory.address > mapreduce.jobhistory.admin.acl > mapreduce.jobhistory.admin.address > mapreduce.jobhistory.cleaner.enable > mapreduce.jobhistory.cleaner.interval-ms > mapreduce.jobhistory.client.thread-count > mapreduce.jobhistory.datestring.cache.size > mapreduce.jobhistory.done-dir > mapreduce.jobhistory.http.policy > mapreduce.jobhistory.intermediate-done-dir > mapreduce.jobhistory.joblist.cache.size > mapreduce.jobhistory.keytab > mapreduce.jobhistory.loadedjobs.cache.size > mapreduce.jobhistory.max-age-ms > mapreduce.jobhistory.minicluster.fixed.ports > mapreduce.jobhistory.move.interval-ms > mapreduce.jobhistory.move.thread-count > mapreduce.jobhistory.principal > mapreduce.jobhistory.recovery.enable > mapreduce.jobhistory.recovery.store.class > mapreduce.jobhistory.recovery.store.fs.uri > mapreduce.jobhistory.store.class > mapreduce.jobhistory.webapp.address > mapreduce.local.clientfactory.class.name > mapreduce.map.skip.proc.count.autoincr > mapreduce.output.fileoutputformat.compress > mapreduce.output.fileoutputformat.compress.codec > mapreduce.output.fileoutputformat.compress.type > mapreduce.reduce.skip.proc.count.autoincr > mapreduce.shuffle.connection-keep-alive.enable > mapreduce.shuffle.connection-keep-alive.timeout > mapreduce.shuffle.max.connections > mapreduce.shuffle.max.threads > mapreduce.shuffle.port > mapreduce.shuffle.ssl.enabled > mapreduce.shuffle.ssl.file.buffer.size > mapreduce.shuffle.transfer.buffer.size > mapreduce.shuffle.transferTo.allowed > yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts > Submitting bug for comment/feedback about which properties should be kept in > mapred-default.xml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6057) Remove obsolete entries from mapred-default.xml
[
https://issues.apache.org/jira/browse/MAPREDUCE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238350#comment-14238350
]
Hadoop QA commented on MAPREDUCE-6057:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685819/MAPREDUCE-6057-03.patch
against trunk revision ffe942b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 100 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-tools/hadoop-sls.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5068//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5068//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5068//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5068//artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5068//console
This message is automatically generated.
> Remove obsolete entries from mapred-default.xml
> ---
>
> Key: MAPREDUCE-6057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.5.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-6057-01.patch, MAPREDUCE-6057-02.patch,
> MAPREDUCE-6057-03.patch
>
>
> The following properties are defined in mapred-default.xml but no longer
> exist in MRJobConfig.
> map.sort.class
> mapred.child.env
> mapred.child.java.opts
> mapreduce.app-submission.cross-platform
> mapreduce.client.completion.pollinterval
> mapreduce.client.output.filter
> mapreduce.client.progressmonitor.pollinterval
> mapreduce.client.submit.file.replication
> mapreduce.cluster.acls.enabled
> mapreduce.cluster.local.dir
> mapreduce.framework.name
> mapreduce.ifile.readahead
> mapreduce.ifile.readahead.bytes
> mapreduce.input.fileinputformat.list-status.num-threads
> mapreduce.input.fileinputformat.split.minsize
> mapreduce.input.lineinputformat.linespermap
> mapreduce.job.counters.limit
> mapreduce.job.max.split.locations
> mapreduce.job.reduce.shuffle.consumer.plugin.class
> mapreduce.jobhistory.address
> mapreduce.jobhistory.admin.acl
> mapreduce.jobhistory.admin.address
> mapreduce.jobhistory.cleaner.enable
> mapreduce.jobhistory.cleaner.interval-ms
> mapreduce.jobhistory.client.thread-count
> mapreduce.jobhistory.datestring.cache.size
> mapreduce.jobhistory.done-dir
> mapreduce.jobhistory.http.policy
> mapreduce.jobhistory.intermediate-done-dir
> mapreduce.jobhistory.joblist.cache.size
> mapreduce.jobhistory.keytab
> mapreduce.jobhistory.loadedjobs.cache.size
> mapreduce.jobhistory.max-age-ms
> mapreduce.jobhistory.minicluster.fixed.ports
> mapreduce.jobhistory.move.interval-ms
> mapreduce.jobhistory.move.thread-count
> mapreduce.jobhistory.principal
> mapreduce.jobhistory.recovery.enable
> mapreduce.jobhistory.recovery.store.class
> mapreduce.jobhistory.recovery.store.fs.uri
> mapreduce.jobhistory.store.class
> mapreduce.jobhistory.webapp.address
> mapreduce.local.clientfactory.class.name
> mapreduce.map.skip.proc.count.autoincr
> mapreduce.output.fileoutputformat.compress
> mapreduce.output.fileoutputformat.compress.codec
> mapreduce.output.fileoutputformat.compress.type
> mapreduce.reduce.skip.proc.count.autoincr
> mapreduce.shuffle.connection-keep-alive.enable
> mapreduce.shuffle.connection-keep-alive.timeout
> mapreduce.shuffle.max.connections
> mapreduce.shuffle.max.threads
> mapreduce.shuffle.port
[jira] [Updated] (MAPREDUCE-6057) Remove obsolete entries from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6057: -- Attachment: MAPREDUCE-6057-03.patch Update patch against trunk. > Remove obsolete entries from mapred-default.xml > --- > > Key: MAPREDUCE-6057 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6057 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Ray Chiang >Assignee: Ray Chiang >Priority: Minor > Labels: newbie > Attachments: MAPREDUCE-6057-01.patch, MAPREDUCE-6057-02.patch, > MAPREDUCE-6057-03.patch > > > The following properties are defined in mapred-default.xml but no longer > exist in MRJobConfig. > map.sort.class > mapred.child.env > mapred.child.java.opts > mapreduce.app-submission.cross-platform > mapreduce.client.completion.pollinterval > mapreduce.client.output.filter > mapreduce.client.progressmonitor.pollinterval > mapreduce.client.submit.file.replication > mapreduce.cluster.acls.enabled > mapreduce.cluster.local.dir > mapreduce.framework.name > mapreduce.ifile.readahead > mapreduce.ifile.readahead.bytes > mapreduce.input.fileinputformat.list-status.num-threads > mapreduce.input.fileinputformat.split.minsize > mapreduce.input.lineinputformat.linespermap > mapreduce.job.counters.limit > mapreduce.job.max.split.locations > mapreduce.job.reduce.shuffle.consumer.plugin.class > mapreduce.jobhistory.address > mapreduce.jobhistory.admin.acl > mapreduce.jobhistory.admin.address > mapreduce.jobhistory.cleaner.enable > mapreduce.jobhistory.cleaner.interval-ms > mapreduce.jobhistory.client.thread-count > mapreduce.jobhistory.datestring.cache.size > mapreduce.jobhistory.done-dir > mapreduce.jobhistory.http.policy > mapreduce.jobhistory.intermediate-done-dir > mapreduce.jobhistory.joblist.cache.size > mapreduce.jobhistory.keytab > mapreduce.jobhistory.loadedjobs.cache.size > mapreduce.jobhistory.max-age-ms > mapreduce.jobhistory.minicluster.fixed.ports > mapreduce.jobhistory.move.interval-ms > mapreduce.jobhistory.move.thread-count > mapreduce.jobhistory.principal > mapreduce.jobhistory.recovery.enable > mapreduce.jobhistory.recovery.store.class > mapreduce.jobhistory.recovery.store.fs.uri > mapreduce.jobhistory.store.class > mapreduce.jobhistory.webapp.address > mapreduce.local.clientfactory.class.name > mapreduce.map.skip.proc.count.autoincr > mapreduce.output.fileoutputformat.compress > mapreduce.output.fileoutputformat.compress.codec > mapreduce.output.fileoutputformat.compress.type > mapreduce.reduce.skip.proc.count.autoincr > mapreduce.shuffle.connection-keep-alive.enable > mapreduce.shuffle.connection-keep-alive.timeout > mapreduce.shuffle.max.connections > mapreduce.shuffle.max.threads > mapreduce.shuffle.port > mapreduce.shuffle.ssl.enabled > mapreduce.shuffle.ssl.file.buffer.size > mapreduce.shuffle.transfer.buffer.size > mapreduce.shuffle.transferTo.allowed > yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts > Submitting bug for comment/feedback about which properties should be kept in > mapred-default.xml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6183) Fix findbugs warnings in mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith reassigned MAPREDUCE-6183: - Assignee: Rohith > Fix findbugs warnings in mapreduce-examples > --- > > Key: MAPREDUCE-6183 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6183 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: examples >Reporter: Akira AJISAKA >Assignee: Rohith > Labels: findbugs, newbie > > Work on MAPREDUCE-5800 has exposed some findbugs warnings. > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5063//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors
[
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238108#comment-14238108
]
Hadoop QA commented on MAPREDUCE-5800:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647047/MAPREDUCE-5800.2.patch
against trunk revision 8963515.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 141 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-gridmix
hadoop-tools/hadoop-streaming.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5067//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5067//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5067//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5067//artifact/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5067//console
This message is automatically generated.
> Use Job#getInstance instead of deprecated constructors
> --
>
> Key: MAPREDUCE-5800
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5800
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Affects Versions: 2.3.0, 2.6.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-5800.2.patch, MAPREDUCE-5800.patch
>
>
> There're some methods calling deprecated constructors such as {{new Job()}},
> which causes javac warnings.
> We should use {{Job.getInstance()}} to get an instance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237968#comment-14237968 ] Hudson commented on MAPREDUCE-6177: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #1965 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1965/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm * hadoop-mapreduce-project/CHANGES.txt > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237958#comment-14237958 ] Hudson commented on MAPREDUCE-6177: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #36 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/36/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm * hadoop-mapreduce-project/CHANGES.txt > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5420) Remove mapreduce.task.tmp.dir from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237961#comment-14237961 ] Harsh J commented on MAPREDUCE-5420: Went over them individually, and all of the failed tests appear irrelevant. They all have had some form of ClassNotFoundExceptions, varying in classes between YARN and MR - but this patch removes no classes from either MR or YARN. The failures are likely due to the wide nature of patch and tests or due to other build issues. > Remove mapreduce.task.tmp.dir from mapred-default.xml > - > > Key: MAPREDUCE-5420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5420 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.1.0-beta >Reporter: Sandy Ryza >Assignee: James Carman > Labels: newbie > Attachments: MAPREDUCE-5420.patch, MAPREDUCE-5420.patch > > > mapreduce.task.tmp.dir no longer has any effect, so it should no longer be > documented in mapred-default. (There is no YARN equivalent for the property. > It now is just always ./tmp). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5420) Remove mapreduce.task.tmp.dir from mapred-default.xml
[
https://issues.apache.org/jira/browse/MAPREDUCE-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237907#comment-14237907
]
Hadoop QA commented on MAPREDUCE-5420:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685737/MAPREDUCE-5420.patch
against trunk revision 8963515.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 91 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.TestLocalRunner
org.apache.hadoop.mapreduce.TestLargeSort
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapreduce.TestMapReduce
org.apache.hadoop.mapreduce.TestMRJobClient
The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5066//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5066//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5066//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5066//console
This message is automatically generated.
> Remove mapreduce.task.tmp.dir from mapred-default.xml
> -
>
> Key: MAPREDUCE-5420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5420
> Project: Hadoop Map/Reduce
> Issue Type: Task
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: James Carman
> Labels: newbie
> Attachments: MAPREDUCE-5420.patch, MAPREDUCE-5420.patch
>
>
> mapreduce.task.tmp.dir no longer has any effect, so it should no longer be
> documented in mapred-default. (There is no YARN equivalent for the property.
> It now is just always ./tmp).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237888#comment-14237888 ] Hudson commented on MAPREDUCE-6177: --- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #32 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/32/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237859#comment-14237859 ] Hudson commented on MAPREDUCE-6177: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1986 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1986/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors
[
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237834#comment-14237834
]
Hadoop QA commented on MAPREDUCE-5800:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647047/MAPREDUCE-5800.2.patch
against trunk revision 120e1de.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 141 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-gridmix
hadoop-tools/hadoop-streaming.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5065//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5065//artifact/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5065//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5065//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5065//console
This message is automatically generated.
> Use Job#getInstance instead of deprecated constructors
> --
>
> Key: MAPREDUCE-5800
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5800
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Affects Versions: 2.3.0, 2.6.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-5800.2.patch, MAPREDUCE-5800.patch
>
>
> There're some methods calling deprecated constructors such as {{new Job()}},
> which causes javac warnings.
> We should use {{Job.getInstance()}} to get an instance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237812#comment-14237812 ] Hudson commented on MAPREDUCE-6177: --- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #33 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/33/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm * hadoop-mapreduce-project/CHANGES.txt > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237801#comment-14237801 ] Hudson commented on MAPREDUCE-6177: --- SUCCESS: Integrated in Hadoop-Yarn-trunk #769 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/769/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5420) Remove mapreduce.task.tmp.dir from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-5420: --- Attachment: MAPREDUCE-5420.patch Hey [~sandyr], Since this got a bit stale, I updated [~jwcarman]'s patch here to include the test removal (and also added in the deprecation message removal). Can you take a re-look to see if this can now go in? > Remove mapreduce.task.tmp.dir from mapred-default.xml > - > > Key: MAPREDUCE-5420 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5420 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.1.0-beta >Reporter: Sandy Ryza >Assignee: James Carman > Labels: newbie > Attachments: MAPREDUCE-5420.patch, MAPREDUCE-5420.patch > > > mapreduce.task.tmp.dir no longer has any effect, so it should no longer be > documented in mapred-default. (There is no YARN equivalent for the property. > It now is just always ./tmp). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237738#comment-14237738 ] Hudson commented on MAPREDUCE-6177: --- FAILURE: Integrated in Hadoop-trunk-Commit #6667 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6667/]) MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) (harsh: rev 8963515b880b78068791f11abe4f5df332553be1) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm * hadoop-mapreduce-project/CHANGES.txt > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved MAPREDUCE-6177. Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Thank you for reporting this, good catch! I transformed your comment into the attached patch and have pushed it to trunk and branch-2. > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Fix For: 2.7.0 > > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-6177: --- Attachment: MAPREDUCE-6177.patch > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Attachments: MAPREDUCE-6177.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6177) Minor typo in the EncryptedShuffle document about ssl-client.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-6177: --- Summary: Minor typo in the EncryptedShuffle document about ssl-client.xml (was: Documentation has some error entry) > Minor typo in the EncryptedShuffle document about ssl-client.xml > > > Key: MAPREDUCE-6177 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6177 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: yangping wu >Assignee: yangping wu >Priority: Trivial > Original Estimate: 8h > Remaining Estimate: 8h > > The installation documentation for Hadoop MapReduce EncryptedShuffle document > has some error > entry(http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) > in the *ssl-client.xml (Reducer/Fetcher) Configuration:* section the *The > mapred user should own the ssl-server.xml file and it should have default > permissions.* should be *The mapred user should own the ssl-client.xml file > and it should have default permissions.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6185) YARN M/R may return NaN for reducer progress after Job completion
[
https://issues.apache.org/jira/browse/MAPREDUCE-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237635#comment-14237635
]
sam liu commented on MAPREDUCE-6185:
This issue is about method 'jobClient.getJob(jobid).reduceProgress()' and only
happens on the job which only has map tasks, but no reduce task, such as
Teragen.
[A] My Test Code:
// Using hadoop-mapreduce-client-core\org.apache.hadoop.mapred.JobClient.java
JobClient jobClient = new JobClient(new Configuration());
for (int i = 0; i < 120; i++){
Thread.sleep(1000);
System.out.println("jobClient.getJob(" + args[0] +
").reduceProgress()=" + jobClient.getJob(args[0]).reduceProgress());
}
[B] WordCount: Got expected result
Can get reduceProgress during job running(0.0 to 1.0) and after job
completion(1.0).
$ hadoop jar yarn.hadoop.jar hadoop.yarn.proj.test.YarnClientTest
job_1417679271770_0169
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.2084
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.2084
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.2084
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.3334
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.3334
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.3334
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.6765683
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.6765683
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.6765683
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.76213264
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.76213264
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.76213264
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.8284452
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.8284452
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.8284452
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.9177592
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.9177592
jobClient.getJob(job_1417679271770_0169).reduceProgress()=0.9177592
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
14/12/04 22:54:21 INFO mapred.ClientServiceDelegate: Application state is
completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
jobClient.getJob(job_1417679271770_0169).reduceProgress()=1.0
[C] Teragen: Failed to get expected result
- Can get reduceProgress during job running(0.0 )
- But get NaN after job completion
As Teragen only has Mapper, but does not has Reducer, so I think it's correct.
$ hadoop jar yarn.hadoop.jar hadoop.yarn.proj.test.YarnClientTest
job_1417679271770_0168
14/12/04 22:51:53 INFO client.RMProxy: Connecting to ResourceManager at
cluster.nn1/9.31.111.201:8032
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
jobClient.getJob(job_1417679271770_0168).reduceProgress()=0.0
14/12/04 22:52:17 INFO mapred.ClientServiceDelegate: Application state is
completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
jobClient.getJob(job_1417679271770_0168).reduceProgress()=NaN
jobClient.getJob(job_1417679271770_0168).reduceProgress()=NaN
jobClient.getJob(job_1417679271770_0168).reduceProgress()=NaN
jobClient.getJob(job_1417679271770_0168).reduceProgress()=NaN
jobClient.getJob(job_1417679271770_0168).reduceProgress()=NaN
Expected result: '0.0'.
Actual result: 'NaN'.
Comment:
- Even after job completion, method reduceProgress() still should return same
result '0.0', but not 'NaN'.
- As 'Redirecting to job history server', marked the component of this jira as
'jobhistoryserver'
> YARN M/R may return NaN for reducer progress after Job completion
> -
>
>
[jira] [Commented] (MAPREDUCE-6181) WordCount.java in mapreduce-examples.jar uses deprecated job object creation API
[
https://issues.apache.org/jira/browse/MAPREDUCE-6181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237633#comment-14237633
]
Suraj Nayak M commented on MAPREDUCE-6181:
--
Should this JIRA be marked Duplicate and close it ?
> WordCount.java in mapreduce-examples.jar uses deprecated job object creation
> API
>
>
> Key: MAPREDUCE-6181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6181
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: sachin
>Assignee: sachin
>Priority: Trivial
>
> In the trunk branch the mapreduce-examples.jar contains a WordCount class. In
> which deprecated Job object has been used. So this Jira is to fix the
> deprecation warning.
> {code:title=WordCount.java|borderStyle=solid}
> // Currently : Job job = new Job(conf, "word count");
> Job job = Job.getInstance(conf);
> job.setJobName("word count");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6185) YARN M/R may return NaN for reducer progress after Job completion
sam liu created MAPREDUCE-6185: -- Summary: YARN M/R may return NaN for reducer progress after Job completion Key: MAPREDUCE-6185 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6185 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.4.1 Reporter: sam liu Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
