[jira] [Commented] (MAPREDUCE-6269) improve JobConf to add option to not reference same credentials between jobs.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536771#comment-14536771
]
zhihai xu commented on MAPREDUCE-6269:
--
The following checkstyle issue is the file JobConf.java too big. It is already
more than 2,000 before my change.
{code}
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java:1:
File length is 2,164 lines (max allowed is 2,000).
{code}
improve JobConf to add option to not reference same credentials between jobs.
-
Key: MAPREDUCE-6269
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6269
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Labels: BB2015-05-RFC
Attachments: MAPREDUCE-6269.000.patch, MAPREDUCE-6269.001.patch
Improve JobConf to add constructor to avoid sharing Credentials between jobs.
By default the Credentials will be shared to keep the backward compatibility.
We can add a new constructor with a new parameter to decide whether to share
Credentials. Some issues reported in cascading is due to corrupted credentials
at
https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e
If we add this support in JobConf, it will benefit all job clients.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6269) improve JobConf to add option to not reference same credentials between jobs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536251#comment-14536251 ] zhihai xu commented on MAPREDUCE-6269: -- I uploaded a new patch MAPREDUCE-6269.001.patch based on [~tgraves]'s comments for review. improve JobConf to add option to not reference same credentials between jobs. - Key: MAPREDUCE-6269 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6269 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Reporter: zhihai xu Assignee: zhihai xu Labels: BB2015-05-RFC Attachments: MAPREDUCE-6269.000.patch, MAPREDUCE-6269.001.patch Improve JobConf to add constructor to avoid sharing Credentials between jobs. By default the Credentials will be shared to keep the backward compatibility. We can add a new constructor with a new parameter to decide whether to share Credentials. Some issues reported in cascading is due to corrupted credentials at https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e If we add this support in JobConf, it will benefit all job clients. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6269) improve JobConf to add option to not reference same credentials between jobs.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536266#comment-14536266
]
Hadoop QA commented on MAPREDUCE-6269:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 29s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 31s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 47s | The applied patch generated 1
new checkstyle issues (total was 96, now 96). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 37s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 37m 47s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731683/MAPREDUCE-6269.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 02a4a22 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5694/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5694/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5694/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5694/console |
This message was automatically generated.
improve JobConf to add option to not reference same credentials between jobs.
-
Key: MAPREDUCE-6269
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6269
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Labels: BB2015-05-RFC
Attachments: MAPREDUCE-6269.000.patch, MAPREDUCE-6269.001.patch
Improve JobConf to add constructor to avoid sharing Credentials between jobs.
By default the Credentials will be shared to keep the backward compatibility.
We can add a new constructor with a new parameter to decide whether to share
Credentials. Some issues reported in cascading is due to corrupted credentials
at
https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e
If we add this support in JobConf, it will benefit all job clients.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
