[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372568#comment-14372568
]
Hadoop QA commented on MAPREDUCE-6286:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706131/MAPREDUCE-6286.000.patch
against trunk revision e1feb4e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5323//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5323//console
This message is automatically generated.
A typo in HistoryViewer makes some code useless.
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6286:
-
Description:
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
was:
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits is reset with the old
configuration.
So this typo is apparent.
A typo in HistoryViewer makes some code useless.
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6286:
-
Description:
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
was:
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
Summary: A typo in HistoryViewer makes some code useless, which causes
counter limits are not reset correctly. (was: A typo in HistoryViewer makes
some code useless.)
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6286:
-
Attachment: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless.
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless.
zhihai xu created MAPREDUCE-6286:
Summary: A typo in HistoryViewer makes some code useless.
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6282) Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372828#comment-14372828 ] Hudson commented on MAPREDUCE-6282: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #139 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/139/]) MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) (rkanter: rev d4f7e2507f4bb02d172f94e74431bc2f319c) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- Key: MAPREDUCE-6282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6282 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: zhihai xu Assignee: zhihai xu Priority: Trivial Labels: optimization Fix For: 2.8.0 Attachments: MAPREDUCE-6282.000.patch Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6286:
-
Description:
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits is reset with the old
configuration.
So this typo is apparent.
was:
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file.
So this typo is apparent.
A typo in HistoryViewer makes some code useless.
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits is reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6286:
-
Status: Patch Available (was: Open)
A typo in HistoryViewer makes some code useless.
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6287) Deprecated methods in org.apache.hadoop.examples.Sort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Zhang updated MAPREDUCE-6287: -- Fix Version/s: 2.7.0 Assignee: Chao Zhang Labels: newbie (was: ) Target Version/s: 2.7.0 Status: Patch Available (was: Open) Deprecated methods in org.apache.hadoop.examples.Sort - Key: MAPREDUCE-6287 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6287 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Chao Zhang Assignee: Chao Zhang Priority: Minor Labels: newbie Fix For: 2.7.0 org.apache.hadoop.examples.Sort is still using deprecated methods like Path#makeQualified and DistributedCache. It is better to replace them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6282) Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372741#comment-14372741 ] Hudson commented on MAPREDUCE-6282: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2089 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2089/]) MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) (rkanter: rev d4f7e2507f4bb02d172f94e74431bc2f319c) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * hadoop-mapreduce-project/CHANGES.txt Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- Key: MAPREDUCE-6282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6282 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: zhihai xu Assignee: zhihai xu Priority: Trivial Labels: optimization Fix For: 2.8.0 Attachments: MAPREDUCE-6282.000.patch Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5782) Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd.
[ https://issues.apache.org/jira/browse/MAPREDUCE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372783#comment-14372783 ] Chao Zhang commented on MAPREDUCE-5782: --- Can't reproduce that again. Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd. Key: MAPREDUCE-5782 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5782 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.10, 2.4.0 Reporter: Bing Jiang Priority: Minor Attachments: MAPREDUCE-5782.001.patch hadoop jar hadoop-mapreduce-examples.jar sort sort \ -inFormat org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat \ -outFormat org.apache.hadoop.mapreduce.lib.output.TextOutputFormat \ -outKey org.apache.hadoop.io.Text \ -outValue org.apache.hadoop.io.Text \ -totalOrder 0.1 1 10 \ input output It prints cannot find the file _partition.lst in the container's cwd. Error: java.lang.IllegalArgumentException: Can't read partitions file at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:116) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:642) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:710) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5782) Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd.
[ https://issues.apache.org/jira/browse/MAPREDUCE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Zhang updated MAPREDUCE-5782: -- Assignee: (was: Chao Zhang) Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd. Key: MAPREDUCE-5782 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5782 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.10, 2.4.0 Reporter: Bing Jiang Priority: Minor Attachments: MAPREDUCE-5782.001.patch hadoop jar hadoop-mapreduce-examples.jar sort sort \ -inFormat org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat \ -outFormat org.apache.hadoop.mapreduce.lib.output.TextOutputFormat \ -outKey org.apache.hadoop.io.Text \ -outValue org.apache.hadoop.io.Text \ -totalOrder 0.1 1 10 \ input output It prints cannot find the file _partition.lst in the container's cwd. Error: java.lang.IllegalArgumentException: Can't read partitions file at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:116) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:642) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:710) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6282) Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372767#comment-14372767 ] Hudson commented on MAPREDUCE-6282: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #2071 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2071/]) MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) (rkanter: rev d4f7e2507f4bb02d172f94e74431bc2f319c) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * hadoop-mapreduce-project/CHANGES.txt Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- Key: MAPREDUCE-6282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6282 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: zhihai xu Assignee: zhihai xu Priority: Trivial Labels: optimization Fix For: 2.8.0 Attachments: MAPREDUCE-6282.000.patch Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6282) Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372648#comment-14372648 ] Hudson commented on MAPREDUCE-6282: --- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #139 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/139/]) MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) (rkanter: rev d4f7e2507f4bb02d172f94e74431bc2f319c) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- Key: MAPREDUCE-6282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6282 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: zhihai xu Assignee: zhihai xu Priority: Trivial Labels: optimization Fix For: 2.8.0 Attachments: MAPREDUCE-6282.000.patch Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5782) Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd.
[ https://issues.apache.org/jira/browse/MAPREDUCE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Zhang updated MAPREDUCE-5782: -- Status: Open (was: Patch Available) Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd. Key: MAPREDUCE-5782 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5782 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.10, 2.4.0 Reporter: Bing Jiang Priority: Minor Attachments: MAPREDUCE-5782.001.patch hadoop jar hadoop-mapreduce-examples.jar sort sort \ -inFormat org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat \ -outFormat org.apache.hadoop.mapreduce.lib.output.TextOutputFormat \ -outKey org.apache.hadoop.io.Text \ -outValue org.apache.hadoop.io.Text \ -totalOrder 0.1 1 10 \ input output It prints cannot find the file _partition.lst in the container's cwd. Error: java.lang.IllegalArgumentException: Can't read partitions file at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:116) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:642) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:710) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6282) Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372793#comment-14372793 ] Hudson commented on MAPREDUCE-6282: --- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #130 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/130/]) MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) (rkanter: rev d4f7e2507f4bb02d172f94e74431bc2f319c) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * hadoop-mapreduce-project/CHANGES.txt Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- Key: MAPREDUCE-6282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6282 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: zhihai xu Assignee: zhihai xu Priority: Trivial Labels: optimization Fix For: 2.8.0 Attachments: MAPREDUCE-6282.000.patch Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6287) Deprecated methods in org.apache.hadoop.examples.Sort
Chao Zhang created MAPREDUCE-6287: - Summary: Deprecated methods in org.apache.hadoop.examples.Sort Key: MAPREDUCE-6287 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6287 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Chao Zhang Priority: Minor org.apache.hadoop.examples.Sort is still using deprecated methods like Path#makeQualified and DistributedCache. It is better to replace them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6282) Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372662#comment-14372662 ] Hudson commented on MAPREDUCE-6282: --- FAILURE: Integrated in Hadoop-Yarn-trunk #873 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/873/]) MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) (rkanter: rev d4f7e2507f4bb02d172f94e74431bc2f319c) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- Key: MAPREDUCE-6282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6282 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: zhihai xu Assignee: zhihai xu Priority: Trivial Labels: optimization Fix For: 2.8.0 Attachments: MAPREDUCE-6282.000.patch Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6239: Status: Open (was: Patch Available) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core - Key: MAPREDUCE-6239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6239: Attachment: MAPREDUCE-6239.branch-2.patch Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core - Key: MAPREDUCE-6239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372954#comment-14372954
]
Hadoop QA commented on MAPREDUCE-6239:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706181/MAPREDUCE-6239.branch-2.patch
against trunk revision e1feb4e.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5324//console
This message is automatically generated.
Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and
hadoop-mapreduce-client-core
-
Key: MAPREDUCE-6239
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Affects Versions: 2.6.0
Reporter: Varun Saxena
Assignee: Varun Saxena
Priority: Minor
Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6213) NullPointerException caused by job history server addr not resolvable
[
https://issues.apache.org/jira/browse/MAPREDUCE-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373020#comment-14373020
]
Harsh J commented on MAPREDUCE-6213:
+1 for the change. Committing shortly. Thanks Peng!
NullPointerException caused by job history server addr not resolvable
-
Key: MAPREDUCE-6213
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6213
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: applicationmaster
Affects Versions: 2.6.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Priority: Minor
Attachments: MAPREDUCE-6213.patch
When DNS failed for a time, all MapReduce jobs which completed during that
time got failed. Log as below:
{noformat}
2015-01-08 01:26:44,968 ERROR [Thread-856]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Exception while
unregistering
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithoutScheme(MRWebAppUtil.java:135)
at
org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithScheme(MRWebAppUtil.java:150)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.doUnregistration(RMCommunicator.java:212)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.unregister(RMCommunicator.java:182)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStop(RMCommunicator.java:255)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStop(RMContainerAllocator.java:257)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStop(MRAppMaster.java:823)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1471)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1085)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372952#comment-14372952 ] Varun Saxena commented on MAPREDUCE-6239: - [~qwertymaniac], uploaded branch-2 patch Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core - Key: MAPREDUCE-6239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6239: Status: Patch Available (was: Open) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core - Key: MAPREDUCE-6239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-6239: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: (was: 2.8.0) Status: Resolved (was: Patch Available) Thanks again Varun! +1. Committed to branch-2 and trunk. Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core - Key: MAPREDUCE-6239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Fix For: 2.8.0 Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6239) Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373016#comment-14373016 ] Hudson commented on MAPREDUCE-6239: --- FAILURE: Integrated in Hadoop-trunk-Commit #7393 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7393/]) MAPREDUCE-6239. Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core. Contributed by Varun Saxena. (harsh: rev 7a678db3accf9480f3799dcf6fd7ffef09a311cc) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/conf/TestJobConf.java Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core - Key: MAPREDUCE-6239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Fix For: 2.8.0 Attachments: MAPREDUCE-6239.01.patch, MAPREDUCE-6239.branch-2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5660) Log info about possible thrashing (when using memory-based scheduling in Capacity Scheduler) is not printed
[
https://issues.apache.org/jira/browse/MAPREDUCE-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-5660:
---
Resolution: Fixed
Fix Version/s: 1.3.0
Assignee: Adam Kawa
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Adam, sorry on delay!
+1, Committed to branch-1 (patch needed a source path change, but applied
clean).
Log info about possible thrashing (when using memory-based scheduling in
Capacity Scheduler) is not printed
---
Key: MAPREDUCE-5660
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5660
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: capacity-sched, mrv1, tasktracker
Affects Versions: 1.2.1
Reporter: Adam Kawa
Assignee: Adam Kawa
Priority: Trivial
Fix For: 1.3.0
Attachments: MAPREDUCE-5660.1.patch.txt
There is a tiny, but confusing when troubleshooting, bug in TaskTracker code:
{code}
if (totalMemoryAllottedForTasks totalPhysicalMemoryOnTT) {
LOG.info(totalMemoryAllottedForTasks totalPhysicalMemoryOnTT.
+ Thrashing might happen.);
} else if (totalMemoryAllottedForTasks totalVirtualMemoryOnTT) {
LOG.info(totalMemoryAllottedForTasks totalVirtualMemoryOnTT.
+ Thrashing might happen.);
}
{code}
totalMemoryAllottedForTasks is calculated in megabytes, while
totalPhysicalMemoryOnTT (and totalVirtualMemoryOnTT) is calculated in bytes.
totalMemoryAllottedForTasks should be converted to bytes for a correct
comparison.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6213) NullPointerException caused by job history server addr not resolvable
[
https://issues.apache.org/jira/browse/MAPREDUCE-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-6213:
---
Resolution: Fixed
Fix Version/s: 2.8.0
Target Version/s: (was: 2.7.0)
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to branch-2 and trunk.
NullPointerException caused by job history server addr not resolvable
-
Key: MAPREDUCE-6213
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6213
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: applicationmaster
Affects Versions: 2.6.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-6213.patch
When DNS failed for a time, all MapReduce jobs which completed during that
time got failed. Log as below:
{noformat}
2015-01-08 01:26:44,968 ERROR [Thread-856]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Exception while
unregistering
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithoutScheme(MRWebAppUtil.java:135)
at
org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithScheme(MRWebAppUtil.java:150)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.doUnregistration(RMCommunicator.java:212)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.unregister(RMCommunicator.java:182)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStop(RMCommunicator.java:255)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStop(RMContainerAllocator.java:257)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStop(MRAppMaster.java:823)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1471)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1085)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373027#comment-14373027
]
Harsh J commented on MAPREDUCE-6286:
+1, nice catch. Committing shortly. Thanks Zhihai!
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6213) NullPointerException caused by job history server addr not resolvable
[
https://issues.apache.org/jira/browse/MAPREDUCE-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373028#comment-14373028
]
Hudson commented on MAPREDUCE-6213:
---
FAILURE: Integrated in Hadoop-trunk-Commit #7394 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7394/])
MAPREDUCE-6213. NullPointerException caused by job history server addr not
resolvable. Contributed by Peng Zhang. (harsh: rev
e1e09052e861926112493d6041aae01ab594b547)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
* hadoop-mapreduce-project/CHANGES.txt
NullPointerException caused by job history server addr not resolvable
-
Key: MAPREDUCE-6213
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6213
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: applicationmaster
Affects Versions: 2.6.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-6213.patch
When DNS failed for a time, all MapReduce jobs which completed during that
time got failed. Log as below:
{noformat}
2015-01-08 01:26:44,968 ERROR [Thread-856]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Exception while
unregistering
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithoutScheme(MRWebAppUtil.java:135)
at
org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithScheme(MRWebAppUtil.java:150)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.doUnregistration(RMCommunicator.java:212)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.unregister(RMCommunicator.java:182)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStop(RMCommunicator.java:255)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStop(RMContainerAllocator.java:257)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStop(MRAppMaster.java:823)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1471)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1085)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373032#comment-14373032
]
Hudson commented on MAPREDUCE-6286:
---
FAILURE: Integrated in Hadoop-trunk-Commit #7395 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7395/])
MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes
counter limits are not reset correctly. Contributed by Zhihai Xu. (harsh: rev
433542904aba5ddebf9bd9d299378647351eb13a)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
* hadoop-mapreduce-project/CHANGES.txt
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.8.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373093#comment-14373093
]
Gera Shegalov commented on MAPREDUCE-6286:
--
+1, Thanks for spotting the bug, [~zxu]!
[~qwertymaniac], do you mind committing to branch-2.7 as well? It looks like a
safe change.
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.8.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-6286:
---
Resolution: Fixed
Fix Version/s: 2.8.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.8.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-6286:
---
Affects Version/s: 2.6.0
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.8.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5183) In, TaskTracker#reportProgress logging of 0.0-1.0 progress is followed by percent sign
[ https://issues.apache.org/jira/browse/MAPREDUCE-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-5183: --- Resolution: Fixed Fix Version/s: 1.3.0 Target Version/s: (was: 1.0.4) Release Note: (was: The progress numbers are between 0 and 100 along with the percent sign) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1, LGTM too. Committed to branch-1. Thanks Niranjan for the fix! Thanks Sandy for the report and review earlier! In, TaskTracker#reportProgress logging of 0.0-1.0 progress is followed by percent sign -- Key: MAPREDUCE-5183 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5183 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, tasktracker Affects Versions: 1.1.2 Reporter: Sandy Ryza Priority: Minor Labels: newbie Fix For: 1.3.0 Attachments: MAPREDUCE-5183.patch, MAPREDUCE-5183.patch This makes looking at progress in the logs unnecessarily confusing. It would probably look prettiest to keep the percentage sign and have the numbers between 0 and 100. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14373082#comment-14373082
]
zhihai xu commented on MAPREDUCE-6286:
--
[~qwertymaniac], many thanks for reviewing and committing the patch!
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.8.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-5448:
---
Attachment: MAPREDUCE-5448.patch
+1, I've just cleaned up the indentation and the comment typo in the test case
added. Changed diff attached for ref.
Committing shortly. Thanks Maysam!
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Attachments: MAPREDUCE-5448.patch, MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374787#comment-14374787
]
Hudson commented on MAPREDUCE-5448:
---
FAILURE: Integrated in Hadoop-trunk-Commit #7396 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7396/])
MAPREDUCE-5448. MapFileOutputFormat#getReaders bug with invisible
files/folders. Contributed by Maysam Yabandeh. (harsh: rev
b46c2bb51ae524e6640756620f70e5925cda7592)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/MapFileOutputFormat.java
* hadoop-mapreduce-project/CHANGES.txt
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-5448.patch, MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-5448:
---
Component/s: mrv2
Target Version/s: 2.8.0
Affects Version/s: 2.6.0
Hadoop Flags: Reviewed
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Attachments: MAPREDUCE-5448.patch, MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-5448:
---
Resolution: Fixed
Fix Version/s: 2.8.0
Target Version/s: (was: 2.8.0)
Status: Resolved (was: Patch Available)
Tests and compilation for core module passed locally. Committed to branch-2 and
trunk.
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-5448.patch, MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5190) Unnecessary condition test in RandomSampler
[ https://issues.apache.org/jira/browse/MAPREDUCE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-5190: --- Component/s: (was: mrv1) mrv2 Unnecessary condition test in RandomSampler --- Key: MAPREDUCE-5190 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5190 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.6.0 Reporter: Jingguo Yao Assignee: Jingguo Yao Priority: Minor Attachments: MAPREDUCE-5190.patch Original Estimate: 20m Remaining Estimate: 20m In getSampe method, there is a condition test after int ind = r.nextInt(numSamples);. The test is (ind != numSamples). This test is unneeded since nextInt(numSamples) will not return numSamples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5190) Unnecessary condition test in RandomSampler
[ https://issues.apache.org/jira/browse/MAPREDUCE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374788#comment-14374788 ] Harsh J commented on MAPREDUCE-5190: +1, change looks good to me (http://docs.oracle.com/javase/7/docs/api/java/util/Random.html#nextInt(int)) Committing shortly. Thanks Jingguo! Unnecessary condition test in RandomSampler --- Key: MAPREDUCE-5190 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5190 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.6.0 Reporter: Jingguo Yao Assignee: Jingguo Yao Priority: Minor Attachments: MAPREDUCE-5190.patch Original Estimate: 20m Remaining Estimate: 20m In getSampe method, there is a condition test after int ind = r.nextInt(numSamples);. The test is (ind != numSamples). This test is unneeded since nextInt(numSamples) will not return numSamples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5190) Unnecessary condition test in RandomSampler
[ https://issues.apache.org/jira/browse/MAPREDUCE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-5190: --- Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Unnecessary condition test in RandomSampler --- Key: MAPREDUCE-5190 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5190 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.6.0 Reporter: Jingguo Yao Assignee: Jingguo Yao Priority: Minor Fix For: 2.8.0 Attachments: MAPREDUCE-5190.patch Original Estimate: 20m Remaining Estimate: 20m In getSampe method, there is a condition test after int ind = r.nextInt(numSamples);. The test is (ind != numSamples). This test is unneeded since nextInt(numSamples) will not return numSamples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5190) Unnecessary condition test in RandomSampler
[ https://issues.apache.org/jira/browse/MAPREDUCE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-5190: --- Affects Version/s: (was: 1.0.4) 2.6.0 Unnecessary condition test in RandomSampler --- Key: MAPREDUCE-5190 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5190 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.6.0 Reporter: Jingguo Yao Assignee: Jingguo Yao Priority: Minor Attachments: MAPREDUCE-5190.patch Original Estimate: 20m Remaining Estimate: 20m In getSampe method, there is a condition test after int ind = r.nextInt(numSamples);. The test is (ind != numSamples). This test is unneeded since nextInt(numSamples) will not return numSamples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5190) Unnecessary condition test in RandomSampler
[ https://issues.apache.org/jira/browse/MAPREDUCE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374790#comment-14374790 ] Hudson commented on MAPREDUCE-5190: --- FAILURE: Integrated in Hadoop-trunk-Commit #7397 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7397/]) MAPREDUCE-5190. Unnecessary condition test in RandomSampler. Contributed by Jingguo Yao. (harsh: rev 1d5c796d654c8959972d15cc6742731a99380bfc) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java * hadoop-mapreduce-project/CHANGES.txt Unnecessary condition test in RandomSampler --- Key: MAPREDUCE-5190 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5190 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.6.0 Reporter: Jingguo Yao Assignee: Jingguo Yao Priority: Minor Fix For: 2.8.0 Attachments: MAPREDUCE-5190.patch Original Estimate: 20m Remaining Estimate: 20m In getSampe method, there is a condition test after int ind = r.nextInt(numSamples);. The test is (ind != numSamples). This test is unneeded since nextInt(numSamples) will not return numSamples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374793#comment-14374793
]
Harsh J commented on MAPREDUCE-6286:
Sure. Pushed into 2.7.0 (amended CHANGES.txt in trunk and branch-2 too).
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.7.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-6286:
---
Fix Version/s: (was: 2.8.0)
2.7.0
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.7.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374797#comment-14374797
]
Hadoop QA commented on MAPREDUCE-5448:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706366/MAPREDUCE-5448.patch
against trunk revision 4335429.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1157 javac
compiler warnings (more than the trunk's current 1155 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5325//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5325//artifact/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5325//console
This message is automatically generated.
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-5448.patch, MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374798#comment-14374798
]
Hudson commented on MAPREDUCE-6286:
---
FAILURE: Integrated in Hadoop-trunk-Commit #7398 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7398/])
MAPREDUCE-6286. Amend commit to CHANGES.txt for backport into 2.7.0. (harsh:
rev 8770c82acc948bc5127afb1c59072718fd04630c)
* hadoop-mapreduce-project/CHANGES.txt
A typo in HistoryViewer makes some code useless, which causes counter limits
are not reset correctly.
-
Key: MAPREDUCE-6286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.6.0
Reporter: zhihai xu
Assignee: zhihai xu
Fix For: 2.7.0
Attachments: MAPREDUCE-6286.000.patch
A typo in HistoryViewer makes some code useless and it causes counter limits
are not reset correctly.
The typo is
Limits.reset(conf);
We should use jobConf instead of conf.
With the typo, the following code becomes useless:
{code}
final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
+ _ + jobDetails[1] + _ + jobDetails[2] + _conf.xml);
final Configuration jobConf = new Configuration(conf);
jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
{code}
The code wants to load the configuration from the Job configuration file and
reset the Limits based on the new configuration loaded from the Job
configuration file. But with the typo, the Limits are reset with the old
configuration.
So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374800#comment-14374800
]
Harsh J commented on MAPREDUCE-5448:
Ah, didn't realise junit.Assert import could cause this. Adding an addendum
patch to resolve the import and fall back to using fail(…) from the TestCase
inherited method.
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-5448.patch, MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-5448:
---
Attachment: MAPREDUCE-5448.addendum.patch
Addendum should clear up the error. Changed on branch-2 and trunk. Sorry for
the noise!
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-5448.addendum.patch, MAPREDUCE-5448.patch,
MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6287) Deprecated methods in org.apache.hadoop.examples.Sort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-6287: --- Fix Version/s: (was: 2.7.0) Deprecated methods in org.apache.hadoop.examples.Sort - Key: MAPREDUCE-6287 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6287 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Chao Zhang Assignee: Chao Zhang Priority: Minor Labels: newbie org.apache.hadoop.examples.Sort is still using deprecated methods like Path#makeQualified and DistributedCache. It is better to replace them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6287) Deprecated methods in org.apache.hadoop.examples.Sort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-6287: --- Status: Open (was: Patch Available) Thanks for reporting and working on this Chao. Please attach the patch via More - Attach Files before marking it as Patch Available, so we may review and commit! Please also set only the target version, as the fix version field is to be used only by committers to track actual committed branches. Deprecated methods in org.apache.hadoop.examples.Sort - Key: MAPREDUCE-6287 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6287 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Chao Zhang Assignee: Chao Zhang Priority: Minor Labels: newbie org.apache.hadoop.examples.Sort is still using deprecated methods like Path#makeQualified and DistributedCache. It is better to replace them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5448) MapFileOutputFormat#getReaders bug with invisible files/folders
[
https://issues.apache.org/jira/browse/MAPREDUCE-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14374804#comment-14374804
]
Hudson commented on MAPREDUCE-5448:
---
FAILURE: Integrated in Hadoop-trunk-Commit #7399 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7399/])
MAPREDUCE-5448. Addendum fix to remove deprecation warning by junit.Assert
import in TestFileOutputCommitter. (harsh: rev
4cd54d9a297435150ab61803284eb05603f114e2)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
MapFileOutputFormat#getReaders bug with invisible files/folders
---
Key: MAPREDUCE-5448
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5448
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.6.0
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
Priority: Minor
Fix For: 2.8.0
Attachments: MAPREDUCE-5448.addendum.patch, MAPREDUCE-5448.patch,
MAPREDUCE-5448.patch
Original Estimate: 1h
Remaining Estimate: 1h
MapReduce jobs also produce some invisible files such as _SUCCESS, even when
the output format is MapFileOutputFormat. MapFileOutputFormat#getReaders
however reads the entire content of the job output, assming that they are
MapFiles.
{code}
Path[] names = FileUtil.stat2Paths(fs.listStatus(dir));
{code}
It should use a filter to skip the files that start with . or _.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
