[jira] [Issue Comment Deleted] (MAPREDUCE-6729) Accurately compute the test execute time in DFSIO
[
https://issues.apache.org/jira/browse/MAPREDUCE-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mingleizhang updated MAPREDUCE-6729:
Comment: was deleted
(was: Thanks to all and I am glad to enjoy this time.)
> Accurately compute the test execute time in DFSIO
> -
>
> Key: MAPREDUCE-6729
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6729
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: benchmarks, performance, test
>Affects Versions: 2.9.0
>Reporter: mingleizhang
>Assignee: mingleizhang
>Priority: Minor
> Labels: performance, test
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6729.001.patch, MAPREDUCE-6729.002.patch
>
>
> When doing DFSIO test as a distributed i/o benchmark tool. Then especially
> writes plenty of files to disk or read from, both can cause performance issue
> and imprecise value in a way. The question is that existing practices needs
> to delete files when before running a job and that will cause extra time
> consumption and furthermore cause performance issue, statistical time error
> and imprecise throughput while the files are lots of. So we need to replace
> or improve this hack to prevent this from happening in the future.
> {code}
> public static void testWrite() throws Exception {
> FileSystem fs = cluster.getFileSystem();
> long tStart = System.currentTimeMillis();
> bench.writeTest(fs); // this line of code will cause extra time
> consumption because of fs.delete(*,*) by the writeTest method
> long execTime = System.currentTimeMillis() - tStart;
> bench.analyzeResult(fs, TestType.TEST_TYPE_WRITE, execTime);
> }
> private void writeTest(FileSystem fs) throws IOException {
> Path writeDir = getWriteDir(config);
> fs.delete(getDataDir(config), true);
> fs.delete(writeDir, true);
> runIOTest(WriteMapper.class, writeDir);
> }
> {code}
> [https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6747) TestMapReduceJobControl#testJobControlWithKillJob times out in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410263#comment-15410263
]
Vrushali C commented on MAPREDUCE-6747:
---
So this is interesting. The warning messages above are not causing the timeout.
I observe that if I were to put a logging statement in
TestMapReduceJobControl#waitTillAllFinished as
{code}
private void waitTillAllFinished(JobControl theControl) {
while (!theControl.allFinished()) {
try {
LOG.info("Sleeping..");
Thread.sleep(1000);
} catch (Exception e) {}
}
}
{code}
The tests pass for me. It appears that without this logging message, the job is
stuck at map phase in the testJobControlWithKillJob test
{code}
2016-08-05 16:08:32,831 INFO [pool-35-thread-1] mapred.LocalJobRunner
(LocalJobRunner.java:run(353)) - Finishing task:
attempt_local1592761666_0012_r_00_0
2016-08-05 16:08:32,831 INFO [Thread-498] mapred.LocalJobRunner
(LocalJobRunner.java:runTasks(486)) - reduce task executor complete.
2016-08-05 16:08:38,421 INFO [communication thread] mapred.LocalJobRunner
(LocalJobRunner.java:statusUpdate(628)) - map > map
{code}
Do you see the same behavior [~ctrezzo] that all tests pass in
TestMapReduceJobControl if you put a logging statement before the Thread.sleep
in waitTillAllFinished?
> TestMapReduceJobControl#testJobControlWithKillJob times out in trunk
>
>
> Key: MAPREDUCE-6747
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6747
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Chris Trezzo
>Priority: Minor
>
> TestMapReduceJobControl#testJobControlWithKillJob seems to time out while
> waiting for all jobs to complete. This seems to only happen if the test is
> run with the other tests in the class (specifically
> testJobControlWithFailJob). If testJobControlWithKillJob is run by itself,
> the test passes.
> Looking into the test logs, when run with another test from the class, the
> test runs into an issue while setting permissions on the local file system:
> {noformat}
> 2016-08-05 11:40:32,101 WARN [Thread-100] util.Shell
> (Shell.java:joinThread(1023)) - Interrupted while joining on:
> Thread[Thread-105,5,main]
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1245)
> at java.lang.Thread.join(Thread.java:1319)
> at org.apache.hadoop.util.Shell.joinThread(Shell.java:1020)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:969)
> at org.apache.hadoop.util.Shell.run(Shell.java:878)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:1266)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:1248)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:781)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:566)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:565)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:565)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:696)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:343)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:541)
> {noformat}
> Conversely, when the test is run by itself, this issue is not hit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6747) TestMapReduceJobControl#testJobControlWithKillJob times out in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410119#comment-15410119
]
Vrushali C commented on MAPREDUCE-6747:
---
Going to try to see if I can fix this. Will update the jira shortly.
> TestMapReduceJobControl#testJobControlWithKillJob times out in trunk
>
>
> Key: MAPREDUCE-6747
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6747
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Chris Trezzo
>Priority: Minor
>
> TestMapReduceJobControl#testJobControlWithKillJob seems to time out while
> waiting for all jobs to complete. This seems to only happen if the test is
> run with the other tests in the class (specifically
> testJobControlWithFailJob). If testJobControlWithKillJob is run by itself,
> the test passes.
> Looking into the test logs, when run with another test from the class, the
> test runs into an issue while setting permissions on the local file system:
> {noformat}
> 2016-08-05 11:40:32,101 WARN [Thread-100] util.Shell
> (Shell.java:joinThread(1023)) - Interrupted while joining on:
> Thread[Thread-105,5,main]
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1245)
> at java.lang.Thread.join(Thread.java:1319)
> at org.apache.hadoop.util.Shell.joinThread(Shell.java:1020)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:969)
> at org.apache.hadoop.util.Shell.run(Shell.java:878)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:1266)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:1248)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:781)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:566)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:565)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:565)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:696)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:343)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:541)
> {noformat}
> Conversely, when the test is run by itself, this issue is not hit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6690) Limit the number of resources a single map reduce job can submit for localization
[ https://issues.apache.org/jira/browse/MAPREDUCE-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409997#comment-15409997 ] Chris Trezzo commented on MAPREDUCE-6690: - Filed jira to fix unrelated test failure: MAPREDUCE-6747 > Limit the number of resources a single map reduce job can submit for > localization > - > > Key: MAPREDUCE-6690 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6690 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Chris Trezzo >Assignee: Chris Trezzo > Attachments: MAPREDUCE-6690-trunk-v1.patch, > MAPREDUCE-6690-trunk-v2.patch, MAPREDUCE-6690-trunk-v3.patch, > MAPREDUCE-6690-trunk-v4.patch, MAPREDUCE-6690-trunk-v5.patch, > MAPREDUCE-6690-trunk-v6.patch > > > Users will sometimes submit a large amount of resources to be localized as > part of a single map reduce job. This can cause issues with YARN localization > that destabilize the cluster and potentially impact other user jobs. These > resources are specified via the files, libjars, archives and jobjar command > line arguments or directly through the configuration (i.e. distributed cache > api). The resources specified could be too large in multiple dimensions: > # Total size > # Number of files > # Size of an individual resource (i.e. a large fat jar) > We would like to encourage good behavior on the client side by having the > option of enforcing resource limits along the above dimensions. > There should be a separate effort to enforce limits at the YARN layer on the > server side, but this jira is only covering the map reduce layer on the > client side. In practice, having these client side limits will get us a long > way towards preventing these localization anti-patterns. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6747) TestMapReduceJobControl#testJobControlWithKillJob times out in trunk
Chris Trezzo created MAPREDUCE-6747:
---
Summary: TestMapReduceJobControl#testJobControlWithKillJob times
out in trunk
Key: MAPREDUCE-6747
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6747
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Chris Trezzo
Priority: Minor
TestMapReduceJobControl#testJobControlWithKillJob seems to time out while
waiting for all jobs to complete. This seems to only happen if the test is run
with the other tests in the class (specifically testJobControlWithFailJob). If
testJobControlWithKillJob is run by itself, the test passes.
Looking into the test logs, when run with another test from the class, the test
runs into an issue while setting permissions on the local file system:
{noformat}
2016-08-05 11:40:32,101 WARN [Thread-100] util.Shell
(Shell.java:joinThread(1023)) - Interrupted while joining on:
Thread[Thread-105,5,main]
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1245)
at java.lang.Thread.join(Thread.java:1319)
at org.apache.hadoop.util.Shell.joinThread(Shell.java:1020)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:969)
at org.apache.hadoop.util.Shell.run(Shell.java:878)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1266)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1248)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:781)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:566)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:565)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:565)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:538)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:696)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:343)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:541)
{noformat}
Conversely, when the test is run by itself, this issue is not hit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6690) Limit the number of resources a single map reduce job can submit for localization
[ https://issues.apache.org/jira/browse/MAPREDUCE-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409982#comment-15409982 ] Chris Trezzo commented on MAPREDUCE-6690: - TestMapReduceJobControl#testJobControlWithKillJob times out in trunk without this patch. The broken test is unrelated. I will file another jira to fix the test, but this patch should be ready for review. Thanks! > Limit the number of resources a single map reduce job can submit for > localization > - > > Key: MAPREDUCE-6690 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6690 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Chris Trezzo >Assignee: Chris Trezzo > Attachments: MAPREDUCE-6690-trunk-v1.patch, > MAPREDUCE-6690-trunk-v2.patch, MAPREDUCE-6690-trunk-v3.patch, > MAPREDUCE-6690-trunk-v4.patch, MAPREDUCE-6690-trunk-v5.patch, > MAPREDUCE-6690-trunk-v6.patch > > > Users will sometimes submit a large amount of resources to be localized as > part of a single map reduce job. This can cause issues with YARN localization > that destabilize the cluster and potentially impact other user jobs. These > resources are specified via the files, libjars, archives and jobjar command > line arguments or directly through the configuration (i.e. distributed cache > api). The resources specified could be too large in multiple dimensions: > # Total size > # Number of files > # Size of an individual resource (i.e. a large fat jar) > We would like to encourage good behavior on the client side by having the > option of enforcing resource limits along the above dimensions. > There should be a separate effort to enforce limits at the YARN layer on the > server side, but this jira is only covering the map reduce layer on the > client side. In practice, having these client side limits will get us a long > way towards preventing these localization anti-patterns. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add MR support to redact job conf properties from JHS
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409939#comment-15409939 ] Haibo Chen commented on MAPREDUCE-6741: --- Thanks a lot for your review, [~jlowe]! I will upload a new patch later to address your comments. For the replacement value, will do a constant number of asterisks plus ''". > add MR support to redact job conf properties from JHS > - > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: mapreduce6741.001.patch, mapreduce6741.002.patch, > mapreduce6741.003.patch > > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6716) Hadoop shell "mapred job -list" shows improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409445#comment-15409445
]
Hadoop QA commented on MAPREDUCE-6716:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core:
The patch generated 3 new + 36 unchanged - 2 fixed = 39 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 6s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 1s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822287/MAPREDUCE_6716.patch |
| JIRA Issue | MAPREDUCE-6716 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 58b7960212e5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 4a26221 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6660/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6660/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6660/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Hadoop shell "mapred job -list" shows imprope
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" shows improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated MAPREDUCE-6716:
---
Summary: Hadoop shell "mapred job -list" shows improper time format (was:
Hadoop shell "mapred job -list" show improper time format )
> Hadoop shell "mapred job -list" shows improper time format
> ---
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch, MAPREDUCE_6716.patch
>
>
> when client excute shell "mapred job -list",
> {code} JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs default DEFAULT 1
> 0 {code}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated MAPREDUCE-6716:
---
Attachment: MAPREDUCE_6716.patch
> Hadoop shell "mapred job -list" show improper time format
> --
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch, MAPREDUCE_6716.patch
>
>
> when client excute shell "mapred job -list",
> {code} JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs default DEFAULT 1
> 0 {code}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated MAPREDUCE-6716:
---
Description:
when client excute shell "mapred job -list",
{code} JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
root.default DEFAULT 1 0 {code}
the result shows StartTime is a long type，which is not user-friendly.
was:
when client excute shell "mapred job -list",
{code} JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
root.default 0 1 0 {code}
the result shows StartTime is a long type，which is not user-friendly.
> Hadoop shell "mapred job -list" show improper time format
> --
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch
>
>
> when client excute shell "mapred job -list",
> {code} JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs root.default DEFAULT 1
> 0 {code}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated MAPREDUCE-6716:
---
Description:
when client excute shell "mapred job -list",
{code} JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
default DEFAULT 1 0
{code}
the result shows StartTime is a long type，which is not user-friendly.
was:
when client excute shell "mapred job -list",
{code} JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
root.default DEFAULT 1 0 {code}
the result shows StartTime is a long type，which is not user-friendly.
> Hadoop shell "mapred job -list" show improper time format
> --
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch
>
>
> when client excute shell "mapred job -list",
> {code} JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs default DEFAULT 1
> 0 {code}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409420#comment-15409420
]
Shen Yinjie commented on MAPREDUCE-6716:
after fix ,"mapred job -list“ shows as follows :
{code}JobId JobName State
StartTime UserName Queue...
job_1470400756100_0003 TeraGen PREP2016-08-05
21:03 hdfsdefault {code}
> Hadoop shell "mapred job -list" show improper time format
> --
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch
>
>
> when client excute shell "mapred job -list",
> {code} JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs default DEFAULT 1
> 0 {code}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated MAPREDUCE-6716:
---
Description:
when client excute shell "mapred job -list",
{code} JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
root.default 0 1 0 {code}
the result shows StartTime is a long type，which is not user-friendly.
was:
when client excute shell "mapred job -list",
{{ JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
root.default 0 1 0 }}
the result shows StartTime is a long type，which is not user-friendly.
> Hadoop shell "mapred job -list" show improper time format
> --
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch
>
>
> when client excute shell "mapred job -list",
> {code} JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs root.default 0 1
> 0 {code}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[
https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated MAPREDUCE-6716:
---
Description:
when client excute shell "mapred job -list",
{{ JobIdState
StartTime UserNameQueue Priority
UsedContainers RsvdContainers
job_1467197069300_0002RUNNING 1470405744358 hdfs
root.default 0 1 0 }}
the result shows StartTime is a long type，which is not user-friendly.
was:
when client excute shell "mapred job -list",
the result shows " job's startTime" is a long type，which is not user-friendly.
> Hadoop shell "mapred job -list" show improper time format
> --
>
> Key: MAPREDUCE-6716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0, 2.8.0
> Environment: centos6.5 x64
> hadoop 2.6.0 /hadoop2.8.0
>Reporter: Shen Yinjie
>Assignee: Shen Yinjie
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6716_trunk.patch
>
>
> when client excute shell "mapred job -list",
> {{ JobId State
> StartTime UserNameQueue Priority
> UsedContainers RsvdContainers
> job_1467197069300_0002 RUNNING 1470405744358
> hdfs root.default 0 1
> 0 }}
> the result shows StartTime is a long type，which is not user-friendly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[ https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shen Yinjie updated MAPREDUCE-6716: --- Description: when client excute shell "mapred job -list", the result shows " job's startTime" is a long type，which is not user-friendly. was: when client excute shell "mapred job -list", the result shows " job's startTime" is a long type，which is not userfriendly. > Hadoop shell "mapred job -list" show improper time format > -- > > Key: MAPREDUCE-6716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 2.6.0, 2.8.0 > Environment: centos6.5 x64 > hadoop 2.6.0 /hadoop2.8.0 >Reporter: Shen Yinjie >Assignee: Shen Yinjie >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6716_trunk.patch > > > when client excute shell "mapred job -list", > the result shows " job's startTime" is a long type，which is not > user-friendly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Hadoop shell "mapred job -list" show improper time format
[ https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shen Yinjie updated MAPREDUCE-6716: --- Summary: Hadoop shell "mapred job -list" show improper time format (was: Shell command "mapred job -list" show improper time format ) > Hadoop shell "mapred job -list" show improper time format > -- > > Key: MAPREDUCE-6716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 2.6.0, 2.8.0 > Environment: centos6.5 x64 > hadoop 2.6.0 /hadoop2.8.0 >Reporter: Shen Yinjie >Assignee: Shen Yinjie >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6716_trunk.patch > > > when client excute shell "mapred job -list", > the result shows " job's startTime" is a long type，which is not userfriendly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Shell command "mapred job -list" show improper time format
[ https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shen Yinjie updated MAPREDUCE-6716: --- Description: when client excute shell "mapred job -list", the result shows " job's startTime" is a long type，which is not userfriendly. was: when client excute shell "mapred job -list", the result shows " job's startTime" is a long type，which is not user-friendly to end-user. > Shell command "mapred job -list" show improper time format > --- > > Key: MAPREDUCE-6716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 2.6.0, 2.8.0 > Environment: centos6.5 x64 > hadoop 2.6.0 /hadoop2.8.0 >Reporter: Shen Yinjie >Assignee: Shen Yinjie >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6716_trunk.patch > > > when client excute shell "mapred job -list", > the result shows " job's startTime" is a long type，which is not userfriendly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6716) Shell command "mapred job -list" show improper time format
[ https://issues.apache.org/jira/browse/MAPREDUCE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shen Yinjie updated MAPREDUCE-6716: --- Attachment: (was: MAPREDUCE-6716.patch) > Shell command "mapred job -list" show improper time format > --- > > Key: MAPREDUCE-6716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6716 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 2.6.0, 2.8.0 > Environment: centos6.5 x64 > hadoop 2.6.0 /hadoop2.8.0 >Reporter: Shen Yinjie >Assignee: Shen Yinjie >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6716_trunk.patch > > > when client excute shell "mapred job -list", > the result shows " job's startTime" is a long type，which is not > user-friendly to end-user. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
