[jira] [Commented] (MAPREDUCE-6401) Container-launch failure gives no debugging output
[ https://issues.apache.org/jira/browse/MAPREDUCE-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589373#comment-14589373 ] Devaraj K commented on MAPREDUCE-6401: -- Have you checked the container/task logs? You could probably find the reason for this failure in the task logs. > Container-launch failure gives no debugging output > -- > > Key: MAPREDUCE-6401 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.6.0 > Environment: HDP 2.2 >Reporter: Hari Sekhon > Attachments: job.log > > > MR jobs are failing on my cluster with Stack trace: ExitCodeException > exitCode=7 but little else in terms of debugging information. Can we please > improve the debugging info? Log file is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6403) Fix typo in the usage of NNBench
[
https://issues.apache.org/jira/browse/MAPREDUCE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589314#comment-14589314
]
Akira AJISAKA commented on MAPREDUCE-6403:
--
It's nice if the unused import is removed.
{code}
import org.apache.hadoop.mapreduce.v2.api.records.TaskId;
{code}
I didn't noticed that when reviewing MAPREDUCE-6363.
> Fix typo in the usage of NNBench
>
>
> Key: MAPREDUCE-6403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Reporter: Akira AJISAKA
>Priority: Trivial
> Labels: newbie
>
> Fix typo 'becnhmarks'.
> {code:title=NNBench.java}
> "\t-baseDir"This is not mandatory>\n" +
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6403) Fix typo in the usage of NNBench
[
https://issues.apache.org/jira/browse/MAPREDUCE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6403:
Assignee: Jagadesh Kiran N
> Fix typo in the usage of NNBench
>
>
> Key: MAPREDUCE-6403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
>Priority: Trivial
> Labels: newbie
>
> Fix typo 'becnhmarks'.
> {code:title=NNBench.java}
> "\t-baseDir"This is not mandatory>\n" +
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6403) Fix typo in the usage of NNBench
Akira AJISAKA created MAPREDUCE-6403:
Summary: Fix typo in the usage of NNBench
Key: MAPREDUCE-6403
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6403
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: documentation
Reporter: Akira AJISAKA
Priority: Trivial
Fix typo 'becnhmarks'.
{code:title=NNBench.java}
"\t-baseDir \n" +
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6373) The logger reports total input paths but it is referring to input files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589306#comment-14589306
]
Bibin A Chundatt commented on MAPREDUCE-6373:
-
[~devaraj.k] Thank you for checking .Will update and upload soon
> The logger reports total input paths but it is referring to input files
> ---
>
> Key: MAPREDUCE-6373
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6373
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Andi Chirita Amdocs
>Assignee: Bibin A Chundatt
>Priority: Trivial
> Labels: newbie
> Attachments: MAPREDUCE-6373.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The log message in the FileInputFormat is misleading :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 6
> {code}
> There is only 1 input path and 6 input files so the log message should be :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to
> process : 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6373) The logger reports total input paths but it is referring to input files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589274#comment-14589274
]
Devaraj K commented on MAPREDUCE-6373:
--
Thanks [~bibinchundatt] for the patch.
Can you also fix the same in
org.apache.hadoop.mapreduce.lib.input.FileInputFormat class as well?
> The logger reports total input paths but it is referring to input files
> ---
>
> Key: MAPREDUCE-6373
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6373
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Andi Chirita Amdocs
>Assignee: Bibin A Chundatt
>Priority: Trivial
> Labels: newbie
> Attachments: MAPREDUCE-6373.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The log message in the FileInputFormat is misleading :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 6
> {code}
> There is only 1 input path and 6 input files so the log message should be :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to
> process : 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6373) The logger reports total input paths but it is referring to input files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-6373:
-
Status: Open (was: Patch Available)
> The logger reports total input paths but it is referring to input files
> ---
>
> Key: MAPREDUCE-6373
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6373
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Andi Chirita Amdocs
>Assignee: Bibin A Chundatt
>Priority: Trivial
> Labels: newbie
> Attachments: MAPREDUCE-6373.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The log message in the FileInputFormat is misleading :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 6
> {code}
> There is only 1 input path and 6 input files so the log message should be :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to
> process : 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3384) Add warning message for underflow or overflow
[
https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589268#comment-14589268
]
Devaraj K commented on MAPREDUCE-3384:
--
This approach seems to be fine for me, please take care of the new variable
names as per conventions and spacing issues in the log message in the updated
patch.
> Add warning message for underflow or overflow
> -
>
> Key: MAPREDUCE-3384
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: JiangKai
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-3384-002.patch, MAPREDUCE-3384-003.patch,
> MAPREDUCE-3384.patch
>
>
> When we call the function reduce() of LongSumReducer,the result may overflow
> or underflow.
> We should have a warning message to users if overflow/underflow occurs for
> all these classes,
> {code:xml}
> /java/org/apache/hadoop/mapred/lib/LongSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/IntSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/LongSumReducer.java
> .../test/java/org/apache/hadoop/fs/AccumulatingReducer.java
> .../hadoop/mapred/nativetask/kvtest/HashSumReducer.java
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6402) Application Master listen on all IPs
Peter Shi created MAPREDUCE-6402: Summary: Application Master listen on all IPs Key: MAPREDUCE-6402 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6402 Project: Hadoop Map/Reduce Issue Type: Bug Components: applicationmaster Affects Versions: 2.7.0 Reporter: Peter Shi The application master listening on all IPs, which may cause security problems. Similar issue: MAPREDUCE-5938 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588557#comment-14588557
]
Gera Shegalov commented on MAPREDUCE-6400:
--
Thanks for pinging [~ajisakaa]. Taking a look!
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 ยป IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588550#comment-14588550 ] Akira AJISAKA commented on MAPREDUCE-6174: -- Hi [~eepayne] and [~jira.shegalov], TestReduceFetchFromPartialMem fails after this commit. Would you check MAPREDUCE-6400? > Combine common stream code into parent class for InMemoryMapOutput and > OnDiskMapOutput. > --- > > Key: MAPREDUCE-6174 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 3.0.0, 2.6.0 >Reporter: Eric Payne >Assignee: Eric Payne > Labels: BB2015-05-RFC > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6174.002.patch, MAPREDUCE-6174.003.patch, > MAPREDUCE-6174.004.patch, MAPREDUCE-6174.005.patch, MAPREDUCE-6174.006.patch, > MAPREDUCE-6174.007.patch, MAPREDUCE-6174.v1.txt > > > Per MAPREDUCE-6166, both InMemoryMapOutput and OnDiskMapOutput will be doing > similar things with regards to IFile streams. > In order to make it explicit that InMemoryMapOutput and OnDiskMapOutput are > different from 3rd-party implementations, this JIRA will make them subclass a > common class (see > https://issues.apache.org/jira/browse/MAPREDUCE-6166?focusedCommentId=14223368&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14223368) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6401) Container-launch failure gives no debugging output
[ https://issues.apache.org/jira/browse/MAPREDUCE-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated MAPREDUCE-6401: --- Attachment: job.log > Container-launch failure gives no debugging output > -- > > Key: MAPREDUCE-6401 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.6.0 > Environment: HDP 2.2 >Reporter: Hari Sekhon > Attachments: job.log > > > MR jobs are failing on my cluster with Stack trace: ExitCodeException > exitCode=7 but little else in terms of debugging information. Can we please > improve the debugging info? Log file is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6401) Container-launch failure gives no debugging output
Hari Sekhon created MAPREDUCE-6401: -- Summary: Container-launch failure gives no debugging output Key: MAPREDUCE-6401 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.6.0 Environment: HDP 2.2 Reporter: Hari Sekhon Attachments: job.log MR jobs are failing on my cluster with Stack trace: ExitCodeException exitCode=7 but little else in terms of debugging information. Can we please improve the debugging info? Log file is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588237#comment-14588237
]
Hadoop QA commented on MAPREDUCE-6395:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 6s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 38s | The applied patch generated 4
new checkstyle issues (total was 75, now 77). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 5s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 13s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 46m 57s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739874/MAPREDUCE-6395-002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b039e69 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5805/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5805/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5805/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5805/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5805/console |
This message was automatically generated.
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395-002.patch, MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588223#comment-14588223
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2176 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2176/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
* hadoop-mapreduce-project/CHANGES.txt
> TestPipeApplication fails by NullPointerException
> -
>
> Key: MAPREDUCE-6396
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6396.patch
>
>
> TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
> on trunk.
> {noformat}
> Running org.apache.hadoop.mapred.pipes.TestPipeApplication
> Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<<
> FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
> testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
> elapsed: 1.157 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
> testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
> 0.035 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588226#comment-14588226 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2176 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2176/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588183#comment-14588183
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #228 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/228/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
> TestPipeApplication fails by NullPointerException
> -
>
> Key: MAPREDUCE-6396
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6396.patch
>
>
> TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
> on trunk.
> {noformat}
> Running org.apache.hadoop.mapred.pipes.TestPipeApplication
> Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<<
> FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
> testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
> elapsed: 1.157 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
> testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
> 0.035 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588186#comment-14588186 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #228 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/228/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java * hadoop-mapreduce-project/CHANGES.txt > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588138#comment-14588138
]
Brahma Reddy Battula commented on MAPREDUCE-6395:
-
Attaching patch to address checkstyle comments.
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395-002.patch, MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6395:
Attachment: MAPREDUCE-6395-002.patch
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395-002.patch, MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588129#comment-14588129 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #219 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/219/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java * hadoop-mapreduce-project/CHANGES.txt > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588126#comment-14588126
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #219 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/219/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
* hadoop-mapreduce-project/CHANGES.txt
> TestPipeApplication fails by NullPointerException
> -
>
> Key: MAPREDUCE-6396
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6396.patch
>
>
> TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
> on trunk.
> {noformat}
> Running org.apache.hadoop.mapred.pipes.TestPipeApplication
> Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<<
> FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
> testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
> elapsed: 1.157 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
> testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
> 0.035 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588090#comment-14588090
]
Hadoop QA commented on MAPREDUCE-6395:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 9s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 46s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 59s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 36s | The applied patch generated 4
new checkstyle issues (total was 75, now 77). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 7s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 33s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 47m 42s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739856/MAPREDUCE-6395.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b039e69 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5804/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5804/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5804/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5804/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5804/console |
This message was automatically generated.
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588006#comment-14588006
]
Brahma Reddy Battula commented on MAPREDUCE-6395:
-
[~jira.shegalov] Thanks a lot for your inputs.. Attached the patch based on
your comments..Kindly review..
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6395:
Attachment: MAPREDUCE-6395.patch
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6395:
Status: Patch Available (was: Open)
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
> Attachments: MAPREDUCE-6395.patch
>
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587977#comment-14587977 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #2158 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2158/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java * hadoop-mapreduce-project/CHANGES.txt > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587974#comment-14587974
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-Hdfs-trunk #2158 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2158/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
* hadoop-mapreduce-project/CHANGES.txt
> TestPipeApplication fails by NullPointerException
> -
>
> Key: MAPREDUCE-6396
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6396.patch
>
>
> TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
> on trunk.
> {noformat}
> Running org.apache.hadoop.mapred.pipes.TestPipeApplication
> Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<<
> FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
> testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
> elapsed: 1.157 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
> testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
> 0.035 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587860#comment-14587860 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #230 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/230/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587857#comment-14587857
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #230 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/230/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
* hadoop-mapreduce-project/CHANGES.txt
> TestPipeApplication fails by NullPointerException
> -
>
> Key: MAPREDUCE-6396
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6396.patch
>
>
> TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
> on trunk.
> {noformat}
> Running org.apache.hadoop.mapred.pipes.TestPipeApplication
> Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<<
> FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
> testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
> elapsed: 1.157 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
> testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
> 0.035 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587832#comment-14587832
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-Yarn-trunk #960 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/960/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
> TestPipeApplication fails by NullPointerException
> -
>
> Key: MAPREDUCE-6396
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6396.patch
>
>
> TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
> on trunk.
> {noformat}
> Running org.apache.hadoop.mapred.pipes.TestPipeApplication
> Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<<
> FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
> testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
> elapsed: 1.157 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
> testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
> 0.035 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
> at
> org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587835#comment-14587835 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-Yarn-trunk #960 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/960/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6395) Improve the commit failure messages in MRAppMaster recovery
[
https://issues.apache.org/jira/browse/MAPREDUCE-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587610#comment-14587610
]
Gera Shegalov commented on MAPREDUCE-6395:
--
I would like to suggest following messages:
# Job commit succeeded in a prior MRAppMaster attempt before it crashed.
Recovering.
# Job commit failed in a prior MRAppMaster attempt before it crashed. Not
retrying.
# Job commit from a prior MRAppMaster attempt is potentially in progress.
Preventing multiple commit executions.
# Job finished cleanly, recording last MRAppMaster retry
> Improve the commit failure messages in MRAppMaster recovery
> ---
>
> Key: MAPREDUCE-6395
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6395
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
>Affects Versions: 2.7.0
>Reporter: Gera Shegalov
>Assignee: Brahma Reddy Battula
> Labels: noob
>
> There are typos, and pluralis majestatis (royal we) or user-including "we"
> messages that are confusing to the users:
> {code}
> if (commitSuccess) {
> shutDownMessage = "We crashed after successfully committing.
> Recovering.";
> forcedState = JobStateInternal.SUCCEEDED;
> } else if (commitFailure) {
> shutDownMessage = "We crashed after a commit failure.";
> forcedState = JobStateInternal.FAILED;
> } else {
> //The commit is still pending, commit error
> shutDownMessage = "We crashed durring a commit";
> forcedState = JobStateInternal.ERROR;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
