[jira] [Commented] (MAPREDUCE-6380) AggregatedLogDeletionService will throw exception when there are some other directories in remote-log-dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585711#comment-14585711 ] Mohammad Shahid Khan commented on MAPREDUCE-6380: - Correct the listStatus(Path f) will throw FileNotFoundException, instead of propagating the failing the same catch the FileNotFoundException and should log that the directory does not exits. AggregatedLogDeletionService will throw exception when there are some other directories in remote-log-dir - Key: MAPREDUCE-6380 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6380 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Reporter: Zhang Wei Priority: Trivial AggregatedLogDeletionService will throw FileNotFoundException when there are some extraneous directories put in remote-log-dir. The deletion function will try to listStatus against the extraneous-dir + suffix dir. I think it would be better if the function can ignore these directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3384) Add warning message for underflow or overflow
[
https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585835#comment-14585835
]
Brahma Reddy Battula commented on MAPREDUCE-3384:
-
[~devaraj.k] thanks for review..
{quote}Can we also have key as part of the warning message?{quote}
Ok.we can have.
{quote}
I am thinking, if we have more number of values where overflow/underflow can
happen, we would be filling with these logs. Can we limit having a some flag in
the reduce method to log only once for overflow and underflow for each key? any
thoughts?{quote}
I am thinking like following,, please let me know your opinion...
{code}
public void reduce(Key key, IterableIntWritable values,
Context context) throws IOException, InterruptedException {
int sum = 0;
boolean isOverflow=false;
boolean isUnderflow=false;
for (IntWritable val : values) {
int tempVal = val.get();
int oldSum = sum;
sum += tempVal;
if ((!isOverflow oldSum 0 tempVal 0) (oldSum sum)) {
LOG.warn(Arithmetic overflow occurs while reducing+tempVal);
isOverflow=True;
}
if ((!isUnderflow oldSum 0 tempVal 0) (oldSum sum)) {
LOG.warn(Arithmetic underflow occurs while reducing+tempVal);
isUnderflow=True;
}
}
result.set(sum);
context.write(key, result);
}
{code}
Add warning message for underflow or overflow
-
Key: MAPREDUCE-3384
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: JiangKai
Assignee: Brahma Reddy Battula
Attachments: MAPREDUCE-3384-002.patch, MAPREDUCE-3384-003.patch,
MAPREDUCE-3384.patch
When we call the function reduce() of LongSumReducer,the result may overflow
or underflow.
We should have a warning message to users if overflow/underflow occurs for
all these classes,
{code:xml}
/java/org/apache/hadoop/mapred/lib/LongSumReducer.java
.../apache/hadoop/mapreduce/lib/reduce/IntSumReducer.java
.../apache/hadoop/mapreduce/lib/reduce/LongSumReducer.java
.../test/java/org/apache/hadoop/fs/AccumulatingReducer.java
.../hadoop/mapred/nativetask/kvtest/HashSumReducer.java
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6399) UI Improvement - Counters Page
Kevin J. Price created MAPREDUCE-6399: - Summary: UI Improvement - Counters Page Key: MAPREDUCE-6399 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6399 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Kevin J. Price Priority: Minor Current, the 'counters' page on the web UI will show totals for each counter, and clicking on a counter gets individual values from each task. For the purposes of job performance tuning, it would be very helpful to be able to quickly see statistic aggregations on the counter. Specifically, I'm thinking a view where, for each counter, we get: MAX, MIN, MEAN, MEDIAN, MODE, STDEV (or some subset of those) The max and min values could include links to corresponding task(s) counters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587289#comment-14587289 ] Brahma Reddy Battula commented on MAPREDUCE-6363: - Thanks a lot [~ajisakaa] for review and commiting!!! And thanks to others for looking into this issue.. [NNBench] Lease mismatch error when running with multiple mappers - Key: MAPREDUCE-6363 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 Project: Hadoop Map/Reduce Issue Type: Bug Components: benchmarks Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Priority: Critical Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch Command : ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10 Trace : 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[
https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6316:
-
Status: Open (was: Patch Available)
A problem occurs when you access
http://rmhost:rmport/proxy/application_1434432414806_0003/mapreduce/attempts/job_1434432414806_0003/m/RUNNING
{code}
2015-06-15 22:46:56,369 ERROR [378354947@qtp-596129348-3]
org.apache.hadoop.mapreduce.v2.app.webapp.AppController: Failed to render
attempts page with task type : m for job id : job_1434432414806_0003
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.app.webapp.TaskPage$AttemptsBlock.render(TaskPage.java:74)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.attempts(AppController.java:296)
{code}
You should test all links at least manually. Ideally please try to add some
unit tests.
Task Attempt List entries should link to the task overview
--
Key: MAPREDUCE-6316
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Siqi Li
Assignee: Siqi Li
Labels: BB2015-05-TBR
Attachments: AM attempt page.png, AM task page.png, All Attempts
page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch,
MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch,
MAPREDUCE-6316.v6.patch, Task Overview page.png
Typical workflow is to click on the list of failed attempts. Then you want to
look at the counters, or the list of attempts of just one task in general. If
each entry task attempt id linked the task id portion of it back to the task,
we would not have to go through the list of tasks to search for the task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula reassigned MAPREDUCE-6400:
---
Assignee: Brahma Reddy Battula
TestReduceFetchFromPartialMem fails
---
Key: MAPREDUCE-6400
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.8.0
Reporter: Akira AJISAKA
Assignee: Brahma Reddy Battula
TestReduceFetchFromPartialMem fails.
{noformat}
Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
Time elapsed: 69.214 sec ERROR!
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
Results :
Tests in error:
TestReduceFetchFromPartialMem.testReduceFromPartialMem:93-runJob:300 » IO
Job...
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated MAPREDUCE-6363: Labels: (was: BB2015-05-TBR) [NNBench] Lease mismatch error when running with multiple mappers - Key: MAPREDUCE-6363 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 Project: Hadoop Map/Reduce Issue Type: Bug Components: benchmarks Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Priority: Critical Fix For: 2.8.0 Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch Command : ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10 Trace : 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated MAPREDUCE-6400:
--
Component/s: test
TestReduceFetchFromPartialMem fails
---
Key: MAPREDUCE-6400
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.8.0
Reporter: Akira AJISAKA
TestReduceFetchFromPartialMem fails.
{noformat}
Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
Time elapsed: 69.214 sec ERROR!
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
Results :
Tests in error:
TestReduceFetchFromPartialMem.testReduceFromPartialMem:93-runJob:300 » IO
Job...
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6399) UI Improvement - Counters Statistics Page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin J. Price updated MAPREDUCE-6399: -- Summary: UI Improvement - Counters Statistics Page (was: UI Improvement - Counters Page) UI Improvement - Counters Statistics Page - Key: MAPREDUCE-6399 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6399 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Kevin J. Price Priority: Minor Current, the 'counters' page on the web UI will show totals for each counter, and clicking on a counter gets individual values from each task. For the purposes of job performance tuning, it would be very helpful to be able to quickly see statistic aggregations on the counter. Specifically, I'm thinking a view where, for each counter, we get: MAX, MIN, MEAN, MEDIAN, MODE, STDEV (or some subset of those) The max and min values could include links to corresponding task(s) counters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6363: - Summary: [NNBench] Lease mismatch error when running with multiple mappers (was: [NNBench mark] Lease mismatch error when running with multiple mappers) [NNBench] Lease mismatch error when running with multiple mappers - Key: MAPREDUCE-6363 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 Project: Hadoop Map/Reduce Issue Type: Bug Components: benchmarks Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Priority: Critical Labels: BB2015-05-TBR Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch Command : ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10 Trace : 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587004#comment-14587004 ] Hudson commented on MAPREDUCE-6363: --- FAILURE: Integrated in Hadoop-trunk-Commit #8018 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8018/]) MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. (aajisaka: rev 75a2560e51387ea31ef4609ef434475bbbc628f7) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java * hadoop-mapreduce-project/CHANGES.txt [NNBench] Lease mismatch error when running with multiple mappers - Key: MAPREDUCE-6363 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 Project: Hadoop Map/Reduce Issue Type: Bug Components: benchmarks Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Priority: Critical Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch Command : ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10 Trace : 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated MAPREDUCE-6316: --- Status: Patch Available (was: Open) Task Attempt List entries should link to the task overview -- Key: MAPREDUCE-6316 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Siqi Li Assignee: Siqi Li Labels: BB2015-05-TBR Attachments: AM attempt page.png, AM task page.png, All Attempts page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, MAPREDUCE-6316.v6.patch, Task Overview page.png Typical workflow is to click on the list of failed attempts. Then you want to look at the counters, or the list of attempts of just one task in general. If each entry task attempt id linked the task id portion of it back to the task, we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench mark] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586981#comment-14586981 ] Akira AJISAKA commented on MAPREDUCE-6363: -- Committing this. [NNBench mark] Lease mismatch error when running with multiple mappers -- Key: MAPREDUCE-6363 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 Project: Hadoop Map/Reduce Issue Type: Bug Components: benchmarks Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Priority: Critical Labels: BB2015-05-TBR Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch Command : ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10 Trace : 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6363: - Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks [~brahmareddy] for the contribution and [~haridsv] for the comment. [NNBench] Lease mismatch error when running with multiple mappers - Key: MAPREDUCE-6363 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 Project: Hadoop Map/Reduce Issue Type: Bug Components: benchmarks Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Priority: Critical Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch Command : ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10 Trace : 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587125#comment-14587125
]
Hudson commented on MAPREDUCE-6396:
---
FAILURE: Integrated in Hadoop-trunk-Commit #8019 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8019/])
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed
by Brahma Reddy Battula. (aajisaka: rev
85cc644f92e0e55293993c43e381f161bca5ce32)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
* hadoop-mapreduce-project/CHANGES.txt
TestPipeApplication fails by NullPointerException
-
Key: MAPREDUCE-6396
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Akira AJISAKA
Assignee: Brahma Reddy Battula
Fix For: 3.0.0
Attachments: MAPREDUCE-6396.patch
TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
on trunk.
{noformat}
Running org.apache.hadoop.mapred.pipes.TestPipeApplication
Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec
FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
elapsed: 1.157 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
0.035 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585535#comment-14585535
]
zhihai xu commented on MAPREDUCE-6396:
--
thanks for working on this issue, [~brahmareddy].
the patch looks good to me, +1 (non-binding)
TestPipeApplication fails by NullPointerException
-
Key: MAPREDUCE-6396
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Akira AJISAKA
Assignee: Brahma Reddy Battula
Attachments: MAPREDUCE-6396.patch
TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
on trunk.
{noformat}
Running org.apache.hadoop.mapred.pipes.TestPipeApplication
Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec
FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
elapsed: 1.157 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
0.035 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated MAPREDUCE-6396:
-
Resolution: Fixed
Fix Version/s: 3.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed this to trunk. Thanks all who contributed to this issue.
TestPipeApplication fails by NullPointerException
-
Key: MAPREDUCE-6396
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Akira AJISAKA
Assignee: Brahma Reddy Battula
Fix For: 3.0.0
Attachments: MAPREDUCE-6396.patch
TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
on trunk.
{noformat}
Running org.apache.hadoop.mapred.pipes.TestPipeApplication
Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec
FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
elapsed: 1.157 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
0.035 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6396) TestPipeApplication fails by NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587037#comment-14587037
]
Akira AJISAKA commented on MAPREDUCE-6396:
--
Thanks [~brahmareddy] for creating the patch and thanks [~zxu] for the comment.
+1, committing this.
TestPipeApplication fails by NullPointerException
-
Key: MAPREDUCE-6396
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6396
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Akira AJISAKA
Assignee: Brahma Reddy Battula
Attachments: MAPREDUCE-6396.patch
TestPipeApplication#testApplication and TestPipeApplication#testRunner fail
on trunk.
{noformat}
Running org.apache.hadoop.mapred.pipes.TestPipeApplication
Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec
FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time
elapsed: 1.157 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed:
0.035 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
at
org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
Akira AJISAKA created MAPREDUCE-6400:
Summary: TestReduceFetchFromPartialMem fails
Key: MAPREDUCE-6400
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.8.0
Reporter: Akira AJISAKA
TestReduceFetchFromPartialMem fails.
{noformat}
Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
Time elapsed: 69.214 sec ERROR!
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
Results :
Tests in error:
TestReduceFetchFromPartialMem.testReduceFromPartialMem:93-runJob:300 » IO
Job...
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587218#comment-14587218
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
I used {{git bisect}} and found this issue was broken by MAPREDUCE-6174.
TestReduceFetchFromPartialMem fails
---
Key: MAPREDUCE-6400
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.8.0
Reporter: Akira AJISAKA
TestReduceFetchFromPartialMem fails.
{noformat}
Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
Time elapsed: 69.214 sec ERROR!
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
at
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
Results :
Tests in error:
TestReduceFetchFromPartialMem.testReduceFromPartialMem:93-runJob:300 » IO
Job...
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
