[jira] [Created] (MAPREDUCE-7199) HsJobsBlock reuse JobACLsManager for checkAccess
Bibin A Chundatt created MAPREDUCE-7199:
---
Summary: HsJobsBlock reuse JobACLsManager for checkAccess
Key: MAPREDUCE-7199
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7199
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Bibin A Chundatt
Reuse JobAclManager.checkAccess
{code}
private boolean checkAccess(String userName) {
if(!areAclsEnabled) {
return true;
}
// User could see its own job.
if (ugi.getShortUserName().equals(userName)) {
return true;
}
// Admin could also see all jobs
if (adminAclList != null && adminAclList.isUserAllowed(ugi)) {
return true;
}
return false;
}
{code}
{code}
jobACLsManager
.checkAccess(ugi, JobACL.VIEW_JOB, ..
new AccessControlList()))
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7195) Mapreduce task timeout to zero could cause too many status update
Bibin A Chundatt created MAPREDUCE-7195:
---
Summary: Mapreduce task timeout to zero could cause too many
status update
Key: MAPREDUCE-7195
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7195
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
* mapreduce.task.timeout=0
Could create too many status update
{code}
public static long getTaskProgressReportInterval(final Configuration conf) {
long taskHeartbeatTimeOut = conf.getLong(
MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
(long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO * taskHeartbeatTimeOut));
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7151) RMContainerAllocator#handleJobPriorityChange expects application_priority always
Bibin A Chundatt created MAPREDUCE-7151:
---
Summary: RMContainerAllocator#handleJobPriorityChange expects
application_priority always
Key: MAPREDUCE-7151
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7151
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
As per yarn_service.proto {{AllocateResponseProto}} *application_priority* is
*optional* field.
But {{RMContainerAllocator#handleJobPriorityChange}} expects to have
application priority always in response which is not mandatory.
{code}
private void handleJobPriorityChange(AllocateResponse response) {
Priority priorityFromResponse = Priority.newInstance(response
.getApplicationPriority().getPriority());
// Update the job priority to Job directly.
getJob().setJobPriority(priorityFromResponse);
}
{code}
{code}
message AllocateResponseProto {
..
optional hadoop.common.TokenProto am_rm_token = 12;
optional PriorityProto application_priority = 13;
optional CollectorInfoProto collector_info = 14;
..
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
Bibin A Chundatt created MAPREDUCE-7073:
---
Summary: Optimize TokenCache#obtainTokensForNamenodesInternal
Key: MAPREDUCE-7073
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
FileInputFormat#listStatus is too slow file system cache is disabled.
{{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
{{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
For fileInput with 1k file will reload YarnConfiguration 1k times.
{{Master.getMasterPrincipal(conf)}} can be passed for
{{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt resolved MAPREDUCE-6987.
-
Resolution: Fixed
Fix Version/s: 3.1.0
3.0.0
2.9.0
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.9.0, 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.9.0, 3.0.0, 3.1.0
>
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
Bibin A Chundatt created MAPREDUCE-6987: --- Summary: JHS Log Scanner and Cleaner blocked Key: MAPREDUCE-6987 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Bibin A Chundatt Priority: Critical -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6782) JHS task page search based on each individual column not working
Bibin A Chundatt created MAPREDUCE-6782:
---
Summary: JHS task page search based on each individual column not
working
Key: MAPREDUCE-6782
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6782
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Submit mapreduce pi job with 10 maps
In Jobs history server selection completed job
Select maps to Task Page for job
Search in individual column fields
*Expected*
Search should be working fine in task page for individual columns
*Actual*
Search not working for individual column in task page
In Attempts page the same search is working fine
{noformat}
jquery.dataTables.min.js:109
Uncaught TypeError: Cannot read property 'oFeatures' of null
fnFilter @ jquery.dataTables.min.js:109(anonymous function) @ m:49dispatch
@ jquery-1.8.2.min.js:2h @ jquery-1.8.2.min.js:2
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6779) Mapreduce job failure on submission
Bibin A Chundatt created MAPREDUCE-6779:
---
Summary: Mapreduce job failure on submission
Key: MAPREDUCE-6779
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
Configure Hibench
Try running Enhanced TestDFSIO
{noformat}
2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
attempt_1473943118844_0001_m_01_0, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
... 17 more
{noformat}
*mapreduce.JobResourceUploader: No job jar file set. User classes may not be
found. See Job or Job#setJar(String).*
Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6705) Task failing continuously on trunk
Bibin A Chundatt created MAPREDUCE-6705:
---
Summary: Task failing continuously on trunk
Key: MAPREDUCE-6705
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
Task attempt failing continuously. Submit any mapreduce application
Run the job as below
{code}
./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
-Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
-Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
{code}
{noformat}
2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
client out of cache: org.apache.hadoop.ipc.Client@291ae
2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
22305
2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
Sleeping for 0ms before retrying again. Got null now.
2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy10.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
Caused by: com.google.protobuf.ServiceException: Too many or few parameters for
request. Method: [getTask], Expected: 2, Actual: 1
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
... 2 more
2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
client from cache: org.apache.hadoop.ipc.Client@291ae
2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
client from cache: org.apache.hadoop.ipc.Client@291ae
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce
Bibin A Chundatt created MAPREDUCE-6700:
---
Summary: Jobhistory server attempt and task table not loading
maps/reduce
Key: MAPREDUCE-6700
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
Browser
===
Chrome
Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page
Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.
Column 0 is of *natural sorting* and not working. So waiting for ever to be
sorted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6466) NNBench map task should create files in separate folder
[
https://issues.apache.org/jira/browse/MAPREDUCE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt resolved MAPREDUCE-6466.
-
Resolution: Duplicate
MAPREDUCE- handling the same.
> NNBench map task should create files in separate folder
> ---
>
> Key: MAPREDUCE-6466
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6466
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>
> In case of NNBench with current implementation all files created using
> different maps are created in same directory.
> In this JIRA will provide a flag to create files in same folder or separate
> folder for each task.
> {{dfs.namenode.fs-limits.max-directory-items}} cannot be set to more than
> 6,400,000 as i understand
> Advantage
> # For scenarios like creating fsimage with large size NNBench can be used
> # Creation of multiple files of same size using NNBench folder based.
> No need to run NNBench multiple time to create FSImage of large size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6693) Job history entry missing when JOB name is of 50 character
Bibin A Chundatt created MAPREDUCE-6693:
---
Summary: Job history entry missing when JOB name is of 50 character
Key: MAPREDUCE-6693
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Critical
Job entry history missing when JOB name is of 50 character
{noformat}
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting
Event Handling thread
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for
Event Handling thread to complete
2016-05-10 06:51:00,674 ERROR [eventHandlingThread]
org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
Thread[eventHandlingThread,5,main] threw an Exception.
java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer for Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,676 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
{noformat}
Looks like 50 character check is going wrong
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6606) Findbug issue in org.apache.hadoop.mapred.OutputCommitter
Bibin A Chundatt created MAPREDUCE-6606:
---
Summary: Findbug issue in org.apache.hadoop.mapred.OutputCommitter
Key: MAPREDUCE-6606
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6606
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
{noformat}
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
doesn't override method in superclass because parameter type
org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type
org.apache.hadoop.mapreduce.JobContext
Bug type NM_WRONG_PACKAGE_INTENTIONAL (click for details)
In class org.apache.hadoop.mapred.OutputCommitter
In method
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
superclass is org.apache.hadoop.mapreduce.OutputCommitter
Did you intend to override
org.apache.hadoop.mapreduce.OutputCommitter.isCommitJobRepeatable(JobContext)
Actual type org.apache.hadoop.mapred.JobContext
Expected org.apache.hadoop.mapreduce.JobContext
Overrides
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
At OutputCommitter.java:[line 230]
{noformat}
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6268/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html#Warnings_BAD_PRACTICE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6602) ApplicationResourceUsageReport should return ALL reserved resource
Bibin A Chundatt created MAPREDUCE-6602: --- Summary: ApplicationResourceUsageReport should return ALL reserved resource Key: MAPREDUCE-6602 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6602 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Bibin A Chundatt Assignee: Bibin A Chundatt ApplicationResourceUsageReport reserved resource report is only of default parition should be of all partitions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6591) TestJobHistoryEventHandler#testTimelineEventHandling failing in trunk
Bibin A Chundatt created MAPREDUCE-6591:
---
Summary: TestJobHistoryEventHandler#testTimelineEventHandling
failing in trunk
Key: MAPREDUCE-6591
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6591
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Reporter: Bibin A Chundatt
Assignee: Bibin A Chundatt
org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.testTimelineEventHandling
failing in trunk
{noformat}
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:331)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processEventForTimelineServer(JobHistoryEventHandler.java:1015)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:586)
at
org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.handleEvent(TestJobHistoryEventHandler.java:722)
at
org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.testTimelineEventHandling(TestJobHistoryEventHandler.java:510)
{noformat}
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6240/testReport/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6581) Shuffle failure incase of NativeMapOutputCollectorDelegator with intermediate-data encrypt
Bibin A Chundatt created MAPREDUCE-6581:
---
Summary: Shuffle failure incase of
NativeMapOutputCollectorDelegator with intermediate-data encrypt
Key: MAPREDUCE-6581
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6581
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
*Steps to reproduce*
# Create data with teragen
# Run terasort on data prepared using teragen
Commands used
./yarn jar
../share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar teragen
1024000 /Terainput1
./yarn jar
../share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar terasort
-Dmapreduce.job.encrypted-intermediate-data=true
-Dmapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator
-Dmapreduce.map.output.compress=true
-Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec
-Dmapreduce.output.fileoutputformat.compress=true
-Dmapreduce.output.fileoutputformat.compress.type=BLOCK
-Dmapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec
-Dmapreduce.reduce.memory.mb=1024 /Terainput1/Teraout12
{noformat}
15/12/18 23:07:57 INFO mapreduce.Job: Task Id :
attempt_1450453391718_0017_r_00_2, Status : FAILED
Error: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in
shuffle in fetcher#5
at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.lang.ArrayIndexOutOfBoundsException
at
org.apache.hadoop.io.compress.snappy.SnappyDecompressor.setInput(SnappyDecompressor.java:107)
at
org.apache.hadoop.io.compress.BlockDecompressorStream.decompress(BlockDecompressorStream.java:104)
at
org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)
at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:201)
at
org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput.doShuffle(InMemoryMapOutput.java:90)
at
org.apache.hadoop.mapreduce.task.reduce.IFileWrappedMapOutput.shuffle(IFileWrappedMapOutput.java:63)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:538)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:336)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:193)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
Bibin A Chundatt created MAPREDUCE-6554:
---
Summary: MRAppMaster servicestart failing with NPE in
MRAppMaster#parsePreviousJobHistory
Key: MAPREDUCE-6554
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Assignee: Bibin A Chundatt
Priority: Critical
Create scenario so that MR app master gets preempted.
On next MRAppMaster launch tried to recover previous job history file
{{MRAppMaster#parsePreviousJobHistory}}
{noformat}
2015-11-21 13:52:27,722 INFO [main] org.apache.hadoop.service.AbstractService:
Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
cause: java.lang.NullPointerException
java.lang.NullPointerException
at java.io.StringReader.(StringReader.java:50)
at org.apache.avro.Schema$Parser.parse(Schema.java:917)
at org.apache.avro.Schema.parse(Schema.java:966)
at
org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
2015-11-21 13:52:27,725 INFO [main]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
JobHistoryEventHandler. Size of the outstanding queue size is 0
{noformat}
EventReader(EventReader stream)
{noformat}
this.version = in.readLine();
...
this.schema = parser.parse(in.readLine());
{noformat}
check for null and return {{IOException}} in {{parser.parse(in.readLine())}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6534) Sorting based on applicaiton attempt not working in JHS
Bibin A Chundatt created MAPREDUCE-6534: --- Summary: Sorting based on applicaiton attempt not working in JHS Key: MAPREDUCE-6534 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6534 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Bibin A Chundatt Steps to reproduce === 1.Submit application with 1100 maps 2.Check successful maps in JHS 3.Try sorting based on attempt /jobhistory/attempts/job_1446629175560_0005/m/SUCCESSFUL -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6492) AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask
Bibin A Chundatt created MAPREDUCE-6492:
---
Summary: AsyncDispatcher exit with NPE on
TaskAttemptImpl#sendJHStartEventForAssignedFailTask
Key: MAPREDUCE-6492
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6492
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Assignee: Bibin A Chundatt
For {{TaskAttemptImpl#DeallocateContainerTransition}}
{{sendJHStartEventForAssignedFailTask}} is send for
TaskAttemptStateInternal.UNASSIGNED also .
Causing NPE on {{taskAttempt.container.getNodeHttpAddress()}}
{noformat}
2015-09-28 18:01:48,656 FATAL [AsyncDispatcher event handler]
org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.sendJHStartEventForAssignedFailTask(TaskAttemptImpl.java:1494)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.access$2900(TaskAttemptImpl.java:147)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$DeallocateContainerTransition.transition(TaskAttemptImpl.java:1700)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$DeallocateContainerTransition.transition(TaskAttemptImpl.java:1686)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:290)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1190)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:146)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1415)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1407)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:109)
at java.lang.Thread.run(Thread.java:745)
2015-09-28 18:01:48,660 INFO [AsyncDispatcher ShutDown handler]
org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..
2015-09-28 18:01:48,660 INFO [ContainerLauncher #6]
org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing
the event EventType: CONTAINER_REMOTE_CLEANUP for container
container_e04_1443430524957_0006_01_59 taskAttempt
attempt_1443430524957_0006_m_00_9
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6476) InvalidResourceException when Nodelabel don't have access to queue should be handled
Bibin A Chundatt created MAPREDUCE-6476:
---
Summary: InvalidResourceException when Nodelabel don't have access
to queue should be handled
Key: MAPREDUCE-6476
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6476
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: applicationmaster
Reporter: Bibin A Chundatt
Assignee: Bibin A Chundatt
Steps to reproduce
===
Submit mapreduce job
# map to label x
# reduce to label y
Precondition
# Queue b to which reduce is submitted not having access to label specified
*Impact*
# Jobs fail only of the RM-AM comunication timeout
(About 10 mins i think)
Should kill the job immediately when InvalidResourceException is received on
{{RMContainerRequestor#makeRemoteRequest}}
*Logs*
{noformat}
2015-09-11 16:44:30,116 ERROR [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid
resource request, queue=b1 doesn't have permission to access all labels in
resource request. labelExpression of resource request=1. Queue labels=3
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:304)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:234)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndvalidateRequest(SchedulerUtils.java:250)
at
org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.normalizeAndValidateRequests(RMServerUtils.java:106)
at
org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:457)
at
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
at
org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:636)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:976)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2230)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2224)
at sun.reflect.GeneratedConstructorAccessor39.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at
org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
at
org.apache.hadoop.yarn.ipc.RPCUtil.instantiateYarnException(RPCUtil.java:75)
at
org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:116)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.allocate(ApplicationMasterProtocolPBClientImpl.java:79)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:251)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at com.sun.proxy.$Proxy37.allocate(Unknown Source)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor.makeRemoteRequest(RMContainerRequestor.java:203)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:694)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:263)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:281)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException):
Invalid resource request, queue=b1 doesn't have permission to access all
labels in resource request. labelExpression of resource request=1. Queue
labels=3
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:304)
at
org.apache.hadoop.yarn.server.resourcemanager.
[jira] [Created] (MAPREDUCE-6466) NNBench map task should create files in separate folder
Bibin A Chundatt created MAPREDUCE-6466:
---
Summary: NNBench map task should create files in separate folder
Key: MAPREDUCE-6466
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6466
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Bibin A Chundatt
Assignee: Bibin A Chundatt
In case of NNBench with current implementation all files created using
different maps are created in same directory.
In this JIRA will provide a flag to create files in same folder or separate
folder for each task.
{{dfs.namenode.fs-limits.max-directory-items}} cannot be set to more than
6,400,000 as i understand
Advantage
# For scenarios like creating fsimage with large size NNBench can be used
# Creation of multiple files of same size using NNBench folder based.
No need to run NNBench multiple time to create FSImage of large size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6452) LocalRunner with intermediate encrypt failing with NPE
Bibin A Chundatt created MAPREDUCE-6452:
---
Summary: LocalRunner with intermediate encrypt failing with NPE
Key: MAPREDUCE-6452
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6452
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Minor
Enable the below properties try running mapreduce job
mapreduce.framework.name=local
mapreduce.job.encrypted-intermediate-data=true
{code}
2015-08-14 16:27:25,248 WARN [Thread-21] mapred.LocalJobRunner
(LocalJobRunner.java:run(561)) - job_local473843898_0001
java.lang.Exception: java.lang.NullPointerException
at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:463)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:523)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.crypto.CryptoOutputStream.(CryptoOutputStream.java:92)
at
org.apache.hadoop.fs.crypto.CryptoFSDataOutputStream.(CryptoFSDataOutputStream.java:31)
at
org.apache.hadoop.mapreduce.CryptoUtils.wrapIfNecessary(CryptoUtils.java:112)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1611)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1492)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:723)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:244)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6317) Invalid Resource Exception could be handled properly when cores not available
[
https://issues.apache.org/jira/browse/MAPREDUCE-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt resolved MAPREDUCE-6317.
-
Resolution: Duplicate
Fixed as part of https://issues.apache.org/jira/browse/MAPREDUCE-5279
> Invalid Resource Exception could be handled properly when cores not available
> -
>
> Key: MAPREDUCE-6317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mr-am
> Environment: 1 RM , 1 DN
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Minor
>
> Configure yarn.nodemanager.resource.cpu-vcores=2 for NM
> Set mapreduce.map.cpu.vcores=5 while running sleep job n client
> {code}
> 2015-04-10 20:37:26,111 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid
> resource request, requested virtual cores < 0, or requested virtual cores >
> max configured, requestedVirtualCores=5, maxVirtualCores=2
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:213)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.validateResourceRequests(RMServerUtils.java:97)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:502)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:636)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:976)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2142)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2138)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2136)
> at sun.reflect.GeneratedConstructorAccessor17.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:101)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.allocate(ApplicationMasterProtocolPBClientImpl.java:79)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at com.sun.proxy.$Proxy34.allocate(Unknown Source)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor.makeRemoteRequest(RMContainerRequestor.java:199)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:686)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:257)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$1.run(RMCommunicator.java:281)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: 2015-04-10 20:37:27,117 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Could not contact
> RM after 36 milliseconds.
> 2015-04-10 20:37:27,173 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Error communicating
> with RM: Could not contact RM after 36 milliseconds.
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Could not contact RM
> after 36 milliseconds.
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:712)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:257)
> at
[jira] [Created] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
Bibin A Chundatt created MAPREDUCE-6377: --- Summary: JHS sorting on state column not working in webUi Key: MAPREDUCE-6377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.7.0 Environment: 2 NM, JHS Reporter: Bibin A Chundatt Priority: Minor Steps to reproduce 1. Install and setup HA cluster with JHS 2.Create state in in JHS where few jobs are killed and Success Check sorting State in JHS WebUI Actual = Sorting on state column not working in JHS Expected == Sorting on state column should be working -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6371) Diagnostics shown wrong in Jobhistory page
Bibin A Chundatt created MAPREDUCE-6371:
---
Summary: Diagnostics shown wrong in Jobhistory page
Key: MAPREDUCE-6371
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6371
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver
Affects Versions: 2.7.0
Reporter: Bibin A Chundatt
Priority: Minor
Diagnostics shown wrong in Jobhistory page
Incase of failed task
{code}
Diagnostics : Task failed task_1432651802642_0086_r_00
Job failed as tasks failed. failedMaps:0 failedReduces:1
{code}
Please check the image for detail
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
