[
https://issues.apache.org/jira/browse/MAPREDUCE-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
翟玉勇 updated MAPREDUCE-6863:
---------------------------
Description:
applicationmaster process log is loop on “Waiting for application to be
successfully unregistered.”
ApplicationMaster log
{code}
2017-03-12 01:16:50,854 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1489067586592_112212Job Transitioned from RUNNING to COMMITTING
2017-03-12 01:16:50,854 INFO [CommitterEvent Processor #2]
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the
event EventType: JOB_COMMIT
2017-03-12 01:16:50,884 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for
JobFinishedEvent
2017-03-12 01:16:50,884 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1489067586592_112212Job Transitioned from COMMITTING to SUCCEEDED
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so
this is the last retry
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator
isAMLastRetry: true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator
notified that shouldUnregistered is: true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler:
JobHistoryEventHandler notified that forceJobCompletion is true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the
services
2017-03-12 01:16:50,886 INFO [Thread-402]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
JobHistoryEventHandler. Size of the outstanding queue size is 0
2017-03-12 01:16:50,959 INFO [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling:
PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0
CompletedMaps:260 CompletedReds:0 ContAlloc:261 ContRel:0 HostLocal:115
RackLocal:146
2017-03-12 01:16:51,212 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying
hdfs://bipcluster:8020/data/yarn/stage/master/.staging/job_1489067586592_112212/job_1489067586592_112212_1.jhist
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
2017-03-12 01:16:51,255 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done
location:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
2017-03-12 01:16:51,256 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying
hdfs://bipcluster:8020/data/yarn/stage/master/.staging/job_1489067586592_112212/job_1489067586592_112212_1_conf.xml
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
2017-03-12 01:16:51,270 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done
location:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
2017-03-12 01:16:51,274 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
done:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212.summary_tmp
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212.summary
2017-03-12 01:16:51,276 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
done:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml
2017-03-12 01:16:51,277 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
done:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist
2017-03-12 01:16:51,277 INFO [Thread-402]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped
JobHistoryEventHandler. super.stop()
2017-03-12 01:16:51,278 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING
attempt_1489067586592_112212_m_000217_0
2017-03-12 01:16:51,279 INFO [Thread-402]
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
Opening proxy : sh-hadoop-datanode-128-41.elenet.me:39175
2017-03-12 01:16:51,284 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl:
attempt_1489067586592_112212_m_000217_0 TaskAttempt Transitioned from
SUCCESS_FINISHING_CONTAINER to SUCCEEDED
2017-03-12 01:16:51,294 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job
diagnostics to
2017-03-12 01:16:51,297 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is
http://bigdata-rsm.elenet.me:20020/jobhistory/job/job_1489067586592_112212
2017-03-12 01:16:51,302 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:51,803 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:52,305 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:52,806 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:53,306 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:53,808 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:54,309 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:54,810 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:55,311 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:55,821 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:56,322 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:56,823 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:57,324 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:57,825 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:58,326 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:58,828 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:59,329 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:59,830 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:00,331 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:00,832 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:01,333 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:01,835 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
{code}
was:
applicationmaster process log is loop on “Waiting for application to be
successfully unregistered.”
ApplicationMaster log
2017-03-12 01:16:50,854 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1489067586592_112212Job Transitioned from RUNNING to COMMITTING
2017-03-12 01:16:50,854 INFO [CommitterEvent Processor #2]
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the
event EventType: JOB_COMMIT
2017-03-12 01:16:50,884 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for
JobFinishedEvent
2017-03-12 01:16:50,884 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1489067586592_112212Job Transitioned from COMMITTING to SUCCEEDED
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so
this is the last retry
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator
isAMLastRetry: true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator
notified that shouldUnregistered is: true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler:
JobHistoryEventHandler notified that forceJobCompletion is true
2017-03-12 01:16:50,885 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the
services
2017-03-12 01:16:50,886 INFO [Thread-402]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
JobHistoryEventHandler. Size of the outstanding queue size is 0
2017-03-12 01:16:50,959 INFO [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling:
PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0
CompletedMaps:260 CompletedReds:0 ContAlloc:261 ContRel:0 HostLocal:115
RackLocal:146
2017-03-12 01:16:51,212 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying
hdfs://bipcluster:8020/data/yarn/stage/master/.staging/job_1489067586592_112212/job_1489067586592_112212_1.jhist
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
2017-03-12 01:16:51,255 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done
location:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
2017-03-12 01:16:51,256 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying
hdfs://bipcluster:8020/data/yarn/stage/master/.staging/job_1489067586592_112212/job_1489067586592_112212_1_conf.xml
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
2017-03-12 01:16:51,270 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done
location:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
2017-03-12 01:16:51,274 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
done:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212.summary_tmp
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212.summary
2017-03-12 01:16:51,276 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
done:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml
2017-03-12 01:16:51,277 INFO [eventHandlingThread]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
done:
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
to
hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist
2017-03-12 01:16:51,277 INFO [Thread-402]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped
JobHistoryEventHandler. super.stop()
2017-03-12 01:16:51,278 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING
attempt_1489067586592_112212_m_000217_0
2017-03-12 01:16:51,279 INFO [Thread-402]
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
Opening proxy : sh-hadoop-datanode-128-41.elenet.me:39175
2017-03-12 01:16:51,284 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl:
attempt_1489067586592_112212_m_000217_0 TaskAttempt Transitioned from
SUCCESS_FINISHING_CONTAINER to SUCCEEDED
2017-03-12 01:16:51,294 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job
diagnostics to
2017-03-12 01:16:51,297 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is
http://bigdata-rsm.elenet.me:20020/jobhistory/job/job_1489067586592_112212
2017-03-12 01:16:51,302 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:51,803 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:52,305 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:52,806 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:53,306 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:53,808 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:54,309 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:54,810 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:55,311 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:55,821 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:56,322 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:56,823 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:57,324 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:57,825 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:58,326 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:58,828 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:59,329 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:16:59,830 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:00,331 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:00,832 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:01,333 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
2017-03-12 01:17:01,835 INFO [Thread-402]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
application to be successfully unregistered.
> job finish but yarn list status is accepted and applicationmaster is hang
> on Waiting for application to be successfully unregistered
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6863
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6863
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, resourcemanager
> Affects Versions: 2.6.0
> Reporter: 翟玉勇
> Priority: Minor
> Attachments: jobhistory status.png, yarn resourcemanager job list
> status.png
>
>
> applicationmaster process log is loop on “Waiting for application to be
> successfully unregistered.”
> ApplicationMaster log
> {code}
> 2017-03-12 01:16:50,854 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1489067586592_112212Job Transitioned from RUNNING to COMMITTING
> 2017-03-12 01:16:50,854 INFO [CommitterEvent Processor #2]
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing
> the event EventType: JOB_COMMIT
> 2017-03-12 01:16:50,884 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for
> JobFinishedEvent
> 2017-03-12 01:16:50,884 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1489067586592_112212Job Transitioned from COMMITTING to SUCCEEDED
> 2017-03-12 01:16:50,885 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so
> this is the last retry
> 2017-03-12 01:16:50,885 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator
> isAMLastRetry: true
> 2017-03-12 01:16:50,885 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator
> notified that shouldUnregistered is: true
> 2017-03-12 01:16:50,885 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry:
> true
> 2017-03-12 01:16:50,885 INFO [Thread-402]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler:
> JobHistoryEventHandler notified that forceJobCompletion is true
> 2017-03-12 01:16:50,885 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the
> services
> 2017-03-12 01:16:50,886 INFO [Thread-402]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> 2017-03-12 01:16:50,959 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before
> Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1
> AssignedReds:0 CompletedMaps:260 CompletedReds:0 ContAlloc:261 ContRel:0
> HostLocal:115 RackLocal:146
> 2017-03-12 01:16:51,212 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying
> hdfs://bipcluster:8020/data/yarn/stage/master/.staging/job_1489067586592_112212/job_1489067586592_112212_1.jhist
> to
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
> 2017-03-12 01:16:51,255 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done
> location:
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
> 2017-03-12 01:16:51,256 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying
> hdfs://bipcluster:8020/data/yarn/stage/master/.staging/job_1489067586592_112212/job_1489067586592_112212_1_conf.xml
> to
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
> 2017-03-12 01:16:51,270 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done
> location:
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
> 2017-03-12 01:16:51,274 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
> done:
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212.summary_tmp
> to
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212.summary
> 2017-03-12 01:16:51,276 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
> done:
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml_tmp
> to
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212_conf.xml
> 2017-03-12 01:16:51,277 INFO [eventHandlingThread]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
> done:
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist_tmp
> to
> hdfs://bipcluster:8020/data/yarn/intermediate_done/master/job_1489067586592_112212-1489252563933-master-3934_7823976%3Adw_log_app_page_event_hour_inc.sql%3As4-1489252610863-260-0-SUCCEEDED-root.bigdata.etl.hourlyetl.veryhigh-1489252568534.jhist
> 2017-03-12 01:16:51,277 INFO [Thread-402]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped
> JobHistoryEventHandler. super.stop()
> 2017-03-12 01:16:51,278 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING
> attempt_1489067586592_112212_m_000217_0
> 2017-03-12 01:16:51,279 INFO [Thread-402]
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
> Opening proxy : sh-hadoop-datanode-128-41.elenet.me:39175
> 2017-03-12 01:16:51,284 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl:
> attempt_1489067586592_112212_m_000217_0 TaskAttempt Transitioned from
> SUCCESS_FINISHING_CONTAINER to SUCCEEDED
> 2017-03-12 01:16:51,294 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job
> diagnostics to
> 2017-03-12 01:16:51,297 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is
> http://bigdata-rsm.elenet.me:20020/jobhistory/job/job_1489067586592_112212
> 2017-03-12 01:16:51,302 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:51,803 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:52,305 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:52,806 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:53,306 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:53,808 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:54,309 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:54,810 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:55,311 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:55,821 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:56,322 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:56,823 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:57,324 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:57,825 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:58,326 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:58,828 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:59,329 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:16:59,830 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:17:00,331 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:17:00,832 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:17:01,333 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> 2017-03-12 01:17:01,835 INFO [Thread-402]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for
> application to be successfully unregistered.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
