Ha, Hun Cheol created MAPREDUCE-6736: ----------------------------------------
Summary: import hive table from parquet files, there is no 'job.splitmetainfo' file message Key: MAPREDUCE-6736 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6736 Project: Hadoop Map/Reduce Issue Type: Bug Components: applicationmaster, mrv2 Affects Versions: 2.6.0 Environment: Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) Hadoop 2.6.0-cdh5.7.0 Subversion http://github.com/cloudera/hadoop -r c00978c67b0d3fe9f3b896b5030741bd40bf541a Compiled by jenkins on 2016-03-23T18:36Z Compiled with protoc 2.5.0 >From source with checksum b2eabfa328e763c88cb14168f9b372 This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.7.0.jar Reporter: Ha, Hun Cheol Priority: Blocker same issue : https://issues.apache.org/jira/browse/MAPREDUCE-3056 which is created on 2011.09.21 and fixed 2011.10.04 there is user(Sergey) who has same issue on 2015.05.13 too! (look last comment of above link) on beeline prompt, try import hive table from parquet files that are exported from another hive table, there is no 'job.splitmetainfo' file on staging directory : FileNotFoundException occurs full log messages below ================== Log Type: syslog Log Upload Time: Wed Jul 20 17:57:36 +0900 2016 Log Length: 21439 2016-07-20 17:57:26,139 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1468834620182_0036_000001 2016-07-20 17:57:26,417 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2016-07-20 17:57:26,463 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: 2016-07-20 17:57:26,463 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (org.apache.hadoop.yarn.security.AMRMTokenIdentifier@52af26ee) 2016-07-20 17:57:26,510 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter. 2016-07-20 17:57:26,991 WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 2016-07-20 17:57:27,091 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null 2016-07-20 17:57:27,154 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2016-07-20 17:57:27,159 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.tools.mapred.CopyCommitter 2016-07-20 17:57:27,231 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler 2016-07-20 17:57:27,232 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher 2016-07-20 17:57:27,233 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher 2016-07-20 17:57:27,234 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher 2016-07-20 17:57:27,235 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler 2016-07-20 17:57:27,241 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher 2016-07-20 17:57:27,242 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter 2016-07-20 17:57:27,243 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter 2016-07-20 17:57:27,292 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://da74:8020] 2016-07-20 17:57:27,322 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://da74:8020] 2016-07-20 17:57:27,352 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://da74:8020] 2016-07-20 17:57:27,382 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Perms after creating 448, Expected: 448 2016-07-20 17:57:27,387 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled 2016-07-20 17:57:27,428 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler 2016-07-20 17:57:27,645 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2016-07-20 17:57:27,703 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2016-07-20 17:57:27,704 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started 2016-07-20 17:57:27,712 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1468834620182_0036 to jobTokenSecretManager 2016-07-20 17:57:27,723 WARN [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Job init failed org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.FileNotFoundException: File does not exist: hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1579) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1443) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1401) at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:997) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:139) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1333) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1101) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1544) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1540) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1473) Caused by: java.io.FileNotFoundException: File does not exist: hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1219) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211) at org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:51) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1574) ... 17 more 2016-07-20 17:57:27,726 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1468834620182_0036. 2016-07-20 17:57:27,753 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2016-07-20 17:57:27,761 INFO [Socket Reader #1 for port 35855] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 35855 2016-07-20 17:57:27,778 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server 2016-07-20 17:57:27,779 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2016-07-20 17:57:27,779 INFO [IPC Server listener on 35855] org.apache.hadoop.ipc.Server: IPC Server listener on 35855: starting 2016-07-20 17:57:27,780 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at da74/115.68.67.98:35855 2016-07-20 17:57:27,837 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-07-20 17:57:27,842 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2016-07-20 17:57:27,846 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined 2016-07-20 17:57:27,855 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2016-07-20 17:57:27,860 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce 2016-07-20 17:57:27,860 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static 2016-07-20 17:57:27,862 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/* 2016-07-20 17:57:27,862 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/* 2016-07-20 17:57:27,870 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 42206 2016-07-20 17:57:27,870 INFO [main] org.mortbay.log: jetty-6.1.26.cloudera.4 2016-07-20 17:57:27,899 INFO [main] org.mortbay.log: Extract jar:file:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.6.0-cdh5.7.0.jar!/webapps/mapreduce to /tmp/Jetty_0_0_0_0_42206_mapreduce____p5x30f/webapp 2016-07-20 17:57:28,159 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:42206 2016-07-20 17:57:28,160 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app /mapreduce started at 42206 2016-07-20 17:57:28,493 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules 2016-07-20 17:57:28,497 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE job_1468834620182_0036 2016-07-20 17:57:28,499 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2016-07-20 17:57:28,499 INFO [Socket Reader #1 for port 41561] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 41561 2016-07-20 17:57:28,506 INFO [IPC Server listener on 41561] org.apache.hadoop.ipc.Server: IPC Server listener on 41561: starting 2016-07-20 17:57:28,502 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2016-07-20 17:57:28,540 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true 2016-07-20 17:57:28,540 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3 2016-07-20 17:57:28,540 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33 2016-07-20 17:57:28,605 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at da74/115.68.67.98:8030 2016-07-20 17:57:28,687 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: <memory:8192, vCores:4> 2016-07-20 17:57:28,687 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.hdfs 2016-07-20 17:57:28,692 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500 2016-07-20 17:57:28,692 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10 2016-07-20 17:57:28,694 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 2016-07-20 17:57:28,699 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1468834620182_0036Job Transitioned from NEW to FAIL_ABORT 2016-07-20 17:57:28,700 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_ABORT 2016-07-20 17:57:28,727 INFO [CommitterEvent Processor #0] org.apache.hadoop.tools.mapred.CopyCommitter: Cleaning up temporary work folder: /user/hive/staging/hdfs/.staging/_distcp70301392 2016-07-20 17:57:28,758 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1468834620182_0036Job Transitioned from FAIL_ABORT to FAILED 2016-07-20 17:57:28,759 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry 2016-07-20 17:57:28,759 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true 2016-07-20 17:57:28,759 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that shouldUnregistered is: true 2016-07-20 17:57:28,760 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true 2016-07-20 17:57:28,760 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true 2016-07-20 17:57:28,760 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services 2016-07-20 17:57:28,760 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 3 2016-07-20 17:57:28,850 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1468834620182_0036, File: hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job_1468834620182_0036_1.jhist 2016-07-20 17:57:29,188 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event JOB_SUBMITTED 2016-07-20 17:57:29,223 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://da74:8020] 2016-07-20 17:57:29,253 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event JOB_QUEUE_CHANGED 2016-07-20 17:57:29,253 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event JOB_FAILED 2016-07-20 17:57:29,376 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job_1468834620182_0036_1.jhist to hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist_tmp 2016-07-20 17:57:29,513 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist_tmp 2016-07-20 17:57:29,520 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job_1468834620182_0036_1_conf.xml to hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml_tmp 2016-07-20 17:57:29,561 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml_tmp 2016-07-20 17:57:29,580 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036.summary_tmp to hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036.summary 2016-07-20 17:57:29,586 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml_tmp to hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml 2016-07-20 17:57:29,594 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist_tmp to hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist 2016-07-20 17:57:29,594 INFO [Thread-55] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop() 2016-07-20 17:57:29,596 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job diagnostics to Job init failed : org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.FileNotFoundException: File does not exist: hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1579) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1443) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1401) at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:997) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:139) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1333) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1101) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1544) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1540) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1473) Caused by: java.io.FileNotFoundException: File does not exist: hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1219) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211) at org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:51) at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1574) ... 17 more 2016-07-20 17:57:29,597 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is http://da74:19888/jobhistory/job/job_1468834620182_0036 2016-07-20 17:57:29,604 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for application to be successfully unregistered. 2016-07-20 17:57:30,605 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0 2016-07-20 17:57:30,607 INFO [Thread-55] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://da74:8020/ /user/hive/staging/hdfs/.staging/job_1468834620182_0036 2016-07-20 17:57:30,628 INFO [Thread-55] org.apache.hadoop.ipc.Server: Stopping server on 41561 2016-07-20 17:57:30,629 INFO [IPC Server listener on 41561] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 41561 2016-07-20 17:57:30,629 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder 2016-07-20 17:57:30,629 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted 2016-07-20 17:57:30,629 INFO [Ping Checker] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: TaskAttemptFinishingMonitor thread interrupted -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org