no doubt Sent from my iPhone 6
> On Mar 23, 2014, at 17:37, Fengyun RAO <raofeng...@gmail.com> wrote: > > What does this exception mean? I googled a lot, all the results tell me it's > because the time is not synchronized between datanode and namenode. > However, I checked all the servers, that the ntpd service is on, and the time > differences are less than 1 second. > What's more, the tasks are not always failing on certain datanodes. > It fails and then it restarts and succeeds. If it were the time problem, I > guess it would always fail. > > My hadoop version is CDH5 beta. Below is the detailed log: > > 14/03/23 14:57:06 INFO mapreduce.Job: Running job: job_1394434496930_0032 > 14/03/23 14:57:17 INFO mapreduce.Job: Job job_1394434496930_0032 running in > uber mode : false > 14/03/23 14:57:17 INFO mapreduce.Job: map 0% reduce 0% > 14/03/23 15:08:01 INFO mapreduce.Job: Task Id : > attempt_1394434496930_0032_m_000034_0, Status : FAILED > Container launch failed for container_1394434496930_0032_01_000041 : > org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to > start container. > This token is expired. current time is 1395558481146 found 1395558443384 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:152) > at > org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106) > at > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155) > at > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:370) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > > 14/03/23 15:08:02 INFO mapreduce.Job: map 1% reduce 0% > 14/03/23 15:09:36 INFO mapreduce.Job: Task Id : > attempt_1394434496930_0032_m_000036_0, Status : FAILED > Container launch failed for container_1394434496930_0032_01_000038 : > org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to > start container. > This token is expired. current time is 1395558575889 found 1395558443245 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:152) > at > org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106) > at > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155) > at > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:370) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >
