If you deploy yarn model，you can used yarn logs -applicationId youApplicationId get yarn logs. You can get logs in details。 Then Looking error info, =============================== Name: cen sujun Mobile: 13067874572 Mail: ce...@lotuseed.com
> 在 2017年2月6日，05:33，Evgenii Morozov <evgeny.a.moro...@gmail.com> 写道： > > Hi, > > I see a lot of exceptions like the following during our machine learning > pipeline calculation. Spark version 2.0.2. > Sometimes it’s just few executors that fails with this message, but the job > is successful. > > I’d appreciate any hint you might have. > Thank you. > > 2017-02-05 07:56:47.022 [task-result-getter-1] WARN > o.a.spark.scheduler.TaskSetManager - Lost task 0.0 in stage 151558.0 (TID > 993070, 10.61.12.43): > java.io.FileNotFoundException: File file:/path/to/file does not exist > at > org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:611) > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:824) > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:601) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:142) > at > org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:346) > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:769) > at > org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:109) > at > org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67) > at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:245) > at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208) > at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:283) > at > org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:283) > at > org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:283) > at > org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:283) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47) > at org.apache.spark.scheduler.Task.run(Task.scala:86) > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >
