It was a configuration in kylin.properties: kylin.engine.spark-conf.spark.eventLog.dir=hdfs\:///kylin/spark-history kylin.engine.spark-conf.spark.history.fs.logDirectory=hdfs\:///kylin/spark-history
2017-06-02 16:47 GMT+08:00 qiumingming <csurj...@gmail.com>: > Hi All, I got a error at Build Cube with Spark step, below is my error > info. > I can not search anything which match "hdfs:/kylin/spark-history" in the > project source code: > ERROR spark.SparkContext: Error initializing SparkContext. > java.io.FileNotFoundException: File does not exist: > hdfs:/kylin/spark-history > at > org.apache.hadoop.hdfs.DistributedFileSystem$18. > doCall(DistributedFileSystem.java:1132) > at > org.apache.hadoop.hdfs.DistributedFileSystem$18. > doCall(DistributedFileSystem.java:1124) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve( > FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus( > DistributedFileSystem.java:1124) > at > org.apache.spark.scheduler.EventLoggingListener.start( > EventLoggingListener.scala:100) > at org.apache.spark.SparkContext.<init>(SparkContext.scala:583) > at > org.apache.spark.api.java.JavaSparkContext.<init>( > JavaSparkContext.scala:59) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute( > SparkCubingByLayer.java:154) > at > org.apache.kylin.common.util.AbstractApplication.execute( > AbstractApplication.java:37) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry. > java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$ > deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at org.apache.spark.deploy.SparkSubmit$.doRunMain$1( > SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit( > SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit. > scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/File-does-not-exist-hdfs-kylin-spark-history-tp8154.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋
