stayblank created KYLIN-4904:
--------------------------------
Summary: build cubes error
Key: KYLIN-4904
URL: https://issues.apache.org/jira/browse/KYLIN-4904
Project: Kylin
Issue Type: Bug
Components: Query Engine
Affects Versions: v4.0.0-alpha
Environment: ubuntu20
Reporter: stayblank
1.不添加 `spark-yarn_2.11-2.4.8-SNAPSHOT.jar` 到 $kylin_home/lib,报这个错
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.kylin.engine.spark.job.NSparkExecutable.runLocalMode(NSparkExecutable.java:389)
at
org.apache.kylin.engine.spark.job.NSparkExecutable.doWork(NSparkExecutable.java:153)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:94)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error execute
org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:92)
at
org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob.main(ResourceDetectBeforeCubingJob.java:100)
... 13 more
Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'yarn'
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2784)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:493)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:283)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:89)
... 14 more
2.添加`spark-yarn_2.11-2.4.8-SNAPSHOT.jar` 到 $kylin_home/lib后报这个错
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.kylin.engine.spark.job.NSparkExecutable.runLocalMode(NSparkExecutable.java:389)
at
org.apache.kylin.engine.spark.job.NSparkExecutable.doWork(NSparkExecutable.java:153)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:94)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ExceptionInInitializerError
at
org.apache.spark.scheduler.EventLoggingListener$.initEventLog(EventLoggingListener.scala:307)
at
org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:126)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:523)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:283)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:89)
at
org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob.main(ResourceDetectBeforeCubingJob.java:100)
... 13 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Incompatible
Jackson version: 2.6.7-3
at
com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:67)
at
com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
at
com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:718)
at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala)
... 24 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
