Hi Gao Michael,
Spark 3.2.1 has differences from Spark 3.1.1. If you want to run on Spark 3.2.1 maybe after Kylin adapted it. You can run Spark 3.2.1 and Spark 3.1.1 together, but two sparks will separate the resource of the cluster. And current the package of "kylin-4.0.1-bin-spark3.tar.gz" can only run on Spark 3.1.1. -- Best regards. Tengting Xu 在 2022-05-20 23:24:13，"Gao Michael" <michael.ga...@hotmail.com> 写道： Hi Tengting Xu Thanks for your help, it works! I download spark3.2.1 and modify SPARK_HOME=`path in .bashrc to spark 3.1.1`, now I can import hive tables via kylin web interface. So is there any way to keep spark3.2.1 run on cluster and just keep kylin run on spark3.1.1? Thanks! 发件人: Mukvin <boyboys...@163.com> 发送时间: 2022年5月20日 9:22 收件人: michael.ga...@hotmail.com; user@kylin.apache.org 主题: Re:回复: Re:回复: Re:回复: File does not exist ....... fairscheduler.xml Hi Gao Michael, I mean you can export SPARK_HOME=`path to spark 3.1.1` for KYLIN, do not need to downgrade 3.2.1. You can keep them both. -- Best regards. Tengting Xu At 2022-05-19 22:47:14, "Gao Michael" <michael.ga...@hotmail.com> wrote: Hi Tengting Xu: I already deployed spark3.2.1 on my cluster, you mean I should degrade it to 3.1.1? 发件人: Mukvin <boyboys...@163.com> 发送时间: 2022年5月19日 22:36 收件人:michael.ga...@hotmail.com; user@kylin.apache.org 主题: Re:回复: Re:回复: File does not exist ....... fairscheduler.xml Hi Gao Michael, As I checked the log file, I found two errors in it. "controller.TableController:131 : Failed to load Hive Table java.lang.StackOverflowError" and "File not found exception". They are related to spark. As you have mentioned your env is with spark 3.2.1, but currently, kylin-4.0.1-bin-spark3.tar.gz only supports spark 3.1.1 as Kylin's official site mentioned, so you may need to download spark 3.1.1 and export the path to be `spark home`. Then you can try to restart Kylin again and check. -- Best regards. Tengting Xu At 2022-05-19 22:15:15, "Gao Michael" <michael.ga...@hotmail.com> wrote: Hi Mukin The log file too long and I only can only send pats of it as an attachment. I import table via kylin web interface, and watch log file by command “tail -f”, The log file watch terminal was rolling continuous in the process of import. Attachment cover a complete loading process Thanks 发件人: Mukvin <boyboys...@163.com> 发送时间: 2022年5月19日 21:28 收件人:michael.ga...@hotmail.com; user@kylin.apache.org 主题: Re:回复: File does not exist ....... fairscheduler.xml Hi Gao Michael, Could you give a full log about kylin.log located in $KYLIN_HOME/logs/kylin.log. Let me get the full trace of this error. -- Best regards. Tengting Xu At 2022-05-19 19:41:20, "Gao Michael" <michael.ga...@hotmail.com> wrote: Hi all, Who can Help? 发件人: Gao Michael <michael.ga...@hotmail.com> 发送时间: 2022年5月18日 19:23 收件人:user@kylin.apache.org 主题: File does not exist ....... fairscheduler.xml Hi all I got follow error when I try to add table to my kylin project via web interface: 2022-05-1818:48:37,904INFO [Thread-7] ui.SparkUI:57 : Bound SparkUI to 0.0.0.0, and started at http://hadoop-cluster-001:4040 2022-05-1818:48:38,681ERROR [Thread-7] scheduler.FairSchedulableBuilder:94 : Error while building the fair scheduler pools java.io.FileNotFoundException: File does not exist: /home/michael/opt/modules/kylin/apache-kylin-4.0.1-bin-spark3/conf/fairscheduler.xml and I’m sure file exist! michael@hadoop-cluster-001:~/opt/modules/kylin/apache-kylin-4.0.1-bin-spark3$ ls conf/ fairscheduler.xml kylin_job_conf_inmem.xml kylin.properties setenv.sh spark-executor-log4j.properties kylin_hive_conf.xml kylin_job_conf.xml kylin-server-log4j.properties setenv-tool.sh kylin_job_conf_cube_merge.xml kylin-kafka-consumer.xml kylin-tools-log4j.properties spark-driver-log4j.properties What’s the problem and how to solve? Thanks!
