GitHub user Mr-xie-188 created a discussion: k8s环境下,会话用户USER级别共享情况下,多集群spark查询应该如何配置
配置如下: 配置会话级别USER:kyuubi.engine.share.level=USER 配置使用多个集群:kyuubi.session.conf.advisor=org.apache.kyuubi.session.FileSessionConfAdvisor 配置2个集群hive连接相关配置:kyuubi-session-xx1.conf(kyuubi.engineEnv.SPARK_CONF_DIR=/opt/spark/xx1-spark-conf)、kyuubi-session-xx2.conf(kyuubi.engineEnv.SPARK_CONF_DIR=/opt/spark/xx2-spark-conf) 问题: 相同用户访问不同集群,先连接xx1集群,起了一个pod,但是新建连接xx2集群,访问的是同一个pod,还是连接到了xx1集群,并且查询有问题 连接方式:bin/beeline -u "jdbc:kyuubi://127.0.0.1:10009/default" --conf kyuubi.session.conf.profile=xx1-n user 求助: 怎么可以kyuubi.engine.share.level=USER情况下,同一个用户,可以连接到不同集群 现象: 尝试分离kyuubi.engineEnv.SPARK_CONF_DIR,但是helm启动的,默认看第一次连接启动pod,spark_conf默认是kyuubi挂载的 /opt/spark/conf from spark-conf-volume-driver (rw) 包含第一次启动的hadoop hive相关conf,指定的kyuubi.engineEnv.SPARK_CONF_DIR无效 GitHub link: https://github.com/apache/kyuubi/discussions/7315 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
