zane.young created SPARK-53447:
----------------------------------
Summary: How to configure multiple Hive Metastores in a single
Spark session?
Key: SPARK-53447
URL: https://issues.apache.org/jira/browse/SPARK-53447
Project: Spark
Issue Type: Question
Components: Connect, Security, SQL
Affects Versions: 3.3.1
Environment: Spark 3.3.1
Reporter: zane.young
In some of our scenarios, we need to perform *cross-cluster queries* by reading
from multiple HDFS clusters within the same Spark session.
To achieve this, we would like to configure *multiple Hive Metastores* in
Spark, so that tables from different clusters can be queried without restarting
the session.
However, when I tried to configure Spark with multiple Hive Metastore URIs, it
did not take effect.
In addition, if multiple HMS are supported, how should *Kerberos
authentication* be configured for each metastore?
*Question:*
* How can we configure multiple Hive Metastores in a single Spark session (for
example, one for Hive tables and another for Hudi tables)? If this is possible,
how can we query data from these different metastores using Spark SQL catalogs？
* Is there a recommended way to handle *Kerberos authentication* for multiple
HMS?
Looking forward to your suggestions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
