Thanks Elliot ! Let me check it out !
On Mon, 17 Apr, 2023, 10:08 pm Elliot West, wrote:
> Hi Ankit,
>
> While not a part of Spark, there is a project called 'WaggleDance' that
> can federate multiple Hive metastores so that they are accessible via a
> single URI:
There is a DSv2-based Hive connector in Apache Kyuubi[1] that supports
connecting multiple HMS in a single Spark application.
Some limitations
- currently only supports Spark 3.3
- has a known issue when using w/ `spark-sql`, but OK w/ spark-shell and
normal jar-based Spark application.
[1]
Hi Ankit,
While not a part of Spark, there is a project called 'WaggleDance' that can
federate multiple Hive metastores so that they are accessible via a single
URI: https://github.com/ExpediaGroup/waggle-dance
This may be useful or perhaps serve as inspiration.
Thanks,
Elliot.
On Mon, 17 Apr
++
User Mailing List
Just a reminder, anyone who can help on this.
Thanks a lot !
Ankit Prakash Gupta
On Wed, Apr 12, 2023 at 8:22 AM Ankit Gupta wrote:
> Hi All
>
> The question is regarding the support of multiple Remote Hive Metastore
> catalogs with Spark. Starting Spark 3, multiple
