I am having the a similar problem to the original post. I am trying to run in
Spark 3 and connect to CDH Hive 2.1.1. I have run with the same
spark.sql.hive.metastore options. The main difference in my environment is
that I am trying to run in Spark on K8S using spark-operator, making it a
little
Hi Michael,
Sure will give it a try once more.
Regards
Pradyumn Agrawal
Media.net (India)
On Sun, Jan 10, 2021 at 9:35 PM michael.yang
wrote:
> Hi Pradyumn,
>
> It seems you did not configure spark-default.conf file well.
> Below configurations are needed to use hive 2.1.1 as metastore and
>
Hi Pradyumn,
It seems you did not configure spark-default.conf file well.
Below configurations are needed to use hive 2.1.1 as metastore and execution
engine.
spark.sql.hive.metastore.version=2.1.1
spark.sql.hive.metastore.jars=/opt/cloudera/parcels/CDH/lib/hive/lib/*
Thanks.
Michael Yang
--
Hi DB Tsai,
Thanks for the JIRA link. I think this blocks me to the Hive end instead of
Spark.
Regards
Pradyumn Agrawal
Media.net (India)
On Sun, Jan 10, 2021 at 10:43 AM DB Tsai wrote:
> Hi Pradyumn,
>
> I think it’s because of a HMS client backward compatibility issue
> described here,
Hi Pradyumn,
I think it’s because of a HMS client backward compatibility issue described
here, https://issues.apache.org/jira/browse/HIVE-24608
Thanks,
DB Tsai | ACI Spark Core | Apple, Inc
> On Jan 9, 2021, at 9:53 AM, Pradyumn Agrawal wrote:
>
> Hi Michael,
> Thanks for references,
Hi Pradyumn,
We integrated Spark 3.0.1 with hive 2.1.1-cdh6.1.0 and it works fine to use
spark-sql to query hive tables.
Make sure you config spark-defaults.conf and spark-env.sh well and copy
hive/hadoop related config files to spark conf folder.
You can refer to below refrences for detail.
Hi All,
I am facing an issue while connecting Spark 3.0.1 with Hive 2.1.1 version
(to be precise Hive 2.1.1-cdh6.3.1 - CDH distribution).
Invoking SQL query through *spark-sql* and *spark-shell* dont logs any
error on stderr and stdout and the application just stucks.
Is there any minimum
