dinesh created SPARK-32988:
------------------------------
Summary: Spark 2.3 and 2.4 backward compatibility problems
Key: SPARK-32988
URL: https://issues.apache.org/jira/browse/SPARK-32988
Project: Spark
Issue Type: Bug
Components: Java API
Affects Versions: 2.4.0, 2.3.2
Reporter: dinesh
With Spark 2.3
spark.sharedState.externalCatalog used to give an object of ExternalCatalog
which was actually an instance of HiveExternalCatalog
With Spark 2.4
spark.sharedState.externalCatalog gives an object of
ExternalCatalogWithListener which has HiveExternalCatalog as unwrapped delegate.
We use HiveExternalCatalog to get HiveClient to perform Hive Operations.
This change in method Signature is giving us backward compatibility problems
such that code compiled with Spark 2.3 can not work with 2.4 due to the change.
This results in Runtime exceptions including ClassCastException,
NoSuchMethodError
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
