pan3793 commented on code in PR #4825:
URL: https://github.com/apache/zeppelin/pull/4825#discussion_r1758093920
##########
spark/spark-shims/src/main/java/org/apache/zeppelin/spark/SparkShims.java:
##########
@@ -58,7 +58,7 @@ public SparkShims(Properties properties) {
private static SparkShims loadShims(int sparkMajorVersion, Properties
properties, Object entryPoint)
throws Exception {
Class<?> sparkShimsClass;
- if (sparkMajorVersion == 3) {
+ if (sparkMajorVersion == 3 || sparkMajorVersion == 4) {
LOGGER.info("Initializing shims for Spark 3.x");
sparkShimsClass = Class.forName("org.apache.zeppelin.spark.Spark3Shims");
Review Comment:
Yes, I think we should remove the shim layer to simplify the codebase given
the Spark API is pretty stable in recent versions, the corner compatibility
cases could be resolved by using reflection directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
