[
https://issues.apache.org/jira/browse/CASSANALYTICS-55?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Guerrero updated CASSANALYTICS-55:
--------------------------------------------
Reviewers: Francisco Guerrero
Status: Review In Progress (was: Needs Committer)
> Replace NotImplementedException with UnsupportedOperationException
> ------------------------------------------------------------------
>
> Key: CASSANALYTICS-55
> URL: https://issues.apache.org/jira/browse/CASSANALYTICS-55
> Project: Apache Cassandra Analytics
> Issue Type: Bug
> Components: Bridge
> Reporter: Stefan Miklosovic
> Assignee: Stefan Miklosovic
> Priority: Normal
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was trying to submit a spark job with all dependencies etc. but it was
> constantly failing on this:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/apache/commons/lang/NotImplementedException
> at
> org.apache.cassandra.spark.data.converter.SparkSqlTypeConverterImplementation.<clinit>(SparkSqlTypeConverterImplementation.java:109)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at
> org.apache.cassandra.bridge.CassandraBridgeFactory.create(CassandraBridgeFactory.java:126)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
> at
> org.apache.cassandra.bridge.CassandraBridgeFactory.get(CassandraBridgeFactory.java:77)
> at
> org.apache.cassandra.spark.KryoRegister.registerClasses(KryoRegister.java:87)
> at
> org.apache.spark.serializer.KryoSerializer.$anonfun$newKryo$8(KryoSerializer.scala:180)
> at
> org.apache.spark.serializer.KryoSerializer.$anonfun$newKryo$8$adapted(KryoSerializer.scala:180)
> at
> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
> {code}
> The reason is that SparkType contains this:
> {code:java}
> import org.apache.commons.lang.NotImplementedException
> ...
> default int compareTo(Object first, Object second)
> {
> throw new NotImplementedException("compareTo not implemented");
> } {code}
>
> This class is located in commons-lang:commons-lang:2.6 but this jar is not in
> Spark 3.2.2 itself nor it is bundled by anything. I am not sure how / from
> where it is taken but it is clearly not present upon deployment and I am
> super hesitant to patch Spark itself to contain this jar.
>
> Just replace it with UnsupportedOperationException from Java.
>
> I am including these jars to upon spark-submit so far via --jars /
> --driver-library-path
>
> {code:java}
> ./cassandra-analytics-core_spark3_2.12-1.0.0.jar
> ./cassandra-analytics-common_spark3_2.12-1.0.0.jar
> ./cassandra-analytics-spark-converter_spark3_2.12-1.0.0.jar
> ./cassandra-bridge_spark3_2.12-1.0.0.jar
> ./build.jar <- my app
> ./bridges
> ./bridges/four-zero-types.jar
> ./bridges/four-zero-bridge.jar
> ./bridges/four-zero.jar
> ./bridges/four-zero-avro.jar
> ./bridges/four-zero-sparksql.jar
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
