Dear, We used CDH5.5.0 Hue Oozie to run Spark action, and in the spark job action, the job used spark-cassandra-connector_2.10-1.5.0-M2.jar.
The job can run successfully with spark-submit command, but it was failed to run in oozie. It said oozie can not find guava-16.0.1.jar(guava-16.0.1.jar is an dependency of DSE4.8.3 Cassandra) as following exception. Do you know how to add external guava-16.0.1.jar in oozie class path? Thanks! ----- >>> Invoking Spark class now >>> <<< Invocation of Main class completed <<< Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, com.google.common.reflect.TypeToken.isPrimitive()Z java.lang.NoSuchMethodError: com.google.common.reflect.TypeToken.isPrimitive()Z at com.datastax.driver.core.TypeCodec.<init>(TypeCodec.java:142) at com.datastax.driver.core.TypeCodec.<init>(TypeCodec.java:136) at com.datastax.driver.core.TypeCodec$BlobCodec.<init>(TypeCodec.java:609) at com.datastax.driver.core.TypeCodec$BlobCodec.<clinit>(TypeCodec.java:606) at com.datastax.driver.core.CodecRegistry.<clinit>(CodecRegistry.java:147) at com.datastax.driver.core.Configuration$Builder.build(Configuration.java:259) at com.datastax.driver.core.Cluster$Builder.getConfiguration(Cluster.java:1135) at com.datastax.driver.core.Cluster.<init>(Cluster.java:111) at com.datastax.driver.core.Cluster.buildFrom(Cluster.java:178) at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:1152) at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createCluster(CassandraConnectionFactory.scala:85) at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:155) at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:150) at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:150) at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31) at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56) at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:81) at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:109) at com.datastax.spark.connector.cql.CassandraConnector.withClusterDo(CassandraConnector.scala:120) at com.datastax.spark.connector.cql.Schema$.fromCassandra(Schema.scala:241) at com.datastax.spark.connector.writer.TableWriter$.apply(TableWriter.scala:263) at com.datastax.spark.connector.RDDFunctions.saveToCassandra(RDDFunctions.scala:36) at TestCassandra$.main(TestCassandra.scala:44) at TestCassandra.main(TestCassandra.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ----- -- Cheers, ----- Big Data - Big Wisdom - Big Value -------------- Michelle Zhang (张莉苹)
