Peter Mead created SPARK-20252: ---------------------------------- Summary: java.lang.ClassNotFoundException: $line22.$read$$iwC$$iwC$movie_row Key: SPARK-20252 URL: https://issues.apache.org/jira/browse/SPARK-20252 Project: Spark Issue Type: Bug Components: Spark Shell Affects Versions: 1.6.3 Environment: Datastax DSE dual node SPARK cluster [cqlsh 5.0.1 | Cassandra 3.0.12.1586 | DSE 5.0.7 | CQL spec 3.4.0 | Native protocol v4] Reporter: Peter Mead
After starting a spark shell using DSE -u xxxxxxxx -p xxxxxxxxx spark scala> case class movie_row (actor: String, character_name: String, video_id: java.util.UUID, video_year: Int, title: String) defined class movie_row scala> val vids=sc.cassandraTable("hcl","videos_by_actor") vids: com.datastax.spark.connector.rdd.CassandraTableScanRDD[com.datastax.spark.connector.CassandraRow] = CassandraTableScanRDD[0] at RDD at CassandraRDD.scala:15 scala> val vids=sc.cassandraTable[movie_row]("hcl","videos_by_actor") vids: com.datastax.spark.connector.rdd.CassandraTableScanRDD[movie_row] = CassandraTableScanRDD[1] at RDD at CassandraRDD.scala:15 scala> vids.count res0: Long = 114961 Works OK!! BUT if the spark context is stopped and recreated THEN: scala> sc.stop() scala> import org.apache.spark.SparkContext, org.apache.spark.SparkContext._, org.apache.spark.SparkConf import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import org.apache.spark.SparkConf scala> :paste // Entering paste mode (ctrl-D to finish) val conf = new SparkConf(true) .set("spark.cassandra.connection.host", "redacted") .set("spark.cassandra.auth.username", "redacted") .set("spark.cassandra.auth.password", "redacted") // Exiting paste mode, now interpreting. conf: org.apache.spark.SparkConf = org.apache.spark.SparkConf@e207342 scala> val sc = new SparkContext("spark://192.168.1.84:7077", "pjm", conf) sc: org.apache.spark.SparkContext = org.apache.spark.SparkContext@12b8b7c8 scala> case class movie_row (actor: String, character_name: String, video_id: java.util.UUID, video_year: Int, title: String) defined class movie_row scala> val vids=sc.cassandraTable[movie_row]("hcl","videos_by_actor") vids: com.datastax.spark.connector.rdd.CassandraTableScanRDD[movie_row] = CassandraTableScanRDD[0] at RDD at CassandraRDD.scala:15 scala> vids.count [Stage 0:> (0 + 2) / 2]WARN 2017-04-07 12:52:03,277 org.apache.spark.scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, cassandra183): java.lang.ClassNotFoundException: $line22.$read$$iwC$$iwC$movie_row FAILS!! I have been unable to get this to work from a remote SPARK shell! -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org