Spark 1.2.1: How to convert SchemaRDD to CassandraRDD?

2015-04-27 Thread Tash Chainar
Hi all, following the import com.datastax.spark.connector.SelectableColumnRef; import com.datastax.spark.connector.japi.CassandraJavaUtil; import org.apache.spark.sql.SchemaRDD; import static com.datastax.spark.connector.util.JavaApiHelper.toScalaSeq; import scala.collection.Seq; SchemaRDD

When the old data dropped from the cache?

2015-04-20 Thread Tash Chainar
Hi all, On https://spark.apache.org/docs/latest/programming-guide.html under the RDD Persistence Removing Data, it states Spark automatically monitors cache usage on each node and drops out old data partitions in a least-recently-used (LRU) fashion. Can it be understood that the cache will

ClassCastException while caching a query

2015-04-17 Thread Tash Chainar
Hi all, Spark 1.2.1. I have a Cassandra column family and doing the following SchemaRDD s = cassandraSQLContext.sql(select user.id as user_id from user); // user.id is UUID in table definition s.registerTempTable( my_user ); s.cache(); // throws following exception // tried the