Thanks for responding Jonathan I was inclined to think that it is a jackson issue. I can see in the distribution build of Zeppelin that the libs folder has references to jackson 2.5.3. How do I confirm what the downloaded spark-1.5.1-hadoop2.6 uses? Also I have 0 issues on spark shell so how do I fix zeppelin?
Thank you On Fri, Oct 16, 2015 at 8:58 PM, Jonathan Kelly <jonathaka...@gmail.com> wrote: > That exception looks like one that I see when using too new of a version > of Jackson. See https://issues.apache.org/jira/browse/SPARK-8332. The > short answer is that you probably need to make sure you are using Jackson > 2.4.x rather than 2.5.x. > > ~ Jonathan > > On Fri, Oct 16, 2015 at 7:37 AM, Armand Naude <armandna...@gmail.com> > wrote: > >> I have setup a standalone single node "cluster" running the following: >> >> - Cassandra 2.2.2 >> - Spark 1.5.1 >> - List item >> - Compiled fat jar for Spark-Cassandra-Connector 1.5.0-M2 >> - Compiled Zeppelin 0.6 snapshot compiled with: mvn -Pspark-1.5 >> -Dspark.version=1.5.1 -Dhadoop.version=2.6.0 -Phadoop-2.4 -DskipTests >> clean >> package >> >> I can work perfectly fine with spark shell retrieving data from cassandra >> >> I have altered the Zeppelin-env.sh as follow: >> >> export MASTER=spark://localhost:7077 >> export SPARK_HOME=/root/spark-1.5.1-bin-hadoop2.6/ >> export ZEPPELIN_PORT=8880 >> export >> ZEPPELIN_JAVA_OPTS="-Dspark.jars=/opt/sparkconnector/spark-cassandra-connector-assembly-1.5.0-M2-SNAPSHOT.jar >> -Dspark.cassandra.connection.host=localhost" >> export ZEPPELIN_NOTEBOOK_DIR="/root/gowalla-spark-demo/notebooks/zeppelin" >> export SPARK_SUBMIT_OPTIONS="--jars >> /opt/sparkconnector/spark-cassandra-connector-assembly-1.5.0-M2-SNAPSHOT.jar >> --deploy-mode cluster" >> export ZEPPELIN_INTP_JAVA_OPTS=$ZEPPELIN_JAVA_OPTS >> >> I then start adding paragraphs to a notebook and import the following >> first: >> >> import com.datastax.spark.connector._ >> import com.datastax.spark.connector.cql._ >> import com.datastax.spark.connector.rdd.CassandraRDD >> import org.apache.spark.rdd.RDD >> import org.apache.spark.SparkContext >> import org.apache.spark.SparkConf >> >> Not sure if all of these are necessary. This paragraph runs fine. >> >> Then I do the following: >> >> val checkins = sc.cassandraTable("lbsn", "checkins") >> >> This runs fine and returns: >> >> checkins: >> com.datastax.spark.connector.rdd.CassandraTableScanRDD[com.datastax.spark.connector.CassandraRow] >> = CassandraTableScanRDD[0] at RDD at CassandraRDD.scala:15 >> >> Then the next paragraph - the follow 2 statements are run -the first >> succeeds and the second fails: >> >> checkins.count >> checkins.first >> >> Result: >> >> res13: Long = 138449 >> com.fasterxml.jackson.databind.JsonMappingException: Could not find creator >> property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope) >> at [Source: {"id":"4","name":"first"}; line: 1, column: 1] >> at >> com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148) >> at >> com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:843) >> at >> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:533) >> at >> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:220) >> at >> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143) >> at >> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409) >> at >> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:358) >> at >> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:265) >> at >> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:245) >> at >> com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:143) >> at >> com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:439) >> at >> com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666) >> at >> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558) >> at >> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2578) >> at >> org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:82) >> at org.apache.spark.rdd.RDD$$anonfun$34.apply(RDD.scala:1582) >> at org.apache.spark.rdd.RDD$$anonfun$34.apply(RDD.scala:1582) >> at scala.Option.map(Option.scala:145) >> at org.apache.spark.rdd.RDD.<init>(RDD.scala:1582) >> at >> com.datastax.spark.connector.rdd.CassandraRDD.<init>(CassandraRDD.scala:15) >> at >> com.datastax.spark.connector.rdd.CassandraTableScanRDD.<init>(CassandraTableScanRDD.scala:59) >> at >> com.datastax.spark.connector.rdd.CassandraTableScanRDD.copy(CassandraTableScanRDD.scala:92) >> at >> com.datastax.spark.connector.rdd.CassandraTableScanRDD.copy(CassandraTableScanRDD.scala:59) >> at >> com.datastax.spark.connector.rdd.CassandraRDD.limit(CassandraRDD.scala:103) >> at com.datastax.spark.connector.rdd.CassandraRDD.take(CassandraRDD.scala:122) >> at org.apache.spark.rdd.RDD$$anonfun$first$1.apply(RDD.scala:1312) >> at >> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >> at >> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) >> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306) >> at org.apache.spark.rdd.RDD.first(RDD.scala:1311) >> at >> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36) >> at >> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:41) >> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:43) >> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:45) >> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:47) >> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:49) >> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:51) >> at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:53) >> at $iwC$$iwC$$iwC$$iwC.<init>(<console>:55) >> at $iwC$$iwC$$iwC.<init>(<console>:57) >> at $iwC$$iwC.<init>(<console>:59) >> at $iwC.<init>(<console>:61) >> at <init>(<console>:63) >> at .<init>(<console>:67) >> at .<clinit>(<console>) >> at .<init>(<console>:7) >> at .<clinit>(<console>) >> at $print(<console>) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065) >> at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340) >> at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840) >> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871) >> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819) >> at >> org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:655) >> at >> org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:620) >> at >> org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:613) >> at >> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57) >> at >> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93) >> at >> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:276) >> at org.apache.zeppelin.scheduler.Job.run(Job.java:170) >> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:118) >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> Why does the call to first fail. Calls such as sc.fromTextFile also fail. >> >> the following also works: >> >> checkins.where("year = 2010 and month=2 and day>12 and day<15").count() >> >> but this does not: >> >> checkins.where("year = 2010 and month=2 and day>12 and day<15").first() >> >> Please assist as this is driving me insane. Especially since the spark >> shell works but this does not or at least seems partially broken. >> >> Thanks >> -- >> Armand Naudé >> Software Engineer >> >> Cell: +27 082 552 0421 >> >> ------------------------------------------------------------ >> To see a world in a grain of sand >> And a heaven in a wild flower, >> Hold infinity in the palm of your hand >> And eternity in an hour. >> -- William Blake >> > > -- Armand Naudé Software Engineer Cell: +27 082 552 0421 ------------------------------------------------------------ To see a world in a grain of sand And a heaven in a wild flower, Hold infinity in the palm of your hand And eternity in an hour. -- William Blake
