Re: Building and running on Mac OS 11.11
I always use sudo and works every time (on 10.11.3): sudo mvn clean package-Pspark-1.6 -Phadoop-2.6 -Dhadoop.version=2.7.2 -Pscalding -Ppyspark -Pyarn -DskipTests And it works every time (I get a bunch of npm errors, but somehow they are recuperated and do not affect the install). Enzo e...@smartinsightsfromdata.com > On 3 Mar 2016, at 18:31, Jose Celaya <jcel...@slb.com> wrote: > > Chris, > Thank you for looking into this. I tried several combinations. These are a > few I can recall. > > 1) man clean package -Pyarn -Pspark-1.6 -Phadoop-2.6 -DskipTests > > 2) man clean package -DskipTests > > 3) man clean package -Pspark-1.6 -Phadoop-2.6 -DskipTests > > I always get the failed build. It always related to some “npm” installation > fail. > cheers > José > > > > > >> On Mar 3, 2016, at 10:08 AM, Christopher Matta <cma...@mapr.com >> <mailto:cma...@mapr.com>> wrote: >> >> Jose, >> What’s your build command look like? The last time I built zeppelin on OS X >> 10.11 I used this command: >> >> mvn -Pmapr50 -Pyarn -Pbuild-distr -Pspark-1.5 -Phadoop-2.6 -Ppyspark package >> -DskipTests -B >> >> Chris Matta >> cma...@mapr.com <mailto:cma...@mapr.com> >> 215-701-3146 >> >> On Thu, Mar 3, 2016 at 9:11 AM, Jose Celaya <jcel...@slb.com >> <mailto:jcel...@slb.com>> wrote: >> Hi, >> >> Are there any suggestions or links to tutorials of building and installing >> for Mac OS 11.11. >> >> I have been trying for two days with no success on building. I have Java 7 >> installed and working, latest Maven installed and working, installed a few >> side packages as well but I am still having issues building barebones and >> also with spark 1.6 and hadoop. >> >> I must say I am not an expert of building open source software so I would >> prefer a binary that I can install. We are quite excited about this tool and >> would like to give it a try. >> >> In any case, any help will be greatly appreciated. >> >> cheers >> >> José >> >> >> >> >> >> José R. Celaya, Ph.D. >> >> Senior Data Scientist >> Schlumberger Software Technology Innovation Center >> Menlo Park, CA >> >> jcel...@slb.com >> <mailto:jcel...@slb.com> >> >
Re: error "Could not find creator property with name 'id' "
Hi Moon
Thanks!! The fixes proposed in the post resolved my problem.
On the other hand, if this is happening to everybody (as I assume), maybe this
should be addressed a bit more systematically??
Thanks again!
Enzo
e...@smartinsightsfromdata.com
> On 1 Mar 2016, at 19:13, moon soo Lee <m...@apache.org> wrote:
>
> Hi Enzo,
>
> It happens when you have multiple version of jackson library in your
> classpath. Please check following email thread
> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/com-fasterxml-jackson-databind-JsonMappingException-td1607.html
>
> <http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/com-fasterxml-jackson-databind-JsonMappingException-td1607.html>
>
> Thanks,
> moon
>
> On Tue, Mar 1, 2016 at 8:46 AM enzo <e...@smartinsightsfromdata.com
> <mailto:e...@smartinsightsfromdata.com>> wrote:
> I get the following euro in a variety of circumstances.
>
> I’ve downloaded zeppelin a couple of days ago. I use Spark 1.6.0.
>
>
> For example:
>
> %spark
>
> val raw = sc.textFile("/tmp/github.json”) // reading a 25Mb file from /tmp
>
> Gives the following error. Hey please!!
>
>
> com.fasterxml.jackson.databind.JsonMappingException: Could not find creator
> property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
> at [Source: {"id":"0","name":"textFile"}; line: 1, column: 1]
> at
> com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
> at
> com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:843)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:533)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:220)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:358)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:265)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:245)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:143)
> at
> com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:439)
> at
> com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666)
> at
> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558)
> at
> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2578)
> at
> org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:85)
> at
> org.apache.spark.rdd.RDDOperationScope$$anonfun$5.apply(RDDOperationScope.scala:136)
> at
> org.apache.spark.rdd.RDDOperationScope$$anonfun$5.apply(RDDOperationScope.scala:136)
> at scala.Option.map(Option.scala:145)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:136)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
> at org.apache.spark.SparkContext.withScope(SparkContext.scala:714)
> at org.apache.spark.SparkContext.hadoopFile(SparkContext.scala:1011)
> at
> org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:832)
> at
> org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:830)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
> at org.apache.spark.SparkContext.withScope(SparkContext.scala:714)
> at org.apache.spark.SparkContext.textFile(SparkContext.scala:830)
> at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:38)
> at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:43)
> at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:45)
> at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:47)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:49)
> at $iwC$$iwC$$iwC$$iwC$$i
error "Could not find creator property with name 'id' "
terpreter.interpret(SparkInterpreter.java:744) at org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:737) at org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:331) at org.apache.zeppelin.scheduler.Job.run(Job.java:171) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Enzo e...@smartinsightsfromdata.com
Re: R interpreters
+1 For R interpreter. Soon please! Incidentally, I wonder if it will display htmlwidgets family of packages as well? Enzo e...@smartinsightsfromdata.com > On 18 Jan 2016, at 06:53, Sourav Mazumder <sourav.mazumde...@gmail.com> wrote: > > Hi, > > Is there a date/plan to integrate this PR to main branch of Zeppelin ? > > I have found this to be very useful in sharing data/model's prediction result > between SparkR and R and then use visualization packages of R to visualize > the prediction results. Also demonstrated the same to many others who also > liked and found it useful. > > Especially given that in Jupyter the data cannot be shared between R and > Spark without serializing/persisting the data, this feature is pretty much > differentiatior for Zeppelin. So wondering is there a plan/date for this PR > to be available in the main branch ? > > Regards, > Sourav > > > > On Thu, Dec 24, 2015 at 11:01 AM, Amos B. Elberg <amos.elb...@me.com > <mailto:amos.elb...@me.com>> wrote: > Yeah, I’ve looked into it a little bit :p > > rZeppelin (the one in repository Zeppelin-With-R) is an accepted pull request > that is pending integration with Zeppelin. > > In an rZeppelin session, R shares the same Spark backend with the rest of the > Zeppelin Spark-based interpreters. That’s why multiple languages can share > the same Spark data pipeline. > > Datalayer’s R interpreter is not integrated with the rest of Zeppelin. It > launches its own instance of Spark which is separate and distinct from the > one being used by the rest of Zeppelin. > > rZeppelin also supports moving data between languages using the Zeppelin > Context, which allows you to share variables with scala and python. > Datalayer has their own implementation for moving variables between R and > scala. > > rZeppelin supports interactive visualizations. I don’t believe that > Datalayer’s does. > > There are other differences as well. This is explained in the rZeppelin > readme. > > > From: Girish Reddy <gir...@springml.com> <mailto:gir...@springml.com> > Reply: users@zeppelin.incubator.apache.org > <mailto:users@zeppelin.incubator.apache.org> > <users@zeppelin.incubator.apache.org> > <mailto:users@zeppelin.incubator.apache.org> > Date: December 24, 2015 at 1:52:11 PM > To: users@zeppelin.incubator.apache.org > <mailto:users@zeppelin.incubator.apache.org> > <users@zeppelin.incubator.apache.org> > <mailto:users@zeppelin.incubator.apache.org> > Subject: R interpreters > >> Hello: >> >> Looks like there are two R interpreters - >> https://github.com/datalayer/zeppelin-R >> <https://github.com/datalayer/zeppelin-R> and >> https://github.com/elbamos/Zeppelin-With-R >> <https://github.com/elbamos/Zeppelin-With-R> >> >> From the docs, Zeppelin-With-R supports passing objects between languages >> using registerTempTable. Not sure if the other interpreter supports it or >> what other differences are. Has anyone looked into this? >> >
cannot build Spark 1.6 with Pyspark
I’m trying to build Spark 1.6 with Pyspark but somewhere in Zeppelin the old dependency (py4j-0.8.2.1-src.zip) is listed, instead of the new py4j-0.9-src.zip [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (download-and-zip-pyspark-files) on project zeppelin-spark-dependencies: An Ant BuildException has occured: Warning: Could not find file /opt/zeppelin/spark-dependencies/target/spark-dist/spark-1.6.0/python/lib/py4j-0.8.2.1-src.zip to copy. [ERROR] around Ant part .. @ 5:149 in /opt/zeppelin/spark-dependencies/target/antrun/build-main.xml Eliminating -Pyspark Zeppelin builds without problems. Enzo e...@smartinsightsfromdata.com
