Users with independent filters
Hi all, I'm just starting with Zeppelin, so please bare with me. I created a simple notebook that has some filters. The notebook works with pre-aggregated data, so querying it is quite fast. Now I wanted to give access to others to this notebook, but the problem is that one user filter by one thing and another one wishes another filter. So. Is it possible that more than one user use the same notebook, as a "report", independently? If not (as I think it isn't) what are my options (if any)? Or am I just using the wrong tool for the job (which is sad, because it is the one that I could create my views the fastest and more dynamic way). Thanks in advance. -- Matheus de Oliveira
[no subject]
Hi guys,
I'm having a trouble for two days already, and could not find any
solutions on google, Zeppelin docs, Jira or anywhere.
I'm getting ClassNotFoundException: org.apache.zeppelin.spark.ZeppelinContext
when using input value inside spark DataFrame's filter method.
val city = z.select("City",cities).toString
oDF.select("city").filter(r => city.equals(r.getAs[String]("city"))).count()
I even tried coping the input value to another val with new String(bytes[])
but still get the same error.
The same code work seamlessly if instead of getting the value from z.select
I declare as a String literal
city: String = "NY"
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0
in stage 49.0 failed 4 times, most recent failure: Lost task 0.3 in stage
49.0 (TID 277, 10.6.60.217): java.lang.NoClassDefFoundError:
Lorg/apache/zeppelin/spark/ZeppelinContext;
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
at java.lang.Class.getDeclaredField(Class.java:2068)
at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1703)
at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:484)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.(ObjectStreamClass.java:472)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:598)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStr
ClassNotFoundException: org.apache.zeppelin.spark.ZeppelinContext when using input value inside spark DataFrame's filter method.
Hi guys,
I'm having a trouble for two days already, and could not find any
solutions on google, Zeppelin docs, Jira or anywhere.
I'm getting ClassNotFoundException: org.apache.zeppelin.spark.ZeppelinContext
when using input value inside spark DataFrame's filter method.
val city = z.select("City",cities).toString
oDF.select("city").filter(r => city.equals(r.getAs[String]("city"))).count()
I even tried coping the input value to another val with new String(bytes[])
but still get the same error.
The same code work seamlessly if instead of getting the value from z.select
I declare as a String literal
city: String = "NY"
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0
in stage 49.0 failed 4 times, most recent failure: Lost task 0.3 in stage
49.0 (TID 277, 10.6.60.217): java.lang.NoClassDefFoundError:
Lorg/apache/zeppelin/spark/ZeppelinContext;
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
at java.lang.Class.getDeclaredField(Class.java:2068)
at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1703)
at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:484)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.(ObjectStreamClass.java:472)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:598)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStr
RE: Netty error with spark interpreter
Hi all,
I solved this problem by excluding netty-all dependencies from external jar as
well as from spark-dependency project. Spark was also using two different
versions. Adding new netty-all-4.0.29.Final dependency into both project just
worked fine.
Thanks & Regards,
Vikash Kumar
From: Vikash Kumar [mailto:vikash.ku...@resilinc.com]
Sent: Wednesday, October 19, 2016 12:11 PM
To: users@zeppelin.apache.org
Subject: Netty error with spark interpreter
Hi all,
I am trying zeppelin with spark which is throwing me the
following error related to netty jar conflicts. I checked properly my class
path. There are only single versions of netty-3.8.0 and netty-all-4.0.29-Final
jar.
Other information :
Spark 2.0.0
Scala 2.11
Zeppelin .6.2 snapshot
Command to build:
mvn clean install -DskipTests -Drat.ignoreErrors=true
-Dcheckstyle.skip=true -Denforcer.skip=true -Pspark-2.0 -Dspark.version=2.0.0
-Pscala-2.11 -Phadoop-2.7 -Pyarn
Queries:
sc.version :- it works fine
sqlContext.sql("show tables").show :- throws error
Running on local mode.
I am attaching my spark log file[zeppelin-interpreter-spark-root.log] and mvn
dependency:tree result [dependencyTree.txt]
So I am not able to solve this problem.. :(
Thanks & Regards,
Vikash Kumar
ApacheCon is now less than a month away!
Dear Apache Enthusiast, ApacheCon Sevilla is now less than a month out, and we need your help getting the word out. Please tell your colleagues, your friends, and members of related technical communities, about this event. Rates go up November 3rd, so register today! ApacheCon, and Apache Big Data, are the official gatherings of the Apache Software Foundation, and one of the best places in the world to meet other members of your project community, gain deeper knowledge about your favorite Apache projects, learn about the ASF. Your project doesn't live in a vacuum - it's part of a larger family of projects that have a shared set of values, as well as a shared governance model. And many of our project have an overlap in developers, in communities, and in subject matter, making ApacheCon a great place for cross-pollination of ideas and of communities. Some highlights of these events will be: * Many of our board members and project chairs will be present * The lightning talks are a great place to hear, and give, short presentations about what you and other members of the community are working on * The key signing gets you linked into the web of trust, and better able to verify our software releases * Evening receptions and parties where you can meet community members in a less formal setting * The State of the Feather, where you can learn what the ASF has done in the last year, and what's coming next year * BarCampApache, an informal unconference-style event, is another venue for discussing your projects at the ASF We have a great schedule lined up, covering the wide range of ASF projects, including: * CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos - Carlos Sanchez * Inner sourcing 101 - Jim Jagielski * Java Memory Leaks in Modular Environments - Mark Thomas ApacheCon/Apache Big Data will be held in Sevilla, Spain, at the Melia Sevilla, November 14th through 18th. You can find out more at http://apachecon.com/ Other ways to stay up to date with ApacheCon are: * Follow us on Twitter at @apachecon * Join us on IRC, at #apachecon on the Freenode IRC network * Join the apachecon-discuss mailing list by sending email to apachecon-discuss-subscr...@apache.org * Or contact me directly at rbo...@apache.org with questions, comments, or to volunteer to help See you in Sevilla! -- Rich Bowen: VP, Conferences rbo...@apache.org http://apachecon.com/ @apachecon
Lost connection to the JobManager
Hello All, I’m trying to get up and running with Flink on a mac. Setup: Flink 1.1.3 Zeppelin 0.6.2 Oracle JDK 1.8 u 111 — I realize this isn’t officially tested. I can’t get it working with Java 7 at all. When I try with 7, the daemon starts, and the page mostly loads at localhost:8080 but then there are ws errors: WebSocket connection to 'ws://localhost:8080/ws' failed: Error during WebSocket handshake: Unexpected response code: 500 Having more success with 8 I switch back. I’ve got my interpreter configured as localhost:6123. If I run %flink // let Zeppelin know what interpretter to use. benv I get res5: org.apache.flink.api.scala.ExecutionEnvironment = org.apache.flink.api.scala.ExecutionEnvironment@76bde237 But if I run the example from Trevor Grant’s gagillion dollar gist I get the following. text: org.apache.flink.api.scala.DataSet[String] = org.apache.flink.api.scala.DataSet@281e2d1b counts: org.apache.flink.api.scala.AggregateDataSet[(String, Int)] = org.apache.flink.api.scala.AggregateDataSet@57613737 org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Communication with JobManager failed: Lost connection to the JobManager. at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:405) at org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:95) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:378) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:365) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:340) at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:211) at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:188) at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:172) at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:896) at org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:637) at org.apache.flink.api.scala.DataSet.collect(DataSet.scala:547) at .(:22) at .() at .(:7) at .() at $print() at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:734) at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:983) at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:604) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:568) at org.apache.zeppelin.flink.FlinkInterpreter$1.apply(FlinkInterpreter.java:299) at org.apache.zeppelin.flink.FlinkInterpreter$1.apply(FlinkInterpreter.java:296) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) at scala.Console$.withOut(Console.scala:107) at scala.Console.withOut(Console.scala) at org.apache.zeppelin.flink.FlinkInterpreter.interpret(FlinkInterpreter.java:294) at org.apache.zeppelin.flink.FlinkInterpreter.interpret(FlinkInterpreter.java:239) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flink.runtime.client.JobExecutionException: Communication with JobManager failed: Lost connection to the JobManager. at org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:137) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:401) ... 42 more Caused by: org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException: Lost conne
