All: Ok more details after sifting through logs. This appears to be a
dependency issue. In the interpreter log I
am seeing:
ERROR [2018-05-08 23:27:27,331] ({SparkListenerBus}
Logging.scala[logError]:91) - uncaught error in thread
SparkListenerBus, stopping SparkContext
java.lang.NoSuchMethodError:
org.json4s.Formats.emptyValueStrategy()Lorg/json4s/prefs/EmptyValueStrategy;
at org.json4s.jackson.JsonMethods$class.render(JsonMethods.scala:32)
at org.json4s.jackson.JsonMethods$.render(JsonMethods.scala:50)
at
org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$1.apply(EventLoggingListener.scala:136)
at
org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$1.apply(EventLoggingListener.scala:136)
at scala.Option.foreach(Option.scala:257)
at
org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:136)
at
org.apache.spark.scheduler.EventLoggingListener.onExecutorAdded(EventLoggingListener.scala:187)
at
org.apache.spark.scheduler.SparkListenerBus$class.doPostEvent(SparkListenerBus.scala:61)
at
org.apache.spark.scheduler.LiveListenerBus.doPostEvent(LiveListenerBus.scala:36)
at
org.apache.spark.scheduler.LiveListenerBus.doPostEvent(LiveListenerBus.scala:36)
at
org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:63)
at
org.apache.spark.scheduler.LiveListenerBus.postToAll(LiveListenerBus.scala:36)
at
org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(LiveListenerBus.scala:94)
at
org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(LiveListenerBus.scala:79)
at
org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(LiveListenerBus.scala:79)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(LiveListenerBus.scala:78)
at
org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1245)
at
org.apache.spark.scheduler.LiveListenerBus$$anon$1.run(LiveListenerBus.scala:77)
Based on that and after chasing down things in Maven I added the
following excludes to my dependency:
org.json4s:json4s-core_2.11, org.json4s:json4s-jackson_2.11,
org.json4s:json4s-native_2.11
That did not make a difference.
Googling it appears to be an incompatibility between 3.2.10 and 3.2.11
of json4s.
On 5/8/18 6:46 PM, David Boyd wrote:
All:
I am following the instructions here:
http://www.geomesa.org/documentation/current/user/spark/zeppelin.html
To use geomesa spark with zeppelin.
Whenever I add the artifact dependency I get the following error on
any code I try to run (this includes the basic features -spark
tutorial).
java.lang.NullPointerException
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:398)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:387)
at
org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:146)
at
org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:843)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
I have tried specifying the jar as a maven artifact and as an absolute
path. I have tried multiple versions (1.3.4, 1.3.5, and 2.0.0) of
the artifact. As soon as I remove the dependency the code works again.
Is there another external dependency I can try to see if that is the
problem.
I have tried this with both the 0.7.3 Zeppelin binary distribution,
and also with a 0.7.4 distribution I built specifically.
I am running spark 2.1 on my cluster. Like I said without this
dependency the example code works just fine.
Here is the build command I used for the distribution I am running:
mvn clean package -DskipTests -Pspark-2.1 -Phadoop-2.7 -Pyarn
-Ppyspark -Psparkr -Pr -Pscala-2.11 -Pexamples -Pbuild-distr
From looking at the code around the trace it appears either a class is
not found or something is getting dorked with SparkContext.
Any help would be appreciated.
--
=========mailto:db...@incadencecorp.com ============
David W. Boyd
VP, Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office: +1-703-552-2862
cell: +1-703-402-7908
==============http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC -www.iliterobotics.org
Board Member- USSTEM Foundation -www.usstem.org
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
--
========= mailto:db...@incadencecorp.com ============
David W. Boyd
VP, Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office: +1-703-552-2862
cell: +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
