I was able to resolve the serialization issue. The root cause was, I was
accessing the config values within foreachRDD{}.
The solution was to extract the values from config outside the foreachRDD
scope and send in values to the loop directly. Probably something obvious
as we cannot have nested dist
That looks like a classpath problem. You should not have to include
the kafka_2.10 artifact in your pom, spark-streaming-kafka_2.10
already has a transitive dependency on it. That being said, 0.8.2.1
is the correct version, so that's a little strange.
How are you building and submitting your app
Also, just to keep it simple, I am trying to use 1.6.0CDH5.7.0 in the
pom.xml as the cluster I am trying to run on is CDH5.7.0 with spark 1.6.0.
Here is my pom setting:
1.6.0-cdh5.7.0
org.apache.spark
spark-core_2.10
${cdh.spark.version}
compile
org.apache.spark
spark-
Which Scala version / Spark release are you using ?
Cheers
On Wed, Jun 22, 2016 at 8:20 PM, Sunita Arvind
wrote:
> Hello Experts,
>
> I am getting this error repeatedly:
>
> 16/06/23 03:06:59 ERROR streaming.StreamingContext: Error starting the
> context, marking it as stopped
> java.lang.Null
Hello Experts,
I am getting this error repeatedly:
16/06/23 03:06:59 ERROR streaming.StreamingContext: Error starting the
context, marking it as stopped
java.lang.NullPointerException
at
com.typesafe.config.impl.SerializedConfigValue.writeOrigin(SerializedConfigValue.java:202)
at
