Maybe try including the jar with
--driver-class-path jar
On Feb 26, 2015, at 12:16 PM, Akshat Aranya aara...@gmail.com wrote:
My guess would be that you are packaging too many things in your job, which
is causing problems with the classpath. When your jar goes in first, you get
the
Hello everyone,
We are trying to decode a message inside a Spark job that we receive from
Kafka. The message is encoded using Proto Buff. The problem is when
decoding we get class-not-found exceptions. We have tried remedies we found
online in Stack Exchange and mail list archives but nothing
My guess would be that you are packaging too many things in your job, which
is causing problems with the classpath. When your jar goes in first, you
get the correct version of protobuf, but some other version of something
else. When your jar goes in later, other things work, but protobuf
breaks.
I assume this is a difference between your local driver classpath and
remote worker classpath. It may not be a question of whether the class
is there, but classpath visibility issues. Have you looked into
settings like spark.files.userClassPathFirst?
On Tue, Feb 24, 2015 at 4:43 AM, necro351 .
Hello,
I am trying to deserialize some data encoded using proto buff from within
Spark and am getting class-not-found exceptions. I have narrowed the
program down to something very simple that shows the problem exactly (see
'The Program' below) and hopefully someone can tell me the easy fix :)
bq. Caused by: java.lang.ClassNotFoundException: com.rick.reports.Reports$
SensorReports
Is Reports$SensorReports class in rick-processors-assembly-1.0.jar ?
Thanks
On Mon, Feb 23, 2015 at 8:43 PM, necro351 . necro...@gmail.com wrote:
Hello,
I am trying to deserialize some data encoded
Hi Ted,
Yes it appears to be:
rick@ubuntu:~/go/src/rick/sparksprint/containers/tests/StreamingReports$
jar tvf
../../../analyzer/spark/target/scala-2.10/rick-processors-assembly-1.0.jar|grep
SensorReports
1128 Mon Feb 23 17:34:46 PST 2015
com/defend7/reports/Reports$SensorReports$1.class
13507
Sorry Ted, that was me clumsily trying to redact my organization's name
from the computer output (in my e-mail editor). I can assure you that
basically defend7 and rick are the same thing in this case so the class is
present in the jar.
On Mon Feb 23 2015 at 9:39:09 PM Ted Yu yuzhih...@gmail.com
The classname given in stack trace was com.rick.reports.Reports
In the output from jar command the class is com.defend7.reports.Reports.
FYI
On Mon, Feb 23, 2015 at 9:33 PM, necro351 . necro...@gmail.com wrote:
Hi Ted,
Yes it appears to be:
