RE: Protobuf error when streaming from Kafka

2015-08-25 Thread java8964
file it is loaded from, and verify if it is in 2.5 version or below. Yong Date: Tue, 25 Aug 2015 13:44:17 -0700 Subject: Re: Protobuf error when streaming from Kafka From: lcas...@gmail.com To: java8...@hotmail.com CC: yuzhih...@gmail.com; user@spark.apache.org Do you think this binary would have i

Re: Protobuf error when streaming from Kafka

2015-08-25 Thread Cassa L
spark build with Hive? >> >> I met the same problem before because the hive-exec jar in the maven >> itself include "protobuf" class, which will be included in the Spark jar. >> >> Yong >> >> ---------- >> Date: Tue, 25 A

Re: Protobuf error when streaming from Kafka

2015-08-25 Thread Cassa L
luded in the Spark jar. > > Yong > > -- > Date: Tue, 25 Aug 2015 12:39:46 -0700 > Subject: Re: Protobuf error when streaming from Kafka > From: lcas...@gmail.com > To: yuzhih...@gmail.com > CC: user@spark.apache.org > > > Hi,

RE: Protobuf error when streaming from Kafka

2015-08-25 Thread java8964
Did your spark build with Hive? I met the same problem before because the hive-exec jar in the maven itself include "protobuf" class, which will be included in the Spark jar. Yong Date: Tue, 25 Aug 2015 12:39:46 -0700 Subject: Re: Protobuf error when streaming from Kafka From: lcas...

Re: Protobuf error when streaming from Kafka

2015-08-25 Thread Cassa L
Hi, I am using Spark-1.4 and Kafka-0.8.2.1 As per google suggestions, I rebuilt all the classes with protobuff-2.5 dependencies. My new protobuf is compiled using 2.5. However now, my spark job does not start. Its throwing different error. Does Spark or any other its dependencies uses old protobuf

Re: Protobuf error when streaming from Kafka

2015-08-24 Thread Ted Yu
Can you show the complete stack trace ? Which Spark / Kafka release are you using ? Thanks On Mon, Aug 24, 2015 at 4:58 PM, Cassa L wrote: > Hi, > I am storing messages in Kafka using protobuf and reading them into > Spark. I upgraded protobuf version from 2.4.1 to 2.5.0. I got > "java.lang.U