i am very surprised this works, since spark depends on akka remote which
uses protobuf 2.5
On Mon, Nov 9, 2015 at 2:28 PM, Lan Jiang wrote:
> I have not run into any linkage problem, but maybe I was lucky. :-). The
> reason I wanted to use protobuf 3 is mainly for Map type support.
>
> On Thu, N
I have not run into any linkage problem, but maybe I was lucky. :-). The
reason I wanted to use protobuf 3 is mainly for Map type support.
On Thu, Nov 5, 2015 at 4:43 AM, Steve Loughran
wrote:
>
> > On 5 Nov 2015, at 00:12, Lan Jiang wrote:
> >
> > I have used protobuf 3 successfully with Spark
> On 5 Nov 2015, at 00:12, Lan Jiang wrote:
>
> I have used protobuf 3 successfully with Spark on CDH 5.4, even though Hadoop
> itself comes with protobuf 2.5. I think the steps apply to HDP too. You need
> to do the following
Protobuf.jar has been so brittle in the past that the entire hado
I have used protobuf 3 successfully with Spark on CDH 5.4, even though Hadoop
itself comes with protobuf 2.5. I think the steps apply to HDP too. You need to
do the following
1. Set the below parameter
spark.executor.userClassPathFirst=true
spark.driver.userClassPathFirst=true
2. Include proto
Hi,
Does spark support protobuff 3.0? I used protobuff 2.5 with spark-1.4
built for HDP 2.3. Given that protobuff has compatibility issues , want to
know if spark supports protbuff 3.0
LCassa
