Re: Spark SQL and Kryo registration

[Olivier Girardot]

Hi everyone, it seems that it works now out of the box. So nevermind, 
registration is compatible with spark 2.0 when using dataframes.
Regards,
Olivier.





On Fri, Aug 5, 2016 10:07 AM, Maciej Bryński mac...@brynski.pl wrote:
Hi Olivier, Did you check performance of Kryo ? I have observations that Kryo 
is slightly slower than Java Serializer.
Regards, Maciek
2016-08-04 17:41 GMT+02:00 Amit Sela < amitsel...@gmail.com > :
It should. Codegen uses the SparkConf in SparkEnv when instantiating a new
Serializer.
On Thu, Aug 4, 2016 at 6:14 PM Jacek Laskowski < ja...@japila.pl > wrote:
Hi Olivier,

I don't know either, but am curious what you've tried already.

Jacek


On 3 Aug 2016 10:50 a.m., "Olivier Girardot" < o.girardot@lateral-thoughts. com 
> wrote:
Hi everyone, I'm currently to use Spark 2.0.0 and making Dataframes work with 
kryo. registrationRequired=true Is it even possible at all considering the 
codegen ?
Regards,
Olivier Girardot | Associé
o.girardot@lateral-thoughts. com
+33 6 24 09 17 94


--
Maciek Bryński

Olivier Girardot | Associé
o.girar...@lateral-thoughts.com
+33 6 24 09 17 94

Re: Spark SQL and Kryo registration

[Maciej Bryński]

Hi Olivier,
Did you check performance of Kryo ?
I have observations that Kryo is slightly slower than Java Serializer.

Regards,
Maciek

2016-08-04 17:41 GMT+02:00 Amit Sela :

> It should. Codegen uses the SparkConf in SparkEnv when instantiating a new
> Serializer.
>
> On Thu, Aug 4, 2016 at 6:14 PM Jacek Laskowski  wrote:
>
>> Hi Olivier,
>>
>> I don't know either, but am curious what you've tried already.
>>
>> Jacek
>>
>> On 3 Aug 2016 10:50 a.m., "Olivier Girardot" <
>> o.girar...@lateral-thoughts.com> wrote:
>>
>>> Hi everyone,
>>> I'm currently to use Spark 2.0.0 and making Dataframes work with kryo.
>>> registrationRequired=true
>>> Is it even possible at all considering the codegen ?
>>>
>>> Regards,
>>>
>>> *Olivier Girardot* | Associé
>>> o.girar...@lateral-thoughts.com
>>> +33 6 24 09 17 94
>>>
>>


-- 
Maciek Bryński

Re: Spark SQL and Kryo registration

[Amit Sela]

It should. Codegen uses the SparkConf in SparkEnv when instantiating a new
Serializer.

On Thu, Aug 4, 2016 at 6:14 PM Jacek Laskowski  wrote:

> Hi Olivier,
>
> I don't know either, but am curious what you've tried already.
>
> Jacek
>
> On 3 Aug 2016 10:50 a.m., "Olivier Girardot" <
> o.girar...@lateral-thoughts.com> wrote:
>
>> Hi everyone,
>> I'm currently to use Spark 2.0.0 and making Dataframes work
>> with kryo.registrationRequired=true
>> Is it even possible at all considering the codegen ?
>>
>> Regards,
>>
>> *Olivier Girardot* | Associé
>> o.girar...@lateral-thoughts.com
>> +33 6 24 09 17 94
>>
>

Re: Spark SQL and Kryo registration

[Jacek Laskowski]

Hi Olivier,

I don't know either, but am curious what you've tried already.

Jacek

On 3 Aug 2016 10:50 a.m., "Olivier Girardot" <
o.girar...@lateral-thoughts.com> wrote:

> Hi everyone,
> I'm currently to use Spark 2.0.0 and making Dataframes work with kryo.
> registrationRequired=true
> Is it even possible at all considering the codegen ?
>
> Regards,
>
> *Olivier Girardot* | Associé
> o.girar...@lateral-thoughts.com
> +33 6 24 09 17 94
>

Spark SQL and Kryo registration

[Olivier Girardot]

Hi everyone, I'm currently to use Spark 2.0.0 and making Dataframes work with
kryo.registrationRequired=true Is it even possible at all considering the 
codegen ?
Regards,
Olivier Girardot | Associé
o.girar...@lateral-thoughts.com
+33 6 24 09 17 94