Re: Serializing collections in Datasets

2016-03-03 Thread Daniel Siegmann
I have confirmed this is fixed in Spark 1.6.1 RC 1. Thanks. On Tue, Feb 23, 2016 at 1:32 PM, Daniel Siegmann < daniel.siegm...@teamaol.com> wrote: > Yes, I will test once 1.6.1 RC1 is released. Thanks. > > On Mon, Feb 22, 2016 at 6:24 PM, Michael Armbrust > wrote: > >> I think this will be fixed

Re: Serializing collections in Datasets

2016-02-23 Thread Daniel Siegmann
Yes, I will test once 1.6.1 RC1 is released. Thanks. On Mon, Feb 22, 2016 at 6:24 PM, Michael Armbrust wrote: > I think this will be fixed in 1.6.1. Can you test when we post the first > RC? (hopefully later today) > > On Mon, Feb 22, 2016 at 1:51 PM, Daniel Siegmann < > daniel.siegm...@teamaol

Re: Serializing collections in Datasets

2016-02-22 Thread Koert Kuipers
it works in 2.0.0-SNAPSHOT On Mon, Feb 22, 2016 at 6:24 PM, Michael Armbrust wrote: > I think this will be fixed in 1.6.1. Can you test when we post the first > RC? (hopefully later today) > > On Mon, Feb 22, 2016 at 1:51 PM, Daniel Siegmann < > daniel.siegm...@teamaol.com> wrote: > >> Experime

Re: Serializing collections in Datasets

2016-02-22 Thread Michael Armbrust
I think this will be fixed in 1.6.1. Can you test when we post the first RC? (hopefully later today) On Mon, Feb 22, 2016 at 1:51 PM, Daniel Siegmann < daniel.siegm...@teamaol.com> wrote: > Experimenting with datasets in Spark 1.6.0 I ran into a serialization > error when using case classes cont

Serializing collections in Datasets

2016-02-22 Thread Daniel Siegmann
Experimenting with datasets in Spark 1.6.0 I ran into a serialization error when using case classes containing a Seq member. There is no problem when using Array instead. Nor is there a problem using RDD or DataFrame (even if converting the DF to a DS later). Here's an example you can test in the