ode that you
> are running?
>
> Thanks
> Best Regards
>
> On Mon, Jun 22, 2015 at 5:37 PM, Sean Barzilay
> wrote:
>
>> I am trying to run a function on every line of a parquet file. The
>> function is in an object. When I run the program, I get an exception th
I am trying to run a function on every line of a parquet file. The function
is in an object. When I run the program, I get an exception that the object
is not serializable. I read around the internet and found that I should use
Kryo Serializer. I changed the setting in the spark conf and registered
tput a massive volume of frequent item
> sets. If that's the case, then it may be expected that it's taking
> ages to write terabytes of data.
>
> On Wed, Mar 11, 2015 at 8:34 AM, Sean Barzilay
> wrote:
> > The program spends its time when I am writing the output t
you haven't tried already.
> >
> > Thanks
> > Best Regards
> >
> > On Wed, Mar 11, 2015 at 12:27 PM, Sean Barzilay
> > wrote:
> >>
> >> I am running on a 4 workers cluster each having between 16 to 30 cores
> and
> >> 50 GB of ram
Hi I am currently using spark 1.3.0-snapshot to run the fpg algorithm from
the mllib library. When I am trying to run the algorithm over a large
basket(over 1000 items) the program seems to never finish. Did anyone find
a workaround for this problem?
