I am trying to run a function on every line of a parquet file. The function
is in an object. When I run the program, I get an exception that the object
is not serializable. I read around the internet and found that I should use
Kryo Serializer. I changed the setting in the spark conf and
you paste the code that you
are running?
Thanks
Best Regards
On Mon, Jun 22, 2015 at 5:37 PM, Sean Barzilay sesnbarzi...@gmail.com
wrote:
I am trying to run a function on every line of a parquet file. The
function is in an object. When I run the program, I get an exception
Hi I am currently using spark 1.3.0-snapshot to run the fpg algorithm from
the mllib library. When I am trying to run the algorithm over a large
basket(over 1000 items) the program seems to never finish. Did anyone find
a workaround for this problem?
.
Thanks
Best Regards
On Wed, Mar 11, 2015 at 12:27 PM, Sean Barzilay sesnbarzi...@gmail.com
wrote:
I am running on a 4 workers cluster each having between 16 to 30 cores
and
50 GB of ram
On Wed, 11 Mar 2015 8:55 am Akhil Das ak...@sigmoidanalytics.com
wrote:
Depending
will output a massive volume of frequent item
sets. If that's the case, then it may be expected that it's taking
ages to write terabytes of data.
On Wed, Mar 11, 2015 at 8:34 AM, Sean Barzilay sesnbarzi...@gmail.com
wrote:
The program spends its time when I am writing the output to a text file
