Any help? Not sure what I am doing wrong.
Best Regards,
Ram
From: Ram VISWANADHA
mailto:ram.viswana...@dailymotion.com>>
Date: Friday, January 22, 2016 at 10:25 AM
To: user mailto:user@spark.apache.org>>
Subject: StackOverflow when computing
MatrixFactorizationModel.recommendProd
Hi,
I am getting this StackOverflowError when fetching recommendations from ALS.
Any help is much appreciated
int features = 100;
double alpha = 0.1;
double lambda = 0.001;
boolean implicit = true;
int iterations = 10;
ALS als = new ALS()
.s
m>>
Date: Saturday, December 5, 2015 at 1:32 AM
To: Ram VISWANADHA
mailto:ram.viswana...@dailymotion.com>>
Cc: user mailto:user@spark.apache.org>>
Subject: Re: Improve saveAsTextFile performance
Which version of spark are you using? Can you look at the event timeline and
the DAG o
0 8 3.9 MB / 95334
Best Regards,
Ram
From: Akhil Das mailto:ak...@sigmoidanalytics.com>>
Date: Saturday, December 5, 2015 at 1:32 AM
To: Ram VISWANADHA
mailto:ram.viswana...@dailymotion.com>>
Cc: user mailto:user@spark.apache.org>>
Subject: Re: Improve saveAsTex
That didn’t work :(
Any help I have documented some steps here.
http://stackoverflow.com/questions/34048340/spark-saveastextfile-last-stage-almost-never-finishes
Best Regards,
Ram
From: Sahil Sareen mailto:sareen...@gmail.com>>
Date: Wednesday, December 2, 2015 at 10:18 PM
To: Ram VISW
Yes. That did not help.
Best Regards,
Ram
From: Ted Yu mailto:yuzhih...@gmail.com>>
Date: Wednesday, December 2, 2015 at 3:25 PM
To: Ram VISWANADHA
mailto:ram.viswana...@dailymotion.com>>
Cc: user mailto:user@spark.apache.org>>
Subject: Re: Improve saveAsTextFile performan
JavaRDD.saveAsTextFile is taking a long time to succeed. There are 10 tasks,
the first 9 complete in a reasonable time but the last task is taking a long
time to complete. The last task contains the maximum number of records like 90%
of the total number of records. Is there any way to paralleli
Hi ,
I am trying to load 931MB file into an RDD, then create a DataFrame and store
the data in a Parquet file. The save method of Parquet file is hanging. I have
set the timeout to 1800 but still the system fails to respond and hangs. I
can’t spot any errors in my code. Can someone help me? Than
