Re: Efficiency of leftOuterJoin a cassandra rdd

2015-07-15 Thread Deepak Jain
Leftouterjoin and join apis are super slow in spark. 100x slower than hadoop Sent from my iPhone On 14-Jul-2015, at 10:59 PM, Wush Wu wush...@gmail.com wrote: I don't understand. By the way, the `joinWithCassandraTable` does improve my query time from 40 mins to 3 mins. 2015-07-15

Re: Spark Job #of attempts ?

2015-04-09 Thread Deepak Jain
Can I see current values of all configs. Similar to configuration in Hadoop world from ui ? Sent from my iPhone On 09-Apr-2015, at 11:07 pm, Marcelo Vanzin van...@cloudera.com wrote: Set spark.yarn.maxAppAttempts=1 if you don't want retries. On Thu, Apr 9, 2015 at 10:31 AM, ÐΞ€ρ@Ҝ (๏̯͡๏)

Re: Spark Job Failed - Class not serializable

2015-04-03 Thread Deepak Jain
I was able to write record that extends specificrecord (avro) this class was not auto generated. Do we need to do something extra for auto generated classes Sent from my iPhone On 03-Apr-2015, at 5:06 pm, Akhil Das ak...@sigmoidanalytics.com wrote: This thread might give you some insights

Re: Spark Job Failed - Class not serializable

2015-04-03 Thread Deepak Jain
I meant that I did not have to use kyro. Why will kyro help fix this issue now ? Sent from my iPhone On 03-Apr-2015, at 5:36 pm, Deepak Jain deepuj...@gmail.com wrote: I was able to write record that extends specificrecord (avro) this class was not auto generated. Do we need to do