Re: using spark context in map funciton TASk not serilizable error

2016-01-20 Thread Giri P
utputDir) >>> >>> Method1 needs spark context to query cassandra. But I see below error >>> >>> java.io.NotSerializableException: org.apache.spark.SparkContext >>> >>> Is there a way we can fix this ? >>> >>> Thanks >>>

Re: using spark context in map funciton TASk not serilizable error

2016-01-20 Thread Shixiong(Ryan) Zhu
n fix this ? >> >> Thanks >> >> -------------- >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://apache-spark-user-list.1001560.n3.nabble.com/using-spark-context-in-map-funciton-TASk-not-serili

Re: using spark context in map funciton TASk not serilizable error

2016-01-19 Thread Ricardo Paiva
ption: org.apache.spark.SparkContext > > Is there a way we can fix this ? > > Thanks > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/using-spark-context-in-map

using spark context in map funciton TASk not serilizable error

2016-01-18 Thread gpatcham
fix this ? Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/using-spark-context-in-map-funciton-TASk-not-serilizable-error-tp25998.html Sent from the Apache Spark User List mailing list archive at Nabble.

Re: using spark context in map funciton TASk not serilizable error

2016-01-18 Thread Ted Yu
s there a way we can fix this ? > > Thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/using-spark-context-in-map-funciton-TASk-not-serilizable-error-tp25998.html > Se

Re: using spark context in map funciton TASk not serilizable error

2016-01-18 Thread Giri P
>> >> Is there a way we can fix this ? >> >> Thanks >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/using-spark-context-in-map-funciton-TASk-not-serilizable-error-tp25998.html >>

Re: using spark context in map funciton TASk not serilizable error

2016-01-18 Thread Giri P
row =>method1(row,sc)).saveAsTextFile(outputDir) >>> >>> Method1 needs spark context to query cassandra. But I see below error >>> >>> java.io.NotSerializableException: org.apache.spark.SparkContext >>> >>> Is there a way we can fix this ? >>> >

Re: using spark context in map funciton TASk not serilizable error

2016-01-18 Thread Ted Yu
gt;>> reRDD.map(row =>method1(row,sc)).saveAsTextFile(outputDir) >>>> >>>> Method1 needs spark context to query cassandra. But I see below error >>>> >>>> java.io.NotSerializableException: org.apache.spark.SparkContext >>>>

Re: using spark context in map funciton TASk not serilizable error

2016-01-18 Thread Giri P
in map function >>>>> >>>>> reRDD.map(row =>method1(row,sc)).saveAsTextFile(outputDir) >>>>> >>>>> Method1 needs spark context to query cassandra. But I see below error >>&g

Re: using spark context in map funciton TASk not serilizable error

2016-01-18 Thread Ted Yu
Mon, Jan 18, 2016 at 12:29 PM, gpatcham <gpatc...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have a use case where I need to pass sparkcontext in map function >>>>>> >>>>>> reRDD.map(row