Right, you can use SparkContext and SQLContext in multiple threads. They
are thread safe.
Best Regards,
Shixiong Zhu
2015-10-01 4:57 GMT+08:00 :
> Hi all,
>
> I have a process where I do some calculations on each one of the columns
> of a dataframe.
>
Hi all,
I have a process where I do some calculations on each one of the columns of a
dataframe.
Intrinsecally, I run across each column with a for loop. On the other hand,
each process itself is non-entirely-distributable.
To speed up the process, I would like to submit a spark program for