SparkR does not include SparkContext

2015-07-28 Thread Siegfried Bilstein
Hi, I'm starting R on Spark via the sparkR script but I can't access the sparkcontext as described in the programming guide. Any ideas? Thanks, Siegfried

Deleting HDFS files from Pyspark

2015-06-11 Thread Siegfried Bilstein
I've seen plenty of examples for creating HDFS files from pyspark but haven't been able to figure out how to delete files from pyspark. Is there an API I am missing for filesystem management? Or should I be including the HDFS python modules? Thanks, Siegfried