How does .jsonFile() work?

2016-04-19 Thread resonance
Hi, this is more of a theoretical question and I'm asking it here because I have no idea where to find documentation for this stuff. I am currently working with Spark SQL and am considering using data contained within JSON datasets. I am aware of the .jsonFile() method in Spark SQL. What is the

Spark 1.4.1 py4j.Py4JException: Method read([]) does not exist

2015-08-12 Thread resonance
Hello everybody, I am programming with Pyspark in the Eclipse IDE and have been trying to transition to Spark 1.4.1 so that I may finally program using Python 3. (Good work with the progress!) The following program works in Spark 1.3.1 but throws an exception in Spark 1.4.1: from pyspark import