Re: Reading JSON in Pyspark throws scala.MatchError

You are correct, that was the issue. On Tue, Oct 20, 2015 at 10:18 PM, Jeff Zhang wrote: > BTW, I think Json Parser should verify the json format at least when > inferring the schema of json. > > On Wed, Oct 21, 2015 at 12:59 PM, Jeff Zhang wrote: > >> I

Troubleshooting "Task not serializable" in Spark/Scala environments

Howdy, I'm a relative novice at Spark/Scala and I'm puzzled by some behavior that I'm seeing in 2 of my local Spark/Scala environments (Scala for Jupyter and Scala IDE) but not the 3rd (Spark Shell). The following code throws the following stack trace error in the former 2 environments but