I had the same issue on AWS EMR with Spark 1.3.1.e (AWS version) passed with
'-h' parameter (it is bootstrap action parameter for spark).
I don't see the problem with Spark 1.3.1.e not passing the parameter.
I am not sure about your env.
Thanks,
--
View this message in context:
Looks like your program somehow picked up a older version of parquet (spark
1.3.1 uses parquet 1.6.0rc3 and seems NO_FILTER field was introduced in
1.6.0rc2). Is it possible that you can check the parquet lib version in
your classpath?
Thanks,
Yin
On Sat, May 30, 2015 at 2:44 PM, ogoh
Hello Bright Sparks,
I was using Spark 1.3.0 to push data out to Parquet files. They have been
working great, super fast, easy way to persist data frames etc.
However I just swapped out Spark 1.3.0 and picked up the tarball for 1.3.1.
I unzipped it, copied my config over and then went to read
