Great, thanks for the extra info!
> On 12 Jun 2015, at 12:41, Cheng Lian wrote:
>
> At the time 1.3.x was released, 1.6.0 hasn't been released yet. And we didn't
> have enough time to upgrade and test Parquet 1.6.0 for Spark 1.4.0. But we've
> already upgraded Parquet to 1.7.0 (which is exact
At the time 1.3.x was released, 1.6.0 hasn't been released yet. And we
didn't have enough time to upgrade and test Parquet 1.6.0 for Spark
1.4.0. But we've already upgraded Parquet to 1.7.0 (which is exactly the
same as 1.6.0 with package name renamed from com.twitter to
org.apache.parquet) on
Hi
What is the reason that Spark still comes with Parquet 1.6.0rc3? It seems like
newer Parquet versions are available (e.g. 1.6.0). This would fix problems with
‘spark.sql.parquet.filterPushdown’, which currently is disabled by default,
because of a bug in Parquet 1.6.0rc3.
Thanks!
Eric
