spark git commit: [SPARK-12546][SQL] Change default number of open parquet files

2016-02-22 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 16f35c4c6 -> 699644c69 [SPARK-12546][SQL] Change default number of open parquet files A common problem that users encounter with Spark 1.6.0 is that writing to a partitioned parquet table OOMs. The root cause is that parquet allocates

spark git commit: [SPARK-12546][SQL] Change default number of open parquet files

2016-02-22 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 4a91806a4 -> 173aa949c [SPARK-12546][SQL] Change default number of open parquet files A common problem that users encounter with Spark 1.6.0 is that writing to a partitioned parquet table OOMs. The root cause is that parquet allocates a