spark git commit: [SPARK-19305][SQL] partitioned table should always put partition columns at the end of table schema

2017-01-20 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f174cdc74 -> 3c2ba9fcc [SPARK-19305][SQL] partitioned table should always put partition columns at the end of table schema ## What changes were proposed in this pull request? For data source tables, we will always reorder the specified ta

spark git commit: [SPARK-14536][SQL] fix to handle null value in array type column for postgres.

2017-01-20 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 54268b42d -> f174cdc74 [SPARK-14536][SQL] fix to handle null value in array type column for postgres. ## What changes were proposed in this pull request? JDBC read is failing with NPE due to missing null value check for array data type

spark git commit: [SPARK-16101][SQL] Refactoring CSV write path to be consistent with JSON data source

2017-01-20 Thread wenchen
Repository: spark Updated Branches: refs/heads/master ea31f92bb -> 54268b42d [SPARK-16101][SQL] Refactoring CSV write path to be consistent with JSON data source ## What changes were proposed in this pull request? This PR refactors CSV write path to be consistent with JSON data source. This

spark git commit: [SPARK-19267][SS] Fix a race condition when stopping StateStore

2017-01-20 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.1 4d286c903 -> 6f0ad575d [SPARK-19267][SS] Fix a race condition when stopping StateStore ## What changes were proposed in this pull request? There is a race condition when stopping StateStore which makes `StateStoreSuite.maintenance` fl

spark git commit: [SPARK-19267][SS] Fix a race condition when stopping StateStore

2017-01-20 Thread tdas
Repository: spark Updated Branches: refs/heads/master 9b7a03f15 -> ea31f92bb [SPARK-19267][SS] Fix a race condition when stopping StateStore ## What changes were proposed in this pull request? There is a race condition when stopping StateStore which makes `StateStoreSuite.maintenance` flaky.

spark git commit: [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join

2017-01-20 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.1 482d361c3 -> 4d286c903 [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join PythonUDF is unevaluable, which can not be used inside a join condition, currently the optimizer will push a PythonUDF which accessing

spark git commit: [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join

2017-01-20 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 552e5f088 -> 9b7a03f15 [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join ## What changes were proposed in this pull request? PythonUDF is unevaluable, which can not be used inside a join condition, currently th

spark git commit: [SPARK-19314][SS][CATALYST] Do not allow sort before aggregation in Structured Streaming plan

2017-01-20 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 9fc053c30 -> 4c2065d0a [SPARK-19314][SS][CATALYST] Do not allow sort before aggregation in Structured Streaming plan ## What changes were proposed in this pull request? Sort in a streaming plan should be allowed only after a aggregati

spark git commit: [SPARK-19314][SS][CATALYST] Do not allow sort before aggregation in Structured Streaming plan

2017-01-20 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.1 7bc3e9ba7 -> 482d361c3 [SPARK-19314][SS][CATALYST] Do not allow sort before aggregation in Structured Streaming plan ## What changes were proposed in this pull request? Sort in a streaming plan should be allowed only after a aggregati

spark git commit: [SPARK-19314][SS][CATALYST] Do not allow sort before aggregation in Structured Streaming plan

2017-01-20 Thread tdas
Repository: spark Updated Branches: refs/heads/master e20d9b156 -> 552e5f088 [SPARK-19314][SS][CATALYST] Do not allow sort before aggregation in Structured Streaming plan ## What changes were proposed in this pull request? Sort in a streaming plan should be allowed only after a aggregation i

spark git commit: [SPARK-19069][CORE] Expose task 'status' and 'duration' in spark history server REST API.

2017-01-20 Thread irashid
Repository: spark Updated Branches: refs/heads/master d50d12b49 -> e20d9b156 [SPARK-19069][CORE] Expose task 'status' and 'duration' in spark history server REST API. ## What changes were proposed in this pull request? Although Spark history server UI shows task ‘status’ and ‘durationâ

spark git commit: [SPARK-19302][DOC][MINOR] Fix the wrong item format in security.md

2017-01-20 Thread srowen
Repository: spark Updated Branches: refs/heads/master 039ed9fe8 -> d50d12b49 [SPARK-19302][DOC][MINOR] Fix the wrong item format in security.md ## What changes were proposed in this pull request? In docs/security.md, there is a description as follows. ``` steps to configure the key-stores a