spark git commit: [SPARK-19723][SQL] create datasource table with an non-existent location should work

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master fb9beda54 -> f6fdf92d0 [SPARK-19723][SQL] create datasource table with an non-existent location should work ## What changes were proposed in this pull request? This JIRA is a follow up work after

spark git commit: [SPARK-19611][SQL] Introduce configurable table schema inference

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 5a2ad4312 -> e481a7381 [SPARK-19611][SQL] Introduce configurable table schema inference Add a new configuration option that allows Spark SQL to infer a case-sensitive schema from a Hive Metastore table's data files when a

spark git commit: [SPARK-19893][SQL] should not run DataFrame set oprations with map type

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 c561e6cfa -> e8426cb5a [SPARK-19893][SQL] should not run DataFrame set oprations with map type In spark SQL, map type can't be used in equality test/comparison, and `Intersect`/`Except`/`Distinct` do need equality test for all

spark git commit: [SPARK-19893][SQL] should not run DataFrame set oprations with map type

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 f0d50fd54 -> 5a2ad4312 [SPARK-19893][SQL] should not run DataFrame set oprations with map type In spark SQL, map type can't be used in equality test/comparison, and `Intersect`/`Except`/`Distinct` do need equality test for all

spark git commit: [SPARK-19893][SQL] should not run DataFrame set oprations with map type

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master ffee4f1ce -> fb9beda54 [SPARK-19893][SQL] should not run DataFrame set oprations with map type ## What changes were proposed in this pull request? In spark SQL, map type can't be used in equality test/comparison, and

spark git commit: [SPARK-19905][SQL] Bring back Dataset.inputFiles for Hive SerDe tables

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master bc3035140 -> ffee4f1ce [SPARK-19905][SQL] Bring back Dataset.inputFiles for Hive SerDe tables ## What changes were proposed in this pull request? `Dataset.inputFiles` works by matching `FileRelation`s in the query plan. In Spark 2.1,

spark git commit: [SPARK-19611][SQL] Preserve metastore field order when merging inferred schema

2017-03-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 8f0490e22 -> bc3035140 [SPARK-19611][SQL] Preserve metastore field order when merging inferred schema ## What changes were proposed in this pull request? The ```HiveMetastoreCatalog.mergeWithMetastoreSchema()``` method added in #16944

spark git commit: [SPARK-17979][SPARK-14453] Remove deprecated SPARK_YARN_USER_ENV and SPARK_JAVA_OPTS

2017-03-10 Thread vanzin
Repository: spark Updated Branches: refs/heads/master dd9049e04 -> 8f0490e22 [SPARK-17979][SPARK-14453] Remove deprecated SPARK_YARN_USER_ENV and SPARK_JAVA_OPTS This fix removes deprecated support for config `SPARK_YARN_USER_ENV`, as is mentioned in SPARK-17979. This fix also removes

spark git commit: [SPARK-19620][SQL] Fix incorrect exchange coordinator id in the physical plan

2017-03-10 Thread yhuai
Repository: spark Updated Branches: refs/heads/master fcb68e0f5 -> dd9049e04 [SPARK-19620][SQL] Fix incorrect exchange coordinator id in the physical plan ## What changes were proposed in this pull request? When adaptive execution is enabled, an exchange coordinator is used in the Exchange

spark git commit: [SPARK-19786][SQL] Facilitate loop optimizations in a JIT compiler regarding range()

2017-03-10 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 501b71119 -> fcb68e0f5 [SPARK-19786][SQL] Facilitate loop optimizations in a JIT compiler regarding range() ## What changes were proposed in this pull request? This PR improves performance of operations with `range()` by changing Java