spark git commit: [SPARK-16489][SQL] Guard against variable reuse mistakes in expression code generation

use problems in expression unit tests. This patch also fixes the bug in crc32 expression. ## How was this patch tested? This is a test harness change, but I also created a new test suite for testing the test harness. Author: Reynold Xin <r...@databricks.com> Closes #14146 from rxin/SPARK-164

spark git commit: [SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter

Repository: spark Updated Branches: refs/heads/master b1e5281c5 -> c9a676215 [SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter ## What changes were proposed in this pull request? It would be useful to support listing the columns that are referenced by a

spark git commit: [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy

Repository: spark Updated Branches: refs/heads/branch-2.0 b37177c22 -> 689261465 [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy This patch fixes a variable namespace collision bug in pmod and partitionBy Regression test for one possible occurrence. A more

spark git commit: [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy

Repository: spark Updated Branches: refs/heads/master e50efd53f -> 9cc74f95e [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy ## What changes were proposed in this pull request? This patch fixes a variable namespace collision bug in pmod and partitionBy ## How

spark git commit: [SPARK-16318][SQL] Implement all remaining xpath functions (branch-2.0)

Repository: spark Updated Branches: refs/heads/branch-2.0 f97dd8a8f -> 72cf74324 [SPARK-16318][SQL] Implement all remaining xpath functions (branch-2.0) ## What changes were proposed in this pull request? This patch implements all remaining xpath functions that Hive supports and not natively

spark git commit: [SPARK-16477] Bump master version to 2.1.0-SNAPSHOT

How was this patch tested? N/A Author: Reynold Xin <r...@databricks.com> Closes #14130 from rxin/SPARK-16477. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ffcb6e05 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ffcb

[1/2] spark git commit: [SPARK-16476] Restructure MimaExcludes for easier union excludes

Repository: spark Updated Branches: refs/heads/branch-2.0 a33643cbf -> 139d5eae0 http://git-wip-us.apache.org/repos/asf/spark/blob/139d5eae/project/MimaExcludes.scala -- diff --git a/project/MimaExcludes.scala

[2/2] spark git commit: [SPARK-16476] Restructure MimaExcludes for easier union excludes

/A - this is a build infra change. Author: Reynold Xin <r...@databricks.com> Closes #14128 from rxin/SPARK-16476. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/52b5bb0b Tree: http://git-wip-us.apache.org/repos/asf/spar

[2/2] spark git commit: [SPARK-16476] Restructure MimaExcludes for easier union excludes

/A - this is a build infra change. Author: Reynold Xin <r...@databricks.com> Closes #14128 from rxin/SPARK-16476. (cherry picked from commit 52b5bb0b7fabe6cc949f514c548f9fbc6a4fa181) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark

[1/2] spark git commit: [SPARK-16476] Restructure MimaExcludes for easier union excludes

Repository: spark Updated Branches: refs/heads/master f12a38b2d -> 52b5bb0b7 http://git-wip-us.apache.org/repos/asf/spark/blob/52b5bb0b/project/MimaExcludes.scala -- diff --git a/project/MimaExcludes.scala

spark git commit: [SPARK-15467][BUILD] update janino version to 3.0.0

Repository: spark Updated Branches: refs/heads/master 7374e518e -> f12a38b2d [SPARK-15467][BUILD] update janino version to 3.0.0 ## What changes were proposed in this pull request? This PR updates version of Janino compiler from 2.7.8 to 3.0.0. This version fixes [an Janino

spark git commit: [SPARK-11857][MESOS] Deprecate fine grained

Repository: spark Updated Branches: refs/heads/branch-2.0 5024c4cb8 -> 50d7002b6 [SPARK-11857][MESOS] Deprecate fine grained ## What changes were proposed in this pull request? Documentation changes to indicate that fine-grained mode is now deprecated. No code changes were made, and all

spark git commit: [SPARK-11857][MESOS] Deprecate fine grained

Repository: spark Updated Branches: refs/heads/master d8b06f18d -> b1db26acc [SPARK-11857][MESOS] Deprecate fine grained ## What changes were proposed in this pull request? Documentation changes to indicate that fine-grained mode is now deprecated. No code changes were made, and all

spark git commit: [SPARK-16432] Empty blocks fail to serialize due to assert in ChunkedByteBuffer

Repository: spark Updated Branches: refs/heads/master 6cef0183c -> d8b06f18d [SPARK-16432] Empty blocks fail to serialize due to assert in ChunkedByteBuffer ## What changes were proposed in this pull request? It's possible to also change the callers to not pass in empty chunks, but it seems

spark git commit: [SPARK-16432] Empty blocks fail to serialize due to assert in ChunkedByteBuffer

Repository: spark Updated Branches: refs/heads/branch-2.0 16202ba68 -> 5024c4cb8 [SPARK-16432] Empty blocks fail to serialize due to assert in ChunkedByteBuffer ## What changes were proposed in this pull request? It's possible to also change the callers to not pass in empty chunks, but it

spark git commit: [SPARK-16387][SQL] JDBC Writer should use dialect to quote field names.

Repository: spark Updated Branches: refs/heads/branch-2.0 07f562f58 -> 463cbf72f [SPARK-16387][SQL] JDBC Writer should use dialect to quote field names. ## What changes were proposed in this pull request? Currently, JDBC Writer uses dialects to get datatypes, but doesn't to quote field

spark git commit: [SPARK-16387][SQL] JDBC Writer should use dialect to quote field names.

Repository: spark Updated Branches: refs/heads/master 60ba436b7 -> 3b22291b5 [SPARK-16387][SQL] JDBC Writer should use dialect to quote field names. ## What changes were proposed in this pull request? Currently, JDBC Writer uses dialects to get datatypes, but doesn't to quote field names.

spark git commit: [SPARK-16281][SQL] Implement parse_url SQL function

Repository: spark Updated Branches: refs/heads/master 142df4834 -> f5fef6914 [SPARK-16281][SQL] Implement parse_url SQL function ## What changes were proposed in this pull request? This PR adds parse_url SQL functions in order to remove Hive fallback. A new implementation of #13999 ## How

spark git commit: [SPARK-16281][SQL] Implement parse_url SQL function

Repository: spark Updated Branches: refs/heads/branch-2.0 0e9333b27 -> e3424fd77 [SPARK-16281][SQL] Implement parse_url SQL function ## What changes were proposed in this pull request? This PR adds parse_url SQL functions in order to remove Hive fallback. A new implementation of #13999 ##

spark git commit: [SPARK-16429][SQL] Include `StringType` columns in `describe()`

Repository: spark Updated Branches: refs/heads/master 67e085ef6 -> 142df4834 [SPARK-16429][SQL] Include `StringType` columns in `describe()` ## What changes were proposed in this pull request? Currently, Spark `describe` supports `StringType`. However, `describe()` returns a dataset for

spark git commit: [SPARK-16420] Ensure compression streams are closed.

Repository: spark Updated Branches: refs/heads/master 38cf8f2a5 -> 67e085ef6 [SPARK-16420] Ensure compression streams are closed. ## What changes were proposed in this pull request? This uses the try/finally pattern to ensure streams are closed after use. `UnsafeShuffleWriter` wasn't

spark git commit: [SPARK-16420] Ensure compression streams are closed.

Repository: spark Updated Branches: refs/heads/branch-2.0 8dee2eca8 -> 0e9333b27 [SPARK-16420] Ensure compression streams are closed. ## What changes were proposed in this pull request? This uses the try/finally pattern to ensure streams are closed after use. `UnsafeShuffleWriter` wasn't

spark git commit: [SPARK-13638][SQL] Add quoteAll option to CSV DataFrameWriter

Repository: spark Updated Branches: refs/heads/branch-2.0 8c8180605 -> 8dee2eca8 [SPARK-13638][SQL] Add quoteAll option to CSV DataFrameWriter ## What changes were proposed in this pull request? Adds an quoteAll option for writing CSV which will quote all fields. See

spark git commit: [SPARK-13638][SQL] Add quoteAll option to CSV DataFrameWriter

Repository: spark Updated Branches: refs/heads/master 255d74fe4 -> 38cf8f2a5 [SPARK-13638][SQL] Add quoteAll option to CSV DataFrameWriter ## What changes were proposed in this pull request? Adds an quoteAll option for writing CSV which will quote all fields. See

spark git commit: [SPARK-16286][SQL] Implement stack table generating function

Repository: spark Updated Branches: refs/heads/branch-2.0 e32c29d86 -> 565e18cf7 [SPARK-16286][SQL] Implement stack table generating function This PR implements `stack` table generating function. Pass the Jenkins tests including new testcases. Author: Dongjoon Hyun

spark git commit: [SPARK-16288][SQL] Implement inline table generating function

Repository: spark Updated Branches: refs/heads/branch-2.0 bb4b0419b -> e32c29d86 [SPARK-16288][SQL] Implement inline table generating function This PR implements `inline` table generating function. Pass the Jenkins tests with new testcase. Author: Dongjoon Hyun Closes

spark git commit: [SPARK-16274][SQL] Implement xpath_boolean

Repository: spark Updated Branches: refs/heads/branch-2.0 144aa84ce -> bb4b0419b [SPARK-16274][SQL] Implement xpath_boolean This patch implements xpath_boolean expression for Spark SQL, a xpath function that returns true or false. The implementation is modelled after Hive's xpath_boolean,

spark git commit: [SPARK-16271][SQL] Implement Hive's UDFXPathUtil

Repository: spark Updated Branches: refs/heads/branch-2.0 a04975457 -> 144aa84ce [SPARK-16271][SQL] Implement Hive's UDFXPathUtil This patch ports Hive's UDFXPathUtil over to Spark, which can be used to implement xpath functionality in Spark in the near future. Added two new test suites

spark git commit: [SPARK-16289][SQL] Implement posexplode table generating function

Repository: spark Updated Branches: refs/heads/branch-2.0 7ef1d1c61 -> a04975457 [SPARK-16289][SQL] Implement posexplode table generating function This PR implements `posexplode` table generating function. Currently, master branch raises the following exception for `map` argument. It's

spark git commit: [SPARK-16278][SPARK-16279][SQL] Implement map_keys/map_values SQL functions

Repository: spark Updated Branches: refs/heads/branch-2.0 88603bd4f -> 7ef1d1c61 [SPARK-16278][SPARK-16279][SQL] Implement map_keys/map_values SQL functions This PR adds `map_keys` and `map_values` SQL functions in order to remove Hive fallback. Pass the Jenkins tests including new

spark git commit: [SPARK-16276][SQL] Implement elt SQL function

Repository: spark Updated Branches: refs/heads/branch-2.0 73c764a04 -> 88603bd4f [SPARK-16276][SQL] Implement elt SQL function This patch implements the elt function, as it is implemented in Hive. Added expression unit test in StringExpressionsSuite and end-to-end test in

spark git commit: [SPARK-16415][SQL] fix catalog string error

Repository: spark Updated Branches: refs/heads/branch-2.0 cbfd94eac -> 30cb3f1d3 [SPARK-16415][SQL] fix catalog string error ## What changes were proposed in this pull request? In #13537 we truncate `simpleString` if it is a long `StructType`. But sometimes we need `catalogString` to

spark git commit: [SPARK-16415][SQL] fix catalog string error

Repository: spark Updated Branches: refs/heads/master 0f7175def -> 28710b42b [SPARK-16415][SQL] fix catalog string error ## What changes were proposed in this pull request? In #13537 we truncate `simpleString` if it is a long `StructType`. But sometimes we need `catalogString` to

spark git commit: [SPARK-16368][SQL] Fix Strange Errors When Creating View With Unmatched Column Num

Repository: spark Updated Branches: refs/heads/branch-2.0 920162a1e -> d63428af6 [SPARK-16368][SQL] Fix Strange Errors When Creating View With Unmatched Column Num What changes were proposed in this pull request? When creating a view, a common user error is the number of columns

spark git commit: [SPARK-16368][SQL] Fix Strange Errors When Creating View With Unmatched Column Num

Repository: spark Updated Branches: refs/heads/master ce3ea9698 -> ab05db0b4 [SPARK-16368][SQL] Fix Strange Errors When Creating View With Unmatched Column Num What changes were proposed in this pull request? When creating a view, a common user error is the number of columns produced by

spark git commit: [SPARK-16021][TEST-MAVEN] Fix the maven build

Repository: spark Updated Branches: refs/heads/master 69f539140 -> 4b5a72c7d [SPARK-16021][TEST-MAVEN] Fix the maven build ## What changes were proposed in this pull request? Fixed the maven build for #13983 ## How was this patch tested? The existing tests. Author: Shixiong Zhu

spark git commit: [SPARK-16398][CORE] Make cancelJob and cancelStage APIs public

Repository: spark Updated Branches: refs/heads/master 42279bff6 -> 69f539140 [SPARK-16398][CORE] Make cancelJob and cancelStage APIs public ## What changes were proposed in this pull request? Make SparkContext `cancelJob` and `cancelStage` APIs public. This allows applications to use

spark git commit: [SPARK-16021] Fill freed memory in test to help catch correctness bugs

Repository: spark Updated Branches: refs/heads/master b8ebf63c1 -> 44c7c62bc [SPARK-16021] Fill freed memory in test to help catch correctness bugs ## What changes were proposed in this pull request? This patches `MemoryAllocator` to fill clean and freed memory with known byte values,

spark git commit: [SPARK-16371][SQL] Two follow-up tasks

est case name. ## How was this patch tested? Updated a test comment. Author: Reynold Xin <r...@databricks.com> Closes #14074 from rxin/parquet-filter. (cherry picked from commit 8e3e4ed6c090d18675d49eec46b3ee572457db95) Signed-off-by: Reynold Xin <r...@databricks.com> Project:

spark git commit: [SPARK-16371][SQL] Two follow-up tasks

ase name. ## How was this patch tested? Updated a test comment. Author: Reynold Xin <r...@databricks.com> Closes #14074 from rxin/parquet-filter. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8e3e4ed6 Tree: http:

spark git commit: [MESOS] expand coarse-grained mode docs

Repository: spark Updated Branches: refs/heads/branch-2.0 88be66b93 -> 2c2b8f121 [MESOS] expand coarse-grained mode docs ## What changes were proposed in this pull request? docs ## How was this patch tested? viewed the docs in github Author: Michael Gummelt

spark git commit: [MESOS] expand coarse-grained mode docs

Repository: spark Updated Branches: refs/heads/master a8f89df3b -> 9c041990c [MESOS] expand coarse-grained mode docs ## What changes were proposed in this pull request? docs ## How was this patch tested? viewed the docs in github Author: Michael Gummelt Closes

spark git commit: [SPARK-16379][CORE][MESOS] Spark on mesos is broken due to race condition in Logging

Repository: spark Updated Branches: refs/heads/master 040f6f9f4 -> a8f89df3b [SPARK-16379][CORE][MESOS] Spark on mesos is broken due to race condition in Logging ## What changes were proposed in this pull request? The commit

spark git commit: [SPARK-16379][CORE][MESOS] Spark on mesos is broken due to race condition in Logging

Repository: spark Updated Branches: refs/heads/branch-2.0 d7926da5e -> 88be66b93 [SPARK-16379][CORE][MESOS] Spark on mesos is broken due to race condition in Logging ## What changes were proposed in this pull request? The commit

spark git commit: [SPARK-16371][SQL] Do not push down filters incorrectly when inner name and outer name are the same in Parquet

Repository: spark Updated Branches: refs/heads/branch-2.0 03f336d89 -> 2465f0728 [SPARK-16371][SQL] Do not push down filters incorrectly when inner name and outer name are the same in Parquet ## What changes were proposed in this pull request? Currently, if there is a schema as below: ```

spark git commit: [SPARK-16371][SQL] Do not push down filters incorrectly when inner name and outer name are the same in Parquet

Repository: spark Updated Branches: refs/heads/master 480357cc6 -> 4f8ceed59 [SPARK-16371][SQL] Do not push down filters incorrectly when inner name and outer name are the same in Parquet ## What changes were proposed in this pull request? Currently, if there is a schema as below: ``` root

spark git commit: [SPARK-16304] LinkageError should not crash Spark executor

Repository: spark Updated Branches: refs/heads/master 4e14199ff -> 480357cc6 [SPARK-16304] LinkageError should not crash Spark executor ## What changes were proposed in this pull request? This patch updates the failure handling logic so Spark executor does not crash when seeing LinkageError.

spark git commit: [MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation

Repository: spark Updated Branches: refs/heads/master b1310425b -> 4e14199ff [MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation ## What changes were proposed in this pull request? This PR fixes wrongly formatted examples in PySpark documentation as below: -

spark git commit: [MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation

Repository: spark Updated Branches: refs/heads/branch-2.0 091cd5f26 -> 03f336d89 [MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation ## What changes were proposed in this pull request? This PR fixes wrongly formatted examples in PySpark documentation as below: -

spark git commit: [DOC][SQL] update out-of-date code snippets using SQLContext in all documents.

Repository: spark Updated Branches: refs/heads/master 23eff5e51 -> b1310425b [DOC][SQL] update out-of-date code snippets using SQLContext in all documents. ## What changes were proposed in this pull request? I search the whole documents directory using SQLContext, and update the following

spark git commit: [DOC][SQL] update out-of-date code snippets using SQLContext in all documents.

Repository: spark Updated Branches: refs/heads/branch-2.0 e956bd775 -> 091cd5f26 [DOC][SQL] update out-of-date code snippets using SQLContext in all documents. ## What changes were proposed in this pull request? I search the whole documents directory using SQLContext, and update the

spark git commit: [SPARK-15979][SQL] Renames CatalystWriteSupport to ParquetWriteSupport

Repository: spark Updated Branches: refs/heads/master 478b71d02 -> 23eff5e51 [SPARK-15979][SQL] Renames CatalystWriteSupport to ParquetWriteSupport ## What changes were proposed in this pull request? PR #13696 renamed various Parquet support classes but left `CatalystWriteSupport` behind.

spark git commit: [SPARK-16383][SQL] Remove `SessionState.executeSql`

Repository: spark Updated Branches: refs/heads/master 1f0d02130 -> 4db63fd2b [SPARK-16383][SQL] Remove `SessionState.executeSql` ## What changes were proposed in this pull request? This PR removes `SessionState.executeSql` in favor of `SparkSession.sql`. We can remove this safely since the

spark git commit: [SPARK-15730][SQL] Respect the --hiveconf in the spark-sql command line

Repository: spark Updated Branches: refs/heads/master 5b7a1770a -> 920cb5fe4 [SPARK-15730][SQL] Respect the --hiveconf in the spark-sql command line ## What changes were proposed in this pull request? This PR makes spark-sql (backed by SparkSQLCLIDriver) respects confs set by hiveconf, which

spark git commit: [SPARK-15730][SQL] Respect the --hiveconf in the spark-sql command line

Repository: spark Updated Branches: refs/heads/branch-2.0 cabee2324 -> 9c1596b6c [SPARK-15730][SQL] Respect the --hiveconf in the spark-sql command line ## What changes were proposed in this pull request? This PR makes spark-sql (backed by SparkSQLCLIDriver) respects confs set by hiveconf,

spark git commit: [HOTFIX] Fix build break.

Repository: spark Updated Branches: refs/heads/master 1fca9da95 -> 5b7a1770a [HOTFIX] Fix build break. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5b7a1770 Tree:

spark git commit: [SPARK-16311][SQL] Metadata refresh should work on temporary views

<petermax...@gmail.com> Closes #14009 from rxin/SPARK-16311. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/16a2a7d7 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/16a2a7d7 Diff: http://git-wip-us.apache.org/repos/asf/spar

spark git commit: [SPARK-16311][SQL] Metadata refresh should work on temporary views

<petermax...@gmail.com> Closes #14009 from rxin/SPARK-16311. (cherry picked from commit 16a2a7d714f945b06978e3bd20a58ea32f0621ac) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: [SPARK-16335][SQL] Structured streaming should fail if source directory does not exist

<r...@databricks.com> Closes #14002 from rxin/SPARK-16335. (cherry picked from commit d601894c0494d415e7f330e02168c43a2dacfb02) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/

spark git commit: [SPARK-16335][SQL] Structured streaming should fail if source directory does not exist

<r...@databricks.com> Closes #14002 from rxin/SPARK-16335. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d601894c Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d601894c Diff: http://git-wip-us.apache.org/repos/asf/s

spark git commit: [SPARK-16331][SQL] Reduce code generation time

Repository: spark Updated Branches: refs/heads/master aa6564f37 -> 14cf61e90 [SPARK-16331][SQL] Reduce code generation time ## What changes were proposed in this pull request? During the code generation, a `LocalRelation` often has a huge `Vector` object as `data`. In the simple example

spark git commit: [SPARK-15954][SQL] Disable loading test tables in Python tests

How was this patch tested? This is a test only change. Author: Reynold Xin <r...@databricks.com> Closes #14005 from rxin/SPARK-15954. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/38f4d6f4 Tree: http://git-wip-us.apache.org/

spark git commit: [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing

ior because it can mask legitimate errors and the resulting plan will silently have 0 rows. This patch changes it to not silently drop the errors. ## How was this patch tested? Manually verified. Author: Reynold Xin <r...@databricks.com> Closes #13987 from rxin/SPARK-16313. (cherry pi

spark git commit: [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing

ior because it can mask legitimate errors and the resulting plan will silently have 0 rows. This patch changes it to not silently drop the errors. ## How was this patch tested? Manually verified. Author: Reynold Xin <r...@databricks.com> Closes #13987 from rxin/SPARK-16313. Project: http:

spark git commit: [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException

Repository: spark Updated Branches: refs/heads/branch-2.0 03008e049 -> 4dc7d377f [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException ## What changes were proposed in this pull request? This patch appends a message to suggest users running refresh table or reloading data

spark git commit: [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException

Repository: spark Updated Branches: refs/heads/master 5d00a7bc1 -> fb41670c9 [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException ## What changes were proposed in this pull request? This patch appends a message to suggest users running refresh table or reloading data

spark git commit: [SPARK-16289][SQL] Implement posexplode table generating function

Repository: spark Updated Branches: refs/heads/master fdf9f94f8 -> 46395db80 [SPARK-16289][SQL] Implement posexplode table generating function ## What changes were proposed in this pull request? This PR implements `posexplode` table generating function. Currently, master branch raises the

spark git commit: [SPARK-16267][TEST] Replace deprecated `CREATE TEMPORARY TABLE ... USING` from testsuites.

Repository: spark Updated Branches: refs/heads/branch-2.0 e1bdf1e02 -> b52bd8070 [SPARK-16267][TEST] Replace deprecated `CREATE TEMPORARY TABLE ... USING` from testsuites. ## What changes were proposed in this pull request? After SPARK-15674, `DDLStrategy` prints out the following

spark git commit: [SPARK-16267][TEST] Replace deprecated `CREATE TEMPORARY TABLE ... USING` from testsuites.

Repository: spark Updated Branches: refs/heads/master d063898be -> 831a04f5d [SPARK-16267][TEST] Replace deprecated `CREATE TEMPORARY TABLE ... USING` from testsuites. ## What changes were proposed in this pull request? After SPARK-15674, `DDLStrategy` prints out the following deprecation

spark git commit: [SPARK-16228][SQL] HiveSessionCatalog should return `double`-param functions for decimal param lookups

Repository: spark Updated Branches: refs/heads/branch-2.0 c4cebd572 -> 011befd20 [SPARK-16228][SQL] HiveSessionCatalog should return `double`-param functions for decimal param lookups ## What changes were proposed in this pull request? This PR supports a fallback lookup by casting

spark git commit: [SPARK-16238] Metrics for generated method and class bytecode size

Repository: spark Updated Branches: refs/heads/master 9b1b3ae77 -> 23c58653f [SPARK-16238] Metrics for generated method and class bytecode size ## What changes were proposed in this pull request? This extends SPARK-15860 to include metrics for the actual bytecode size of janino-generated

spark git commit: [SPARK-16238] Metrics for generated method and class bytecode size

Repository: spark Updated Branches: refs/heads/branch-2.0 ef0253ff6 -> c4cebd572 [SPARK-16238] Metrics for generated method and class bytecode size ## What changes were proposed in this pull request? This extends SPARK-15860 to include metrics for the actual bytecode size of

spark git commit: [SPARK-16006][SQL] Attemping to write empty DataFrame with no fields throws non-intuitive exception

Repository: spark Updated Branches: refs/heads/branch-2.0 a7f66ef62 -> ef0253ff6 [SPARK-16006][SQL] Attemping to write empty DataFrame with no fields throws non-intuitive exception ## What changes were proposed in this pull request? This PR allows `emptyDataFrame.write` since the user

spark git commit: [SPARK-16006][SQL] Attemping to write empty DataFrame with no fields throws non-intuitive exception

Repository: spark Updated Branches: refs/heads/master 8b5a8b25b -> 9b1b3ae77 [SPARK-16006][SQL] Attemping to write empty DataFrame with no fields throws non-intuitive exception ## What changes were proposed in this pull request? This PR allows `emptyDataFrame.write` since the user didn't

spark git commit: [SPARK-16044][SQL] Backport input_file_name() for data source based on NewHadoopRDD to branch 1.6

Repository: spark Updated Branches: refs/heads/branch-1.6 0cb06c993 -> 1ac830aca [SPARK-16044][SQL] Backport input_file_name() for data source based on NewHadoopRDD to branch 1.6 ## What changes were proposed in this pull request? This PR backports

spark git commit: [SPARK-14480][SQL] Remove meaningless StringIteratorReader for CSV data source.

Repository: spark Updated Branches: refs/heads/master 39f2eb1da -> cb1b9d34f [SPARK-14480][SQL] Remove meaningless StringIteratorReader for CSV data source. ## What changes were proposed in this pull request? This PR removes meaningless `StringIteratorReader` for CSV data source. In

spark git commit: [SPARK-16271][SQL] Implement Hive's UDFXPathUtil

Repository: spark Updated Branches: refs/heads/master 0df5ce1bc -> 153c2f9ac [SPARK-16271][SQL] Implement Hive's UDFXPathUtil ## What changes were proposed in this pull request? This patch ports Hive's UDFXPathUtil over to Spark, which can be used to implement xpath functionality in Spark in

spark git commit: [SPARK-16248][SQL] Whitelist the list of Hive fallback functions

ist and adds only a small number of functions to the whitelist, i.e. the ones we intend to support in the long run in Spark. ## How was this patch tested? Updated tests to reflect the change. Author: Reynold Xin <r...@databricks.com> Closes #13939 from rxin/hive-whitelist. (cherry pi

spark git commit: [SPARK-16248][SQL] Whitelist the list of Hive fallback functions

ist and adds only a small number of functions to the whitelist, i.e. the ones we intend to support in the long run in Spark. ## How was this patch tested? Updated tests to reflect the change. Author: Reynold Xin <r...@databricks.com> Closes #13939 from rxin/hive-whitelist. Project: http:

spark git commit: [SPARK-16236][SQL] Add Path Option back to Load API in DataFrameReader

Repository: spark Updated Branches: refs/heads/branch-2.0 43bd612f3 -> 5626a0af5 [SPARK-16236][SQL] Add Path Option back to Load API in DataFrameReader What changes were proposed in this pull request? koertkuipers identified the PR https://github.com/apache/spark/pull/13727/ changed the

spark git commit: [SPARK-16236][SQL] Add Path Option back to Load API in DataFrameReader

Repository: spark Updated Branches: refs/heads/master 35438fb0a -> 25520e976 [SPARK-16236][SQL] Add Path Option back to Load API in DataFrameReader What changes were proposed in this pull request? koertkuipers identified the PR https://github.com/apache/spark/pull/13727/ changed the

spark git commit: [SPARK-16259][PYSPARK] cleanup options in DataFrame read/write API

Repository: spark Updated Branches: refs/heads/master ae14f3623 -> 1aad8c6e5 [SPARK-16259][PYSPARK] cleanup options in DataFrame read/write API ## What changes were proposed in this pull request? There are some duplicated code for options in DataFrame reader/writer API, this PR clean them

spark git commit: [SPARK-16202][SQL][DOC] Correct The Description of CreatableRelationProvider's createRelation

Repository: spark Updated Branches: refs/heads/master dd6b7dbe7 -> 4cbf611c1 [SPARK-16202][SQL][DOC] Correct The Description of CreatableRelationProvider's createRelation What changes were proposed in this pull request? The API description of `createRelation` in

spark git commit: [SPARK-15863][SQL][DOC][FOLLOW-UP] Update SQL programming guide.

Repository: spark Updated Branches: refs/heads/master a0da854fb -> dd6b7dbe7 [SPARK-15863][SQL][DOC][FOLLOW-UP] Update SQL programming guide. ## What changes were proposed in this pull request? This PR makes several updates to SQL programming guide. Author: Yin Huai

spark git commit: [SPARK-15863][SQL][DOC][FOLLOW-UP] Update SQL programming guide.

Repository: spark Updated Branches: refs/heads/branch-2.0 7177e1843 -> af70ad028 [SPARK-15863][SQL][DOC][FOLLOW-UP] Update SQL programming guide. ## What changes were proposed in this pull request? This PR makes several updates to SQL programming guide. Author: Yin Huai

spark git commit: [SPARK-16111][SQL][DOC] Hide SparkOrcNewRecordReader in API docs

Repository: spark Updated Branches: refs/heads/branch-2.0 4c4f7775c -> 7177e1843 [SPARK-16111][SQL][DOC] Hide SparkOrcNewRecordReader in API docs ## What changes were proposed in this pull request? Currently, Spark Scala/Java API documents shows **org.apache.hadoop.hive.ql.io.orc** package

spark git commit: [SPARK-16111][SQL][DOC] Hide SparkOrcNewRecordReader in API docs

Repository: spark Updated Branches: refs/heads/master 1b7fc5817 -> 50fdd866b [SPARK-16111][SQL][DOC] Hide SparkOrcNewRecordReader in API docs ## What changes were proposed in this pull request? Currently, Spark Scala/Java API documents shows **org.apache.hadoop.hive.ql.io.orc** package at

spark git commit: [SPARK-16220][SQL] Add scope to show functions

Repository: spark Updated Branches: refs/heads/branch-2.0 27f3462d0 -> c7704099d [SPARK-16220][SQL] Add scope to show functions ## What changes were proposed in this pull request? Spark currently shows all functions when issue a `SHOW FUNCTIONS` command. This PR refines the `SHOW FUNCTIONS`

spark git commit: [SPARK-16220][SQL] Add scope to show functions

Repository: spark Updated Branches: refs/heads/master c15b552dd -> 02a029df4 [SPARK-16220][SQL] Add scope to show functions ## What changes were proposed in this pull request? Spark currently shows all functions when issue a `SHOW FUNCTIONS` command. This PR refines the `SHOW FUNCTIONS`

svn commit: r1750426 [2/2] - in /spark/site: ./ graphx/ mllib/ news/ releases/ screencasts/ sql/ streaming/

Modified: spark/site/screencasts/3-transformations-and-caching.html URL: http://svn.apache.org/viewvc/spark/site/screencasts/3-transformations-and-caching.html?rev=1750426=1750425=1750426=diff == ---

svn commit: r1750426 [1/2] - in /spark/site: ./ graphx/ mllib/ news/ releases/ screencasts/ sql/ streaming/

Author: rxin Date: Mon Jun 27 23:35:03 2016 New Revision: 1750426 URL: http://svn.apache.org/viewvc?rev=1750426=rev Log: update doc header Modified: spark/site/community.html spark/site/documentation.html spark/site/downloads.html spark/site/examples.html spark/site/faq.html

svn commit: r1750425 - /spark/_layouts/global.html

Author: rxin Date: Mon Jun 27 23:34:10 2016 New Revision: 1750425 URL: http://svn.apache.org/viewvc?rev=1750425=rev Log: update doc header Modified: spark/_layouts/global.html Modified: spark/_layouts/global.html URL: http://svn.apache.org/viewvc/spark/_layouts/global.html?rev=1750425

svn commit: r1750424 - in /spark: documentation.md site/docs/latest site/documentation.html

Author: rxin Date: Mon Jun 27 23:32:42 2016 New Revision: 1750424 URL: http://svn.apache.org/viewvc?rev=1750424=rev Log: doc update for 1.6.2 Modified: spark/documentation.md spark/site/docs/latest spark/site/documentation.html Modified: spark/documentation.md URL: http

svn commit: r1750423 [3/3] - in /spark: ./ js/ news/_posts/ releases/_posts/ site/ site/graphx/ site/js/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

Modified: spark/site/releases/spark-release-1-1-0.html URL: http://svn.apache.org/viewvc/spark/site/releases/spark-release-1-1-0.html?rev=1750423=1750422=1750423=diff == --- spark/site/releases/spark-release-1-1-0.html

svn commit: r1750423 [1/3] - in /spark: ./ js/ news/_posts/ releases/_posts/ site/ site/graphx/ site/js/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

Author: rxin Date: Mon Jun 27 23:22:21 2016 New Revision: 1750423 URL: http://svn.apache.org/viewvc?rev=1750423=rev Log: Add 1.6.2 release Added: spark/news/_posts/2016-06-25-spark-1-6-2-released.md spark/releases/_posts/2016-06-25-spark-release-1-6-2.md spark/site/news/spark-1-6-2

svn commit: r1750423 [2/3] - in /spark: ./ js/ news/_posts/ releases/_posts/ site/ site/graphx/ site/js/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

Added: spark/site/news/spark-1-6-2-released.html URL: http://svn.apache.org/viewvc/spark/site/news/spark-1-6-2-released.html?rev=1750423=auto == --- spark/site/news/spark-1-6-2-released.html (added) +++

svn commit: r1750186 - in /spark/site/docs/1.6.2: ./ api/ api/R/ api/java/ api/java/lib/ api/java/org/ api/java/org/apache/ api/java/org/apache/spark/ api/java/org/apache/spark/annotation/ api/java/or

Author: rxin Date: Sat Jun 25 08:02:58 2016 New Revision: 1750186 URL: http://svn.apache.org/viewvc?rev=1750186=rev Log: add 1.6.2 doc [This commit notification would consist of 931 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

svn commit: r14146 - /dev/spark/spark-1.6.2/ /release/spark/spark-1.6.2/

Author: rxin Date: Sat Jun 25 07:16:01 2016 New Revision: 14146 Log: Spark 1.6.2 release Added: release/spark/spark-1.6.2/ - copied from r14145, dev/spark/spark-1.6.2/ Removed: dev/spark/spark-1.6.2

svn commit: r14145 - /dev/spark/spark-1.6.2/

Author: rxin Date: Sat Jun 25 07:07:25 2016 New Revision: 14145 Log: Add spark-1.6.2 Added: dev/spark/spark-1.6.2/ dev/spark/spark-1.6.2/spark-1.6.2-bin-cdh4.tgz (with props) dev/spark/spark-1.6.2/spark-1.6.2-bin-cdh4.tgz.asc dev/spark/spark-1.6.2/spark-1.6.2-bin-cdh4.tgz.md5

[spark] Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.6.2-rc1 [deleted] 4168d9c94 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

<    5   6   7   8   9   10   11   12   13   14   >