spark git commit: [SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter

2016-07-11 Thread rxin
Repository: spark Updated Branches: refs/heads/master b1e5281c5 -> c9a676215 [SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter ## What changes were proposed in this pull request? It would be useful to support listing the columns that are referenced by a

spark git commit: [SPARK-12639][SQL] Mark Filters Fully Handled By Sources with *

2016-07-11 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 9cc74f95e -> b1e5281c5 [SPARK-12639][SQL] Mark Filters Fully Handled By Sources with * ## What changes were proposed in this pull request? In order to make it clear which filters are fully handled by the underlying datasource we will mark

spark git commit: [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy

2016-07-11 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 b37177c22 -> 689261465 [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy This patch fixes a variable namespace collision bug in pmod and partitionBy Regression test for one possible occurrence. A more

spark git commit: [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy

2016-07-11 Thread rxin
Repository: spark Updated Branches: refs/heads/master e50efd53f -> 9cc74f95e [SPARK-16488] Fix codegen variable namespace collision in pmod and partitionBy ## What changes were proposed in this pull request? This patch fixes a variable namespace collision bug in pmod and partitionBy ## How

spark git commit: [SPARK-16430][SQL][STREAMING] Fixed bug in the maxFilesPerTrigger in FileStreamSource

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 b716e104b -> b37177c22 [SPARK-16430][SQL][STREAMING] Fixed bug in the maxFilesPerTrigger in FileStreamSource ## What changes were proposed in this pull request? Incorrect list of files were being allocated to a batch. This caused a

spark git commit: [SPARK-16430][SQL][STREAMING] Fixed bug in the maxFilesPerTrigger in FileStreamSource

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master 91a443b84 -> e50efd53f [SPARK-16430][SQL][STREAMING] Fixed bug in the maxFilesPerTrigger in FileStreamSource ## What changes were proposed in this pull request? Incorrect list of files were being allocated to a batch. This caused a file

spark git commit: [SPARK-16433][SQL] Improve StreamingQuery.explain when no data arrives

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 81d7f484a -> b716e104b [SPARK-16433][SQL] Improve StreamingQuery.explain when no data arrives ## What changes were proposed in this pull request? Display `No physical plan. Waiting for data.` instead of `N/A` for

spark git commit: [SPARK-16433][SQL] Improve StreamingQuery.explain when no data arrives

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master 05d7151cc -> 91a443b84 [SPARK-16433][SQL] Improve StreamingQuery.explain when no data arrives ## What changes were proposed in this pull request? Display `No physical plan. Waiting for data.` instead of `N/A` for StreamingQuery.explain

spark git commit: [MINOR][STREAMING][DOCS] Minor changes on kinesis integration

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master 9e2c763db -> 05d7151cc [MINOR][STREAMING][DOCS] Minor changes on kinesis integration ## What changes were proposed in this pull request? Some minor changes for documentation page "Spark Streaming + Kinesis Integration". Moved

spark git commit: [MINOR][STREAMING][DOCS] Minor changes on kinesis integration

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 02d584ccb -> 81d7f484a [MINOR][STREAMING][DOCS] Minor changes on kinesis integration ## What changes were proposed in this pull request? Some minor changes for documentation page "Spark Streaming + Kinesis Integration". Moved

spark git commit: [SPARK-16114][SQL] structured streaming event time window example

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 cb463b6db -> 02d584ccb [SPARK-16114][SQL] structured streaming event time window example ## What changes were proposed in this pull request? A structured streaming example with event time windowing. ## How was this patch tested? Run

spark git commit: [SPARK-16114][SQL] structured streaming event time window example

2016-07-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master b4fbe140b -> 9e2c763db [SPARK-16114][SQL] structured streaming event time window example ## What changes were proposed in this pull request? A structured streaming example with event time windowing. ## How was this patch tested? Run

spark git commit: [SPARK-16349][SQL] Fall back to isolated class loader when classes not found.

2016-07-11 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 7f38b9d5f -> b4fbe140b [SPARK-16349][SQL] Fall back to isolated class loader when classes not found. Some Hadoop classes needed by the Hive metastore client jars are not present in Spark's packaging (for example,

spark git commit: [SPARK-16144][SPARKR] update R API doc for mllib

2016-07-11 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 b938ca76e -> cb463b6db [SPARK-16144][SPARKR] update R API doc for mllib ## What changes were proposed in this pull request? >From SPARK-16140/PR #13921 - the issue is we left write.ml doc empty:

spark git commit: [SPARK-16144][SPARKR] update R API doc for mllib

2016-07-11 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 2ad031be6 -> 7f38b9d5f [SPARK-16144][SPARKR] update R API doc for mllib ## What changes were proposed in this pull request? >From SPARK-16140/PR #13921 - the issue is we left write.ml doc empty:

spark git commit: [SPARKR][DOC] SparkR ML user guides update for 2.0

2016-07-11 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 aea33bf05 -> b938ca76e [SPARKR][DOC] SparkR ML user guides update for 2.0 ## What changes were proposed in this pull request? * Update SparkR ML section to make them consistent with SparkR API docs. * Since #13972 adds labelling

spark git commit: [SPARKR][DOC] SparkR ML user guides update for 2.0

2016-07-11 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 840853ed0 -> 2ad031be6 [SPARKR][DOC] SparkR ML user guides update for 2.0 ## What changes were proposed in this pull request? * Update SparkR ML section to make them consistent with SparkR API docs. * Since #13972 adds labelling support

spark git commit: [SPARK-16385][CORE] Catch correct exception when calling method via reflection.

2016-07-11 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 bb92788f9 -> 702178d1f [SPARK-16385][CORE] Catch correct exception when calling method via reflection. Using "Method.invoke" causes an exception to be thrown, not an error, so Utils.waitForProcess() was always throwing an exception

spark git commit: [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables

2016-07-11 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 72cf74324 -> aea33bf05 [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables ## What changes were proposed in this pull request? Temporary tables are used frequently, but `spark.catalog.listColumns` does

spark git commit: [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables

2016-07-11 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master ffcb6e055 -> 840853ed0 [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables ## What changes were proposed in this pull request? Temporary tables are used frequently, but `spark.catalog.listColumns` does not

spark git commit: [SPARK-16318][SQL] Implement all remaining xpath functions (branch-2.0)

2016-07-11 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 f97dd8a8f -> 72cf74324 [SPARK-16318][SQL] Implement all remaining xpath functions (branch-2.0) ## What changes were proposed in this pull request? This patch implements all remaining xpath functions that Hive supports and not natively

spark git commit: [SPARK-16459][SQL] Prevent dropping current database

2016-07-11 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 7e4ba66d9 -> f97dd8a8f [SPARK-16459][SQL] Prevent dropping current database This PR prevents dropping the current database to avoid errors like the followings. ```scala scala> sql("create database delete_db") scala> sql("use

spark git commit: [SPARK-16477] Bump master version to 2.1.0-SNAPSHOT

2016-07-11 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7ac79da0e -> ffcb6e055 [SPARK-16477] Bump master version to 2.1.0-SNAPSHOT ## What changes were proposed in this pull request? After SPARK-16476 (committed earlier today as #14128), we can finally bump the version number. ## How was this

spark git commit: [SPARK-16459][SQL] Prevent dropping current database

2016-07-11 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 9cb1eb7af -> 7ac79da0e [SPARK-16459][SQL] Prevent dropping current database ## What changes were proposed in this pull request? This PR prevents dropping the current database to avoid errors like the followings. ```scala scala>

spark git commit: [SPARK-16381][SQL][SPARKR] Update SQL examples and programming guide for R language binding

2016-07-11 Thread lian
Repository: spark Updated Branches: refs/heads/branch-2.0 aa8cbcd19 -> 7e4ba66d9 [SPARK-16381][SQL][SPARKR] Update SQL examples and programming guide for R language binding https://issues.apache.org/jira/browse/SPARK-16381 ## What changes were proposed in this pull request? Update SQL

spark git commit: [SPARK-16381][SQL][SPARKR] Update SQL examples and programming guide for R language binding

2016-07-11 Thread lian
Repository: spark Updated Branches: refs/heads/master e22627894 -> 9cb1eb7af [SPARK-16381][SQL][SPARKR] Update SQL examples and programming guide for R language binding https://issues.apache.org/jira/browse/SPARK-16381 ## What changes were proposed in this pull request? Update SQL examples

spark git commit: [SPARK-16355][SPARK-16354][SQL] Fix Bugs When LIMIT/TABLESAMPLE is Non-foldable, Zero or Negative

2016-07-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 139d5eae0 -> aa8cbcd19 [SPARK-16355][SPARK-16354][SQL] Fix Bugs When LIMIT/TABLESAMPLE is Non-foldable, Zero or Negative What changes were proposed in this pull request? **Issue 1:** When a query containing LIMIT/TABLESAMPLE 0,

spark git commit: [SPARK-16355][SPARK-16354][SQL] Fix Bugs When LIMIT/TABLESAMPLE is Non-foldable, Zero or Negative

2016-07-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 82f087445 -> e22627894 [SPARK-16355][SPARK-16354][SQL] Fix Bugs When LIMIT/TABLESAMPLE is Non-foldable, Zero or Negative What changes were proposed in this pull request? **Issue 1:** When a query containing LIMIT/TABLESAMPLE 0, the