spark git commit: [SPARK-12922][SPARKR][WIP] Implement gapply() on DataFrame in SparkR

Repository: spark Updated Branches: refs/heads/branch-2.0 f0279b05c -> 35c0a60a6 [SPARK-12922][SPARKR][WIP] Implement gapply() on DataFrame in SparkR ## What changes were proposed in this pull request? gapply() applies an R function on groups grouped by one or more columns of a DataFrame,

spark git commit: [SPARK-12922][SPARKR][WIP] Implement gapply() on DataFrame in SparkR

Repository: spark Updated Branches: refs/heads/master b75f454f9 -> 7c6c69263 [SPARK-12922][SPARKR][WIP] Implement gapply() on DataFrame in SparkR ## What changes were proposed in this pull request? gapply() applies an R function on groups grouped by one or more columns of a DataFrame, and

spark git commit: [SPARK-15824][SQL] Execute WITH .... INSERT ... statements immediately

Repository: spark Updated Branches: refs/heads/branch-2.0 61738a38a -> f0279b05c [SPARK-15824][SQL] Execute WITH INSERT ... statements immediately ## What changes were proposed in this pull request? We currently immediately execute `INSERT` commands when they are issued. This is not the

spark git commit: [SPARK-15824][SQL] Execute WITH .... INSERT ... statements immediately

Repository: spark Updated Branches: refs/heads/master 5a52ba0f9 -> b75f454f9 [SPARK-15824][SQL] Execute WITH INSERT ... statements immediately ## What changes were proposed in this pull request? We currently immediately execute `INSERT` commands when they are issued. This is not the

spark git commit: [SPARK-15851][BUILD] Fix the call of the bash script to enable proper run in Windows

Repository: spark Updated Branches: refs/heads/branch-2.0 21daee439 -> cb3bb1901 [SPARK-15851][BUILD] Fix the call of the bash script to enable proper run in Windows ## What changes were proposed in this pull request? The way bash script `build/spark-build-info` is called from core/pom.xml

spark git commit: [SPARK-15851][BUILD] Fix the call of the bash script to enable proper run in Windows

Repository: spark Updated Branches: refs/heads/master ebdd75127 -> 5a52ba0f9 [SPARK-15851][BUILD] Fix the call of the bash script to enable proper run in Windows ## What changes were proposed in this pull request? The way bash script `build/spark-build-info` is called from core/pom.xml

spark git commit: [SPARK-13498][SQL] Increment the recordsRead input metric for JDBC data source

Repository: spark Updated Branches: refs/heads/master 865e7cc38 -> ebdd75127 [SPARK-13498][SQL] Increment the recordsRead input metric for JDBC data source ## What changes were proposed in this pull request? This patch brings https://github.com/apache/spark/pull/11373 up-to-date and

[2/2] spark git commit: [SPARK-15979][SQL] Rename various Parquet support classes.

[SPARK-15979][SQL] Rename various Parquet support classes. ## What changes were proposed in this pull request? This patch renames various Parquet support classes from CatalystAbc to ParquetAbc. This new naming makes more sense for two reasons: 1. These are not optimizer related (i.e. Catalyst)

spark git commit: [SPARK-12492][SQL] Add missing SQLExecution.withNewExecutionId for hiveResultString

Repository: spark Updated Branches: refs/heads/branch-2.0 382735c41 -> bc83b09ee [SPARK-12492][SQL] Add missing SQLExecution.withNewExecutionId for hiveResultString ## What changes were proposed in this pull request? Add missing SQLExecution.withNewExecutionId for hiveResultString so that

spark git commit: [DOCS] Fix Gini and Entropy scaladocs in context of multiclass classification

Repository: spark Updated Branches: refs/heads/branch-2.0 9acf254ed -> 382735c41 [DOCS] Fix Gini and Entropy scaladocs in context of multiclass classification The PR changes outdated scaladocs for Gini and Entropy classes. Since PR #886 Spark supports multiclass classification, but the docs

spark git commit: Revert "[SPARK-15782][YARN] Set spark.jars system property in client mode"

Repository: spark Updated Branches: refs/heads/master 1a33f2e05 -> a153e41c0 Revert "[SPARK-15782][YARN] Set spark.jars system property in client mode" This reverts commit 4df8df5c2e68f5a5d231c401b04d762d7a648159. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-7848][STREAMING][UPDATE SPARKSTREAMING DOCS TO INCORPORATE IMPORTANT POINTS.]

Repository: spark Updated Branches: refs/heads/master cafc696d0 -> 04d7b3d2b [SPARK-7848][STREAMING][UPDATE SPARKSTREAMING DOCS TO INCORPORATE IMPORTANT POINTS.] Updated the SparkStreaming Doc with some important points. Author: Nirman Narang Closes #4 from

spark git commit: [SPARK-7848][STREAMING][UPDATE SPARKSTREAMING DOCS TO INCORPORATE IMPORTANT POINTS.]

Repository: spark Updated Branches: refs/heads/branch-2.0 2c1aae442 -> 73bf87f3c [SPARK-7848][STREAMING][UPDATE SPARKSTREAMING DOCS TO INCORPORATE IMPORTANT POINTS.] Updated the SparkStreaming Doc with some important points. Author: Nirman Narang Closes #4 from

spark git commit: [HOTFIX][CORE] fix flaky BasicSchedulerIntegrationTest

Repository: spark Updated Branches: refs/heads/master 9bd80ad6b -> cafc696d0 [HOTFIX][CORE] fix flaky BasicSchedulerIntegrationTest ## What changes were proposed in this pull request? SPARK-15927 exacerbated a race in BasicSchedulerIntegrationTest, so it went from very unlikely to fairly

spark git commit: [SPARK-15776][SQL] Divide Expression inside Aggregation function is casted to wrong type

Repository: spark Updated Branches: refs/heads/branch-2.0 fe8ac729e -> 2c1aae442 [SPARK-15776][SQL] Divide Expression inside Aggregation function is casted to wrong type ## What changes were proposed in this pull request? This PR fixes the problem that Divide Expression inside Aggregation

spark git commit: [SPARK-15776][SQL] Divide Expression inside Aggregation function is casted to wrong type

Repository: spark Updated Branches: refs/heads/master 049e639fc -> 9bd80ad6b [SPARK-15776][SQL] Divide Expression inside Aggregation function is casted to wrong type ## What changes were proposed in this pull request? This PR fixes the problem that Divide Expression inside Aggregation

spark git commit: [SPARK-15934] [SQL] Return binary mode in ThriftServer

Repository: spark Updated Branches: refs/heads/master 09925735b -> 049e639fc [SPARK-15934] [SQL] Return binary mode in ThriftServer Returning binary mode to ThriftServer for backward compatibility. Tested with Squirrel and Tableau. Author: Egor Pakhomov Closes #13667

spark git commit: [SPARK-15934] [SQL] Return binary mode in ThriftServer

Repository: spark Updated Branches: refs/heads/branch-2.0 4253ba71b -> fe8ac729e [SPARK-15934] [SQL] Return binary mode in ThriftServer Returning binary mode to ThriftServer for backward compatibility. Tested with Squirrel and Tableau. Author: Egor Pakhomov Closes

spark git commit: [SPARK-15901][SQL][TEST] Verification of CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

Repository: spark Updated Branches: refs/heads/branch-2.0 f1e9d2d92 -> 4253ba71b [SPARK-15901][SQL][TEST] Verification of CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET What changes were proposed in this pull request? So far, we do not have test cases for verifying whether the

spark git commit: [SPARK-15901][SQL][TEST] Verification of CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

Repository: spark Updated Branches: refs/heads/master 4df8df5c2 -> 09925735b [SPARK-15901][SQL][TEST] Verification of CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET What changes were proposed in this pull request? So far, we do not have test cases for verifying whether the external

spark git commit: [SPARK-15782][YARN] Set spark.jars system property in client mode

Repository: spark Updated Branches: refs/heads/branch-2.0 b8e1b7c8a -> f1e9d2d92 [SPARK-15782][YARN] Set spark.jars system property in client mode ## What changes were proposed in this pull request? When `--packages` is specified with `spark-shell` the classes from those packages cannot be

spark git commit: [SPARK-15782][YARN] Set spark.jars system property in client mode

Repository: spark Updated Branches: refs/heads/master 5389013ac -> 4df8df5c2 [SPARK-15782][YARN] Set spark.jars system property in client mode ## What changes were proposed in this pull request? When `--packages` is specified with `spark-shell` the classes from those packages cannot be

spark git commit: [SPARK-15888] [SQL] fix Python UDF with aggregate

Repository: spark Updated Branches: refs/heads/branch-2.0 8ef31fbd7 -> b8e1b7c8a [SPARK-15888] [SQL] fix Python UDF with aggregate ## What changes were proposed in this pull request? After we move the ExtractPythonUDF rule into physical plan, Python UDF can't work on top of aggregate

spark git commit: [SPARK-15888] [SQL] fix Python UDF with aggregate

Repository: spark Updated Branches: refs/heads/master 279bd4aa5 -> 5389013ac [SPARK-15888] [SQL] fix Python UDF with aggregate ## What changes were proposed in this pull request? After we move the ExtractPythonUDF rule into physical plan, Python UDF can't work on top of aggregate anymore,

spark git commit: [SPARK-15826][CORE] PipedRDD to allow configurable char encoding

Repository: spark Updated Branches: refs/heads/master 9b234b55d -> 279bd4aa5 [SPARK-15826][CORE] PipedRDD to allow configurable char encoding ## What changes were proposed in this pull request? Link to jira which describes the problem: https://issues.apache.org/jira/browse/SPARK-15826 The

spark git commit: [SPARK-15826][CORE] PipedRDD to allow configurable char encoding

Repository: spark Updated Branches: refs/heads/branch-2.0 de56ea9bf -> 8ef31fbd7 [SPARK-15826][CORE] PipedRDD to allow configurable char encoding ## What changes were proposed in this pull request? Link to jira which describes the problem: https://issues.apache.org/jira/browse/SPARK-15826

spark git commit: [SPARK-15518][CORE][FOLLOW-UP] Rename LocalSchedulerBackendEndpoint -> LocalSchedulerBackend

Repository: spark Updated Branches: refs/heads/master e1585cc74 -> 9b234b55d [SPARK-15518][CORE][FOLLOW-UP] Rename LocalSchedulerBackendEndpoint -> LocalSchedulerBackend ## What changes were proposed in this pull request? This patch is a follow-up to

spark git commit: [SPARK-15518][CORE][FOLLOW-UP] Rename LocalSchedulerBackendEndpoint -> LocalSchedulerBackend

Repository: spark Updated Branches: refs/heads/branch-2.0 eb1d746c4 -> de56ea9bf [SPARK-15518][CORE][FOLLOW-UP] Rename LocalSchedulerBackendEndpoint -> LocalSchedulerBackend ## What changes were proposed in this pull request? This patch is a follow-up to

spark git commit: [SPARK-15959][SQL] Add the support of hive.metastore.warehouse.dir back

Repository: spark Updated Branches: refs/heads/master 9a5071996 -> e1585cc74 [SPARK-15959][SQL] Add the support of hive.metastore.warehouse.dir back ## What changes were proposed in this pull request? This PR adds the support of conf `hive.metastore.warehouse.dir` back. With this patch, the

[1/3] spark git commit: [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery

Repository: spark Updated Branches: refs/heads/branch-2.0 4c950a757 -> 885e74a38 http://git-wip-us.apache.org/repos/asf/spark/blob/885e74a3/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala

[3/3] spark git commit: [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery

[SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery Renamed for simplicity, so that its obvious that its related to streaming. Existing unit tests. Author: Tathagata Das Closes #13673 from tdas/SPARK-15953. Project:

[2/3] spark git commit: [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery

http://git-wip-us.apache.org/repos/asf/spark/blob/885e74a3/sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListener.scala -- diff --git

[3/3] spark git commit: [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery

[SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery Renamed for simplicity, so that its obvious that its related to streaming. Existing unit tests. Author: Tathagata Das Closes #13673 from tdas/SPARK-15953. (cherry picked from commit

[2/3] spark git commit: [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery

http://git-wip-us.apache.org/repos/asf/spark/blob/9a507199/sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListener.scala -- diff --git

[1/3] spark git commit: [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery

Repository: spark Updated Branches: refs/heads/master d30b7e669 -> 9a5071996 http://git-wip-us.apache.org/repos/asf/spark/blob/9a507199/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala

spark git commit: [SPARK-15637][SPARK-15931][SPARKR] Fix R masked functions checks

Repository: spark Updated Branches: refs/heads/master de99c3d08 -> d30b7e669 [SPARK-15637][SPARK-15931][SPARKR] Fix R masked functions checks ## What changes were proposed in this pull request? Because of the fix in SPARK-15684, this exclusion is no longer necessary. ## How was this patch

spark git commit: [SPARK-15637][SPARK-15931][SPARKR] Fix R masked functions checks

Repository: spark Updated Branches: refs/heads/branch-2.0 5c53442cc -> 4c950a757 [SPARK-15637][SPARK-15931][SPARKR] Fix R masked functions checks ## What changes were proposed in this pull request? Because of the fix in SPARK-15684, this exclusion is no longer necessary. ## How was this

spark git commit: [SPARK-15915][SQL] Logical plans should use subqueries eliminated plan when override sameResult.

Repository: spark Updated Branches: refs/heads/branch-1.6 2f3e327c4 -> cffc0800b [SPARK-15915][SQL] Logical plans should use subqueries eliminated plan when override sameResult. ## What changes were proposed in this pull request? This pr is a backport of #13638 for `branch-1.6`. ## How was

spark git commit: [SPARK-15960][SQL] Rename `spark.sql.enableFallBackToHdfsForStats` config

Repository: spark Updated Branches: refs/heads/master 40eeef952 -> de99c3d08 [SPARK-15960][SQL] Rename `spark.sql.enableFallBackToHdfsForStats` config ## What changes were proposed in this pull request? Since we are probably going to add more statistics related configurations in the future,

spark git commit: [SPARK-15960][SQL] Rename `spark.sql.enableFallBackToHdfsForStats` config

Repository: spark Updated Branches: refs/heads/branch-2.0 7a0ed75ea -> 5c53442cc [SPARK-15960][SQL] Rename `spark.sql.enableFallBackToHdfsForStats` config ## What changes were proposed in this pull request? Since we are probably going to add more statistics related configurations in the

spark git commit: [SPARK-15046][YARN] Parse value of token renewal interval correctly.

Repository: spark Updated Branches: refs/heads/branch-2.0 df9a19fe8 -> 7a0ed75ea [SPARK-15046][YARN] Parse value of token renewal interval correctly. Use the config variable definition both to set and parse the value, avoiding issues with code expecting the value in a different format.

spark git commit: [SPARK-15046][YARN] Parse value of token renewal interval correctly.

Repository: spark Updated Branches: refs/heads/master 0ee9fd9e5 -> 40eeef952 [SPARK-15046][YARN] Parse value of token renewal interval correctly. Use the config variable definition both to set and parse the value, avoiding issues with code expecting the value in a different format. Tested by