spark git commit: [MINOR][SPARKR] Add run command comment in examples

2017-03-29 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 79636054f -> 471de5db5 [MINOR][SPARKR] Add run command comment in examples ## What changes were proposed in this pull request? There are two examples in r folder missing the run commands. In this PR, I just add the missing comment, which

spark git commit: [SPARK-20148][SQL] Extend the file commit API to allow subscribing to task commit messages

2017-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master 60977889e -> 79636054f [SPARK-20148][SQL] Extend the file commit API to allow subscribing to task commit messages ## What changes were proposed in this pull request? The internal FileCommitProtocol interface returns all task commit messag

spark git commit: [SPARK-20136][SQL] Add num files and metadata operation timing to scan operator metrics

2017-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master 22f07fefe -> 60977889e [SPARK-20136][SQL] Add num files and metadata operation timing to scan operator metrics ## What changes were proposed in this pull request? This patch adds explicit metadata operation timing and number of files in da

spark git commit: [SPARK-20146][SQL] fix comment missing issue for thrift server

2017-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master dd2e7d528 -> 22f07fefe [SPARK-20146][SQL] fix comment missing issue for thrift server ## What changes were proposed in this pull request? The column comment was missing while constructing the Hive TableSchema. This fix will preserve the o

spark git commit: [SPARK-19088][SQL] Fix 2.10 build.

2017-03-29 Thread ueshin
Repository: spark Updated Branches: refs/heads/master fe1d6b05d -> dd2e7d528 [SPARK-19088][SQL] Fix 2.10 build. ## What changes were proposed in this pull request? Commit 6c70a38 broke the build for scala 2.10. The commit uses some reflections which are not available in Scala 2.10. This PR f

spark git commit: [SPARK-20120][SQL] spark-sql support silent mode

2017-03-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 5c8ef376e -> fe1d6b05d [SPARK-20120][SQL] spark-sql support silent mode ## What changes were proposed in this pull request? It is similar to Hive silent mode, just show the query result. see: [Hive LanguageManual+Cli](https://cwiki.apache

spark git commit: [SPARK-17075][SQL][FOLLOWUP] Add Estimation of Constant Literal

2017-03-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/master c4008480b -> 5c8ef376e [SPARK-17075][SQL][FOLLOWUP] Add Estimation of Constant Literal ### What changes were proposed in this pull request? `FalseLiteral` and `TrueLiteral` should have been eliminated by optimizer rule `BooleanSimplificati

spark git commit: [SPARK-20009][SQL] Support DDL strings for defining schema in functions.from_json

2017-03-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 142f6d149 -> c4008480b [SPARK-20009][SQL] Support DDL strings for defining schema in functions.from_json ## What changes were proposed in this pull request? This pr added `StructType.fromDDL` to convert a DDL format string into `StructTy

spark git commit: [SPARK-20048][SQL] Cloning SessionState does not clone query execution listeners

2017-03-29 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master d6ddfdf60 -> 142f6d149 [SPARK-20048][SQL] Cloning SessionState does not clone query execution listeners ## What changes were proposed in this pull request? Bugfix from [SPARK-19540.](https://github.com/apache/spark/pull/16826) Cloning Sess

spark git commit: [SPARK-19955][PYSPARK] Jenkins Python Conda based test.

2017-03-29 Thread holden
Repository: spark Updated Branches: refs/heads/master c622a87c4 -> d6ddfdf60 [SPARK-19955][PYSPARK] Jenkins Python Conda based test. ## What changes were proposed in this pull request? Allow Jenkins Python tests to use the installed conda to test Python 2.7 support & test pip installability.

spark git commit: [SPARK-20059][YARN] Use the correct classloader for HBaseCredentialProvider

2017-03-29 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.1 f8c1b3e20 -> 103ff54db [SPARK-20059][YARN] Use the correct classloader for HBaseCredentialProvider ## What changes were proposed in this pull request? Currently we use system classloader to find HBase jars, if it is specified by `--ja

spark git commit: [SPARK-20059][YARN] Use the correct classloader for HBaseCredentialProvider

2017-03-29 Thread vanzin
Repository: spark Updated Branches: refs/heads/master b56ad2b1e -> c622a87c4 [SPARK-20059][YARN] Use the correct classloader for HBaseCredentialProvider ## What changes were proposed in this pull request? Currently we use system classloader to find HBase jars, if it is specified by `--jars`,

spark git commit: [SPARK-19556][CORE] Do not encrypt block manager data in memory.

2017-03-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 9712bd395 -> b56ad2b1e [SPARK-19556][CORE] Do not encrypt block manager data in memory. This change modifies the way block data is encrypted to make the more common cases faster, while penalizing an edge case. As a side effect of the change

spark git commit: [SPARK-20134][SQL] SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates

2017-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master a5c87707e -> 9712bd395 [SPARK-20134][SQL] SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates ## What changes were proposed in this pull request? It is not super intuitive how to update SQLMetric on the driver side. T

spark git commit: [SPARK-20134][SQL] SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates

2017-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 30954806f -> f8c1b3e20 [SPARK-20134][SQL] SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates ## What changes were proposed in this pull request? It is not super intuitive how to update SQLMetric on the driver sid