spark git commit: [SPARK-16632][SQL] Respect Hive schema when merging parquet schema.

2016-07-19 Thread lian
Repository: spark Updated Branches: refs/heads/branch-2.0 6f209c8fa -> c2b5b3ca5 [SPARK-16632][SQL] Respect Hive schema when merging parquet schema. When Hive (or at least certain versions of Hive) creates parquet files containing tinyint or smallint columns, it stores them as int32, but

spark git commit: [SPARK-16632][SQL] Respect Hive schema when merging parquet schema.

2016-07-19 Thread lian
Repository: spark Updated Branches: refs/heads/master fc2326362 -> 75146be6b [SPARK-16632][SQL] Respect Hive schema when merging parquet schema. When Hive (or at least certain versions of Hive) creates parquet files containing tinyint or smallint columns, it stores them as int32, but doesn't

spark git commit: [SPARK-10683][SPARK-16510][SPARKR] Move SparkR include jar test to SparkSubmitSuite

2016-07-19 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 f58fd4620 -> 6f209c8fa [SPARK-10683][SPARK-16510][SPARKR] Move SparkR include jar test to SparkSubmitSuite ## What changes were proposed in this pull request? This change moves the include jar test from R to SparkSubmitSuite and uses

spark git commit: [SPARK-10683][SPARK-16510][SPARKR] Move SparkR include jar test to SparkSubmitSuite

2016-07-19 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 9674af6f6 -> fc2326362 [SPARK-10683][SPARK-16510][SPARKR] Move SparkR include jar test to SparkSubmitSuite ## What changes were proposed in this pull request? This change moves the include jar test from R to SparkSubmitSuite and uses a

spark git commit: [SPARK-16568][SQL][DOCUMENTATION] update sql programming guide refreshTable API in python code

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 307f8922b -> f58fd4620 [SPARK-16568][SQL][DOCUMENTATION] update sql programming guide refreshTable API in python code ## What changes were proposed in this pull request? update `refreshTable` API in python code of the

spark git commit: [SPARK-16568][SQL][DOCUMENTATION] update sql programming guide refreshTable API in python code

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master 004e29cba -> 9674af6f6 [SPARK-16568][SQL][DOCUMENTATION] update sql programming guide refreshTable API in python code ## What changes were proposed in this pull request? update `refreshTable` API in python code of the

spark git commit: [SPARK-14702] Make environment of SparkLauncher launched process more configurable

2016-07-19 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 2ae7b88a0 -> 004e29cba [SPARK-14702] Make environment of SparkLauncher launched process more configurable ## What changes were proposed in this pull request? Adds a few public methods to `SparkLauncher` to allow configuring some extra

[2/2] spark git commit: Preparing development version 2.0.1-SNAPSHOT

2016-07-19 Thread pwendell
Preparing development version 2.0.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/307f8922 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/307f8922 Diff:

[spark] Git Push Summary

2016-07-19 Thread pwendell
Repository: spark Updated Tags: refs/tags/v2.0.0-rc5 [created] 13650fc58 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v2.0.0-rc5

2016-07-19 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-2.0 80ab8b666 -> 307f8922b Preparing Spark release v2.0.0-rc5 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/13650fc5 Tree:

spark git commit: [SPARK-15705][SQL] Change the default value of spark.sql.hive.convertMetastoreOrc to false.

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 f18f9ca5b -> 80ab8b666 [SPARK-15705][SQL] Change the default value of spark.sql.hive.convertMetastoreOrc to false. ## What changes were proposed in this pull request? In 2.0, we add a new logic to convert HiveTableScan on ORC tables

spark git commit: [SPARK-15705][SQL] Change the default value of spark.sql.hive.convertMetastoreOrc to false.

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master 162d04a30 -> 2ae7b88a0 [SPARK-15705][SQL] Change the default value of spark.sql.hive.convertMetastoreOrc to false. ## What changes were proposed in this pull request? In 2.0, we add a new logic to convert HiveTableScan on ORC tables to

spark git commit: [SPARK-16602][SQL] `Nvl` function should support numeric-string cases

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 6ca1d941b -> f18f9ca5b [SPARK-16602][SQL] `Nvl` function should support numeric-string cases ## What changes were proposed in this pull request? `Nvl` function should support numeric-straing cases like Hive/Spark1.6. Currently, `Nvl`

spark git commit: [SPARK-16602][SQL] `Nvl` function should support numeric-string cases

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master 0bd76e872 -> 162d04a30 [SPARK-16602][SQL] `Nvl` function should support numeric-string cases ## What changes were proposed in this pull request? `Nvl` function should support numeric-straing cases like Hive/Spark1.6. Currently, `Nvl`

spark git commit: [SPARK-16620][CORE] Add back the tokenization process in `RDD.pipe(command: String)`

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 2c74b6d73 -> 6ca1d941b [SPARK-16620][CORE] Add back the tokenization process in `RDD.pipe(command: String)` ## What changes were proposed in this pull request? Currently `RDD.pipe(command: String)`: - works only when the command is

spark git commit: [SPARK-16494][ML] Upgrade breeze version to 0.12

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 5d92326be -> 670891496 [SPARK-16494][ML] Upgrade breeze version to 0.12 ## What changes were proposed in this pull request? breeze 0.12 has been released for more than half a year, and it brings lots of new features, performance

spark git commit: [SPARK-16478] graphX (added graph caching in strongly connected components)

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6c4b9f4be -> 5d92326be [SPARK-16478] graphX (added graph caching in strongly connected components) ## What changes were proposed in this pull request? I added caching in every iteration for sccGraph that is returned in strongly connected

spark git commit: [SPARK-16395][STREAMING] Fail if too many CheckpointWriteHandlers are queued up in the fixed thread pool

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8310c0741 -> 6c4b9f4be [SPARK-16395][STREAMING] Fail if too many CheckpointWriteHandlers are queued up in the fixed thread pool ## What changes were proposed in this pull request? Begin failing if checkpoint writes will likely keep up

spark git commit: [SPARK-16600][MLLIB] fix some latex formula syntax error

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 929fa287e -> 2c74b6d73 [SPARK-16600][MLLIB] fix some latex formula syntax error ## What changes were proposed in this pull request? `\partial\x` ==> `\partial x` `har{x_i}` ==> `hat{x_i}` ## How was this patch tested? N/A Author:

spark git commit: [SPARK-16600][MLLIB] fix some latex formula syntax error

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6caa22050 -> 8310c0741 [SPARK-16600][MLLIB] fix some latex formula syntax error ## What changes were proposed in this pull request? `\partial\x` ==> `\partial x` `har{x_i}` ==> `hat{x_i}` ## How was this patch tested? N/A Author:

spark git commit: [MINOR][SQL][STREAMING][DOCS] Fix minor typos, punctuations and grammar

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 eb1c20fa0 -> 929fa287e [MINOR][SQL][STREAMING][DOCS] Fix minor typos, punctuations and grammar ## What changes were proposed in this pull request? Minor fixes correcting some typos, punctuations, grammar. Adding more anchors for easy

spark git commit: [MINOR][SQL][STREAMING][DOCS] Fix minor typos, punctuations and grammar

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 21a6dd2ae -> 6caa22050 [MINOR][SQL][STREAMING][DOCS] Fix minor typos, punctuations and grammar ## What changes were proposed in this pull request? Minor fixes correcting some typos, punctuations, grammar. Adding more anchors for easy

spark git commit: [SPARK-16535][BUILD] In pom.xml, remove groupId which is redundant definition and inherited from the parent

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 556a9437a -> 21a6dd2ae [SPARK-16535][BUILD] In pom.xml, remove groupId which is redundant definition and inherited from the parent https://issues.apache.org/jira/browse/SPARK-16535 ## What changes were proposed in this pull request?

spark git commit: [MINOR][BUILD] Fix Java Linter `LineLength` errors

2016-07-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6ee40d2cc -> 556a9437a [MINOR][BUILD] Fix Java Linter `LineLength` errors ## What changes were proposed in this pull request? This PR fixes four java linter `LineLength` errors. Those are all `LineLength` errors, but we had better remove

spark git commit: [DOC] improve python doc for rdd.histogram and dataframe.join

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master 1426a0805 -> 6ee40d2cc [DOC] improve python doc for rdd.histogram and dataframe.join ## What changes were proposed in this pull request? doc change only ## How was this patch tested? doc change only Author: Mortada Mehyar

spark git commit: [DOC] improve python doc for rdd.histogram and dataframe.join

2016-07-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 ef2a6f131 -> 504aa6f7a [DOC] improve python doc for rdd.histogram and dataframe.join ## What changes were proposed in this pull request? doc change only ## How was this patch tested? doc change only Author: Mortada Mehyar

spark git commit: [SPARK-16303][DOCS][EXAMPLES] Minor Scala/Java example update

2016-07-19 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 24ea87519 -> ef2a6f131 [SPARK-16303][DOCS][EXAMPLES] Minor Scala/Java example update ## What changes were proposed in this pull request? This PR moves one and the last hard-coded Scala example snippet from the SQL programming guide

spark git commit: [SPARK-16303][DOCS][EXAMPLES] Minor Scala/Java example update

2016-07-19 Thread yhuai
Repository: spark Updated Branches: refs/heads/master e5fbb182c -> 1426a0805 [SPARK-16303][DOCS][EXAMPLES] Minor Scala/Java example update ## What changes were proposed in this pull request? This PR moves one and the last hard-coded Scala example snippet from the SQL programming guide into