spark git commit: [SPARK-9340] [SQL] Fixes converting unannotated Parquet lists

Repository: spark Updated Branches: refs/heads/branch-1.5 94692bb14 - 01efa4f27 [SPARK-9340] [SQL] Fixes converting unannotated Parquet lists This PR is inspired by #8063 authored by dguy. Especially, testing Parquet files added here are all taken from that PR. **Committer who merges this

spark git commit: [SPARK-9340] [SQL] Fixes converting unannotated Parquet lists

Repository: spark Updated Branches: refs/heads/master 3c9802d94 - 071bbad5d [SPARK-9340] [SQL] Fixes converting unannotated Parquet lists This PR is inspired by #8063 authored by dguy. Especially, testing Parquet files added here are all taken from that PR. **Committer who merges this PR

spark git commit: [SPARK-9729] [SPARK-9363] [SQL] Use sort merge join for left and right outer join

Repository: spark Updated Branches: refs/heads/master 071bbad5d - 91e9389f3 [SPARK-9729] [SPARK-9363] [SQL] Use sort merge join for left and right outer join This patch adds a new `SortMergeOuterJoin` operator that performs left and right outer joins using sort merge join. It also

spark git commit: [SPARK-9729] [SPARK-9363] [SQL] Use sort merge join for left and right outer join

Repository: spark Updated Branches: refs/heads/branch-1.5 01efa4f27 - f9beef998 [SPARK-9729] [SPARK-9363] [SQL] Use sort merge join for left and right outer join This patch adds a new `SortMergeOuterJoin` operator that performs left and right outer joins using sort merge join. It also

spark git commit: [SPARK-9755] [MLLIB] Add docs to MultivariateOnlineSummarizer methods

Repository: spark Updated Branches: refs/heads/branch-1.5 94b2f5b32 - 3ee2c8d16 [SPARK-9755] [MLLIB] Add docs to MultivariateOnlineSummarizer methods Adds method documentations back to `MultivariateOnlineSummarizer`, which were present in 1.4 but disappeared somewhere along the way to 1.5.

spark git commit: [SPARK-9755] [MLLIB] Add docs to MultivariateOnlineSummarizer methods

Repository: spark Updated Branches: refs/heads/master 0f3366a4c - 00b655cce [SPARK-9755] [MLLIB] Add docs to MultivariateOnlineSummarizer methods Adds method documentations back to `MultivariateOnlineSummarizer`, which were present in 1.4 but disappeared somewhere along the way to 1.5.

spark git commit: [SPARK-9710] [TEST] Fix RPackageUtilsSuite when R is not available.

Repository: spark Updated Branches: refs/heads/master e3fef0f9e - 0f3366a4c [SPARK-9710] [TEST] Fix RPackageUtilsSuite when R is not available. RUtils.isRInstalled throws an exception if R is not installed, instead of returning false. Fix that. Author: Marcelo Vanzin van...@cloudera.com

spark git commit: [SPARK-9743] [SQL] Fixes JSONRelation refreshing

Repository: spark Updated Branches: refs/heads/branch-1.5 f75c64b0c - 94b2f5b32 [SPARK-9743] [SQL] Fixes JSONRelation refreshing PR #7696 added two `HadoopFsRelation.refresh()` calls ([this] [1], and [this] [2]) in `DataSourceStrategy` to make test case `InsertSuite.save directly to the

spark git commit: Fixed AtmoicReference Example

Repository: spark Updated Branches: refs/heads/branch-1.5 3ee2c8d16 - 39493b235 Fixed AtmoicReference Example Author: Mahmoud Lababidi labab...@gmail.com Closes #8076 from lababidi/master and squashes the following commits: af4553b [Mahmoud Lababidi] Fixed AtmoicReference Example (cherry

[1/2] spark git commit: Preparing Spark release v1.5.0-snapshot-20150810

Repository: spark Updated Branches: refs/heads/branch-1.5 d17303a94 - e51779c17 Preparing Spark release v1.5.0-snapshot-20150810 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/22031493 Tree: http://git-wip-us.apache.org

spark git commit: [SPARK-9784] [SQL] Exchange.isUnsafe should check whether codegen and unsafe are enabled

Repository: spark Updated Branches: refs/heads/branch-1.5 39493b235 - d251d9ff0 [SPARK-9784] [SQL] Exchange.isUnsafe should check whether codegen and unsafe are enabled Exchange.isUnsafe should check whether codegen and unsafe are enabled. Author: Josh Rosen joshro...@databricks.com Closes

spark git commit: [SPARK-9784] [SQL] Exchange.isUnsafe should check whether codegen and unsafe are enabled

Repository: spark Updated Branches: refs/heads/master d28521275 - 0fe66744f [SPARK-9784] [SQL] Exchange.isUnsafe should check whether codegen and unsafe are enabled Exchange.isUnsafe should check whether codegen and unsafe are enabled. Author: Josh Rosen joshro...@databricks.com Closes

spark git commit: [SPARK-9633] [BUILD] SBT download locations outdated; need an update

Repository: spark Updated Branches: refs/heads/branch-1.2 1c38d4254 - e23843c06 [SPARK-9633] [BUILD] SBT download locations outdated; need an update Remove 2 defunct SBT download URLs and replace with the 1 known download URL. Also, use https. Follow up on

spark git commit: [SPARK-9620] [SQL] generated UnsafeProjection should support many columns or large exressions

Repository: spark Updated Branches: refs/heads/branch-1.5 c1838e430 - 23842482f [SPARK-9620] [SQL] generated UnsafeProjection should support many columns or large exressions Currently, generated UnsafeProjection can reach 64k byte code limit of Java. This patch will split the generated

spark git commit: [SPARK-9620] [SQL] generated UnsafeProjection should support many columns or large exressions

Repository: spark Updated Branches: refs/heads/master 40ed2af58 - fe2fb7fb7 [SPARK-9620] [SQL] generated UnsafeProjection should support many columns or large exressions Currently, generated UnsafeProjection can reach 64k byte code limit of Java. This patch will split the generated

spark git commit: [SPARK-9633] [BUILD] SBT download locations outdated; need an update

Repository: spark Updated Branches: refs/heads/branch-1.0 d304ef4a9 - 117843f85 [SPARK-9633] [BUILD] SBT download locations outdated; need an update Remove 2 defunct SBT download URLs and replace with the 1 known download URL. Also, use https. Follow up on

spark git commit: [SPARK-9759] [SQL] improve decimal.times() and cast(int, decimalType)

Repository: spark Updated Branches: refs/heads/master fe2fb7fb7 - c4fd2a242 [SPARK-9759] [SQL] improve decimal.times() and cast(int, decimalType) This patch optimize two things: 1. passing MathContext to JavaBigDecimal.multiply/divide/reminder to do right rounding, because

spark git commit: [SPARK-9759] [SQL] improve decimal.times() and cast(int, decimalType)

Repository: spark Updated Branches: refs/heads/branch-1.5 23842482f - d17303a94 [SPARK-9759] [SQL] improve decimal.times() and cast(int, decimalType) This patch optimize two things: 1. passing MathContext to JavaBigDecimal.multiply/divide/reminder to do right rounding, because

[05/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/TestJsonData.scala -- diff --git

[2/2] spark git commit: Preparing development version 1.5.0-SNAPSHOT

Preparing development version 1.5.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e51779c1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e51779c1 Diff:

Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.5.0-snapshot-20150810 [created] 220314935 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.5.0-snapshot-20150803 [deleted] 7e7147f3b - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 1.5.0-SNAPSHOT

Preparing development version 1.5.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0e4f58ed Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0e4f58ed Diff:

[1/2] spark git commit: Preparing Spark release v1.5.0-snapshot-20150810

Repository: spark Updated Branches: refs/heads/branch-1.5 e51779c17 - 0e4f58ed9 Preparing Spark release v1.5.0-snapshot-20150810 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3369ad9b Tree: http://git-wip-us.apache.org

Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.5.0-snapshot-20150810 [created] 3369ad9bc - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-9737] [YARN] Add the suggested configuration when required executor memory is above the max threshold of this cluster on YARN mode

Repository: spark Updated Branches: refs/heads/branch-1.5 0e4f58ed9 - 51406becc [SPARK-9737] [YARN] Add the suggested configuration when required executor memory is above the max threshold of this cluster on YARN mode Author: Yadong Qi qiyadong2...@gmail.com Closes #8028 from

spark git commit: [SPARK-5155] [PYSPARK] [STREAMING] Mqtt streaming support in Python

Repository: spark Updated Branches: refs/heads/master c4fd2a242 - 853809e94 [SPARK-5155] [PYSPARK] [STREAMING] Mqtt streaming support in Python This PR is based on #4229, thanks prabeesh. Closes #4229 Author: Prabeesh K prabsma...@gmail.com Author: zsxwing zsxw...@gmail.com Author: prabs

spark git commit: [SPARK-5155] [PYSPARK] [STREAMING] Mqtt streaming support in Python

Repository: spark Updated Branches: refs/heads/branch-1.5 51406becc - 8f4014fda [SPARK-5155] [PYSPARK] [STREAMING] Mqtt streaming support in Python This PR is based on #4229, thanks prabeesh. Closes #4229 Author: Prabeesh K prabsma...@gmail.com Author: zsxwing zsxw...@gmail.com Author:

spark git commit: [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files.

Repository: spark Updated Branches: refs/heads/branch-1.4 4b5bbc589 - 6dde38026 [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files. Spark streaming deletes the temp file and backup files without checking if they exist or not Author: Hao Zhu viadea...@gmail.com

spark git commit: [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files.

Repository: spark Updated Branches: refs/heads/branch-1.5 8f4014fda - 94692bb14 [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files. Spark streaming deletes the temp file and backup files without checking if they exist or not Author: Hao Zhu viadea...@gmail.com

spark git commit: [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files.

Repository: spark Updated Branches: refs/heads/master 853809e94 - 3c9802d94 [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files. Spark streaming deletes the temp file and backup files without checking if they exist or not Author: Hao Zhu viadea...@gmail.com Closes

spark git commit: [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files.

Repository: spark Updated Branches: refs/heads/branch-1.3 b104501d3 - a98603f8c [SPARK-9801] [STREAMING] Check if file exists before deleting temporary files. Spark streaming deletes the temp file and backup files without checking if they exist or not Author: Hao Zhu viadea...@gmail.com

[04/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaSuite.scala -- diff --git

[04/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaSuite.scala -- diff --git

[10/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLTab.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLTab.scala

[08/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala

[03/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala -- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala

[02/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetIOSuite.scala -- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetIOSuite.scala

[11/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRelation.scala -- diff --git

[11/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRelation.scala -- diff --git

[13/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala -- diff --git

[08/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala

[10/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLTab.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLTab.scala

[14/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

[SPARK-9763][SQL] Minimize exposure of internal SQL classes. There are a few changes in this pull request: 1. Moved all data sources to execution.datasources, except the public JDBC APIs. 2. In order to maintain backward compatibility from 1, added a backward compatibility translation map in

[02/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetIOSuite.scala -- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetIOSuite.scala

spark git commit: [SPARK-9633] [BUILD] SBT download locations outdated; need an update

Repository: spark Updated Branches: refs/heads/branch-1.1 326988e00 - 119d6cf98 [SPARK-9633] [BUILD] SBT download locations outdated; need an update Remove 2 defunct SBT download URLs and replace with the 1 known download URL. Also, use https. Follow up on

[14/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

[SPARK-9763][SQL] Minimize exposure of internal SQL classes. There are a few changes in this pull request: 1. Moved all data sources to execution.datasources, except the public JDBC APIs. 2. In order to maintain backward compatibility from 1, added a backward compatibility translation map in

[09/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/main/scala/org/apache/spark/sql/parquet/CatalystReadSupport.scala -- diff --git

[12/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/CatalystRowConverter.scala -- diff --git

[03/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala -- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala

[01/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

Repository: spark Updated Branches: refs/heads/branch-1.5 d251d9ff0 - c1838e430 http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/test/scala/org/apache/spark/sql/sources/DDLSourceLoadSuite.scala -- diff

[06/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/test/gen-java/org/apache/spark/sql/parquet/test/avro/ParquetAvroCompat.java -- diff --git

[13/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala -- diff --git

[05/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/c1838e43/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/TestJsonData.scala -- diff --git

[07/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/main/scala/org/apache/spark/sql/ui/SparkPlanGraph.scala -- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/ui/SparkPlanGraph.scala

[01/14] spark git commit: [SPARK-9763][SQL] Minimize exposure of internal SQL classes.

Repository: spark Updated Branches: refs/heads/master 0fe66744f - 40ed2af58 http://git-wip-us.apache.org/repos/asf/spark/blob/40ed2af5/sql/core/src/test/scala/org/apache/spark/sql/sources/DDLSourceLoadSuite.scala -- diff --git