spark git commit: [SPARK-9814] [SQL] EqualNotNull not passing to data sources

Repository: spark Updated Branches: refs/heads/branch-1.5 e9d1eab92 - eead87ef2 [SPARK-9814] [SQL] EqualNotNull not passing to data sources Author: hyukjinkwon gurwls...@gmail.com Author: 권혁진 gurwls...@gmail.com Closes #8096 from HyukjinKwon/master. (cherry picked from commit

spark git commit: [SPARK-9824] [CORE] Fix the issue that InternalAccumulator leaks WeakReference

Repository: spark Updated Branches: refs/heads/master 00c02728a - f16bc68df [SPARK-9824] [CORE] Fix the issue that InternalAccumulator leaks WeakReference `InternalAccumulator.create` doesn't call `registerAccumulatorForCleanup` to register itself with ContextCleaner, so `WeakReference`s for

spark git commit: Closes #1290 Closes #4934

Repository: spark Updated Branches: refs/heads/master f16bc68df - 423cdfd83 Closes #1290 Closes #4934 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/423cdfd8 Tree:

spark git commit: [SPARK-9788] [MLLIB] Fix LDA Binary Compatibility

Repository: spark Updated Branches: refs/heads/branch-1.5 cdf781db6 - 2273e7432 [SPARK-9788] [MLLIB] Fix LDA Binary Compatibility 1. Add “asymmetricDocConcentration” and revert docConcentration changes. If the (internal) doc concentration vector is a single value, “getDocConcentration

spark git commit: [SPARK-9788] [MLLIB] Fix LDA Binary Compatibility

Repository: spark Updated Branches: refs/heads/master 423cdfd83 - be3e27164 [SPARK-9788] [MLLIB] Fix LDA Binary Compatibility 1. Add “asymmetricDocConcentration” and revert docConcentration changes. If the (internal) doc concentration vector is a single value, “getDocConcentration

Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.5.0-snapshot-20150811 [created] e9329ef6a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v1.5.0-snapshot-20150811

Repository: spark Updated Branches: refs/heads/branch-1.5 ef961ed48 - 725e5c7a4 Preparing Spark release v1.5.0-snapshot-20150811 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e9329ef6 Tree: http://git-wip-us.apache.org

[2/2] spark git commit: Preparing development version 1.5.0-SNAPSHOT

Preparing development version 1.5.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/725e5c7a Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/725e5c7a Diff:

spark git commit: [SPARK-7726] Add import so Scaladoc doesn't fail.

Repository: spark Updated Branches: refs/heads/master 520ad44b1 - 2a3be4ddf [SPARK-7726] Add import so Scaladoc doesn't fail. This is another import needed so Scala 2.11 doc generation doesn't fail. See SPARK-7726 for more detail. I tested this locally and the 2.11 install goes from failing

spark git commit: [SPARK-7726] Add import so Scaladoc doesn't fail.

Repository: spark Updated Branches: refs/heads/branch-1.5 811d23f1c - e9d1eab92 [SPARK-7726] Add import so Scaladoc doesn't fail. This is another import needed so Scala 2.11 doc generation doesn't fail. See SPARK-7726 for more detail. I tested this locally and the 2.11 install goes from

spark git commit: [SPARK-9814] [SQL] EqualNotNull not passing to data sources

Repository: spark Updated Branches: refs/heads/master 2a3be4ddf - 00c02728a [SPARK-9814] [SQL] EqualNotNull not passing to data sources Author: hyukjinkwon gurwls...@gmail.com Author: 권혁진 gurwls...@gmail.com Closes #8096 from HyukjinKwon/master. Project:

spark git commit: [SPARK-8925] [MLLIB] Add @since tags to mllib.util

Repository: spark Updated Branches: refs/heads/branch-1.5 2273e7432 - ef961ed48 [SPARK-8925] [MLLIB] Add @since tags to mllib.util Went thru the history of changes the file MLUtils.scala and picked up the version that the change went in. Author: Sudhakar Thota sudhakarth...@yahoo.com

spark git commit: [SPARK-8925] [MLLIB] Add @since tags to mllib.util

Repository: spark Updated Branches: refs/heads/master be3e27164 - 017b5de07 [SPARK-8925] [MLLIB] Add @since tags to mllib.util Went thru the history of changes the file MLUtils.scala and picked up the version that the change went in. Author: Sudhakar Thota sudhakarth...@yahoo.com Author:

spark git commit: [SPARK-9649] Fix flaky test MasterSuite again - disable REST

Repository: spark Updated Branches: refs/heads/branch-1.5 ec7a4b9b0 - 0119edf52 [SPARK-9649] Fix flaky test MasterSuite again - disable REST The REST server is not actually used in most tests and so we can disable it. It is a source of flakiness because it tries to bind to a specific port in

spark git commit: [SPARK-9640] [STREAMING] [TEST] Do not run Python Kinesis tests when the Kinesis assembly JAR has not been generated

Repository: spark Updated Branches: refs/heads/branch-1.5 f9beef998 - c7f009040 [SPARK-9640] [STREAMING] [TEST] Do not run Python Kinesis tests when the Kinesis assembly JAR has not been generated Author: Tathagata Das tathagata.das1...@gmail.com Closes #7961 from tdas/SPARK-9640 and

spark git commit: [SPARK-9640] [STREAMING] [TEST] Do not run Python Kinesis tests when the Kinesis assembly JAR has not been generated

Repository: spark Updated Branches: refs/heads/master 91e9389f3 - 0f90d6055 [SPARK-9640] [STREAMING] [TEST] Do not run Python Kinesis tests when the Kinesis assembly JAR has not been generated Author: Tathagata Das tathagata.das1...@gmail.com Closes #7961 from tdas/SPARK-9640 and squashes

spark git commit: [SPARK-9727] [STREAMING] [BUILD] Updated streaming kinesis SBT project name to be more consistent

Repository: spark Updated Branches: refs/heads/master 55752d883 - 600031ebe [SPARK-9727] [STREAMING] [BUILD] Updated streaming kinesis SBT project name to be more consistent Author: Tathagata Das tathagata.das1...@gmail.com Closes #8092 from tdas/SPARK-9727 and squashes the following

spark git commit: [SPARK-9727] [STREAMING] [BUILD] Updated streaming kinesis SBT project name to be more consistent

Repository: spark Updated Branches: refs/heads/branch-1.5 c7f009040 - ebbd3b616 [SPARK-9727] [STREAMING] [BUILD] Updated streaming kinesis SBT project name to be more consistent Author: Tathagata Das tathagata.das1...@gmail.com Closes #8092 from tdas/SPARK-9727 and squashes the following

spark git commit: [SPARK-9810] [BUILD] Remove individual commit messages from the squash commit message

Repository: spark Updated Branches: refs/heads/master 0f90d6055 - 55752d883 [SPARK-9810] [BUILD] Remove individual commit messages from the squash commit message For more information, please see the JIRA ticket and the associated dev list discussion.

[2/2] spark git commit: [SPARK-9815] Rename PlatformDependent.UNSAFE - Platform.

[SPARK-9815] Rename PlatformDependent.UNSAFE - Platform. PlatformDependent.UNSAFE is way too verbose. Author: Reynold Xin r...@databricks.com Closes #8094 from rxin/SPARK-9815 and squashes the following commits: 229b603 [Reynold Xin] [SPARK-9815] Rename PlatformDependent.UNSAFE - Platform.

[2/2] spark git commit: [SPARK-9815] Rename PlatformDependent.UNSAFE - Platform.

[SPARK-9815] Rename PlatformDependent.UNSAFE - Platform. PlatformDependent.UNSAFE is way too verbose. Author: Reynold Xin r...@databricks.com Closes #8094 from rxin/SPARK-9815 and squashes the following commits: 229b603 [Reynold Xin] [SPARK-9815] Rename PlatformDependent.UNSAFE - Platform.

spark git commit: [SPARK-9785] [SQL] HashPartitioning compatibility should consider expression ordering

Repository: spark Updated Branches: refs/heads/master d378396f8 - dfe347d2c [SPARK-9785] [SQL] HashPartitioning compatibility should consider expression ordering HashPartitioning compatibility is currently defined w.r.t the _set_ of expressions, but the ordering of those expressions matters

[1/2] spark git commit: [SPARK-9815] Rename PlatformDependent.UNSAFE - Platform.

Repository: spark Updated Branches: refs/heads/master 600031ebe - d378396f8 http://git-wip-us.apache.org/repos/asf/spark/blob/d378396f/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala

[1/2] spark git commit: [SPARK-9815] Rename PlatformDependent.UNSAFE - Platform.

Repository: spark Updated Branches: refs/heads/branch-1.5 ebbd3b616 - 84ba990f2 http://git-wip-us.apache.org/repos/asf/spark/blob/84ba990f/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala

spark git commit: [SPARK-8345] [ML] Add an SQL node as a feature transformer

Repository: spark Updated Branches: refs/heads/master bce72797f - 8cad854ef [SPARK-8345] [ML] Add an SQL node as a feature transformer Implements the transforms which are defined by SQL statement. Currently we only support SQL syntax like 'SELECT ... FROM __THIS__' where '__THIS__' represents

spark git commit: [SPARK-9572] [STREAMING] [PYSPARK] Added StreamingContext.getActiveOrCreate() in Python

Repository: spark Updated Branches: refs/heads/master dbd778d84 - 5b8bb1b21 [SPARK-9572] [STREAMING] [PYSPARK] Added StreamingContext.getActiveOrCreate() in Python Author: Tathagata Das tathagata.das1...@gmail.com Closes #8080 from tdas/SPARK-9572 and squashes the following commits:

spark git commit: [SPARK-9572] [STREAMING] [PYSPARK] Added StreamingContext.getActiveOrCreate() in Python

Repository: spark Updated Branches: refs/heads/branch-1.5 b077f36ea - 71460b889 [SPARK-9572] [STREAMING] [PYSPARK] Added StreamingContext.getActiveOrCreate() in Python Author: Tathagata Das tathagata.das1...@gmail.com Closes #8080 from tdas/SPARK-9572 and squashes the following commits:

spark git commit: [SPARK-8764] [ML] string indexer should take option to handle unseen values

Repository: spark Updated Branches: refs/heads/master 8cad854ef - dbd778d84 [SPARK-8764] [ML] string indexer should take option to handle unseen values As a precursor to adding a public constructor add an option to handle unseen values by skipping rather than throwing an exception (default

[2/2] spark git commit: [SPARK-9646] [SQL] Add metrics for all join and aggregate operators

[SPARK-9646] [SQL] Add metrics for all join and aggregate operators This PR added metrics for all join and aggregate operators. However, I found the metrics may be confusing in the following two case: 1. The iterator is not totally consumed and the metric values will be less. 2. Recreating the

[2/2] spark git commit: [SPARK-9646] [SQL] Add metrics for all join and aggregate operators

[SPARK-9646] [SQL] Add metrics for all join and aggregate operators This PR added metrics for all join and aggregate operators. However, I found the metrics may be confusing in the following two case: 1. The iterator is not totally consumed and the metric values will be less. 2. Recreating the

[1/2] spark git commit: [SPARK-9646] [SQL] Add metrics for all join and aggregate operators

Repository: spark Updated Branches: refs/heads/branch-1.5 71460b889 - 767ee1884 http://git-wip-us.apache.org/repos/asf/spark/blob/767ee188/sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --

spark git commit: [SPARK-9750] [MLLIB] Improve equals on SparseMatrix and DenseMatrix

Repository: spark Updated Branches: refs/heads/master 5831294a7 - 520ad44b1 [SPARK-9750] [MLLIB] Improve equals on SparseMatrix and DenseMatrix Adds unit test for `equals` on `mllib.linalg.Matrix` class and `equals` to both `SparseMatrix` and `DenseMatrix`. Supports equality testing between

spark git commit: [SPARK-9750] [MLLIB] Improve equals on SparseMatrix and DenseMatrix

Repository: spark Updated Branches: refs/heads/branch-1.5 767ee1884 - 811d23f1c [SPARK-9750] [MLLIB] Improve equals on SparseMatrix and DenseMatrix Adds unit test for `equals` on `mllib.linalg.Matrix` class and `equals` to both `SparseMatrix` and `DenseMatrix`. Supports equality testing

spark git commit: [HOTFIX] Fix style error caused by 017b5de

Repository: spark Updated Branches: refs/heads/master 017b5de07 - 736af95bd [HOTFIX] Fix style error caused by 017b5de Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/736af95b Tree:

spark git commit: [HOTFIX] Fix style error caused by ef961ed48a4f45447f0e0ad256b040c7ab2d78d9

Repository: spark Updated Branches: refs/heads/branch-1.5 725e5c7a4 - 1067c7369 [HOTFIX] Fix style error caused by ef961ed48a4f45447f0e0ad256b040c7ab2d78d9 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1067c736 Tree:

spark git commit: [SPARK-9831] [SQL] fix serialization with empty broadcast

Repository: spark Updated Branches: refs/heads/master 74a293f45 - c3e9a120e [SPARK-9831] [SQL] fix serialization with empty broadcast Author: Davies Liu dav...@databricks.com Closes #8117 from davies/fix_serialization and squashes the following commits: d21ac71 [Davies Liu] fix

spark git commit: [SPARK-9831] [SQL] fix serialization with empty broadcast

Repository: spark Updated Branches: refs/heads/branch-1.5 890c75bc2 - 7024f3eac [SPARK-9831] [SQL] fix serialization with empty broadcast Author: Davies Liu dav...@databricks.com Closes #8117 from davies/fix_serialization and squashes the following commits: d21ac71 [Davies Liu] fix

spark git commit: [SPARK-9854] [SQL] RuleExecutor.timeMap should be thread-safe

Repository: spark Updated Branches: refs/heads/master c3e9a120e - b1581ac28 [SPARK-9854] [SQL] RuleExecutor.timeMap should be thread-safe `RuleExecutor.timeMap` is currently a non-thread-safe mutable HashMap; this can lead to infinite loops if multiple threads are concurrently modifying the

[2/2] spark git commit: Preparing development version 1.5.0-SNAPSHOT

Preparing development version 1.5.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b7497e3a Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b7497e3a Diff:

spark git commit: [SPARK-9849] [SQL] DirectParquetOutputCommitter qualified name should be backward compatible

Repository: spark Updated Branches: refs/heads/master 5a5bbc299 - afa757c98 [SPARK-9849] [SQL] DirectParquetOutputCommitter qualified name should be backward compatible DirectParquetOutputCommitter was moved in SPARK-9763. However, users can explicitly set the class as a config option, so

spark git commit: [SPARK-9849] [SQL] DirectParquetOutputCommitter qualified name should be backward compatible

Repository: spark Updated Branches: refs/heads/branch-1.5 b7497e3a2 - ec7a4b9b0 [SPARK-9849] [SQL] DirectParquetOutputCommitter qualified name should be backward compatible DirectParquetOutputCommitter was moved in SPARK-9763. However, users can explicitly set the class as a config option,

spark git commit: [SPARK-9713] [ML] Document SparkR MLlib glm() integration in Spark 1.5

Repository: spark Updated Branches: refs/heads/branch-1.5 6ea33f5bf - 890c75bc2 [SPARK-9713] [ML] Document SparkR MLlib glm() integration in Spark 1.5 This documents the use of R model formulae in the SparkR guide. Also fixes some bugs in the R api doc. mengxr Author: Eric Liang

spark git commit: [SPARK-9713] [ML] Document SparkR MLlib glm() integration in Spark 1.5

Repository: spark Updated Branches: refs/heads/master 3ef0f3292 - 74a293f45 [SPARK-9713] [ML] Document SparkR MLlib glm() integration in Spark 1.5 This documents the use of R model formulae in the SparkR guide. Also fixes some bugs in the R api doc. mengxr Author: Eric Liang

Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.5.0-snapshot-20150811 [deleted] e9329ef6a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-1517] Refactor release scripts to facilitate nightly publishing

Repository: spark Updated Branches: refs/heads/branch-1.5 0119edf52 - 6ea33f5bf [SPARK-1517] Refactor release scripts to facilitate nightly publishing This update contains some code changes to the release scripts that allow easier nightly publishing. I've been using these new scripts on