spark git commit: [SPARK-7235] [SQL] Refactor the grouping sets

2015-06-23 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 4f7fbefb8 - 7b1450b66 [SPARK-7235] [SQL] Refactor the grouping sets The logical plan `Expand` takes the `output` as constructor argument, which break the references chain. We need to refactor the code, as well as the column pruning.

spark git commit: [SPARK-8498] [TUNGSTEN] fix npe in errorhandling path in unsafeshuffle writer

2015-06-23 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.4 929479675 - 334824505 [SPARK-8498] [TUNGSTEN] fix npe in errorhandling path in unsafeshuffle writer Author: Holden Karau hol...@pigscanfly.ca Closes #6918 from holdenk/SPARK-8498-fix-npe-in-errorhandling-path-in-unsafeshuffle-writer

spark git commit: Revert [SPARK-8498] [TUNGSTEN] fix npe in errorhandling path in unsafeshuffle writer

2015-06-23 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.4 334824505 - 77cb1d5ed Revert [SPARK-8498] [TUNGSTEN] fix npe in errorhandling path in unsafeshuffle writer This reverts commit 33482450559028b907a0473277297cc54429322e. Reverting because `catch (Exception e) ... throw e` doesn't

spark git commit: [SQL] [DOCS] updated the documentation for explode

2015-06-23 Thread sarutak
Repository: spark Updated Branches: refs/heads/branch-1.4 77cb1d5ed - 27693e175 [SQL] [DOCS] updated the documentation for explode the syntax was incorrect in the example in explode Author: lockwobr lockw...@gmail.com Closes #6943 from lockwobr/master and squashes the following commits:

spark git commit: [SPARK-8498] [TUNGSTEN] fix npe in errorhandling path in unsafeshuffle writer

2015-06-23 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 6ceb16960 - 0f92be5b5 [SPARK-8498] [TUNGSTEN] fix npe in errorhandling path in unsafeshuffle writer Author: Holden Karau hol...@pigscanfly.ca Closes #6918 from holdenk/SPARK-8498-fix-npe-in-errorhandling-path-in-unsafeshuffle-writer and

spark git commit: [SQL] [DOCS] updated the documentation for explode

2015-06-23 Thread sarutak
Repository: spark Updated Branches: refs/heads/master 0f92be5b5 - 4f7fbefb8 [SQL] [DOCS] updated the documentation for explode the syntax was incorrect in the example in explode Author: lockwobr lockw...@gmail.com Closes #6943 from lockwobr/master and squashes the following commits:

spark git commit: [SPARK-8265] [MLLIB] [PYSPARK] Add LinearDataGenerator to pyspark.mllib.utils

2015-06-23 Thread meng
Repository: spark Updated Branches: refs/heads/master 2bdd0 - f2022fa0d [SPARK-8265] [MLLIB] [PYSPARK] Add LinearDataGenerator to pyspark.mllib.utils It is useful to generate linear data for easy testing of linear models and in general. Scala already has it. This is just a wrapper around

spark git commit: [SPARK-7888] Be able to disable intercept in linear regression in ml package

2015-06-23 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master 6f4cadf5e - 2bdd0 [SPARK-7888] Be able to disable intercept in linear regression in ml package Author: Holden Karau hol...@pigscanfly.ca Closes #6927 from

spark git commit: [SPARK-8111] [SPARKR] SparkR shell should display Spark logo and version banner on startup.

2015-06-23 Thread shivaram
Repository: spark Updated Branches: refs/heads/master f2022fa0d - f2fb0285a [SPARK-8111] [SPARKR] SparkR shell should display Spark logo and version banner on startup. spark version is taken from the environment variable SPARK_VERSION Author: Alok Singh singhal@Aloks-MacBook-Pro.local

spark git commit: [SPARK-8432] [SQL] fix hashCode() and equals() of BinaryType in Row

2015-06-23 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 7b1450b66 - 6f4cadf5e [SPARK-8432] [SQL] fix hashCode() and equals() of BinaryType in Row Also added more tests in LiteralExpressionSuite Author: Davies Liu dav...@databricks.com Closes #6876 from davies/fix_hashcode and squashes the

spark git commit: [DOC] [SQL] Addes Hive metastore Parquet table conversion section

2015-06-23 Thread lian
Repository: spark Updated Branches: refs/heads/master a8031183a - d96d7b557 [DOC] [SQL] Addes Hive metastore Parquet table conversion section This PR adds a section about Hive metastore Parquet table conversion. It documents: 1. Schema reconciliation rules introduced in #5214 (see [this

spark git commit: [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector

2015-06-23 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.1 36eed2f9e - faa35ca05 [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector fix LabeledPoint parser when there is a whitespace between label and features vector, e.g. (y, [x1, x2,

spark git commit: [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector

2015-06-23 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 27693e175 - 8d6e3636e [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector fix LabeledPoint parser when there is a whitespace between label and features vector, e.g. (y, [x1, x2,

spark git commit: [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector

2015-06-23 Thread meng
Repository: spark Updated Branches: refs/heads/master f2fb0285a - a8031183a [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector fix LabeledPoint parser when there is a whitespace between label and features vector, e.g. (y, [x1, x2, x3])

spark git commit: [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector

2015-06-23 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 716dcf631 - 88e303f6e [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector fix LabeledPoint parser when there is a whitespace between label and features vector, e.g. (y, [x1, x2,

spark git commit: [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector

2015-06-23 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 30789f6ef - 24c2c58c2 [SPARK-8525] [MLLIB] fix LabeledPoint parser when there is a whitespace between label and features vector fix LabeledPoint parser when there is a whitespace between label and features vector, e.g. (y, [x1, x2,

Git Push Summary

2015-06-23 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.1-rc1 [created] 60e08e507 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v1.4.1-rc1

2015-06-23 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 13f7b0a91 - eafbe1345 Preparing Spark release v1.4.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/60e08e50 Tree:

[2/2] spark git commit: Preparing development version 1.4.2-SNAPSHOT

2015-06-23 Thread pwendell
Preparing development version 1.4.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/eafbe134 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/eafbe134 Diff:

spark git commit: [SPARK-6749] [SQL] Make metastore client robust to underlying socket connection loss

2015-06-23 Thread yhuai
Repository: spark Updated Branches: refs/heads/master a458efc66 - 50c3a86f4 [SPARK-6749] [SQL] Make metastore client robust to underlying socket connection loss This works around a bug in the underlying RetryingMetaStoreClient (HIVE-10384) by refreshing the metastore client on thrift

spark git commit: [SPARK-8483] [STREAMING] Remove commons-lang3 dependency from Flume Si…

2015-06-23 Thread tdas
Repository: spark Updated Branches: refs/heads/master 31bd30687 - 9b618fb0d [SPARK-8483] [STREAMING] Remove commons-lang3 dependency from Flume Si… …nk. Also bump Flume version to 1.6.0 Author: Hari Shreedharan hshreedha...@apache.org Closes #6910 from

spark git commit: [SPARK-8359] [SQL] Fix incorrect decimal precision after multiplication

2015-06-23 Thread davies
Repository: spark Updated Branches: refs/heads/master d4f633514 - 31bd30687 [SPARK-8359] [SQL] Fix incorrect decimal precision after multiplication JIRA: https://issues.apache.org/jira/browse/SPARK-8359 Author: Liang-Chi Hsieh vii...@gmail.com Closes #6814 from viirya/fix_decimal2 and

spark git commit: [SPARK-8541] [PYSPARK] test the absolute error in approx doctests

2015-06-23 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 9b618fb0d - f0dcbe8a7 [SPARK-8541] [PYSPARK] test the absolute error in approx doctests A minor change but one which is (presumably) visible on the public api docs webpage. Author: Scott Taylor git...@megatron.me.uk Closes #6942 from

spark git commit: [SPARK-8483] [STREAMING] Remove commons-lang3 dependency from Flume Sink

2015-06-23 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.4 d0943afbc - 929479675 [SPARK-8483] [STREAMING] Remove commons-lang3 dependency from Flume Sink Author: Hari Shreedharan hshreedha...@apache.org Closes #6910 from harishreedharan/remove-commons-lang3 and squashes the following

spark git commit: [SPARK-8431] [SPARKR] Add in operator to DataFrame Column in SparkR

2015-06-23 Thread davies
Repository: spark Updated Branches: refs/heads/master 164fe2aa4 - d4f633514 [SPARK-8431] [SPARKR] Add in operator to DataFrame Column in SparkR [[SPARK-8431] Add in operator to DataFrame Column in SparkR - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-8431) Author: Yu ISHIKAWA

spark git commit: [SPARK-8541] [PYSPARK] test the absolute error in approx doctests

2015-06-23 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.4 22cc1ab66 - d0943afbc [SPARK-8541] [PYSPARK] test the absolute error in approx doctests A minor change but one which is (presumably) visible on the public api docs webpage. Author: Scott Taylor git...@megatron.me.uk Closes #6942

spark git commit: [SPARK-8541] [PYSPARK] test the absolute error in approx doctests

2015-06-23 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.3 45b4527e3 - 716dcf631 [SPARK-8541] [PYSPARK] test the absolute error in approx doctests A minor change but one which is (presumably) visible on the public api docs webpage. Author: Scott Taylor git...@megatron.me.uk Closes #6942

spark git commit: [SPARK-8300] DataFrame hint for broadcast join.

2015-06-23 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master f0dcbe8a7 - 6ceb16960 [SPARK-8300] DataFrame hint for broadcast join. Users can now do ```scala left.join(broadcast(right), joinKey) ``` to give the query planner a hint that right DataFrame is small and should be broadcasted. Author:

spark git commit: Revert [SPARK-7157][SQL] add sampleBy to DataFrame

2015-06-23 Thread rxin
Repository: spark Updated Branches: refs/heads/master 0401cbaa8 - a458efc66 Revert [SPARK-7157][SQL] add sampleBy to DataFrame This reverts commit 0401cbaa8ee51c71f43604f338b65022a479da0a. The new test case on Jenkins is failing. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

Git Push Summary

2015-06-23 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.1-rc1 [deleted] d0a5560ce - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-8573] [SPARK-8568] [SQL] [PYSPARK] raise Exception if column is used in booelan expression

2015-06-23 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.4 8d6e3636e - 13f7b0a91 [SPARK-8573] [SPARK-8568] [SQL] [PYSPARK] raise Exception if column is used in booelan expression It's a common mistake that user will put Column in a boolean expression (together with `and` , `or`), which does

spark git commit: [SPARK-8139] [SQL] Updates docs and comments of data sources and Parquet output committer options

2015-06-23 Thread lian
Repository: spark Updated Branches: refs/heads/master 7fb5ae502 - 111d6b9b8 [SPARK-8139] [SQL] Updates docs and comments of data sources and Parquet output committer options This PR only applies to master branch (1.5.0-SNAPSHOT) since it references `org.apache.parquet` classes which only

spark git commit: [SPARK-8573] [SPARK-8568] [SQL] [PYSPARK] raise Exception if column is used in booelan expression

2015-06-23 Thread davies
Repository: spark Updated Branches: refs/heads/master d96d7b557 - 7fb5ae502 [SPARK-8573] [SPARK-8568] [SQL] [PYSPARK] raise Exception if column is used in booelan expression It's a common mistake that user will put Column in a boolean expression (together with `and` , `or`), which does not