spark git commit: [SPARK-9667][SQL] followup: Use GenerateUnsafeProjection.canSupport to test Exchange supported data types.

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9897cc5e3 - aeddeafc0 [SPARK-9667][SQL] followup: Use GenerateUnsafeProjection.canSupport to test Exchange supported data types. This way we recursively test the data types. cc chenghao-intel Author: Reynold Xin r...@databricks.com

spark git commit: [SPARK-9733][SQL] Improve physical plan explain for data sources

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 1b0f784a1 - d13b5c895 [SPARK-9733][SQL] Improve physical plan explain for data sources All data sources show up as PhysicalRDD in physical plan explain. It'd be better if we can show the name of the data source. Without this patch:

spark git commit: [SPARK-9733][SQL] Improve physical plan explain for data sources

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master aeddeafc0 - 05d04e10a [SPARK-9733][SQL] Improve physical plan explain for data sources All data sources show up as PhysicalRDD in physical plan explain. It'd be better if we can show the name of the data source. Without this patch: ```

spark git commit: Revert [SPARK-8481] [MLLIB] GaussianMixtureModel.predict, GaussianMixtureModel.predictSoft variants for a single vector

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-1.4 e5a994f21 - 4b5bbc589 Revert [SPARK-8481] [MLLIB] GaussianMixtureModel.predict, GaussianMixtureModel.predictSoft variants for a single vector This reverts commit 07f778978d80f0af57d3dafda4c566a813ad2d09. Project:

spark git commit: [SPARK-8481] [MLLIB] GaussianMixtureModel predict accepting single vector

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 881548ab2 - e2fbbe731 [SPARK-8481] [MLLIB] GaussianMixtureModel predict accepting single vector Resubmit of [https://github.com/apache/spark/pull/6906] for adding single-vec predict to GMMs CC: dkobylarz mengxr To be merged with master

spark git commit: [SPARK-8481] [MLLIB] GaussianMixtureModel predict accepting single vector

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-1.5 547120287 - 295266049 [SPARK-8481] [MLLIB] GaussianMixtureModel predict accepting single vector Resubmit of [https://github.com/apache/spark/pull/6906] for adding single-vec predict to GMMs CC: dkobylarz mengxr To be merged with

spark git commit: [SPARK-9748] [MLLIB] Centriod typo in KMeansModel

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master e2fbbe731 - 902334fd5 [SPARK-9748] [MLLIB] Centriod typo in KMeansModel A minor typo (centriod - centroid). Readable variable names help every users. Author: Bertrand Dechoux bertranddech...@users.noreply.github.com Closes #8037 from

spark git commit: [SPARK-9453] [SQL] support records larger than page size in UnsafeShuffleExternalSorter

2015-08-07 Thread davies
Repository: spark Updated Branches: refs/heads/master 4309262ec - 15bd6f338 [SPARK-9453] [SQL] support records larger than page size in UnsafeShuffleExternalSorter This patch follows exactly #7891 (except testing) Author: Davies Liu dav...@databricks.com Closes #8005 from

spark git commit: [SPARK-9700] Pick default page size more intelligently.

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7aaed1b11 - 4309262ec [SPARK-9700] Pick default page size more intelligently. Previously, we use 64MB as the default page size, which was way too big for a lot of Spark applications (especially for single node). This patch changes it so

spark git commit: [SPARK-9700] Pick default page size more intelligently.

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 c34fdaf55 - 0e439c29d [SPARK-9700] Pick default page size more intelligently. Previously, we use 64MB as the default page size, which was way too big for a lot of Spark applications (especially for single node). This patch changes it

spark git commit: [SPARK-9674] Re-enable ignored test in SQLQuerySuite

2015-08-07 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 05d04e10a - 881548ab2 [SPARK-9674] Re-enable ignored test in SQLQuerySuite The original code that this test tests is removed in https://github.com/apache/spark/commit/9270bd06fd0b16892e3f37213b5bc7813ea11fdd. It was ignored shortly

spark git commit: [SPARK-9674] Re-enable ignored test in SQLQuerySuite

2015-08-07 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-1.5 d13b5c895 - 547120287 [SPARK-9674] Re-enable ignored test in SQLQuerySuite The original code that this test tests is removed in https://github.com/apache/spark/commit/9270bd06fd0b16892e3f37213b5bc7813ea11fdd. It was ignored shortly

spark git commit: [SPARK-9667][SQL] followup: Use GenerateUnsafeProjection.canSupport to test Exchange supported data types.

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 70bf170b9 - 1b0f784a1 [SPARK-9667][SQL] followup: Use GenerateUnsafeProjection.canSupport to test Exchange supported data types. This way we recursively test the data types. cc chenghao-intel Author: Reynold Xin r...@databricks.com

spark git commit: [SPARK-8890] [SQL] Fallback on sorting when writing many dynamic partitions

2015-08-07 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.5 295266049 - ea4dfb90a [SPARK-8890] [SQL] Fallback on sorting when writing many dynamic partitions Previously, we would open a new file for each new dynamic written out using `HadoopFsRelation`. For formats like parquet this is very

spark git commit: [SPARK-8890] [SQL] Fallback on sorting when writing many dynamic partitions

2015-08-07 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 902334fd5 - 49702bd73 [SPARK-8890] [SQL] Fallback on sorting when writing many dynamic partitions Previously, we would open a new file for each new dynamic written out using `HadoopFsRelation`. For formats like parquet this is very

spark git commit: [SPARK-9756] [ML] Make constructors in ML decision trees private

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 49702bd73 - cd540c1e5 [SPARK-9756] [ML] Make constructors in ML decision trees private These should be made private until there is a public constructor for providing `rootNode: Node` to use these constructors. jkbradley Author: Feynman

spark git commit: [SPARK-9756] [ML] Make constructors in ML decision trees private

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-1.5 ea4dfb90a - 2a179a94e [SPARK-9756] [ML] Make constructors in ML decision trees private These should be made private until there is a public constructor for providing `rootNode: Node` to use these constructors. jkbradley Author:

spark git commit: [SPARK-9719] [ML] Clean up Naive Bayes doc

2015-08-07 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master cd540c1e5 - 85be65b39 [SPARK-9719] [ML] Clean up Naive Bayes doc Small documentation cleanups, including: * Adds documentation for `pi` and `theta` * setParam to `setModelType` Author: Feynman Liang fli...@databricks.com Closes #8047

spark git commit: [SPARK-9754][SQL] Remove TypeCheck in debug package.

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master 85be65b39 - 998f4ff94 [SPARK-9754][SQL] Remove TypeCheck in debug package. TypeCheck no longer applies in the new Tungsten world. Author: Reynold Xin r...@databricks.com Closes #8043 from rxin/SPARK-9754 and squashes the following

spark git commit: [SPARK-9754][SQL] Remove TypeCheck in debug package.

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 c5d43d6c8 - 5598b6238 [SPARK-9754][SQL] Remove TypeCheck in debug package. TypeCheck no longer applies in the new Tungsten world. Author: Reynold Xin r...@databricks.com Closes #8043 from rxin/SPARK-9754 and squashes the following

spark git commit: [SPARK-9753] [SQL] TungstenAggregate should also accept InternalRow instead of just UnsafeRow

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 5598b6238 - 47e473550 [SPARK-9753] [SQL] TungstenAggregate should also accept InternalRow instead of just UnsafeRow https://issues.apache.org/jira/browse/SPARK-9753 This PR makes TungstenAggregate to accept `InternalRow` instead of

spark git commit: [SPARK-9753] [SQL] TungstenAggregate should also accept InternalRow instead of just UnsafeRow

2015-08-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master 998f4ff94 - c564b2744 [SPARK-9753] [SQL] TungstenAggregate should also accept InternalRow instead of just UnsafeRow https://issues.apache.org/jira/browse/SPARK-9753 This PR makes TungstenAggregate to accept `InternalRow` instead of just