spark git commit: [SPARK-18016][SQL][CATALYST] Code Generation: Constant Pool Limit - Class Splitting

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 205142817 -> b32b2123d [SPARK-18016][SQL][CATALYST] Code Generation: Constant Pool Limit - Class Splitting ## What changes were proposed in this pull request? This pull-request exclusively includes the class splitting feature described

spark git commit: [SPARK-20980][SQL] Rename `wholeFile` to `multiLine` for both CSV and JSON

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 e02e0637f -> af4f89c98 [SPARK-20980][SQL] Rename `wholeFile` to `multiLine` for both CSV and JSON The current option name `wholeFile` is misleading for CSV users. Currently, it is not representing a record per file. Actually, one file

spark git commit: [SPARK-20980][SQL] Rename `wholeFile` to `multiLine` for both CSV and JSON

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/master fffeb6d7c -> 205142817 [SPARK-20980][SQL] Rename `wholeFile` to `multiLine` for both CSV and JSON ### What changes were proposed in this pull request? The current option name `wholeFile` is misleading for CSV users. Currently, it is not

spark git commit: [SPARK-21092][SQL] Wire SQLConf in logical plan and expressions

2017-06-14 Thread rxin
Repository: spark Updated Branches: refs/heads/master 292467440 -> fffeb6d7c [SPARK-21092][SQL] Wire SQLConf in logical plan and expressions ## What changes were proposed in this pull request? It is really painful to not have configs in logical plan and expressions. We had to add all sorts

spark git commit: [SPARK-19900][CORE] Remove driver when relaunching.

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/master e254e868f -> 292467440 [SPARK-19900][CORE] Remove driver when relaunching. This is https://github.com/apache/spark/pull/17888 . Below are some spark ui snapshots. Master, after worker disconnects:

spark git commit: [SPARK-21091][SQL] Move constraint code into QueryPlanConstraints

2017-06-14 Thread rxin
Repository: spark Updated Branches: refs/heads/master 77a2fc5b5 -> e254e868f [SPARK-21091][SQL] Move constraint code into QueryPlanConstraints ## What changes were proposed in this pull request? This patch moves constraint related code into a separate trait QueryPlanConstraints, so we don't

spark git commit: Revert "[SPARK-20941][SQL] Fix SubqueryExec Reuse"

2017-06-14 Thread lixiao
Repository: spark Updated Branches: refs/heads/master df766a471 -> 77a2fc5b5 Revert "[SPARK-20941][SQL] Fix SubqueryExec Reuse" This reverts commit f7cf2096fdecb8edab61c8973c07c6fc877ee32d. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: Revert "[SPARK-20941][SQL] Fix SubqueryExec Reuse"

2017-06-14 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 3dda682c4 -> e02e0637f Revert "[SPARK-20941][SQL] Fix SubqueryExec Reuse" This reverts commit 6a4e023b250a86887475958093f1d3bdcbb49a03. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-21089][SQL] Fix DESC EXTENDED/FORMATTED to Show Table Properties

2017-06-14 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 626511953 -> 3dda682c4 [SPARK-21089][SQL] Fix DESC EXTENDED/FORMATTED to Show Table Properties Since both table properties and storage properties share the same key values, table properties are not shown in the output of DESC

spark git commit: [SPARK-21089][SQL] Fix DESC EXTENDED/FORMATTED to Show Table Properties

2017-06-14 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0c88e8d37 -> df766a471 [SPARK-21089][SQL] Fix DESC EXTENDED/FORMATTED to Show Table Properties ### What changes were proposed in this pull request? Since both table properties and storage properties share the same key values, table

spark git commit: [SPARK-20211][SQL][BACKPORT-2.2] Fix the Precision and Scale of Decimal Values when the Input is BigDecimal between -1.0 and 1.0

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 0f3598820 -> 0239b1673 [SPARK-20211][SQL][BACKPORT-2.2] Fix the Precision and Scale of Decimal Values when the Input is BigDecimal between -1.0 and 1.0 ### What changes were proposed in this pull request? This PR is to backport

spark git commit: [SPARK-20211][SQL][BACKPORT-2.2] Fix the Precision and Scale of Decimal Values when the Input is BigDecimal between -1.0 and 1.0

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 ee0e74e65 -> a890466bc [SPARK-20211][SQL][BACKPORT-2.2] Fix the Precision and Scale of Decimal Values when the Input is BigDecimal between -1.0 and 1.0 ### What changes were proposed in this pull request? This PR is to backport

spark git commit: [SPARK-20211][SQL][BACKPORT-2.2] Fix the Precision and Scale of Decimal Values when the Input is BigDecimal between -1.0 and 1.0

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 9bdc83590 -> 626511953 [SPARK-20211][SQL][BACKPORT-2.2] Fix the Precision and Scale of Decimal Values when the Input is BigDecimal between -1.0 and 1.0 ### What changes were proposed in this pull request? This PR is to backport

spark git commit: [SPARK-21085][SQL] Failed to read the partitioned table created by Spark 2.1

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 42cc83082 -> 9bdc83590 [SPARK-21085][SQL] Failed to read the partitioned table created by Spark 2.1 ### What changes were proposed in this pull request? Before the PR, Spark is unable to read the partitioned table created by Spark 2.1

spark git commit: [SPARK-21085][SQL] Failed to read the partitioned table created by Spark 2.1

2017-06-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d6f76eb34 -> 0c88e8d37 [SPARK-21085][SQL] Failed to read the partitioned table created by Spark 2.1 ### What changes were proposed in this pull request? Before the PR, Spark is unable to read the partitioned table created by Spark 2.1

spark git commit: [SPARK-21057][ML] Do not use a PascalDistribution in countApprox

2017-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4d01aa464 -> d6f76eb34 [SPARK-21057][ML] Do not use a PascalDistribution in countApprox ## What changes were proposed in this pull request? Use Poisson analysis for approx count in all cases. ## How was this patch tested? Existing

spark git commit: [SPARK-20754][SQL][FOLLOWUP] Add Function Alias For MOD/POSITION.

2017-06-14 Thread lixiao
Repository: spark Updated Branches: refs/heads/master dccc0aa3c -> 4d01aa464 [SPARK-20754][SQL][FOLLOWUP] Add Function Alias For MOD/POSITION. ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/18106 Support TRUNC (number), We should also add function