[spark] Git Push Summary

2016-07-25 Thread rxin
Repository: spark Updated Tags: refs/tags/v2.0.0-rc1 [deleted] 0c66ca41a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2016-07-25 Thread rxin
Repository: spark Updated Tags: refs/tags/v2.0.0-rc2 [deleted] 4a55b2326 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2016-07-25 Thread rxin
Repository: spark Updated Tags: refs/tags/v2.0.0-rc3 [deleted] 48d1fa3e7 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2016-07-25 Thread rxin
Repository: spark Updated Tags: refs/tags/v2.0.0-rc4 [deleted] e5f8c1117 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2016-07-25 Thread rxin
Repository: spark Updated Tags: refs/tags/v2.0.0-rc5 [deleted] 13650fc58 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2016-07-25 Thread rxin
Repository: spark Updated Tags: refs/tags/v2.0.0 [created] 13650fc58 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-16686][SQL] Remove PushProjectThroughSample since it is handled by ColumnPruning

2016-07-25 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 815f3eece -> 7b06a8948 [SPARK-16686][SQL] Remove PushProjectThroughSample since it is handled by ColumnPruning ## What changes were proposed in this pull request? We push down `Project` through `Sample` in `Optimizer` by the rule

spark git commit: [SPARK-16633][SPARK-16642][SPARK-16721][SQL] Fixes three issues related to lead and lag functions

2016-07-25 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 4b38a6a53 -> 4391d4a3c [SPARK-16633][SPARK-16642][SPARK-16721][SQL] Fixes three issues related to lead and lag functions ## What changes were proposed in this pull request? This PR contains three changes. First, this PR changes the

spark git commit: [SPARK-16633][SPARK-16642][SPARK-16721][SQL] Fixes three issues related to lead and lag functions

2016-07-25 Thread yhuai
Repository: spark Updated Branches: refs/heads/master f99e34e8e -> 815f3eece [SPARK-16633][SPARK-16642][SPARK-16721][SQL] Fixes three issues related to lead and lag functions ## What changes were proposed in this pull request? This PR contains three changes. First, this PR changes the

spark git commit: [SPARK-16724] Expose DefinedByConstructorParams

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 aeb6d5c05 -> 4b38a6a53 [SPARK-16724] Expose DefinedByConstructorParams We don't generally make things in catalyst/execution private. Instead they are just undocumented due to their lack of stability guarantees. Author: Michael

spark git commit: [SPARK-16724] Expose DefinedByConstructorParams

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master 8a8d26f1e -> f99e34e8e [SPARK-16724] Expose DefinedByConstructorParams We don't generally make things in catalyst/execution private. Instead they are just undocumented due to their lack of stability guarantees. Author: Michael Armbrust

spark git commit: [SPARK-16672][SQL] SQLBuilder should not raise exceptions on EXISTS queries

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 3d3547487 -> aeb6d5c05 [SPARK-16672][SQL] SQLBuilder should not raise exceptions on EXISTS queries ## What changes were proposed in this pull request? Currently, `SQLBuilder` raises `empty.reduceLeft` exceptions on *unoptimized*

spark git commit: [SPARK-16672][SQL] SQLBuilder should not raise exceptions on EXISTS queries

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master ba0aade6d -> 8a8d26f1e [SPARK-16672][SQL] SQLBuilder should not raise exceptions on EXISTS queries ## What changes were proposed in this pull request? Currently, `SQLBuilder` raises `empty.reduceLeft` exceptions on *unoptimized* `EXISTS`

spark git commit: Fix description of spark.speculation.quantile

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master 3fc456694 -> ba0aade6d Fix description of spark.speculation.quantile ## What changes were proposed in this pull request? Minor doc fix regarding the spark.speculation.quantile configuration parameter. It incorrectly states it should be

spark git commit: Fix description of spark.speculation.quantile

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 9d581dc61 -> 3d3547487 Fix description of spark.speculation.quantile ## What changes were proposed in this pull request? Minor doc fix regarding the spark.speculation.quantile configuration parameter. It incorrectly states it should

spark git commit: [SPARK-16678][SPARK-16677][SQL] Fix two View-related bugs

2016-07-25 Thread wenchen
Repository: spark Updated Branches: refs/heads/master e164a04b2 -> 3fc456694 [SPARK-16678][SPARK-16677][SQL] Fix two View-related bugs ## What changes were proposed in this pull request? **Issue 1: Disallow Creating/Altering a View when the same-name Table Exists (without IF NOT EXISTS)**

spark git commit: [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails

2016-07-25 Thread tdas
Repository: spark Updated Branches: refs/heads/master db36e1e75 -> e164a04b2 [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails ## What changes were proposed in this pull request? This PR moves `ssc.stop()` into `finally` for

spark git commit: [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails

2016-07-25 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 b17fe4e41 -> 9d581dc61 [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails ## What changes were proposed in this pull request? This PR moves `ssc.stop()` into `finally` for

spark git commit: [SPARK-15590][WEBUI] Paginate Job Table in Jobs tab

2016-07-25 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master c979c8bba -> db36e1e75 [SPARK-15590][WEBUI] Paginate Job Table in Jobs tab ## What changes were proposed in this pull request? This patch adds pagination support for the Job Tables in the Jobs tab. Pagination is provided for all of the

spark git commit: [SPARK-14131][STREAMING] SQL Improved fix for avoiding potential deadlocks in HDFSMetadataLog

2016-07-25 Thread tdas
Repository: spark Updated Branches: refs/heads/master 12f490b5c -> c979c8bba [SPARK-14131][STREAMING] SQL Improved fix for avoiding potential deadlocks in HDFSMetadataLog ## What changes were proposed in this pull request? Current fix for deadlock disables interrupts in the StreamExecution

spark git commit: [SPARK-16715][TESTS] Fix a potential ExprId conflict for SubexpressionEliminationSuite."Semantic equals and hash"

2016-07-25 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 1b4f7cf13 -> 41e72f659 [SPARK-16715][TESTS] Fix a potential ExprId conflict for SubexpressionEliminationSuite."Semantic equals and hash" ## What changes were proposed in this pull request? SubexpressionEliminationSuite."Semantic

spark git commit: [SPARK-16715][TESTS] Fix a potential ExprId conflict for SubexpressionEliminationSuite."Semantic equals and hash"

2016-07-25 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master f5ea7fe53 -> 12f490b5c [SPARK-16715][TESTS] Fix a potential ExprId conflict for SubexpressionEliminationSuite."Semantic equals and hash" ## What changes were proposed in this pull request? SubexpressionEliminationSuite."Semantic equals

spark git commit: [SPARK-16166][CORE] Also take off-heap memory usage into consideration in log and webui display

2016-07-25 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master cda4603de -> f5ea7fe53 [SPARK-16166][CORE] Also take off-heap memory usage into consideration in log and webui display ## What changes were proposed in this pull request? Currently in the log and UI display, only on-heap storage memory

spark git commit: [SQL][DOC] Fix a default name for parquet compression

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 f0d05f669 -> 1b4f7cf13 [SQL][DOC] Fix a default name for parquet compression ## What changes were proposed in this pull request? This pr is to fix a wrong description for parquet default compression. Author: Takeshi YAMAMURO

spark git commit: [SQL][DOC] Fix a default name for parquet compression

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master fc17121d5 -> cda4603de [SQL][DOC] Fix a default name for parquet compression ## What changes were proposed in this pull request? This pr is to fix a wrong description for parquet default compression. Author: Takeshi YAMAMURO

spark git commit: Revert "[SPARK-15271][MESOS] Allow force pulling executor docker images"

2016-07-25 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 3b6e1d094 -> fc17121d5 Revert "[SPARK-15271][MESOS] Allow force pulling executor docker images" This reverts commit 978cd5f125eb5a410bad2e60bf8385b11cf1b978. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-16485][DOC][ML] Fixed several inline formatting in ml features doc

2016-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 d9bd066b9 -> f0d05f669 [SPARK-16485][DOC][ML] Fixed several inline formatting in ml features doc ## What changes were proposed in this pull request? Fixed several inline formatting in ml features doc. Before:

spark git commit: [SPARK-16485][DOC][ML] Fixed several inline formatting in ml features doc

2016-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master 978cd5f12 -> 3b6e1d094 [SPARK-16485][DOC][ML] Fixed several inline formatting in ml features doc ## What changes were proposed in this pull request? Fixed several inline formatting in ml features doc. Before:

spark git commit: [SPARK-15271][MESOS] Allow force pulling executor docker images

2016-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master dd784a882 -> 978cd5f12 [SPARK-15271][MESOS] Allow force pulling executor docker images ## What changes were proposed in this pull request? Mesos agents by default will not pull docker images which are cached locally already. In order to

spark git commit: [SPARK-16653][ML][OPTIMIZER] update ANN convergence tolerance param default to 1e-6

2016-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master b73defdd7 -> ad3708e78 [SPARK-16653][ML][OPTIMIZER] update ANN convergence tolerance param default to 1e-6 ## What changes were proposed in this pull request? replace ANN convergence tolerance param default from 1e-4 to 1e-6 so that it

spark git commit: [SPARK-16685] Remove audit-release scripts.

2016-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master ad3708e78 -> dd784a882 [SPARK-16685] Remove audit-release scripts. ## What changes were proposed in this pull request? This patch removes dev/audit-release. It was initially created to do basic release auditing. They have been unused by

spark git commit: [SPARKR][DOCS] fix broken url in doc

2016-07-25 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 57d65e511 -> d9bd066b9 [SPARKR][DOCS] fix broken url in doc ## What changes were proposed in this pull request? Fix broken url, also, sparkR.session.stop doc page should have it in the header, instead of saying "sparkR.stop"

spark git commit: [SPARKR][DOCS] fix broken url in doc

2016-07-25 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 7ea6d282b -> b73defdd7 [SPARKR][DOCS] fix broken url in doc ## What changes were proposed in this pull request? Fix broken url, also, sparkR.session.stop doc page should have it in the header, instead of saying "sparkR.stop"

spark git commit: [SPARK-16703][SQL] Remove extra whitespace in SQL generation for window functions

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 b52e639a8 -> 57d65e511 [SPARK-16703][SQL] Remove extra whitespace in SQL generation for window functions ## What changes were proposed in this pull request? This PR fixes a minor formatting issue of `WindowSpecDefinition.sql` when no

spark git commit: [SPARK-16703][SQL] Remove extra whitespace in SQL generation for window functions

2016-07-25 Thread rxin
Repository: spark Updated Branches: refs/heads/master 79826f3c7 -> 7ea6d282b [SPARK-16703][SQL] Remove extra whitespace in SQL generation for window functions ## What changes were proposed in this pull request? This PR fixes a minor formatting issue of `WindowSpecDefinition.sql` when no

spark git commit: [SPARK-16698][SQL] Field names having dots should be allowed for datasources based on FileFormat

2016-07-25 Thread lian
Repository: spark Updated Branches: refs/heads/branch-2.0 fcbb7f653 -> b52e639a8 [SPARK-16698][SQL] Field names having dots should be allowed for datasources based on FileFormat ## What changes were proposed in this pull request? It seems this is a regression assuming from

spark git commit: [SPARK-16698][SQL] Field names having dots should be allowed for datasources based on FileFormat

2016-07-25 Thread lian
Repository: spark Updated Branches: refs/heads/master d6a52176a -> 79826f3c7 [SPARK-16698][SQL] Field names having dots should be allowed for datasources based on FileFormat ## What changes were proposed in this pull request? It seems this is a regression assuming from

spark git commit: [SPARK-16668][TEST] Test parquet reader for row groups containing both dictionary and plain encoded pages

2016-07-25 Thread lian
Repository: spark Updated Branches: refs/heads/master 64529b186 -> d6a52176a [SPARK-16668][TEST] Test parquet reader for row groups containing both dictionary and plain encoded pages ## What changes were proposed in this pull request? This patch adds an explicit test for [SPARK-14217] by

spark git commit: [SPARK-16691][SQL] move BucketSpec to catalyst module and use it in CatalogTable

2016-07-25 Thread lian
Repository: spark Updated Branches: refs/heads/master d27d362eb -> 64529b186 [SPARK-16691][SQL] move BucketSpec to catalyst module and use it in CatalogTable ## What changes were proposed in this pull request? It's weird that we have `BucketSpec` to abstract bucket info, but don't use it in

spark git commit: [SPARK-16660][SQL] CreateViewCommand should not take CatalogTable

2016-07-25 Thread lian
Repository: spark Updated Branches: refs/heads/master 7ffd99ec5 -> d27d362eb [SPARK-16660][SQL] CreateViewCommand should not take CatalogTable ## What changes were proposed in this pull request? `CreateViewCommand` only needs some information of a `CatalogTable`, but not all of them. We

spark git commit: [SPARK-16674][SQL] Avoid per-record type dispatch in JDBC when reading

2016-07-25 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 68b4020d0 -> 7ffd99ec5 [SPARK-16674][SQL] Avoid per-record type dispatch in JDBC when reading ## What changes were proposed in this pull request? Currently, `JDBCRDD.compute` is doing type dispatch for each row to read appropriate

spark git commit: [SPARK-16648][SQL] Make ignoreNullsExpr a child expression of First and Last

2016-07-25 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 d226dce12 -> fcbb7f653 [SPARK-16648][SQL] Make ignoreNullsExpr a child expression of First and Last ## What changes were proposed in this pull request? Default `TreeNode.withNewChildren` implementation doesn't work for `Last` and

spark git commit: [SPARK-16648][SQL] Make ignoreNullsExpr a child expression of First and Last

2016-07-25 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 468a3c3ac -> 68b4020d0 [SPARK-16648][SQL] Make ignoreNullsExpr a child expression of First and Last ## What changes were proposed in this pull request? Default `TreeNode.withNewChildren` implementation doesn't work for `Last` and when