spark git commit: [SPARK-22159][SQL][FOLLOW-UP] Make config names consistently end with "enabled".

2017-10-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/master bebd2e1ce -> af8a34c78 [SPARK-22159][SQL][FOLLOW-UP] Make config names consistently end with "enabled". ## What changes were proposed in this pull request? This is a follow-up of #19384. In the previous pr, only definitions of the config

spark git commit: [SPARK-22222][CORE] Fix the ARRAY_MAX in BufferHolder and add a test

2017-10-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 71c2b81aa -> bebd2e1ce [SPARK-2][CORE] Fix the ARRAY_MAX in BufferHolder and add a test ## What changes were proposed in this pull request? We should not break the assumption that the length of the allocated byte array is word

spark git commit: [SPARK-22230] Swap per-row order in state store restore.

2017-10-09 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 155ab6347 -> 71c2b81aa [SPARK-22230] Swap per-row order in state store restore. ## What changes were proposed in this pull request? In state store restore, for each row, put the saved state before the row in the iterator instead of after.

spark git commit: [SPARK-22170][SQL] Reduce memory consumption in broadcast joins.

2017-10-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/master dadd13f36 -> 155ab6347 [SPARK-22170][SQL] Reduce memory consumption in broadcast joins. ## What changes were proposed in this pull request? This updates the broadcast join code path to lazily decompress pages and iterate through

spark git commit: [SPARK] Misleading error message for missing --proxy-user value

2017-10-09 Thread srowen
Repository: spark Updated Branches: refs/heads/master b650ee026 -> dadd13f36 [SPARK] Misleading error message for missing --proxy-user value Fix misleading error message when argument is expected. ## What changes were proposed in this pull request? Change message to be accurate. ## How was

spark git commit: [INFRA] Close stale PRs.

2017-10-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/master a74ec6d7b -> b650ee026 [INFRA] Close stale PRs. Closes #19423 Closes #19455 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b650ee02 Tree:

spark git commit: [SPARK-22218] spark shuffle services fails to update secret on app re-attempts

2017-10-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 0d3f1667e -> c5889b59d [SPARK-22218] spark shuffle services fails to update secret on app re-attempts This patch fixes application re-attempts when running spark on yarn using the external shuffle service with security on. Currently

spark git commit: [SPARK-22218] spark shuffle services fails to update secret on app re-attempts

2017-10-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/master f31e11404 -> a74ec6d7b [SPARK-22218] spark shuffle services fails to update secret on app re-attempts This patch fixes application re-attempts when running spark on yarn using the external shuffle service with security on. Currently

spark git commit: [SPARK-21568][CORE] ConsoleProgressBar should only be enabled in shells

2017-10-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 98057583d -> f31e11404 [SPARK-21568][CORE] ConsoleProgressBar should only be enabled in shells ## What changes were proposed in this pull request? This PR disables console progress bar feature in non-shell environment by overriding the

svn commit: r22186 - /dev/spark/spark-2.1.2-rc4-bin/ /release/spark/spark-2.1.2/

2017-10-09 Thread mlnick
Author: mlnick Date: Mon Oct 9 19:37:45 2017 New Revision: 22186 Log: Release Spark 2.1.2 Added: release/spark/spark-2.1.2/ - copied from r22185, dev/spark/spark-2.1.2-rc4-bin/ Removed: dev/spark/spark-2.1.2-rc4-bin/

svn commit: r22176 - /dev/spark/spark-2.1.2-rc4-bin/

2017-10-09 Thread holden
Author: holden Date: Mon Oct 9 17:56:14 2017 New Revision: 22176 Log: Add spark-2.1.2-rc4 Added: dev/spark/spark-2.1.2-rc4-bin/ dev/spark/spark-2.1.2-rc4-bin/SparkR_2.1.2.tar.gz (with props) dev/spark/spark-2.1.2-rc4-bin/SparkR_2.1.2.tar.gz.asc

spark git commit: [SPARK-20679][ML] Support recommending for a subset of users/items in ALSModel

2017-10-09 Thread mlnick
Repository: spark Updated Branches: refs/heads/master fe7b219ae -> 98057583d [SPARK-20679][ML] Support recommending for a subset of users/items in ALSModel This PR adds methods `recommendForUserSubset` and `recommendForItemSubset` to `ALSModel`. These allow recommending for a specified set

spark git commit: [SPARK-22074][CORE] Task killed by other attempt task should not be resubmitted

2017-10-09 Thread jshao
Repository: spark Updated Branches: refs/heads/master c998a2ae0 -> fe7b219ae [SPARK-22074][CORE] Task killed by other attempt task should not be resubmitted ## What changes were proposed in this pull request? As the detail scenario described in