[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223438072 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,255 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223436337 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,255 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223440106 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestTestHelper.scala --- @@ -0,0 +1,90 @@ +/* + * Licensed t

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223438810 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,255 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223434738 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -288,9 +296,14 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223436523 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,255 @@ +/* + * Licensed

[GitHub] spark pull request #22674: [SPARK-25680][SQL] SQL execution listener shouldn...

2018-10-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22674#discussion_r223443736 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala --- @@ -75,95 +76,70 @@ trait QueryExecutionListener { */

[GitHub] spark pull request #22674: [SPARK-25680][SQL] SQL execution listener shouldn...

2018-10-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22674#discussion_r223443327 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -39,7 +39,14 @@ case class SparkListenerSQLExecutionStart(

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3800/

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22674 **[Test build #97123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97123/testReport)** for PR 22674 at commit [`28f64d0`](https://github.com/apache/spark/commit/28

[GitHub] spark issue #22672: [SPARK-25605][TESTS] Alternate take. Run cast string to ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22672 **[Test build #97113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97113/testReport)** for PR 22672 at commit [`4565859`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22672: [SPARK-25605][TESTS] Alternate take. Run cast string to ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97113/ Test PASSed. ---

[GitHub] spark issue #22672: [SPARK-25605][TESTS] Alternate take. Run cast string to ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22672 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223433455 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed to t

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223440836 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala --- @@ -134,6 +163,29 @@ class YarnAllocatorSuite exte

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223439004 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestTestHelper.scala --- @@ -0,0 +1,90 @@ +/* + * Licensed t

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223436285 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,255 @@ +/* + * Licensed

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22657 Then we don't need any randomness here, just pick one timezone(like PST?) and test it. --- - To unsubscribe, e-mail: reviews-u

[GitHub] spark issue #22649: [SPARK-25644][SS][FOLLOWUP][BUILD] Fix Scala 2.12 build ...

2018-10-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22649 This kind of stuff did fail when we were updating for 2.12 and we had to make a lot of similar changes to the Java code for this reason, yeah. --- --

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22630 **[Test build #97111 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97111/testReport)** for PR 22630 at commit [`eac31b2`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22630 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97111/ Test PASSed. ---

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22630 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97122/ Test FAILed. ---

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22674 **[Test build #97122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97122/testReport)** for PR 22674 at commit [`1701f3b`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223447961 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile var exitedC

[GitHub] spark issue #22557: [SPARK-25535][core] Work around bad error handling in co...

2018-10-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22557 Ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22641 **[Test build #97117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97117/testReport)** for PR 22641 at commit [`fdab980`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22641 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97117/ Test PASSed. ---

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22641 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22624: [SPARK-23781][CORE] Add base class for token renewal fun...

2018-10-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22624 > ...calls fetchDelegationTokens() twice since the tokenRenewalInterval... That is true - the second fetch is only needed for YARN because reasons. But that's also completely unrelated to thi

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97123/ Test FAILed. ---

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22674 **[Test build #97123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97123/testReport)** for PR 22674 at commit [`28f64d0`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #22674: [SPARK-25680][SQL] SQL execution listener shouldn...

2018-10-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22674#discussion_r223450058 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala --- @@ -75,95 +76,70 @@ trait QueryExecutionListener {

[GitHub] spark issue #22672: [SPARK-25605][TESTS] Alternate take. Run cast string to ...

2018-10-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22672 The original change in #22631 made the test time go down from about 2:30 to 0:17. See build 96945: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96945/testReport/junit/org.ap

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222810125 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r223421728 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +505,92 @@ class TaskSchedulerImplSuite extends SparkF

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222785940 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -117,6 +117,12 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222811152 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222816923 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #97124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97124/testReport)** for PR 22295 at commit [`55f1b03`](https://github.com/apache/spark/commit/55

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222810004 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222814235 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222811527 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222816658 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222785870 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -117,6 +117,12 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222783407 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -597,6 +597,17 @@ package object config { .checkValue(v =>

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r223421506 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +505,92 @@ class TaskSchedulerImplSuite extends SparkF

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222813512 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -597,6 +597,17 @@ package object config { .checkValue(v =>

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r223451952 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -597,6 +597,17 @@ package object config { .checkValue(v =>

[GitHub] spark issue #22673: [SPARK-20144] Allow reading files in order with spark.sq...

2018-10-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22673 @darabos . Thank you for trying to make a contribution. However, we had better discuss on that JIRA first before making a PR. Especially, for SPARK-20144 which is discussed already, it does.

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3801/

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97114/testReport)** for PR 22598 at commit [`a4ab4f5`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97114/ Test PASSed. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread devaraj-kavali
Github user devaraj-kavali commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223453234 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile var

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223453846 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile var exitedC

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21710 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3802/

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21710 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21710 **[Test build #97125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97125/testReport)** for PR 21710 at commit [`0393076`](https://github.com/apache/spark/commit/03

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223454115 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile var exitedC

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97115/testReport)** for PR 22598 at commit [`8c860fe`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...

2018-10-08 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 pulled into master, thanks @redsanket --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97115/ Test PASSed. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223456079 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1989,22 +2026,211 @@ head(sql("select * from aggregates")) -# Using F

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223456294 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -2709,6 +2935,78 @@ write.stream(aggDF, "memory", outputMode = "complete", checkpointLocation =

[GitHub] spark issue #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch, python...

2018-10-08 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/22627 @holdenk yeah, i intend to backport this to 2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark pull request #22628: [SPARK-25641] Change the spark.shuffle.server.chu...

2018-10-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22628 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22645: [SPARK-25566][SPARK-25567][WEBUI][SQL]Support pagination...

2018-10-08 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/22645 Hi @srowen , There is one behavior change this PR introduces, which is correct. Sorting Job Ids in the previous versions of spark was not proper. After the PR the sorting is proper. ![sc

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-10-08 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r223459186 --- Diff: docs/sql-programming-guide.md --- @@ -1890,6 +1890,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see # Mi

[GitHub] spark issue #22635: [SPARK-25591][PySpark][SQL] Avoid overwriting deserializ...

2018-10-08 Thread AbdealiJK
Github user AbdealiJK commented on the issue: https://github.com/apache/spark/pull/22635 @cloud-fan @viirya Any chance of this making it into 2.4 ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org F

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-10-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r223460909 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,248 @@ object ArrayUnion

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #97124 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97124/testReport)** for PR 22295 at commit [`55f1b03`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97124/ Test PASSed. ---

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223461492 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -0,0 +1,161 @

[GitHub] spark issue #22624: [SPARK-23781][CORE] Add base class for token renewal fun...

2018-10-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22624 Given the lack of reviews let me ping some people: @skonto @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apa

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223461972 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/HadoopConfExecutorFeatureStep.scala --- @@ -0,0 +1,51 @@

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223461273 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,45 @@ private[client] class Shim_v0_13 extends Shim_v0_

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223459567 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -544,6 +544,15 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223462348 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -79,12 +82,30 @@ class HiveClientSuite(version: String)

[GitHub] spark issue #21755: Doc fix: The Imputer is an Estimator

2018-10-08 Thread zoltanctoth
Github user zoltanctoth commented on the issue: https://github.com/apache/spark/pull/21755 @srowen Just about to submit a new doc relates pull request. Wondering if your `PS see https://spark.apache.org/contributing.html` line referred to anything specific about how I should issue

[GitHub] spark issue #21755: Doc fix: The Imputer is an Estimator

2018-10-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21755 Minor stuff: we usually tag this with `[MINOR]` in the title to be clear there's no JIRA. Also ideal to batch together small related changes but I don't know that there was anything else to do here.

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

2018-10-08 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/22612#discussion_r223466260 --- Diff: core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala --- @@ -59,6 +60,43 @@ case object JVMOffHeapMemory extends ExecutorMetricT

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223467868 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -0,0 +1,161 @

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223468018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -518,56 +521,81 @@ class SQLMetricsSuite extends SparkFu

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread kmanamcheri
Github user kmanamcheri commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223469446 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -79,12 +82,30 @@ class HiveClientSuite(version: String)

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22466 **[Test build #97118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97118/testReport)** for PR 22466 at commit [`c50e99b`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22466 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97118/ Test PASSed. ---

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22466 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22615: [SPARK-25016][BUILD][CORE] Remove support for Hadoop 2.6

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22615 **[Test build #97116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97116/testReport)** for PR 22615 at commit [`fb2c90d`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread kmanamcheri
Github user kmanamcheri commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223473324 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,45 @@ private[client] class Shim_v0_13 extends Shi

[GitHub] spark issue #22615: [SPARK-25016][BUILD][CORE] Remove support for Hadoop 2.6

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22615 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22614: [SPARK-25561][SQL] Implement a new config to control par...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22614 **[Test build #97126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97126/testReport)** for PR 22614 at commit [`01e2123`](https://github.com/apache/spark/commit/01

[GitHub] spark issue #22615: [SPARK-25016][BUILD][CORE] Remove support for Hadoop 2.6

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22615 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97116/ Test PASSed. ---

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474853 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,73 @@ case class RangeExec(range: org.apac

<    1   2   3   4   5   6   >