[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649991405 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649991400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649991400 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649912241 **[Test build #124525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124525/testReport)** for PR 28898 at commit [`4c705bd`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649990865 **[Test build #124525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124525/testReport)** for PR 28898 at commit [`4c705bd`](https://github.co

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28928: [SPARK-32098][PYTHON] Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

HyukjinKwon commented on a change in pull request #28928: URL: https://github.com/apache/spark/pull/28928#discussion_r445977049 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -413,7 +413,7 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone):

[GitHub] [spark] gatorsmile commented on a change in pull request #28928: [SPARK-32098][PYTHON] Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

gatorsmile commented on a change in pull request #28928: URL: https://github.com/apache/spark/pull/28928#discussion_r445976618 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -413,7 +413,7 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649976751 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649976746 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649976746 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649936846 **[Test build #124527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124527/testReport)** for PR 28898 at commit [`ab39d24`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649976637 **[Test build #124527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124527/testReport)** for PR 28898 at commit [`ab39d24`](https://github.co

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-649968228 In the latest update, there are three changes: 1. `failedEpoch` and `fileLostEpoch` are renamed and comments explaining what they are are expanded, largely based on suggestions

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is l

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-649963283 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-649963283 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28805: [SPARK-28169][SQL] Convert scan predicate condition to CNF

AmplabJenkins removed a comment on pull request #28805: URL: https://github.com/apache/spark/pull/28805#issuecomment-649963207 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28805: [SPARK-28169][SQL] Convert scan predicate condition to CNF

AmplabJenkins commented on pull request #28805: URL: https://github.com/apache/spark/pull/28805#issuecomment-649963207 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] wypoon commented on a change in pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r445965785 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -177,6 +177,8 @@ private[spark] class DAGScheduler( // TODO: Garba

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-649962628 **[Test build #124530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124530/testReport)** for PR 28848 at commit [`d09ef93`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28805: [SPARK-28169][SQL] Convert scan predicate condition to CNF

SparkQA commented on pull request #28805: URL: https://github.com/apache/spark/pull/28805#issuecomment-649962618 **[Test build #124531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124531/testReport)** for PR 28805 at commit [`270324e`](https://github.com

[GitHub] [spark] wypoon commented on a change in pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r445965319 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -177,6 +177,8 @@ private[spark] class DAGScheduler( // TODO: Garba

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649954884 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649954884 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649954378 **[Test build #124529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124529/testReport)** for PR 28898 at commit [`652c77f`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-649944669 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-649944669 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-649944172 **[Test build #124528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124528/testReport)** for PR 28676 at commit [`488e051`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649942950 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649942879 **[Test build #124524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124524/testReport)** for PR 28898 at commit [`3c8cf11`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649942946 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649942946 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649907459 **[Test build #124524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124524/testReport)** for PR 28898 at commit [`3c8cf11`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649937189 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649937189 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649936846 **[Test build #124527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124527/testReport)** for PR 28898 at commit [`ab39d24`](https://github.com

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445946556 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

AmplabJenkins removed a comment on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649930413 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

SparkQA removed a comment on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649924942 **[Test build #124526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124526/testReport)** for PR 28927 at commit [`a0756db`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

AmplabJenkins commented on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649930413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

SparkQA commented on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649930269 **[Test build #124526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124526/testReport)** for PR 28927 at commit [`a0756db`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

AmplabJenkins removed a comment on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649925471 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

AmplabJenkins commented on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649925471 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

SparkQA commented on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649924942 **[Test build #124526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124526/testReport)** for PR 28927 at commit [`a0756db`](https://github.com

[GitHub] [spark] HyukjinKwon commented on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

HyukjinKwon commented on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649923516 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28927: [SPARK-32099][DOCS] Remove broken link in cloud integration documentation

AmplabJenkins removed a comment on pull request #28927: URL: https://github.com/apache/spark/pull/28927#issuecomment-649467834 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] viirya commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

viirya commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445935640 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class NestedColu

[GitHub] [spark] viirya commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

viirya commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445935343 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class NestedColu

[GitHub] [spark] viirya commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

viirya commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445935219 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,14 @@ object NestedColumnAlia

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649912241 **[Test build #124525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124525/testReport)** for PR 28898 at commit [`4c705bd`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649909806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649909806 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649907836 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649907836 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window/sort functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649907459 **[Test build #124524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124524/testReport)** for PR 28898 at commit [`3c8cf11`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28897: [SPARK-32058][BUILD] Use Apache Hadoop 3.2.0 dependency by default

AmplabJenkins removed a comment on pull request #28897: URL: https://github.com/apache/spark/pull/28897#issuecomment-649906877 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28897: [SPARK-32058][BUILD] Use Apache Hadoop 3.2.0 dependency by default

AmplabJenkins commented on pull request #28897: URL: https://github.com/apache/spark/pull/28897#issuecomment-649906877 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dilipbiswal commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

dilipbiswal commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-649906157 @maropu Resolved the conflicts. Thank you. This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #28897: [SPARK-32058][BUILD] Use Apache Hadoop 3.2.0 dependency by default

SparkQA commented on pull request #28897: URL: https://github.com/apache/spark/pull/28897#issuecomment-649906173 **[Test build #124523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124523/testReport)** for PR 28897 at commit [`2434365`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28897: [SPARK-32058][BUILD] Use Apache Hadoop 3.2.0 dependency by default

SparkQA removed a comment on pull request #28897: URL: https://github.com/apache/spark/pull/28897#issuecomment-649865700 **[Test build #124523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124523/testReport)** for PR 28897 at commit [`2434365`](https://gi

[GitHub] [spark] HyukjinKwon closed pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

HyukjinKwon closed pull request #28896: URL: https://github.com/apache/spark/pull/28896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

HyukjinKwon commented on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649901085 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] AmplabJenkins commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

AmplabJenkins commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-649898619 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

AmplabJenkins removed a comment on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-649898619 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

SparkQA removed a comment on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-649795495 **[Test build #124520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124520/testReport)** for PR 28425 at commit [`7ce28a2`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

SparkQA commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-649898054 **[Test build #124520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124520/testReport)** for PR 28425 at commit [`7ce28a2`](https://github.co

[GitHub] [spark] TJX2014 edited a comment on pull request #28918: [SPARK-32068][WEBUI] Task lauchtime in stage tab not correct

TJX2014 edited a comment on pull request #28918: URL: https://github.com/apache/spark/pull/28918#issuecomment-649862045 > According to the following documents, this change seems work with recent browsers. > https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445916122 ## File path: python/pyspark/sql/readwriter.py ## @@ -1048,6 +1048,128 @@ def jdbc(self, url, table, mode=None, properties=None): self.mode(m

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445910093 ## File path: python/pyspark/sql/readwriter.py ## @@ -1048,6 +1048,128 @@ def jdbc(self, url, table, mode=None, properties=None): self.mode(m

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #28805: [SPARK-28169][SQL] Convert scan predicate condition to CNF

AngersZh commented on a change in pull request #28805: URL: https://github.com/apache/spark/pull/28805#discussion_r445914791 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruneFileSourcePartitionsSuite.scala ## @@ -108,4 +109,54 @@ class PruneFi

[GitHub] [spark] HyukjinKwon edited a comment on pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon edited a comment on pull request #27331: URL: https://github.com/apache/spark/pull/27331#issuecomment-649888157 > I haven't replied because I don't see how it is an important concern. @rdblue, I explained multiple times why I think this is relevant and important - once yo

[GitHub] [spark] HyukjinKwon commented on pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on pull request #27331: URL: https://github.com/apache/spark/pull/27331#issuecomment-649888157 > I haven't replied because I don't see how it is an important concern. @rdblue, I explained multiple times why I think this is relevant and important - once you add t

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445910093 ## File path: python/pyspark/sql/readwriter.py ## @@ -1048,6 +1048,128 @@ def jdbc(self, url, table, mode=None, properties=None): self.mode(m

[GitHub] [spark] maropu commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649884335 This is not a bugfix, so we will merge this commit only into master(v3.1.0). This is an automated message from th

[GitHub] [spark] frankyin-factual commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

frankyin-factual commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649883695 Also, how likely this will get backported to 2.4.x versions? This is an automated message from the Apa

[GitHub] [spark] HyukjinKwon commented on pull request #28928: [SPARK-32098][PYTHON] Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

HyukjinKwon commented on pull request #28928: URL: https://github.com/apache/spark/pull/28928#issuecomment-649883425 Thank you @BryanCutler and @ueshin! This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] github-actions[bot] closed pull request #26816: [SPARK-30191][YARN] optimize yarn allocator

github-actions[bot] closed pull request #26816: URL: https://github.com/apache/spark/pull/26816 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] github-actions[bot] commented on pull request #27377: [SPARK-30666][Core][WIP] Reliable single-stage accumulators

github-actions[bot] commented on pull request #27377: URL: https://github.com/apache/spark/pull/27377#issuecomment-649881487 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] commented on pull request #25721: [WIP][SPARK-29018][SQL] Implement Spark Thrift Server with it's own code base on PROTOCOL_VERSION_V9

github-actions[bot] commented on pull request #25721: URL: https://github.com/apache/spark/pull/25721#issuecomment-649881504 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] closed pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to PythonUDF.

github-actions[bot] closed pull request #18906: URL: https://github.com/apache/spark/pull/18906 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] github-actions[bot] closed pull request #26711: [SPARK-30069][CORE][YARN] Clean up non-shuffle disk block manager files following executor exists on YARN

github-actions[bot] closed pull request #26711: URL: https://github.com/apache/spark/pull/26711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445904606 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +129,39 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445903970 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +129,39 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445903857 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -32,7 +32,9 @@ object NestedC

[GitHub] [spark] maropu commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

maropu commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649877864 @alismess-db Looks the valid test failures. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445903069 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -32,7 +32,9 @@ object NestedColumnAlias

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445902694 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -32,7 +32,9 @@ object NestedColumnAlias

[GitHub] [spark] holdenk commented on pull request #28864: [SPARK-32004][ALL] Drop references to slave

holdenk commented on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-649877003 If there are no more comments by EOW I'll merge this. This is an automated message from the Apache Git Service.

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-649874411 > @wypoon if you have not started extending the test with the multiple fetch failures case you can use this I you agree with it: > [attilapiros@be14a51](https://github.com/att

[GitHub] [spark] sap1ens commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

sap1ens commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445898815 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +129,39 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445885603 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +131,40 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28929: [SPARK-32100][CORE][TESTS] Add WorkerDecommissionExtendedSuite

AmplabJenkins removed a comment on pull request #28929: URL: https://github.com/apache/spark/pull/28929#issuecomment-649870481 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28929: [SPARK-32100][CORE][TESTS] Add WorkerDecommissionExtendedSuite

AmplabJenkins commented on pull request #28929: URL: https://github.com/apache/spark/pull/28929#issuecomment-649870481 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445894796 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2656,6 +2656,16 @@ object SQLConf { .checkValue(_ > 0,

[GitHub] [spark] SparkQA removed a comment on pull request #28929: [SPARK-32100][CORE][TESTS] Add WorkerDecommissionExtendedSuite

SparkQA removed a comment on pull request #28929: URL: https://github.com/apache/spark/pull/28929#issuecomment-649809047 **[Test build #124522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124522/testReport)** for PR 28929 at commit [`3da70ec`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28929: [SPARK-32100][CORE][TESTS] Add WorkerDecommissionExtendedSuite

SparkQA commented on pull request #28929: URL: https://github.com/apache/spark/pull/28929#issuecomment-649869826 **[Test build #124522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124522/testReport)** for PR 28929 at commit [`3da70ec`](https://github.co

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445894796 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2656,6 +2656,16 @@ object SQLConf { .checkValue(_ > 0,

[GitHub] [spark] dongjoon-hyun commented on pull request #28897: [SPARK-32058][BUILD] Use Apache Hadoop 3.2.0 dependency by default

dongjoon-hyun commented on pull request #28897: URL: https://github.com/apache/spark/pull/28897#issuecomment-649868767 Hi, @srowen , @HyukjinKwon , @gatorsmile , @holdenk , @dbtsai . According to your comments and advices, I updated the PR description clearly and focused on only Apache-s

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445893610 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2656,6 +2656,16 @@ object SQLConf { .checkValue(_ > 0,

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r445884598 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +129,39 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] AmplabJenkins commented on pull request #28897: [SPARK-32058][BUILD] Use Apache Hadoop 3.2.0 dependency by default

AmplabJenkins commented on pull request #28897: URL: https://github.com/apache/spark/pull/28897#issuecomment-649866080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

  1   2   3   >