[GitHub] [spark] SparkQA commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

SparkQA commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792555098 **[Test build #135851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135851/testReport)** for PR 31711 at commit [`279d7c3`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #31765: [SPARK-34615][SQL] Support `java.time.Period` as an external type of the year-month interval type

cloud-fan commented on a change in pull request #31765: URL: https://github.com/apache/spark/pull/31765#discussion_r589229213 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/IntervalUtils.scala ## @@ -791,4 +791,35 @@ object IntervalUtils { * @

[GitHub] [spark] maropu commented on a change in pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

maropu commented on a change in pull request #31775: URL: https://github.com/apache/spark/pull/31775#discussion_r589225142 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -962,7 +962,7 @@ abstract class AnsiCastSuiteBas

[GitHub] [spark] cloud-fan commented on pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

cloud-fan commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792548773 late LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] LuciferYang commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

LuciferYang commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792548254 Thx ~ @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] maropu commented on pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

maropu commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792546605 late lgtm. This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] amandeep-sharma commented on a change in pull request #31769: [SPARK-34649][SQL] org.apache.spark.sql.DataFrameNaFunctions.replace() fails for column name having a dot

amandeep-sharma commented on a change in pull request #31769: URL: https://github.com/apache/spark/pull/31769#discussion_r589222586 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala ## @@ -485,4 +485,25 @@ class DataFrameNaFunctionsSuit

[GitHub] [spark] dongjoon-hyun closed pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

dongjoon-hyun closed pull request #31775: URL: https://github.com/apache/spark/pull/31775 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

dongjoon-hyun commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792544591 This is a test-only PR and I tested this manually. Merged to branch-3.1 to recover the branch. This is an

[GitHub] [spark] dongjoon-hyun closed pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

dongjoon-hyun closed pull request #31711: URL: https://github.com/apache/spark/pull/31711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] kiszk commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

kiszk commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589219424 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -961,7 +961,7 @@ abstract class AnsiCastSuiteBase

[GitHub] [spark] dongjoon-hyun commented on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

dongjoon-hyun commented on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792541188 Anyway, I'll make this as a draft for now. This is an automated message from the Apache Git Service. To re

[GitHub] [spark] kiszk commented on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

kiszk commented on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792540776 I see. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

dongjoon-hyun edited a comment on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792540196 This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] SparkQA removed a comment on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

SparkQA removed a comment on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792480080 **[Test build #135856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135856/testReport)** for PR 30763 at commit [`abcf8f3`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

SparkQA commented on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792540315 **[Test build #135856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135856/testReport)** for PR 30763 at commit [`abcf8f3`](https://github.co

[GitHub] [spark] amandeep-sharma commented on a change in pull request #31769: [SPARK-34649][SQL] org.apache.spark.sql.DataFrameNaFunctions.replace() fails for column name having a dot

amandeep-sharma commented on a change in pull request #31769: URL: https://github.com/apache/spark/pull/31769#discussion_r589218756 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala ## @@ -359,6 +359,16 @@ final class DataFrameNaFunctions pr

[GitHub] [spark] dongjoon-hyun commented on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

dongjoon-hyun commented on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792540196 No~, @kiszk . `testFallback` is required for the two test cases of SPARK-34596 and SPARK34607 because we had better **explicitly** use `testFallback` for those test case w

[GitHub] [spark] kiszk commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

kiszk commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589215812 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -961,7 +961,7 @@ abstract class AnsiCastSuiteBase

[GitHub] [spark] kiszk commented on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

kiszk commented on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792535973 @dongjoon-hyun Should we move #31775 for branch-3.1? Then, will you close this PR? This is an automated message f

[GitHub] [spark] dongjoon-hyun commented on pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

dongjoon-hyun commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792533499 Thank you, @kiszk ! This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] dongjoon-hyun commented on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

dongjoon-hyun commented on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792531944 I made a PR for recovering branch-3.1. - https://github.com/apache/spark/pull/31775 This is an automate

[GitHub] [spark] kiszk commented on pull request #31775: [SPARK-34660][TESTS][3.1] Don't use ParVector with `withExistingConf` which is not thread-safe

kiszk commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792531482 Great analysis! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun commented on pull request #31775: [SPARK-34660][TESTS] Don't use ParVector with `withExistingConf` which is not thread-safe

dongjoon-hyun commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792529201 cc @leanken , @cloud-fan , @rednaxelafx , @maropu , @kiszk , @viirya This is an automated message from th

[GitHub] [spark] SparkQA commented on pull request #31775: [SPARK-34660][TESTS] Don't use ParVector with `withExistingConf` which is not thread-safe

SparkQA commented on pull request #31775: URL: https://github.com/apache/spark/pull/31775#issuecomment-792527967 **[Test build #135857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135857/testReport)** for PR 31775 at commit [`6bfb870`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

AmplabJenkins removed a comment on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792526742 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40438/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31712: [SPARK-34598][SQL] RewritePredicateSubquery Rule must not update Filters without subqueries

AmplabJenkins removed a comment on pull request #31712: URL: https://github.com/apache/spark/pull/31712#issuecomment-792526744 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40435/

[GitHub] [spark] AmplabJenkins commented on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

AmplabJenkins commented on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792526742 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40438/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31712: [SPARK-34598][SQL] RewritePredicateSubquery Rule must not update Filters without subqueries

AmplabJenkins commented on pull request #31712: URL: https://github.com/apache/spark/pull/31712#issuecomment-792526744 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40435/ -

[GitHub] [spark] dongjoon-hyun opened a new pull request #31775: [SPARK-34660][TESTS] Don't use ParVector with `withExistingConf` which is not thread-safe

dongjoon-hyun opened a new pull request #31775: URL: https://github.com/apache/spark/pull/31775 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was t

[GitHub] [spark] HyukjinKwon commented on pull request #31721: [SPARK-34603][SQL] Support ADD ARCHIVE and LIST ARCHIVES command

HyukjinKwon commented on pull request #31721: URL: https://github.com/apache/spark/pull/31721#issuecomment-792521268 cc @maropu too This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] maropu commented on pull request #31769: [SPARK-34649][SQL] org.apache.spark.sql.DataFrameNaFunctions.replace() fails for column name having a dot

maropu commented on pull request #31769: URL: https://github.com/apache/spark/pull/31769#issuecomment-792521209 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #31769: [SPARK-34649][SQL] org.apache.spark.sql.DataFrameNaFunctions.replace() fails for column name having a dot

maropu commented on a change in pull request #31769: URL: https://github.com/apache/spark/pull/31769#discussion_r589205084 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala ## @@ -485,4 +485,25 @@ class DataFrameNaFunctionsSuite extends

[GitHub] [spark] dongjoon-hyun commented on pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

dongjoon-hyun commented on pull request #31764: URL: https://github.com/apache/spark/pull/31764#issuecomment-792519186 Hi, All. Currently, this PR is related two stuff. 1. `testFallback` is designed to clarify the goal of test case. 2. Recovering `branch-3.1`. For (2),

[GitHub] [spark] SparkQA commented on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

SparkQA commented on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792518903 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40438/ ---

[GitHub] [spark] baibaichen commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

baibaichen commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-792516852 Thanks @huaxingao we did some tests on aggregate push down in real product environment last month. here are results 1. datasets: 550M records 2. 4 click-hous

[GitHub] [spark] SparkQA commented on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

SparkQA commented on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792500440 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40438/ -

[GitHub] [spark] zhengruifeng commented on pull request #31693: [SPARK-34448][ML] Binary logistic regression incorrectly computes the intercept and coefficients with small var features

zhengruifeng commented on pull request #31693: URL: https://github.com/apache/spark/pull/31693#issuecomment-792496074 @srowen I think it is ok. @dbtsai @mengxr Could I ping you here? Since it seems that existing behavior of standardization without removing centers originated from th

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31666: [SPARK-34527][SQL] Resolve duplicated common columns from USING/NATURAL JOIN

AmplabJenkins removed a comment on pull request #31666: URL: https://github.com/apache/spark/pull/31666#issuecomment-792494995 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40436/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

AmplabJenkins removed a comment on pull request #31102: URL: https://github.com/apache/spark/pull/31102#issuecomment-792495018 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40437/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31691: [SPARK-34575][SQL] Push down limit through window when partitionSpec is empty

AmplabJenkins removed a comment on pull request #31691: URL: https://github.com/apache/spark/pull/31691#issuecomment-792495017 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40434/

[GitHub] [spark] AmplabJenkins commented on pull request #31666: [SPARK-34527][SQL] Resolve duplicated common columns from USING/NATURAL JOIN

AmplabJenkins commented on pull request #31666: URL: https://github.com/apache/spark/pull/31666#issuecomment-792494995 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40436/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31691: [SPARK-34575][SQL] Push down limit through window when partitionSpec is empty

AmplabJenkins commented on pull request #31691: URL: https://github.com/apache/spark/pull/31691#issuecomment-792495017 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40434/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

AmplabJenkins commented on pull request #31102: URL: https://github.com/apache/spark/pull/31102#issuecomment-792495018 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40437/ -

[GitHub] [spark] SparkQA commented on pull request #31666: [SPARK-34527][SQL] Resolve duplicated common columns from USING/NATURAL JOIN

SparkQA commented on pull request #31666: URL: https://github.com/apache/spark/pull/31666#issuecomment-792494964 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40436/ ---

[GitHub] [spark] Ngone51 commented on a change in pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

Ngone51 commented on a change in pull request #31763: URL: https://github.com/apache/spark/pull/31763#discussion_r589188420 ## File path: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ## @@ -52,6 +52,13 @@ private[spark] sealed trait MapStatus { * partition

[GitHub] [spark] SparkQA commented on pull request #31666: [SPARK-34527][SQL] Resolve duplicated common columns from USING/NATURAL JOIN

SparkQA commented on pull request #31666: URL: https://github.com/apache/spark/pull/31666#issuecomment-792490190 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40436/ -

[GitHub] [spark] SparkQA commented on pull request #31691: [SPARK-34575][SQL] Push down limit through window when partitionSpec is empty

SparkQA commented on pull request #31691: URL: https://github.com/apache/spark/pull/31691#issuecomment-792487328 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40434/ ---

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

dongjoon-hyun commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589183759 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala ## @@ -51,6 +51,15 @@ trait CodegenInterpretedPlanTest

[GitHub] [spark] maropu commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

maropu commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589182718 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala ## @@ -214,7 +214,8 @@ class ExpressionEncod

[GitHub] [spark] SparkQA commented on pull request #30763: [SPARK-31801][API][SHUFFLE] Register map output metadata

SparkQA commented on pull request #30763: URL: https://github.com/apache/spark/pull/30763#issuecomment-792480080 **[Test build #135856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135856/testReport)** for PR 30763 at commit [`abcf8f3`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31712: [SPARK-34598][SQL] RewritePredicateSubquery Rule must not update Filters without subqueries

SparkQA commented on pull request #31712: URL: https://github.com/apache/spark/pull/31712#issuecomment-792480053 **[Test build #135853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135853/testReport)** for PR 31712 at commit [`1de70e4`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

SparkQA commented on pull request #31102: URL: https://github.com/apache/spark/pull/31102#issuecomment-792479964 **[Test build #135855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135855/testReport)** for PR 31102 at commit [`9678bff`](https://github.com

[GitHub] [spark] maropu commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

maropu commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589179924 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala ## @@ -51,6 +51,15 @@ trait CodegenInterpretedPlanTest extend

[GitHub] [spark] maropu commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

maropu commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589179603 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala ## @@ -51,6 +51,15 @@ trait CodegenInterpretedPlanTest extend

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31666: [SPARK-34527][SQL] Resolve duplicated common columns from USING/NATURAL JOIN

AmplabJenkins removed a comment on pull request #31666: URL: https://github.com/apache/spark/pull/31666#issuecomment-792473332 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135854/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31666: [SPARK-34527][SQL] Resolve duplicated common columns from USING/NATURAL JOIN

AmplabJenkins commented on pull request #31666: URL: https://github.com/apache/spark/pull/31666#issuecomment-792473332 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135854/ -

[GitHub] [spark] SparkQA commented on pull request #31691: [SPARK-34575][SQL] Push down limit through window when partitionSpec is empty

SparkQA commented on pull request #31691: URL: https://github.com/apache/spark/pull/31691#issuecomment-792470565 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40434/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31773: [SPARK-34657][PYTHON][DOCS] Replace the tag of release to the hash to hide RC tags in Binder

AmplabJenkins removed a comment on pull request #31773: URL: https://github.com/apache/spark/pull/31773#issuecomment-792466725 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135850/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31773: [SPARK-34657][PYTHON][DOCS] Replace the tag of release to the hash to hide RC tags in Binder

AmplabJenkins commented on pull request #31773: URL: https://github.com/apache/spark/pull/31773#issuecomment-792466725 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135850/ -

[GitHub] [spark] cloud-fan commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

cloud-fan commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589170423 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala ## @@ -51,6 +51,15 @@ trait CodegenInterpretedPlanTest ext

[GitHub] [spark] cloud-fan commented on a change in pull request #31764: [SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add `PlanTest. testFallback` and use it

cloud-fan commented on a change in pull request #31764: URL: https://github.com/apache/spark/pull/31764#discussion_r589168308 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -961,7 +961,7 @@ abstract class AnsiCastSuite

[GitHub] [spark] hiboyang commented on a change in pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

hiboyang commented on a change in pull request #31763: URL: https://github.com/apache/spark/pull/31763#discussion_r589166884 ## File path: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ## @@ -52,6 +52,13 @@ private[spark] sealed trait MapStatus { * partitio

[GitHub] [spark] cloud-fan commented on pull request #31750: [SPARK-34628][SQL] Remove GlobalLimit operator if its child max rows not larger than limit number

cloud-fan commented on pull request #31750: URL: https://github.com/apache/spark/pull/31750#issuecomment-792457702 late LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] SparkQA commented on pull request #31691: [SPARK-34575][SQL] Push down limit through window when partitionSpec is empty

SparkQA commented on pull request #31691: URL: https://github.com/apache/spark/pull/31691#issuecomment-792457052 **[Test build #135852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135852/testReport)** for PR 31691 at commit [`ee3a782`](https://github.com

[GitHub] [spark] Ngone51 commented on pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

Ngone51 commented on pull request #31102: URL: https://github.com/apache/spark/pull/31102#issuecomment-792456347 retest this please This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] Ngone51 commented on a change in pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

Ngone51 commented on a change in pull request #31102: URL: https://github.com/apache/spark/pull/31102#discussion_r589165321 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -66,86 +66,103 @@ private[storage] class BlockManagerDe

[GitHub] [spark] SparkQA removed a comment on pull request #31773: [SPARK-34657][PYTHON][DOCS] Replace the tag of release to the hash to hide RC tags in Binder

SparkQA removed a comment on pull request #31773: URL: https://github.com/apache/spark/pull/31773#issuecomment-792407784 **[Test build #135850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135850/testReport)** for PR 31773 at commit [`9847fd1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31773: [SPARK-34657][PYTHON][DOCS] Replace the tag of release to the hash to hide RC tags in Binder

SparkQA commented on pull request #31773: URL: https://github.com/apache/spark/pull/31773#issuecomment-792454416 **[Test build #135850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135850/testReport)** for PR 31773 at commit [`9847fd1`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

AmplabJenkins removed a comment on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792447483 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135848/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

AmplabJenkins removed a comment on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792447482 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40433/

[GitHub] [spark] AmplabJenkins commented on pull request #31774: [SPARK-34659] Fix that Web UI always correctly get appId

AmplabJenkins commented on pull request #31774: URL: https://github.com/apache/spark/pull/31774#issuecomment-792447646 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

AmplabJenkins commented on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792447483 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135848/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

AmplabJenkins commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792447482 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40433/ -

[GitHub] [spark] SparkQA commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

SparkQA commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792445317 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40433/ ---

[GitHub] [spark] SparkQA commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

SparkQA commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792443250 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40433/ -

[GitHub] [spark] SparkQA commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

SparkQA commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792437428 **[Test build #135851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135851/testReport)** for PR 31711 at commit [`279d7c3`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

SparkQA removed a comment on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792397759 **[Test build #135848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135848/testReport)** for PR 31664 at commit [`cd907bb`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

SparkQA commented on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792434848 **[Test build #135848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135848/testReport)** for PR 31664 at commit [`cd907bb`](https://github.co

[GitHub] [spark] Ngone51 commented on a change in pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

Ngone51 commented on a change in pull request #31763: URL: https://github.com/apache/spark/pull/31763#discussion_r589148712 ## File path: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ## @@ -52,6 +52,13 @@ private[spark] sealed trait MapStatus { * partition

[GitHub] [spark] ornew opened a new pull request #31774: [SPARK-34659] Fix that Web UI always correctly get appId

ornew opened a new pull request #31774: URL: https://github.com/apache/spark/pull/31774 Web UI does not correctly get appId when it has `proxy` or `history` in URL. In my case, it happens on `https://jupyterhub.hosted.us/my-name/proxy/4040/executors/`. Web developer console says:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

AmplabJenkins removed a comment on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792428715 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40430/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31652: [SPARK-34546][SQL] AlterViewAs.query should be analyzed during the analysis phase, and AlterView should invalidate the cache

AmplabJenkins removed a comment on pull request #31652: URL: https://github.com/apache/spark/pull/31652#issuecomment-792428808 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135849/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31652: [SPARK-34546][SQL] AlterViewAs.query should be analyzed during the analysis phase, and AlterView should invalidate the cache

AmplabJenkins commented on pull request #31652: URL: https://github.com/apache/spark/pull/31652#issuecomment-792428808 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135849/ -

[GitHub] [spark] AngersZhuuuu commented on pull request #31611: [SPARK-34488][CORE] Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

AngersZh commented on pull request #31611: URL: https://github.com/apache/spark/pull/31611#issuecomment-792428772 gentle ping @dongjoon-hyun @srowen @sarutak Could you help to review on this PR. Thanks This is an automat

[GitHub] [spark] AmplabJenkins commented on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

AmplabJenkins commented on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792428715 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40430/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31652: [SPARK-34546][SQL] AlterViewAs.query should be analyzed during the analysis phase, and AlterView should invalidate the cache

AmplabJenkins removed a comment on pull request #31652: URL: https://github.com/apache/spark/pull/31652#issuecomment-792428717 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40431/

[GitHub] [spark] AmplabJenkins commented on pull request #31652: [SPARK-34546][SQL] AlterViewAs.query should be analyzed during the analysis phase, and AlterView should invalidate the cache

AmplabJenkins commented on pull request #31652: URL: https://github.com/apache/spark/pull/31652#issuecomment-792428717 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40431/ -

[GitHub] [spark] Ngone51 commented on a change in pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

Ngone51 commented on a change in pull request #31763: URL: https://github.com/apache/spark/pull/31763#discussion_r589144809 ## File path: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ## @@ -52,6 +52,13 @@ private[spark] sealed trait MapStatus { * partition

[GitHub] [spark] AngersZhuuuu commented on pull request #31165: [SPARK-34092][SQL] Support Stage level restful api filter task details by task status

AngersZh commented on pull request #31165: URL: https://github.com/apache/spark/pull/31165#issuecomment-792428427 gentle ping @dongjoon-hyun @srowen @sarutak This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #31652: [SPARK-34546][SQL] AlterViewAs.query should be analyzed during the analysis phase, and AlterView should invalidate the cache

SparkQA removed a comment on pull request #31652: URL: https://github.com/apache/spark/pull/31652#issuecomment-792397775 **[Test build #135849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135849/testReport)** for PR 31652 at commit [`51c66e8`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31652: [SPARK-34546][SQL] AlterViewAs.query should be analyzed during the analysis phase, and AlterView should invalidate the cache

SparkQA commented on pull request #31652: URL: https://github.com/apache/spark/pull/31652#issuecomment-792428162 **[Test build #135849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135849/testReport)** for PR 31652 at commit [`51c66e8`](https://github.co

[GitHub] [spark] maropu commented on a change in pull request #31712: [SPARK-34598] RewritePredicateSubquery Rule must not update Filters without subqueries

maropu commented on a change in pull request #31712: URL: https://github.com/apache/spark/pull/31712#discussion_r589142505 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteSubquerySuite.scala ## @@ -67,4 +68,12 @@ class RewriteSubquerySu

[GitHub] [spark] SparkQA commented on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

SparkQA commented on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792423016 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40430/ ---

[GitHub] [spark] MrPowers commented on pull request #29935: [SPARK-33055][PYTHON][SQL] Add Python CalendarIntervalType

MrPowers commented on pull request #29935: URL: https://github.com/apache/spark/pull/29935#issuecomment-792419286 Just wanted to check in and see if we can potentially get this merged, so we can add `make_interval` to the PySpark API. Seems like @HyukjinKwon is [cool with this gettin

[GitHub] [spark] SparkQA commented on pull request #31664: [SPARK-34541][CORE] Fixed an issue where data could not be cleaned up when unregisterShuffle.

SparkQA commented on pull request #31664: URL: https://github.com/apache/spark/pull/31664#issuecomment-792414969 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40430/ -

[GitHub] [spark] LuciferYang commented on pull request #31711: [SPARK-34597][SQL] Replaces `ParquetFileReader.readFooter` with `ParquetFileReader.open and getFooter`

LuciferYang commented on pull request #31711: URL: https://github.com/apache/spark/pull/31711#issuecomment-792414623 Thank you for your review ~ @maropu @dongjoon-hyun @sunchao, 279d7c3 fix @sunchao's comment ~ This is an

[GitHub] [spark] beliefer commented on pull request #31757: [SPARK-33602][SQL] Group exception messages in execution/datasources

beliefer commented on pull request #31757: URL: https://github.com/apache/spark/pull/31757#issuecomment-792413105 cc @allisonwang-db This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31736: [SPARK-34620][SQL] Code-gen broadcast nested loop join (inner/cross)

AmplabJenkins removed a comment on pull request #31736: URL: https://github.com/apache/spark/pull/31736#issuecomment-792411421 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135847/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31773: [SPARK-34657][PYTHON][DOCS] Replace the tag of release to the hash to hide RC tags in Binder

AmplabJenkins removed a comment on pull request #31773: URL: https://github.com/apache/spark/pull/31773#issuecomment-792411417 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40432/

  1   2   3   >