[GitHub] [spark] SparkQA commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

SparkQA commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690916890 **[Test build #128550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128550/testReport)** for PR 29677 at commit [`5680d48`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

SparkQA commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690916884 **[Test build #128547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128547/testReport)** for PR 29721 at commit [`8b9b5e4`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690917086 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

AmplabJenkins commented on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690917210 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

SparkQA commented on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690916885 **[Test build #128549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128549/testReport)** for PR 29316 at commit [`65f781a`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

SparkQA commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690916888 **[Test build #128555 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128555/testReport)** for PR 29721 at commit [`0484de4`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690916889 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690916891 **[Test build #128554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128554/testReport)** for PR 29585 at commit [`e014a13`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

SparkQA commented on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690916883 **[Test build #128551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128551/testReport)** for PR 25840 at commit [`f78ff9c`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690917065 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

SparkQA commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690916896 **[Test build #128553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128553/testReport)** for PR 29723 at commit [`b8b38ec`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

SparkQA commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690916887 **[Test build #128552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128552/testReport)** for PR 29724 at commit [`069ad73`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690917205 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins removed a comment on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690917362 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690917439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128547/

[GitHub] [spark] SparkQA removed a comment on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

SparkQA removed a comment on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690863425 **[Test build #128549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128549/testReport)** for PR 29316 at commit [`65f781a`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins removed a comment on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690917086 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

SparkQA removed a comment on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690877598 **[Test build #128551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128551/testReport)** for PR 25840 at commit [`f78ff9c`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690917454 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] c21 commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

c21 commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690917366 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690917065 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

SparkQA removed a comment on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690845696 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690832325 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] SparkQA removed a comment on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

SparkQA removed a comment on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690889186 **[Test build #128553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128553/testReport)** for PR 29723 at commit [`b8b38ec`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690917362 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

AmplabJenkins removed a comment on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690917210 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins removed a comment on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690917318 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

SparkQA removed a comment on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690889136 **[Test build #128552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128552/testReport)** for PR 29724 at commit [`069ad73`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690894145 **[Test build #128554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128554/testReport)** for PR 29585 at commit [`e014a13`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690917710 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins removed a comment on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690917205 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690917432 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] SparkQA removed a comment on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

SparkQA removed a comment on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690873403 **[Test build #128550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128550/testReport)** for PR 29677 at commit [`5680d48`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690917211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128553/

[GitHub] [spark] AmplabJenkins commented on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins commented on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690917326 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128549/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690917070 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins removed a comment on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690917091 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690917643 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins removed a comment on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690917439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins removed a comment on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690917710 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins removed a comment on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690917462 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins removed a comment on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690917211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690917643 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins removed a comment on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690917454 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

AmplabJenkins removed a comment on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690917222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins removed a comment on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690917326 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690917695 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] sarutak commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

sarutak commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690918981 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] xuzikun2003 opened a new pull request #29725: Improve sorting performance of Spark SQL window function by removing window partition key from sort order

xuzikun2003 opened a new pull request #29725: URL: https://github.com/apache/spark/pull/29725 ### What changes were proposed in this pull request? Spark SQL rank window function needs to sort the data in each window  partition, and it relies on the execution operator SortExec to do th

[GitHub] [spark] AmplabJenkins commented on pull request #29725: Improve sorting performance of Spark SQL window function by removing window partition key from sort order

AmplabJenkins commented on pull request #29725: URL: https://github.com/apache/spark/pull/29725#issuecomment-690920155 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

SparkQA commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690920356 **[Test build #128557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128557/testReport)** for PR 29677 at commit [`5680d48`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

SparkQA commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690920318 **[Test build #128556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128556/testReport)** for PR 29724 at commit [`069ad73`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins removed a comment on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690917369 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins commented on pull request #29725: Improve sorting performance of Spark SQL window function by removing window partition key from sort order

AmplabJenkins commented on pull request #29725: URL: https://github.com/apache/spark/pull/29725#issuecomment-690920657 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins removed a comment on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690920965 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29725: Improve sorting performance of Spark SQL window function by removing window partition key from sort order

AmplabJenkins removed a comment on pull request #29725: URL: https://github.com/apache/spark/pull/29725#issuecomment-690920155 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690920965 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan commented on a change in pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

cloud-fan commented on a change in pull request #29564: URL: https://github.com/apache/spark/pull/29564#discussion_r486821529 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala ## @@ -393,6 +393,29 @@ class DataSourceV2Suite extends Q

[GitHub] [spark] wangyum opened a new pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

wangyum opened a new pull request #29726: URL: https://github.com/apache/spark/pull/29726 ### What changes were proposed in this pull request? For some filtering side can not broadcast by join type but can broadcast by size, then we should not consider reuse broadcast only, for examp

[GitHub] [spark] AmplabJenkins commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690923986 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins removed a comment on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690923986 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

SparkQA commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690926561 **[Test build #128559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128559/testReport)** for PR 29722 at commit [`7efad16`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

SparkQA commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-690926543 **[Test build #128558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128558/testReport)** for PR 29726 at commit [`2986aad`](https://github.com

[GitHub] [spark] tdas closed pull request #29700: [SPARK-32794][SS] Fixed rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 sources

tdas closed pull request #29700: URL: https://github.com/apache/spark/pull/29700 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AmplabJenkins commented on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

AmplabJenkins commented on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-690927120 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

cloud-fan commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r486827015 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,204 @@ +/* + * Licen

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29726: [SPARK-32855][SQL] Improve DPP for some join type do not support broadcast filtering side

AmplabJenkins removed a comment on pull request #29726: URL: https://github.com/apache/spark/pull/29726#issuecomment-690927120 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

SparkQA commented on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690929637 **[Test build #128560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128560/testReport)** for PR 29316 at commit [`965ed5a`](https://github.com

[GitHub] [spark] cloud-fan commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

cloud-fan commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690930039 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HyukjinKwon commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

HyukjinKwon commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-690930268 Will take a look too @zero323 within few days. Thanks for doing this. This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins commented on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690930195 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins removed a comment on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690930195 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

cloud-fan commented on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-690931387 If the user manually adds a shuffle (DISTRIBUTE BY) in the query before broadcast join, I think we can take care of the skew. Spark query optimizer should not add the extra sh

[GitHub] [spark] cloud-fan edited a comment on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

cloud-fan edited a comment on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-690931387 If the user manually adds a shuffle (DISTRIBUTE BY) in the query before broadcast join, I think we can take care of the skew. Spark query optimizer should not add the e

[GitHub] [spark] peter-toth opened a new pull request #29727: [SPARK-32730][SQL] Improve LeftAnti SortMergeJoin right side buffering

peter-toth opened a new pull request #29727: URL: https://github.com/apache/spark/pull/29727 ### What changes were proposed in this pull request? This is a follow up to https://github.com/apache/spark/pull/29572. LeftAnti SortMergeJoin should not buffer all matching right side

[GitHub] [spark] SparkQA commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

SparkQA commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690932680 **[Test build #128562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128562/testReport)** for PR 29723 at commit [`b8b38ec`](https://github.com

[GitHub] [spark] peter-toth commented on a change in pull request #29572: [SPARK-32730][SQL] Improve LeftSemi and Existence SortMergeJoin right side buffering

peter-toth commented on a change in pull request #29572: URL: https://github.com/apache/spark/pull/29572#discussion_r486833087 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala ## @@ -251,7 +251,8 @@ case class SortMergeJoinExec

[GitHub] [spark] gemelen commented on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

gemelen commented on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-690932507 @HyukjinKwon these settings are for tests (or compile, few lines below) phase, while we now have java.lang.StackOverflowError during import of project into sbt, so that place of

[GitHub] [spark] SparkQA commented on pull request #29727: [SPARK-32730][SQL] Improve LeftAnti SortMergeJoin right side buffering

SparkQA commented on pull request #29727: URL: https://github.com/apache/spark/pull/29727#issuecomment-690932636 **[Test build #128561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128561/testReport)** for PR 29727 at commit [`8f048ee`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29727: [SPARK-32730][SQL] Improve LeftAnti SortMergeJoin right side buffering

AmplabJenkins commented on pull request #29727: URL: https://github.com/apache/spark/pull/29727#issuecomment-690933238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] peter-toth commented on pull request #29727: [SPARK-32730][SQL] Improve LeftAnti SortMergeJoin right side buffering

peter-toth commented on pull request #29727: URL: https://github.com/apache/spark/pull/29727#issuecomment-690933379 cc @cloud-fan, @juliuszsompolski, @gatorsmile This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690933219 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29727: [SPARK-32730][SQL] Improve LeftAnti SortMergeJoin right side buffering

AmplabJenkins removed a comment on pull request #29727: URL: https://github.com/apache/spark/pull/29727#issuecomment-690933238 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins removed a comment on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690933219 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

AngersZh commented on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-690936960 > Spark query optimizer should not add the extra shuffle by itself, as it's likely to cause perf regression. With this rule, we can't handle such data skew case autom

[GitHub] [spark] cloud-fan commented on a change in pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

cloud-fan commented on a change in pull request #29724: URL: https://github.com/apache/spark/pull/29724#discussion_r486837991 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala ## @@ -56,8 +56,8 @@ import org.apa

[GitHub] [spark] cloud-fan commented on a change in pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

cloud-fan commented on a change in pull request #29724: URL: https://github.com/apache/spark/pull/29724#discussion_r486837991 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala ## @@ -56,8 +56,8 @@ import org.apa

[GitHub] [spark] cloud-fan commented on a change in pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

cloud-fan commented on a change in pull request #29724: URL: https://github.com/apache/spark/pull/29724#discussion_r486840286 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala ## @@ -246,13 +244,14 @@ case class

[GitHub] [spark] dongjoon-hyun commented on pull request #29700: [SPARK-32794][SS] Fixed rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 sources

dongjoon-hyun commented on pull request #29700: URL: https://github.com/apache/spark/pull/29700#issuecomment-690942010 Thank you, @tdas and @zsxwing . cc @HeartSaVioR This is an automated message from the Apache Git Servi

[GitHub] [spark] cloud-fan commented on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

cloud-fan commented on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-690945537 Then we need some estimation work, as the shuffle/scan node may be far away from the join node. We also need to carefully justify if the extra shuffle cost worths the skew eli

[GitHub] [spark] dongjoon-hyun commented on pull request #29723: [SPARK-32853][SQL] Consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

dongjoon-hyun commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690945970 Thank you for pinging me, @cloud-fan . This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] maropu commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

maropu commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690949498 GA passed. cc: @cloud-fan @viirya This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [spark] cloud-fan commented on pull request #29497: [WIP][SPARK-32670][SQL]Group exception messages in Catalyst Analyzer in one file

cloud-fan commented on pull request #29497: URL: https://github.com/apache/spark/pull/29497#issuecomment-690952463 > that they are different package names + same object name. This is a good point. Is it possible that we put all the error messages in the catalyst module? Other modules

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r486858033 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,204 @@ +/* + * Licensed

[GitHub] [spark] cloud-fan commented on a change in pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

cloud-fan commented on a change in pull request #29721: URL: https://github.com/apache/spark/pull/29721#discussion_r486860621 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala ## @@ -2555,6 +2555,24 @@ class DataFrameSuite extends QueryTest va

[GitHub] [spark] maropu commented on a change in pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

maropu commented on a change in pull request #29587: URL: https://github.com/apache/spark/pull/29587#discussion_r486860843 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -17,29 +17,101 @@ package org.apache.spark.sq

[GitHub] [spark] maropu commented on a change in pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

maropu commented on a change in pull request #29587: URL: https://github.com/apache/spark/pull/29587#discussion_r486860843 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -17,29 +17,101 @@ package org.apache.spark.sq

[GitHub] [spark] maropu commented on a change in pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

maropu commented on a change in pull request #29721: URL: https://github.com/apache/spark/pull/29721#discussion_r486866208 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala ## @@ -2555,6 +2555,24 @@ class DataFrameSuite extends QueryTest val d

[GitHub] [spark] AngersZhuuuu commented on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AngersZh commented on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-690967408 ping @maropu @cloud-fan Is ok for this pr to merge? @alfozan will raise pr about spark native serde after this pr merged.

[GitHub] [spark] AngersZhuuuu commented on pull request #29421: [SPARK-32388][SQL][test-hadoop2.7][test-hive1.2] TRANSFORM with schema-less mode should keep the same with hive

AngersZh commented on pull request #29421: URL: https://github.com/apache/spark/pull/29421#issuecomment-690967606 ping @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, pleas

  1   2   3   4   5   >