[GitHub] [spark] SparkQA commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better

SparkQA commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-882008438 **[Test build #141208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141208/testReport)** for PR 33078 at commit [`1de689b`](https://github.com

[GitHub] [spark] venkata91 edited a comment on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 edited a comment on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-882008124 > Some tests seem to fail. Could you fix them? @sarutak it seems like something is wrong with the Github actions on my account therefore the tests are not able to

[GitHub] [spark] venkata91 edited a comment on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 edited a comment on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-882008124 > Some tests seem to fail. Could you fix them? @sarutak it seems like something is wrong with the Github actions on my account therefore the tests are not able to

[GitHub] [spark] venkata91 commented on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 commented on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-882008124 > Some tests seem to fail. Could you fix them? @sarutak it seems like something is wrong with the Github actions on my account therefore the tests are not able to run. I

[GitHub] [spark] venkata91 opened a new pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 opened a new pull request #33253: URL: https://github.com/apache/spark/pull/33253 ### What changes were proposed in this pull request? Currently there are no speculation metrics available for Spark either at application/job/stage level. This PR is to add some basic

[GitHub] [spark] venkata91 closed pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 closed pull request #33253: URL: https://github.com/apache/spark/pull/33253 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] gengliangwang commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow

gengliangwang commented on pull request #33341: URL: https://github.com/apache/spark/pull/33341#issuecomment-882006202 @dongjoon-hyun Yes currently all the TimestampNTZ changes are targeting Spark 3.2 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] zhouyejoe commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

zhouyejoe commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r671788857 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/PushBlockStream.java ## @@ -85,12 +96,13 @@ public boolean

[GitHub] [spark] zhouyejoe commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a bett

zhouyejoe commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-882004998 > Are there any UTs that verify the changes to `finalizeShuffleMerge` when the message is from old attempt? Added a unit test for this one yesterday. -- This is an au

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

AmplabJenkins removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881996770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

AmplabJenkins commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881996772 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] SparkQA removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

SparkQA removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881984416 **[Test build #141206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141206/testReport)** for PR 33405 at commit [`e50a8d2`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881994653 **[Test build #141206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141206/testReport)** for PR 33405 at commit [`e50a8d2`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

SparkQA removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881950484 **[Test build #141204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141204/testReport)** for PR 33405 at commit [`304ed04`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881993795 **[Test build #141204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141204/testReport)** for PR 33405 at commit [`304ed04`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33361: [SPARK-36155][SQL] Eliminate outer join base uniqueness

AmplabJenkins removed a comment on pull request #33361: URL: https://github.com/apache/spark/pull/33361#issuecomment-881993395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45720/

[GitHub] [spark] SparkQA commented on pull request #33361: [SPARK-36155][SQL] Eliminate outer join base uniqueness

SparkQA commented on pull request #33361: URL: https://github.com/apache/spark/pull/33361#issuecomment-881993393 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45720/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33361: [SPARK-36155][SQL] Eliminate outer join base uniqueness

AmplabJenkins commented on pull request #33361: URL: https://github.com/apache/spark/pull/33361#issuecomment-881993395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45720/ -- T

[GitHub] [spark] SparkQA commented on pull request #33361: [SPARK-36155][SQL] Eliminate outer join base uniqueness

SparkQA commented on pull request #33361: URL: https://github.com/apache/spark/pull/33361#issuecomment-881991431 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45720/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

AmplabJenkins removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881989761 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45719/

[GitHub] [spark] AmplabJenkins commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

AmplabJenkins commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881989761 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45719/ -- T

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881989755 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45719/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33361: [SPARK-36155][SQL] Eliminate outer join base uniqueness

SparkQA commented on pull request #33361: URL: https://github.com/apache/spark/pull/33361#issuecomment-881988727 **[Test build #141207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141207/testReport)** for PR 33361 at commit [`6339da0`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

viirya commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671771189 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* + * Lice

[GitHub] [spark] dongjoon-hyun commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

dongjoon-hyun commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881987742 Also, cc @cloud-fan , @maropu , @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671770195 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671770115 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671769998 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 2g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881987488 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45719/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671769829 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671769310 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671769310 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] wangyum commented on pull request #33361: [SPARK-36155][SQL] Eliminate join base uniqueness

wangyum commented on pull request #33361: URL: https://github.com/apache/spark/pull/33361#issuecomment-881985013 I plan to remove support **Elimination of left semi -> inner if uniqueness can be guaranteed on the right side** because it may introduce SMJ as [it can not estimate `EqualNullS

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881984416 **[Test build #141206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141206/testReport)** for PR 33405 at commit [`e50a8d2`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

AmplabJenkins removed a comment on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881984040 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141205/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

AmplabJenkins commented on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881984040 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141205/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

SparkQA removed a comment on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881957803 **[Test build #141205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141205/testReport)** for PR 31905 at commit [`ec7f727`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

SparkQA commented on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881981056 **[Test build #141205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141205/testReport)** for PR 31905 at commit [`ec7f727`](https://github.co

[GitHub] [spark] wangyum commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

wangyum commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671763303 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* + * Lic

[GitHub] [spark] wangyum commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

wangyum commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671763262 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* + * Lic

[GitHub] [spark] wangyum commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

wangyum commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671762429 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* + * Lic

[GitHub] [spark] wangyum commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

wangyum commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671761972 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* + * Lic

[GitHub] [spark] venkata91 commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

venkata91 commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r671759804 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/PushBlockStream.java ## @@ -85,12 +96,13 @@ public boolean

[GitHub] [spark] github-actions[bot] closed pull request #30930: [SPARK-33070][SQL] Optimize higher order functions

github-actions[bot] closed pull request #30930: URL: https://github.com/apache/spark/pull/30930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

AmplabJenkins removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881959523 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45717/

[GitHub] [spark] dongjoon-hyun commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

dongjoon-hyun commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881975762 I'm observing. ``` java.lang.OutOfMemoryError: Metaspace Error: Exception in thread "dispatcher-event-loop-110" java.lang.OutOfMemoryError: Metaspace ``` --

[GitHub] [spark] xuanyuanking commented on pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

xuanyuanking commented on pull request #33220: URL: https://github.com/apache/spark/pull/33220#issuecomment-881974743 Deleted in https://github.com/apache/spark/pull/33401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] xuanyuanking closed pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

xuanyuanking closed pull request #33220: URL: https://github.com/apache/spark/pull/33220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-u

[GitHub] [spark] xuanyuanking commented on pull request #33401: [SPARK-35785][SS][FOLLOWUP] Remove ignored test from RocksDBSuite

xuanyuanking commented on pull request #33401: URL: https://github.com/apache/spark/pull/33401#issuecomment-881974736 Agree to delete this first. Refer to https://github.com/apache/spark/pull/33220, even with the exception var fix, you may still found the test is flaky in Jenkins env.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

AmplabJenkins removed a comment on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881963631 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45718/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

AmplabJenkins removed a comment on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881963630 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141202/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

AmplabJenkins commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881963630 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141202/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

AmplabJenkins commented on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881963631 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45718/ -- T

[GitHub] [spark] SparkQA commented on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

SparkQA commented on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881962345 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45718/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

SparkQA removed a comment on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881924781 **[Test build #141202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141202/testReport)** for PR 33404 at commit [`5486d64`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

SparkQA commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881961913 **[Test build #141202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141202/testReport)** for PR 33404 at commit [`5486d64`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881959518 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45717/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

AmplabJenkins commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881959523 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45717/ -- T

[GitHub] [spark] SparkQA commented on pull request #31905: [SPARK-34806][SQL] Add Observation helper for Dataset.observe

SparkQA commented on pull request #31905: URL: https://github.com/apache/spark/pull/31905#issuecomment-881957803 **[Test build #141205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141205/testReport)** for PR 31905 at commit [`ec7f727`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

AmplabJenkins removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881957463 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

AmplabJenkins commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881957464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] viirya commented on pull request #33401: [SPARK-35785][SS][FOLLOWUP] Remove ignored test from RocksDBSuite

viirya commented on pull request #33401: URL: https://github.com/apache/spark/pull/33401#issuecomment-881956895 Thanks @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881956008 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45717/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881954944 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45716/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881954507 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45715/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881950518 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45716/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881950484 **[Test build #141204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141204/testReport)** for PR 33405 at commit [`304ed04`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

AmplabJenkins removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881950205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141203/ -

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881950227 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45715/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

AmplabJenkins commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881950205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141203/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA removed a comment on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881945012 **[Test build #141203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141203/testReport)** for PR 33405 at commit [`8d7a987`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881946927 **[Test build #141203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141203/testReport)** for PR 33405 at commit [`8d7a987`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

dongjoon-hyun commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881945408 Hi, @kbendick . Thank you for your contribution. I made this PR and added you as a co-author. You will be marked as one of the author of this commit when this PR is merged

[GitHub] [spark] SparkQA commented on pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

SparkQA commented on pull request #33405: URL: https://github.com/apache/spark/pull/33405#issuecomment-881945012 **[Test build #141203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141203/testReport)** for PR 33405 at commit [`8d7a987`](https://github.com

[GitHub] [spark] dongjoon-hyun opened a new pull request #33405: [SPARK-36195][BUILD] Set MaxMetaspaceSize JVM option to 1g

dongjoon-hyun opened a new pull request #33405: URL: https://github.com/apache/spark/pull/33405 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### H

[GitHub] [spark] dongjoon-hyun commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

dongjoon-hyun commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881941519 Thank you, @wangyum ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671729749 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671729433 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671729433 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671729299 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left si

dongjoon-hyun commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r671729083 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveAggsThroughLeftSemiAntiJoin.scala ## @@ -0,0 +1,40 @@ +/* +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33350: [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

dongjoon-hyun commented on a change in pull request #33350: URL: https://github.com/apache/spark/pull/33350#discussion_r671725543 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitionsSuite.scala ## @@ -42,35 +43,27 @@ class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33350: [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

dongjoon-hyun commented on a change in pull request #33350: URL: https://github.com/apache/spark/pull/33350#discussion_r671725543 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitionsSuite.scala ## @@ -42,35 +43,27 @@ class

[GitHub] [spark] dongjoon-hyun commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow

dongjoon-hyun commented on pull request #33341: URL: https://github.com/apache/spark/pull/33341#issuecomment-881932237 To @beliefer , please resolve the conflict. To @gengliangwang , is this targeting Apache Spark 3.2? -- This is an automated message from the Apache Git Service. To res

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow

dongjoon-hyun commented on a change in pull request #33341: URL: https://github.com/apache/spark/pull/33341#discussion_r671723488 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala ## @@ -352,4 +463,31 @@ class DataFrameTimeWindowingSu

[GitHub] [spark] imback82 commented on a change in pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

imback82 commented on a change in pull request #33200: URL: https://github.com/apache/spark/pull/33200#discussion_r671595283 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statements.scala ## @@ -229,22 +228,13 @@ case class ReplaceTableA

[GitHub] [spark] SparkQA commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

SparkQA commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881924781 **[Test build #141202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141202/testReport)** for PR 33404 at commit [`5486d64`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

AmplabJenkins removed a comment on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881924636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45714/

[GitHub] [spark] AmplabJenkins commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

AmplabJenkins commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881924636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45714/ -- T

[GitHub] [spark] SparkQA commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

SparkQA commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-881924186 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45714/ -- This

[GitHub] [spark] wangyum opened a new pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

wangyum opened a new pull request #33404: URL: https://github.com/apache/spark/pull/33404 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this pa

[GitHub] [spark] sarutak edited a comment on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

sarutak edited a comment on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-881910368 Some tests seem to fail. Could you fix them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] sarutak commented on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

sarutak commented on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-881910368 Some tests seem to fail. Could you fix it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] venkata91 commented on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 commented on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-881909197 @sarutak @AngersZh can you please take a look again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] srowen commented on pull request #33301: [SPARK-36122][CORE] Passing on needClientAuth to Jetty SSLContextFactory

srowen commented on pull request #33301: URL: https://github.com/apache/spark/pull/33301#issuecomment-881903788 Merged to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [spark] srowen closed pull request #33301: [SPARK-36122][CORE] Passing on needClientAuth to Jetty SSLContextFactory

srowen closed pull request #33301: URL: https://github.com/apache/spark/pull/33301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] AmplabJenkins commented on pull request #33388: [SPARK-36176][PYTHON] Expose tableExists in pyspark.sql.catalog

AmplabJenkins commented on pull request #33388: URL: https://github.com/apache/spark/pull/33388#issuecomment-881902160 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #33392: [SPARK-36178][PYTHON] List pyspark.sql.catalog APIs in documentation

AmplabJenkins commented on pull request #33392: URL: https://github.com/apache/spark/pull/33392#issuecomment-881902148 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #33394: [SPARK-36181][PYTHON] updating pyspark sql readwriter documentation

AmplabJenkins commented on pull request #33394: URL: https://github.com/apache/spark/pull/33394#issuecomment-881901283 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

AmplabJenkins removed a comment on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881895142 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45713/

  1   2   >