[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

2021-07-01 Thread GitBox
dongjoon-hyun commented on a change in pull request #33116: URL: https://github.com/apache/spark/pull/33116#discussion_r662659993 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java ## @@ -323,10 +323,13 @@ private

[GitHub] [spark] aokolnychyi commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872625668 Thanks for reviewing, @viirya @cloud-fan @sunchao @rdblue @dongjoon-hyun @holdenk! -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
SparkQA commented on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872625473 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45053/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

2021-07-01 Thread GitBox
dongjoon-hyun commented on a change in pull request #33116: URL: https://github.com/apache/spark/pull/33116#discussion_r662659993 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java ## @@ -323,10 +323,13 @@ private

[GitHub] [spark] rahulsmahadev commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
rahulsmahadev commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662659646 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1306,22 +1316,22 @@ class

[GitHub] [spark] dongjoon-hyun commented on pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33116: URL: https://github.com/apache/spark/pull/33116#issuecomment-872624452 Thank you for updates, @xkrogen . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] github-actions[bot] commented on pull request #29754: [SPARK-32875][CORE][TEST] TaskSchedulerImplSuite: For the pattern of submitTasks + resourceOffers + assert, extract the general m

2021-07-01 Thread GitBox
github-actions[bot] commented on pull request #29754: URL: https://github.com/apache/spark/pull/29754#issuecomment-872622534 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872621711 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45056/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872621501 Retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872621339 Let me try. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872614796 **[Test build #140545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140545/testReport)** for PR 33093 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872620965 In this case, we used to use `[test-hadoop3.2][test-java11]` combination which is still working in Jenkins. However, Jenkins PR Builder seems to be broken for

[GitHub] [spark] SparkQA commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
SparkQA commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872620140 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45055/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872619983 It seems to be a Spark bug which works only at maven and not on sbt. ``` $ build/mvn dependency:tree -Phadoop-3.2 -Dhadoop.version=3.1.1 -pl core | grep

[GitHub] [spark] viirya closed pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
viirya closed pull request #32921: URL: https://github.com/apache/spark/pull/32921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872619549 **[Test build #140545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140545/testReport)** for PR 33093 at commit

[GitHub] [spark] viirya commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
viirya commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872619510 Thanks @aokolnychyi for this work and all for the review! Merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] viirya commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
viirya commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872619166 In GA, all tests are passed. Only "Hadoop 2 build with SBT" failed which seems unrelated: ``` [error]

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872619023 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45054/ --

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872619011 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45054/ -- This is an automated message from the

[GitHub] [spark] tdas commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
tdas commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662654731 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1330,6 +1340,59 @@ class

[GitHub] [spark] tdas commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
tdas commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662654516 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1306,22 +1316,22 @@ class

[GitHub] [spark] tdas commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
tdas commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662653872 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1278,9 +1276,21 @@ class

[GitHub] [spark] tdas commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
tdas commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662652717 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1278,9 +1276,21 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872615854 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140539/

[GitHub] [spark] AmplabJenkins commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872615854 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140539/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872567952 **[Test build #140539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140539/testReport)** for PR 33164 at commit

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872615156 **[Test build #140539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140539/testReport)** for PR 33164 at commit

[GitHub] [spark] SparkQA commented on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

2021-07-01 Thread GitBox
SparkQA commented on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-872614842 **[Test build #140547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140547/testReport)** for PR 32944 at commit

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872614787 **[Test build #140546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140546/testReport)** for PR 33070 at commit

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872614796 **[Test build #140545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140545/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

2021-07-01 Thread GitBox
SparkQA commented on pull request #33116: URL: https://github.com/apache/spark/pull/33116#issuecomment-872614565 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45057/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872613924 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140542/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872613920 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140532/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872613922 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140541/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872613923 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140534/

[GitHub] [spark] c21 commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

2021-07-01 Thread GitBox
c21 commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r662649382 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/costing.scala ## @@ -17,16 +17,32 @@ package

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872613921 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140540/

[GitHub] [spark] AmplabJenkins commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872613923 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140534/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872613921 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140540/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872613922 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140541/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872613924 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140542/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872613920 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140532/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872572607 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140537/

[GitHub] [spark] SparkQA commented on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
SparkQA commented on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872613696 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45053/ -- This is an automated message from the Apache

[GitHub] [spark] sunchao commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
sunchao commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872613657 Yeah, but somehow it references Hadoop 3.3.1 class like `org.apache.hadoop.http.HttpServer2` according to the line number. -- This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872612897 So, `SBT` with the following conf still fails? ``` -Phadoop-3.2 -Dhadoop.version=3.1.1 -Pno-shaded-hadoop-client -Phive-2.3 -Pmesos -Pspark-ganglia-lgpl -Pyarn

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872611855 Let me check the PR builder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] allisonwang-db commented on a change in pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
allisonwang-db commented on a change in pull request #33070: URL: https://github.com/apache/spark/pull/33070#discussion_r662647204 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/DecorrelateInnerQuery.scala ## @@ -212,14 +214,38 @@ object

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872610763 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45056/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
SparkQA commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872610309 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45055/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872609423 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45054/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872470969 **[Test build #140534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140534/testReport)** for PR 33070 at commit

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872607352 **[Test build #140534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140534/testReport)** for PR 33070 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872593526 **[Test build #140542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140542/testReport)** for PR 33095 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872468498 **[Test build #140532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140532/testReport)** for PR 33176 at commit

[GitHub] [spark] SparkQA commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
SparkQA commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872606020 **[Test build #140542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140542/testReport)** for PR 33095 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872593472 **[Test build #140540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140540/testReport)** for PR 33179 at commit

[GitHub] [spark] SparkQA commented on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
SparkQA commented on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872605841 **[Test build #140540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140540/testReport)** for PR 33179 at commit

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872605503 **[Test build #140532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140532/testReport)** for PR 33176 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872593487 **[Test build #140541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140541/testReport)** for PR 33162 at commit

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872604064 **[Test build #140541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140541/testReport)** for PR 33162 at commit

[GitHub] [spark] rahulsmahadev commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
rahulsmahadev commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662635048 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1268,12 +1267,252 @@ class

[GitHub] [spark] SparkQA commented on pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

2021-07-01 Thread GitBox
SparkQA commented on pull request #33116: URL: https://github.com/apache/spark/pull/33116#issuecomment-872596176 **[Test build #140544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140544/testReport)** for PR 33116 at commit

[GitHub] [spark] xkrogen commented on pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

2021-07-01 Thread GitBox
xkrogen commented on pull request #33116: URL: https://github.com/apache/spark/pull/33116#issuecomment-872595353 @dongjoon-hyun updated per your comments, PTAL. I also updated the documentation to address the discrepancy. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872593557 **[Test build #140543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140543/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
SparkQA commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872593526 **[Test build #140542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140542/testReport)** for PR 33095 at commit

[GitHub] [spark] SparkQA commented on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
SparkQA commented on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872593472 **[Test build #140540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140540/testReport)** for PR 33179 at commit

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-01 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-872593487 **[Test build #140541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140541/testReport)** for PR 33162 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872545780 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872592352 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45051/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872592351 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45052/

[GitHub] [spark] AmplabJenkins commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872592351 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45052/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872592350 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140529/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872592352 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45051/ --

[GitHub] [spark] sunchao commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
sunchao commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872590845 ``` Cause: java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [info] at

[GitHub] [spark] ueshin commented on pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
ueshin commented on pull request #33179: URL: https://github.com/apache/spark/pull/33179#issuecomment-872589948 cc @HyukjinKwon @itholic @xinrong-databricks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] ueshin opened a new pull request #33179: [SPARK-35981][PYTHON][TEST] Use check_exact=False to loosen the check precision

2021-07-01 Thread GitBox
ueshin opened a new pull request #33179: URL: https://github.com/apache/spark/pull/33179 ### What changes were proposed in this pull request? We should use `check_exact=False` because the value check in `StatsTest.test_cov_corr_meta` is too strict. ### Why are the changes

[GitHub] [spark] xinrong-databricks commented on a change in pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
xinrong-databricks commented on a change in pull request #33095: URL: https://github.com/apache/spark/pull/33095#discussion_r662624343 ## File path: python/pyspark/pandas/tests/data_type_ops/test_string_ops.py ## @@ -217,6 +220,20 @@ def test_from_to_pandas(self): def

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872584362 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45052/ -- This is an automated message from the

[GitHub] [spark] ueshin commented on a change in pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
ueshin commented on a change in pull request #33095: URL: https://github.com/apache/spark/pull/33095#discussion_r662623005 ## File path: python/pyspark/pandas/tests/data_type_ops/test_string_ops.py ## @@ -217,6 +220,20 @@ def test_from_to_pandas(self): def

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872578159 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45051/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872509503 **[Test build #140537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140537/testReport)** for PR 33160 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872433609 **[Test build #140529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140529/testReport)** for PR 33176 at commit

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872575005 **[Test build #140529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140529/testReport)** for PR 33176 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872572607 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140537/ -- This

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872572033 **[Test build #140537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140537/testReport)** for PR 33160 at commit

[GitHub] [spark] viirya commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
viirya commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872569891 It failed on pyspark test? ``` == FAIL: test_repartitionByRange_dataframe

[GitHub] [spark] AmplabJenkins commented on pull request #33178: [SPARK-35980][CORE] ThreadAudit logs whether thread is daemon

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33178: URL: https://github.com/apache/spark/pull/33178#issuecomment-872569405 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872569292 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45052/ -- This is an automated message from the Apache

[GitHub] [spark] timarmstrong opened a new pull request #33178: [SPARK-35980][CORE] ThreadAudit logs whether thread is daemon

2021-07-01 Thread GitBox
timarmstrong opened a new pull request #33178: URL: https://github.com/apache/spark/pull/33178 ### What changes were proposed in this pull request? Add `daemon={true|false}` to the POSSIBLE THREAD LEAK IN SUITE warning printed by test framework. ### Why are the changes

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872567952 **[Test build #140539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140539/testReport)** for PR 33164 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872567674 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140530/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872567671 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45050/

[GitHub] [spark] AmplabJenkins commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872567674 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140530/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872567671 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45050/ --

[GitHub] [spark] SparkQA removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872433669 **[Test build #140530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140530/testReport)** for PR 33172 at commit

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872565233 **[Test build #140530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140530/testReport)** for PR 33172 at commit

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872563811 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45051/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

2021-07-01 Thread GitBox
SparkQA commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872556292 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45050/ -- This is an automated message from the

<    1   2   3   4   5   6   7   8   9   10   >