[GitHub] [spark] AmplabJenkins removed a comment on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins removed a comment on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918263335 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47710/

[GitHub] [spark] SparkQA commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

SparkQA commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918269822 **[Test build #143211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143211/testReport)** for PR 33958 at commit [`f856a10`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33973: [SPARK-36730][SQL] Use V2 Filter in V2 file source

SparkQA commented on pull request #33973: URL: https://github.com/apache/spark/pull/33973#issuecomment-918269921 **[Test build #143210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143210/testReport)** for PR 33973 at commit [`fc060c8`](https://github.com

[GitHub] [spark] cloud-fan commented on pull request #33966: [SPARK-36736][SQL] Support ILIKE (ALL | ANY | SOME) - case insensitive LIKE

cloud-fan commented on pull request #33966: URL: https://github.com/apache/spark/pull/33966#issuecomment-918272251 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan closed pull request #33966: [SPARK-36736][SQL] Support ILIKE (ALL | ANY | SOME) - case insensitive LIKE

cloud-fan closed pull request #33966: URL: https://github.com/apache/spark/pull/33966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] SparkQA commented on pull request #33978: [SPARK-36738][SQL][DOC] Fixed the wrong documentation on Cot API

SparkQA commented on pull request #33978: URL: https://github.com/apache/spark/pull/33978#issuecomment-918280685 **[Test build #143201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143201/testReport)** for PR 33978 at commit [`d732715`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33978: [SPARK-36738][SQL][DOC] Fixed the wrong documentation on Cot API

SparkQA removed a comment on pull request #33978: URL: https://github.com/apache/spark/pull/33978#issuecomment-918068035 **[Test build #143201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143201/testReport)** for PR 33978 at commit [`d732715`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33978: [SPARK-36738][SQL][DOC] Fixed the wrong documentation on Cot API

AmplabJenkins commented on pull request #33978: URL: https://github.com/apache/spark/pull/33978#issuecomment-918282985 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143201/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33978: [SPARK-36738][SQL][DOC] Fixed the wrong documentation on Cot API

AmplabJenkins removed a comment on pull request #33978: URL: https://github.com/apache/spark/pull/33978#issuecomment-918282985 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143201/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

AmplabJenkins commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918285153 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] sarutak commented on pull request #33979: [SPARK-36739][DOCS][PYTHON] Add apache license headers to makefiles

sarutak commented on pull request #33979: URL: https://github.com/apache/spark/pull/33979#issuecomment-918286059 @yoda-mon You changed `bat` files so it's better to ensure that we can successfully build the docs on Windows. Did you confirm that? If so, could you note in the description t

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918288358 **[Test build #143212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143212/testReport)** for PR 33955 at commit [`db8159e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918294963 **[Test build #143213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143213/testReport)** for PR 33955 at commit [`eb1f028`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33956: [SPARK-36715][SQL] InferFiltersFromGenerate should not infer filter for udf

SparkQA commented on pull request #33956: URL: https://github.com/apache/spark/pull/33956#issuecomment-918310169 **[Test build #143202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143202/testReport)** for PR 33956 at commit [`d7aa7c0`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33956: [SPARK-36715][SQL] InferFiltersFromGenerate should not infer filter for udf

SparkQA removed a comment on pull request #33956: URL: https://github.com/apache/spark/pull/33956#issuecomment-918068124 **[Test build #143202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143202/testReport)** for PR 33956 at commit [`d7aa7c0`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-91831 **[Test build #143207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143207/testReport)** for PR 33974 at commit [`15f470c`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA removed a comment on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918164048 **[Test build #143207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143207/testReport)** for PR 33974 at commit [`15f470c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

SparkQA commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918320313 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47713/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33956: [SPARK-36715][SQL] InferFiltersFromGenerate should not infer filter for udf

AmplabJenkins commented on pull request #33956: URL: https://github.com/apache/spark/pull/33956#issuecomment-918321602 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143202/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918321607 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143207/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins removed a comment on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918321607 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143207/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33956: [SPARK-36715][SQL] InferFiltersFromGenerate should not infer filter for udf

AmplabJenkins removed a comment on pull request #33956: URL: https://github.com/apache/spark/pull/33956#issuecomment-918321602 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143202/ -

[GitHub] [spark] SparkQA commented on pull request #33973: [SPARK-36730][SQL] Use V2 Filter in V2 file source

SparkQA commented on pull request #33973: URL: https://github.com/apache/spark/pull/33973#issuecomment-918326934 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47712/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

SparkQA commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918328157 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47713/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

AmplabJenkins commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918328236 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47713/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

AmplabJenkins removed a comment on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918328236 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47713/

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918332492 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47714/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33973: [SPARK-36730][SQL] Use V2 Filter in V2 file source

AmplabJenkins commented on pull request #33973: URL: https://github.com/apache/spark/pull/33973#issuecomment-918334889 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47712/ -- T

[GitHub] [spark] SparkQA commented on pull request #33973: [SPARK-36730][SQL] Use V2 Filter in V2 file source

SparkQA commented on pull request #33973: URL: https://github.com/apache/spark/pull/33973#issuecomment-918334837 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47712/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33973: [SPARK-36730][SQL] Use V2 Filter in V2 file source

AmplabJenkins removed a comment on pull request #33973: URL: https://github.com/apache/spark/pull/33973#issuecomment-918334889 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47712/

[GitHub] [spark] AmplabJenkins commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918340039 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47714/ -- T

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918339982 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47714/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins removed a comment on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918340039 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47714/

[GitHub] [spark] SparkQA commented on pull request #33934: [SPARK-36691][PYTHON] PythonRunner failed should pass error message to ApplicationMaster too

SparkQA commented on pull request #33934: URL: https://github.com/apache/spark/pull/33934#issuecomment-918343435 **[Test build #143208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143208/testReport)** for PR 33934 at commit [`1fc50d7`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33934: [SPARK-36691][PYTHON] PythonRunner failed should pass error message to ApplicationMaster too

SparkQA removed a comment on pull request #33934: URL: https://github.com/apache/spark/pull/33934#issuecomment-918164135 **[Test build #143208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143208/testReport)** for PR 33934 at commit [`1fc50d7`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918343433 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47715/ -- This is an automated message from the Apache

[GitHub] [spark] srowen closed pull request #33948: [SPARK-36712][Build] Make scala-parallel-collections in 2.13 POM a direct dependency (not in maven profile)

srowen closed pull request #33948: URL: https://github.com/apache/spark/pull/33948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] AmplabJenkins commented on pull request #33934: [SPARK-36691][PYTHON] PythonRunner failed should pass error message to ApplicationMaster too

AmplabJenkins commented on pull request #33934: URL: https://github.com/apache/spark/pull/33934#issuecomment-918345899 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143208/ -- This

[GitHub] [spark] huaxingao commented on pull request #33973: [SPARK-36730][SQL] Use V2 Filter in V2 file source

huaxingao commented on pull request #33973: URL: https://github.com/apache/spark/pull/33973#issuecomment-918346449 cc @cloud-fan @sunchao @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] srowen commented on pull request #33948: [SPARK-36712][Build] Make scala-parallel-collections in 2.13 POM a direct dependency (not in maven profile)

srowen commented on pull request #33948: URL: https://github.com/apache/spark/pull/33948#issuecomment-918346578 Merged to master / 3.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33934: [SPARK-36691][PYTHON] PythonRunner failed should pass error message to ApplicationMaster too

AmplabJenkins removed a comment on pull request #33934: URL: https://github.com/apache/spark/pull/33934#issuecomment-918345899 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143208/ -

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918354595 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47715/ -- This is an automated message from the A

[GitHub] [spark] viirya commented on a change in pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

viirya commented on a change in pull request #33975: URL: https://github.com/apache/spark/pull/33975#discussion_r707494550 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/dynamicpruning/PartitionPruning.scala ## @@ -177,6 +177,26 @@ object PartitionPruning

[GitHub] [spark] AmplabJenkins commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918374932 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47715/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918374959 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins removed a comment on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918374932 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47715/

[GitHub] [spark] SparkQA commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918377571 **[Test build #143214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143214/testReport)** for PR 33975 at commit [`5c79527`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918381879 **[Test build #143214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143214/testReport)** for PR 33975 at commit [`5c79527`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

AmplabJenkins commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918381931 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143214/ -- This

[GitHub] [spark] SparkQA commented on pull request #33966: [SPARK-36736][SQL] Support ILIKE (ALL | ANY | SOME) - case insensitive LIKE

SparkQA commented on pull request #33966: URL: https://github.com/apache/spark/pull/33966#issuecomment-918382628 **[Test build #143204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143204/testReport)** for PR 33966 at commit [`967b8c2`](https://github.co

[GitHub] [spark] viirya commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

viirya commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918382760 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

AmplabJenkins removed a comment on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918381931 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143214/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA removed a comment on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918377571 **[Test build #143214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143214/testReport)** for PR 33975 at commit [`5c79527`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33966: [SPARK-36736][SQL] Support ILIKE (ALL | ANY | SOME) - case insensitive LIKE

SparkQA removed a comment on pull request #33966: URL: https://github.com/apache/spark/pull/33966#issuecomment-918112827 **[Test build #143204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143204/testReport)** for PR 33966 at commit [`967b8c2`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33966: [SPARK-36736][SQL] Support ILIKE (ALL | ANY | SOME) - case insensitive LIKE

AmplabJenkins commented on pull request #33966: URL: https://github.com/apache/spark/pull/33966#issuecomment-918384228 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143204/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33966: [SPARK-36736][SQL] Support ILIKE (ALL | ANY | SOME) - case insensitive LIKE

AmplabJenkins removed a comment on pull request #33966: URL: https://github.com/apache/spark/pull/33966#issuecomment-918384228 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143204/ -

[GitHub] [spark] SparkQA commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918387254 **[Test build #143215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143215/testReport)** for PR 33975 at commit [`5c79527`](https://github.com

[GitHub] [spark] sarutak opened a new pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

sarutak opened a new pull request #33981: URL: https://github.com/apache/spark/pull/33981 ### What changes were proposed in this pull request? This PR fixes a perf issue in `SchemaPruning` when a struct has many fields (e.g. >10K fields). The root cause is `SchemaPruning.sortLeftF

[GitHub] [spark] SparkQA commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918422617 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47717/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

AmplabJenkins commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918423631 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47717/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

AmplabJenkins removed a comment on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918423631 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47717/

[GitHub] [spark] SparkQA commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

SparkQA commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918424155 **[Test build #143211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143211/testReport)** for PR 33958 at commit [`f856a10`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

SparkQA commented on pull request #33981: URL: https://github.com/apache/spark/pull/33981#issuecomment-918424792 **[Test build #143216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143216/testReport)** for PR 33981 at commit [`80bf864`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

SparkQA commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-918424718 **[Test build #143206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143206/testReport)** for PR 33893 at commit [`cd3c449`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918424863 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47716/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

SparkQA removed a comment on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918269822 **[Test build #143211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143211/testReport)** for PR 33958 at commit [`f856a10`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

SparkQA removed a comment on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-918155391 **[Test build #143206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143206/testReport)** for PR 33893 at commit [`cd3c449`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

AmplabJenkins commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918425145 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143211/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

AmplabJenkins removed a comment on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918425145 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143211/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

AmplabJenkins commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-918426518 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143206/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

AmplabJenkins removed a comment on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-918426518 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143206/ -

[GitHub] [spark] mridulm commented on pull request #33977: [SPARK-36737][BUILD][CORE][SQL][SS] Upgrade Apache commons-io to 2.11.0 and revert change of SPARK-36456

mridulm commented on pull request #33977: URL: https://github.com/apache/spark/pull/33977#issuecomment-918427668 Do we want to preserve this behavior (of suppressing log message) and rely on commons for something `JavaUtils` already does ? The intent of `closeQuietly` is more to ensure clo

[GitHub] [spark] SparkQA commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

SparkQA commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918432985 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47716/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

AmplabJenkins commented on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918433033 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47716/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33975: [SPARK-36735][SQL] Adjust overhead of cached relation for DPP

AmplabJenkins removed a comment on pull request #33975: URL: https://github.com/apache/spark/pull/33975#issuecomment-918433033 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47716/

[GitHub] [spark] mridulm commented on pull request #32900: [SPARK-35751]Support Joint eviction strategies for cached RDD partitions

mridulm commented on pull request #32900: URL: https://github.com/apache/spark/pull/32900#issuecomment-918443507 Closing stale pr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [spark] mridulm closed pull request #32900: [SPARK-35751]Support Joint eviction strategies for cached RDD partitions

mridulm closed pull request #32900: URL: https://github.com/apache/spark/pull/32900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] rmcyang commented on a change in pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

rmcyang commented on a change in pull request #33976: URL: https://github.com/apache/spark/pull/33976#discussion_r707567642 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -2597,14 +2597,29 @@ private[spark] object Utils extends Logging { } /**

[GitHub] [spark] SparkQA commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

SparkQA commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918445362 **[Test build #143217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143217/testReport)** for PR 33976 at commit [`9f4e4f4`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

AmplabJenkins removed a comment on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918285153 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] rmcyang commented on a change in pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

rmcyang commented on a change in pull request #33976: URL: https://github.com/apache/spark/pull/33976#discussion_r707568147 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -2597,14 +2597,29 @@ private[spark] object Utils extends Logging { } /**

[GitHub] [spark] mridulm commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

mridulm commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918449062 Ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] mridulm commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

mridulm commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918449493 Will merge after tests pass, thanks for working on this @rmcyang ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] SparkQA commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

SparkQA commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918451599 **[Test build #143218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143218/testReport)** for PR 33976 at commit [`9f4e4f4`](https://github.com

[GitHub] [spark] venkata91 commented on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level

venkata91 commented on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-918458484 Gentle reminder @sarutak Have addressed the review comments. Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [spark] SparkQA commented on pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

SparkQA commented on pull request #33981: URL: https://github.com/apache/spark/pull/33981#issuecomment-918466217 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47718/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

SparkQA commented on pull request #33981: URL: https://github.com/apache/spark/pull/33981#issuecomment-918473173 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47718/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

AmplabJenkins commented on pull request #33981: URL: https://github.com/apache/spark/pull/33981#issuecomment-918473336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47718/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

AmplabJenkins removed a comment on pull request #33981: URL: https://github.com/apache/spark/pull/33981#issuecomment-918473336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47718/

[GitHub] [spark] zhouyejoe commented on a change in pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

zhouyejoe commented on a change in pull request #33976: URL: https://github.com/apache/spark/pull/33976#discussion_r707591152 ## File path: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ## @@ -1510,9 +1510,16 @@ class UtilsSuite extends SparkFunSuite with ResetSys

[GitHub] [spark] zhouyejoe commented on a change in pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

zhouyejoe commented on a change in pull request #33976: URL: https://github.com/apache/spark/pull/33976#discussion_r707591530 ## File path: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ## @@ -1510,9 +1510,16 @@ class UtilsSuite extends SparkFunSuite with ResetSys

[GitHub] [spark] SparkQA commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

SparkQA commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918484908 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47719/ -- This is an automated message from the Apache

[GitHub] [spark] holdenk commented on pull request #33257: [SPARK-36039][K8S] Fix executor pod hadoop conf mount

holdenk commented on pull request #33257: URL: https://github.com/apache/spark/pull/33257#issuecomment-918488366 Ping @ScrapCodes / @dongjoon-hyun , I can also take a look if y'all are busy. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] SparkQA commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

SparkQA commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918490304 **[Test build #143219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143219/testReport)** for PR 33976 at commit [`0e5e229`](https://github.com

[GitHub] [spark] holdenk commented on pull request #33675: [SPARK-27997][K8S] Add support for kubernetes OAuth Token refresh

holdenk commented on pull request #33675: URL: https://github.com/apache/spark/pull/33675#issuecomment-918490869 Jenkins retest this please. @haodemon have you run the K8s integration tests locally on your env? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] rmcyang commented on a change in pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

rmcyang commented on a change in pull request #33976: URL: https://github.com/apache/spark/pull/33976#discussion_r707605971 ## File path: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ## @@ -1510,9 +1510,16 @@ class UtilsSuite extends SparkFunSuite with ResetSyste

[GitHub] [spark] holdenk commented on a change in pull request #33675: [SPARK-27997][K8S] Add support for kubernetes OAuth Token refresh

holdenk commented on a change in pull request #33675: URL: https://github.com/apache/spark/pull/33675#discussion_r707606545 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala ## @@ -92,7 +101,9 @@ priva

[GitHub] [spark] SparkQA commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

SparkQA commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918492673 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47719/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

AmplabJenkins commented on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918492791 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47719/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33976: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

AmplabJenkins removed a comment on pull request #33976: URL: https://github.com/apache/spark/pull/33976#issuecomment-918492791 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47719/

<    1   2   3   4   5   6   7   >