[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524947360 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g

[GitHub] [spark] HyukjinKwon opened a new pull request #30394: Check SPARK-33464

HyukjinKwon opened a new pull request #30394: URL: https://github.com/apache/spark/pull/30394 Check if .m2 is required. This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand

AmplabJenkins commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728754094 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand

SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728754079 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35804/

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524945402 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g

[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728753070 **[Test build #131207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131207/testReport)** for PR 30391 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524945013 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524944927 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524943408 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment:

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524943842 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g

[GitHub] [spark] HyukjinKwon commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

HyukjinKwon commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728751123 Addressed! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728750100 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35805/

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524942149 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment:

[GitHub] [spark] maropu commented on a change in pull request #30372: [SPARK-33172][SQL] Adding support for UserDefinedType for Spark SQL Code generator

maropu commented on a change in pull request #30372: URL: https://github.com/apache/spark/pull/30372#discussion_r524941668 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala ## @@ -1734,15 +1734,17 @@ object

[GitHub] [spark] maropu commented on a change in pull request #30372: [SPARK-33172][SQL] Adding support for UserDefinedType for Spark SQL Code generator

maropu commented on a change in pull request #30372: URL: https://github.com/apache/spark/pull/30372#discussion_r524941488 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala ## @@ -1734,15 +1734,17 @@ object

[GitHub] [spark] dongjoon-hyun commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728749215 Please check https://github.com/apache/spark/pull/30391#discussion_r524939916 . This is an automated

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524940363 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524939916 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728747106 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728747106 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728724756 **[Test build #131202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131202/testReport)** for PR 28386 at commit

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728746848 **[Test build #131202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131202/testReport)** for PR 28386 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743942 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728746560 **[Test build #131206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131206/testReport)** for PR 30391 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.

AmplabJenkins removed a comment on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728745887 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] mridulm commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

mridulm commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728746062 I am not sure I follow - the `reduce` will reduce it at driver - based on the individual priority queues per partition - while `treeReduce` will progressively reduce it in

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728745377 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.

AmplabJenkins commented on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728745887 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

AmplabJenkins removed a comment on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745183 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728745377 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728745363 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35803/

[GitHub] [spark] AmplabJenkins commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

AmplabJenkins commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745179 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

AmplabJenkins removed a comment on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745179 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745161 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35802/

[GitHub] [spark] SparkQA commented on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.

SparkQA commented on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728744819 **[Test build #131189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131189/testReport)** for PR 2 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.

SparkQA removed a comment on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728649075 **[Test build #131189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131189/testReport)** for PR 2 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743938 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] mridulm commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

mridulm commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-728744238 Thanks for the +1 and review @Ngone51 ! I dont have any further comments, the recent changes proposed look good to me too. Any other review comments @tgravescs,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743881 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743938 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743881 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743346 **[Test build #131205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131205/testReport)** for PR 30391 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

AmplabJenkins commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728740932 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand

SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728740967 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35804/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

AmplabJenkins removed a comment on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728740932 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

SparkQA commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728740916 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35801/

[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728740285 **[Test build #131204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131204/testReport)** for PR 30391 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

AmplabJenkins removed a comment on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738673 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

SparkQA removed a comment on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728737413 **[Test build #131203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131203/testReport)** for PR 30393 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

AmplabJenkins commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738666 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

SparkQA commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738655 **[Test build #131203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131203/testReport)** for PR 30393 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

AmplabJenkins removed a comment on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738666 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] BryanCutler commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

BryanCutler commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728737383 ping @HyukjinKwon to please take a look when you can, thanks! This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

SparkQA commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728737413 **[Test build #131203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131203/testReport)** for PR 30393 at commit

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728736268 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35803/

[GitHub] [spark] tanelk edited a comment on pull request #30040: [WIP][SPARK-33124][SQL][DOCS] Adds a group tag in all the expressions for built-in functions

tanelk edited a comment on pull request #30040: URL: https://github.com/apache/spark/pull/30040#issuecomment-728736015 > @tanelk Are you still working on this? I haven't work on it for a while, but I'll get back to it this week.

[GitHub] [spark] tanelk commented on pull request #30040: [WIP][SPARK-33124][SQL][DOCS] Adds a group tag in all the expressions for built-in functions

tanelk commented on pull request #30040: URL: https://github.com/apache/spark/pull/30040#issuecomment-728736015 > @tanelk Are you still working on this? I havent work on it for a while, but I'll get back to it this week.

[GitHub] [spark] viirya commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

viirya commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728735611 I will run benchmark later if the test is passed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values

AmplabJenkins removed a comment on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728735429 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728735386 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35802/

[GitHub] [spark] AmplabJenkins commented on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values

AmplabJenkins commented on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728735429 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values

SparkQA removed a comment on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728642372 **[Test build #131186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131186/testReport)** for PR 30325 at commit

[GitHub] [spark] SparkQA commented on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values

SparkQA commented on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728734485 **[Test build #131186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131186/testReport)** for PR 30325 at commit

[GitHub] [spark] BryanCutler opened a new pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

BryanCutler opened a new pull request #30393: URL: https://github.com/apache/spark/pull/30393 ### What changes were proposed in this pull request? This change adds MapType support for PySpark with Arrow, if using pyarrow >= 2.0.0. ### Why are the changes needed?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

AmplabJenkins removed a comment on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728733454 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

AmplabJenkins commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728733454 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

AmplabJenkins removed a comment on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728732814 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728732787 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35800/

[GitHub] [spark] SparkQA removed a comment on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

SparkQA removed a comment on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728642439 **[Test build #131187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131187/testReport)** for PR 30145 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

AmplabJenkins commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728732814 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

SparkQA commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728732426 **[Test build #131187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131187/testReport)** for PR 30145 at commit

[GitHub] [spark] SparkQA commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

SparkQA commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728732186 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35801/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728731347 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728731332 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728731332 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] LuciferYang commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

LuciferYang commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r524923149 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala ## @@ -894,10 +893,6 @@ trait ScalaReflection extends

[GitHub] [spark] LuciferYang commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

LuciferYang commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r524923149 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala ## @@ -894,10 +893,6 @@ trait ScalaReflection extends

[GitHub] [spark] SparkQA removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

SparkQA removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728683503 **[Test build #131194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131194/testReport)** for PR 30391 at commit

[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728730647 **[Test build #131194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131194/testReport)** for PR 30391 at commit

[GitHub] [spark] MichaelChirico commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

MichaelChirico commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728726963 > Adding magrittr as a dependency Best saved for a different thread, but TL;DR: yes, I think that's a good idea, `magrittr` will help writing `SparkR` code that's

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524918338 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment:

[GitHub] [spark] HyukjinKwon commented on pull request #30378: [SPARK-33454][INFRA] Add GitHub Action job for Hadoop 2

HyukjinKwon commented on pull request #30378: URL: https://github.com/apache/spark/pull/30378#issuecomment-728725027 There's a discussion thread in the mailing list about JDK 14 FYI: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-on-JDK-14-td30348.html

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728724756 **[Test build #131202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131202/testReport)** for PR 28386 at commit

[GitHub] [spark] MichaelChirico commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

MichaelChirico commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728724666 > Ideally, we'd like both to behave the same way (possibly resolving the problem with mutate), that's however a breaking change. A reasonable request (though

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723077 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723065 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728718374 **[Test build #131200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131200/testReport)** for PR 28386 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723065 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723045 **[Test build #131200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131200/testReport)** for PR 28386 at commit

[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand

SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728721434 **[Test build #131201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131201/testReport)** for PR 28647 at commit

[GitHub] [spark] jsoref commented on pull request #30323: Spelling

jsoref commented on pull request #30323: URL: https://github.com/apache/spark/pull/30323#issuecomment-728721289 ok, so, part one is merged. `bin`, `repl`, and `streaming` seem to be the next least recently touched... What's do you recommend doing next?

[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728720126 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35800/

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728718374 **[Test build #131200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131200/testReport)** for PR 28386 at commit

[GitHub] [spark] MichaelChirico commented on a change in pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

MichaelChirico commented on a change in pull request #28386: URL: https://github.com/apache/spark/pull/28386#discussion_r524910801 ## File path: R/pkg/R/DataFrame.R ## @@ -3444,7 +3445,8 @@ setMethod("as.data.frame", #' @note attach since 1.6.0 setMethod("attach", Review

[GitHub] [spark] maropu commented on pull request #30342: [MINOR][GRAPHX][3.0] Correct typos in the sub-modules: graphx, external, and examples

maropu commented on pull request #30342: URL: https://github.com/apache/spark/pull/30342#issuecomment-728715902 Thanks! Merged to branch-3.0. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] maropu closed pull request #30342: [MINOR][GRAPHX][3.0] Correct typos in the sub-modules: graphx, external, and examples

maropu closed pull request #30342: URL: https://github.com/apache/spark/pull/30342 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row

SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728715339 **[Test build #131199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131199/testReport)** for PR 30368 at commit

[GitHub] [spark] MichaelChirico commented on a change in pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

MichaelChirico commented on a change in pull request #28386: URL: https://github.com/apache/spark/pull/28386#discussion_r524908881 ## File path: R/pkg/R/DataFrame.R ## @@ -3444,7 +3445,8 @@ setMethod("as.data.frame", #' @note attach since 1.6.0 setMethod("attach",

[GitHub] [spark] SparkQA commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead

SparkQA commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728712269 **[Test build #131198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131198/testReport)** for PR 30392 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

AmplabJenkins removed a comment on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-728710246 Test FAILed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   7   8   9   10   >