[GitHub] [spark] AmplabJenkins removed a comment on pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

AmplabJenkins removed a comment on pull request #28027: URL: https://github.com/apache/spark/pull/28027#issuecomment-721445646 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

SparkQA commented on pull request #28027: URL: https://github.com/apache/spark/pull/28027#issuecomment-721445631 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35178/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

AmplabJenkins commented on pull request #28027: URL: https://github.com/apache/spark/pull/28027#issuecomment-721445646 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

SparkQA commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-721445300 **[Test build #130581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130581/testReport)** for PR 29247 at commit [`5a55030`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30186: [SPARK-23432][UI] Add executor peak jvm memory metrics in executors page

SparkQA commented on pull request #30186: URL: https://github.com/apache/spark/pull/30186#issuecomment-721445258 **[Test build #130580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130580/testReport)** for PR 30186 at commit [`30da8f1`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

AmplabJenkins removed a comment on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721441460 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] SparkQA removed a comment on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

SparkQA removed a comment on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721389401 **[Test build #130576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130576/testReport)** for PR 30062 at commit [`27e0d31`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

AmplabJenkins removed a comment on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721440342 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

AmplabJenkins removed a comment on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721439694 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

SparkQA removed a comment on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721436214 **[Test build #130578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130578/testReport)** for PR 30241 at commit [`6678bb9`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

AmplabJenkins commented on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721441452 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

AmplabJenkins commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721440342 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

AmplabJenkins commented on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721439694 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

SparkQA commented on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721439675 **[Test build #130578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130578/testReport)** for PR 30241 at commit [`6678bb9`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

SparkQA commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721439586 **[Test build #130576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130576/testReport)** for PR 30062 at commit [`27e0d31`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

SparkQA commented on pull request #28027: URL: https://github.com/apache/spark/pull/28027#issuecomment-721439184 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35178/ -

[GitHub] [spark] SparkQA commented on pull request #30240: [SPARK-33333][BUILD][3.0] Upgrade Jetty to 9.4.28.v20200408

SparkQA commented on pull request #30240: URL: https://github.com/apache/spark/pull/30240#issuecomment-721436241 **[Test build #130579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130579/testReport)** for PR 30240 at commit [`e3c6722`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

SparkQA commented on pull request #30241: URL: https://github.com/apache/spark/pull/30241#issuecomment-721436214 **[Test build #130578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130578/testReport)** for PR 30241 at commit [`6678bb9`](https://github.com

[GitHub] [spark] dongjoon-hyun opened a new pull request #30241: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

dongjoon-hyun opened a new pull request #30241: URL: https://github.com/apache/spark/pull/30241 ### What changes were proposed in this pull request? This PR aims to upgrade Jetty to 9.4.28.v20200408 like `master` branch (Apache Spark 3.1). ### Why are the changes needed?

[GitHub] [spark] dongjoon-hyun opened a new pull request #30240: [SPARK-33333][BUILD][3.0] Upgrade Jetty to 9.4.28.v20200408

dongjoon-hyun opened a new pull request #30240: URL: https://github.com/apache/spark/pull/30240 ### What changes were proposed in this pull request? This PR aims to upgrade Jetty to 9.4.28.v20200408 like `master` branch. ### Why are the changes needed? To bring the bug f

[GitHub] [spark] ahshahid commented on pull request #30185: [SPARK-33152][SQL] This PR proposes a new logic to maintain & track constraints which solves the OOM or performance issues in query compilat

ahshahid commented on pull request #30185: URL: https://github.com/apache/spark/pull/30185#issuecomment-721431403 @srowen Fixed the merge issues. I will rebase it into a single commit. This is an automated message from the A

[GitHub] [spark] kbendick commented on pull request #30239: Test pr labeler by commenting out presumed broken part

kbendick commented on pull request #30239: URL: https://github.com/apache/spark/pull/30239#issuecomment-721430959 Oops. Should be in my local. Sorry! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] kbendick closed pull request #30239: Test pr labeler by commenting out presumed broken part

kbendick closed pull request #30239: URL: https://github.com/apache/spark/pull/30239 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] kbendick opened a new pull request #30239: Test pr labeler by commenting out presumed broken part

kbendick opened a new pull request #30239: URL: https://github.com/apache/spark/pull/30239 When I changed BUILD to use `any` and `all`, it made the config file invalid. I'm commenting it out to see if that's the source of the problem or if there are other issues that I'm not seeing.

[GitHub] [spark] holdenk commented on a change in pull request #30206: [SPARK-33261][K8S] Add a developer API for custom feature steps

holdenk commented on a change in pull request #30206: URL: https://github.com/apache/spark/pull/30206#discussion_r517017815 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -211,6 +211,24 @@ private[spark] object Con

[GitHub] [spark] rdblue commented on a change in pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

rdblue commented on a change in pull request #28027: URL: https://github.com/apache/spark/pull/28027#discussion_r517016147 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala ## @@ -48,6 +48,15 @@ case class DataSo

[GitHub] [spark] kbendick commented on pull request #30238: Comment out other gh actions while testing labeler

kbendick commented on pull request #30238: URL: https://github.com/apache/spark/pull/30238#issuecomment-721429006 Woops. Closing this PR as I intended to create it in my local fork. Need more coffee today 😅 This is an autom

[GitHub] [spark] kbendick closed pull request #30238: Comment out other gh actions while testing labeler

kbendick closed pull request #30238: URL: https://github.com/apache/spark/pull/30238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] kbendick opened a new pull request #30238: Comment out other gh actions while testing labeler

kbendick opened a new pull request #30238: URL: https://github.com/apache/spark/pull/30238 In my personal fork, tests on PRs don't stop running if you push to that branch (which should invalidate the previous runs so as to not waste test runner resource and to make running the tests faster

[GitHub] [spark] rdblue commented on a change in pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

rdblue commented on a change in pull request #28027: URL: https://github.com/apache/spark/pull/28027#discussion_r517015556 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala ## @@ -48,6 +48,15 @@ case class DataSo

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins removed a comment on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721427657 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins commented on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721427657 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

SparkQA commented on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721426992 **[Test build #130574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130574/testReport)** for PR 30229 at commit [`c80e8eb`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

SparkQA removed a comment on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721313475 **[Test build #130574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130574/testReport)** for PR 30229 at commit [`c80e8eb`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

SparkQA commented on pull request #28027: URL: https://github.com/apache/spark/pull/28027#issuecomment-721424926 **[Test build #130577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130577/testReport)** for PR 28027 at commit [`d661246`](https://github.com

[GitHub] [spark] dongjoon-hyun closed pull request #30235: [SPARK-33293][SQL][FOLLOW-UP] Rename TableWriteExec to TableWriteExecHelper

dongjoon-hyun closed pull request #30235: URL: https://github.com/apache/spark/pull/30235 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #30235: [SPARK-33293][SQL][FOLLOW-UP] Rename TableWriteExec to TableWriteExecHelper

dongjoon-hyun commented on pull request #30235: URL: https://github.com/apache/spark/pull/30235#issuecomment-721416193 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

AmplabJenkins removed a comment on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721414386 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

AmplabJenkins removed a comment on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721414383 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #24513: [SPARK-27620][BUILD] Upgrade jetty to 9.4.18.v20190429

dongjoon-hyun edited a comment on pull request #24513: URL: https://github.com/apache/spark/pull/24513#issuecomment-721413416 ~Oh, Sorry. I realized that this is effectively fixed via SPARK-30318.~ It seems that we need this for a different issue. ---

[GitHub] [spark] kbendick closed pull request #30237: Fix any and all indentation in build

kbendick closed pull request #30237: URL: https://github.com/apache/spark/pull/30237 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] kbendick commented on pull request #30237: Fix any and all indentation in build

kbendick commented on pull request #30237: URL: https://github.com/apache/spark/pull/30237#issuecomment-721414324 Woops, Closing this PR as I meant to do it in my local for testing purposes 😅 This is an automated message fr

[GitHub] [spark] AmplabJenkins commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

AmplabJenkins commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721414383 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

SparkQA commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721414299 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35177/ ---

[GitHub] [spark] kbendick opened a new pull request #30237: Fix any and all indentation in build

kbendick opened a new pull request #30237: URL: https://github.com/apache/spark/pull/30237 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How wa

[GitHub] [spark] dongjoon-hyun commented on pull request #24513: [SPARK-27620][BUILD] Upgrade jetty to 9.4.18.v20190429

dongjoon-hyun commented on pull request #24513: URL: https://github.com/apache/spark/pull/24513#issuecomment-721413416 Oh, Sorry. I realized that this is effectively fixed via SPARK-30318. This is an automated message from th

[GitHub] [spark] dongjoon-hyun commented on pull request #24513: [SPARK-27620][BUILD] Upgrade jetty to 9.4.18.v20190429

dongjoon-hyun commented on pull request #24513: URL: https://github.com/apache/spark/pull/24513#issuecomment-721412084 Hi, Guys. Can we have this in branch-2.4 because Apache Spark 2.4 is LTS? This is an automated message

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516991423 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -0,0 +1,905 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

SparkQA commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721404842 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35177/ -

[GitHub] [spark] otterc commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721401673 I am resolving all the older comments which have been addressed. This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30236: [SPARK-33330] Add support for Enums on CatalystTypeConverters

AmplabJenkins removed a comment on pull request #30236: URL: https://github.com/apache/spark/pull/30236#issuecomment-721392129 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #30236: [SPARK-33330] Add support for Enums on CatalystTypeConverters

AmplabJenkins commented on pull request #30236: URL: https://github.com/apache/spark/pull/30236#issuecomment-721392574 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #30236: [SPARK-33330] Add support for Enums on CatalystTypeConverters

AmplabJenkins commented on pull request #30236: URL: https://github.com/apache/spark/pull/30236#issuecomment-721392129 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] malduarte opened a new pull request #30236: [SPARK-33330] Add support for Enums on CatalystTypeConverters

malduarte opened a new pull request #30236: URL: https://github.com/apache/spark/pull/30236 ### What changes were proposed in this pull request? Adds support for Converting Java Enums on CatalystTypeConverters. ### Why are the changes needed? Given that:

[GitHub] [spark] SparkQA commented on pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

SparkQA commented on pull request #30062: URL: https://github.com/apache/spark/pull/30062#issuecomment-721389401 **[Test build #130576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130576/testReport)** for PR 30062 at commit [`27e0d31`](https://github.com

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516969302 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -0,0 +1,966 @@ +/* + * Licensed to

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516969385 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -0,0 +1,966 @@ +/* + * Licensed to

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516969462 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -0,0 +1,966 @@ +/* + * Licensed to

[GitHub] [spark] venkata91 commented on a change in pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

venkata91 commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r516969382 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1927,4 +1927,34 @@ package object config { .version("3.0.

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516969116 ## File path: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java ## @@ -0,0 +1,490 @@ +/* + * License

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516960321 ## File path: common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java ## @@ -363,4 +363,39 @@ public boolean useOldFetchProto

[GitHub] [spark] planga82 commented on pull request #29837: [SPARK-32463][SQL][DOCS] Add "Type Conversion" section in "Supported Data Types" of SQL docs

planga82 commented on pull request #29837: URL: https://github.com/apache/spark/pull/29837#issuecomment-721376889 @gengliangwang I have introduced descriptions to clarify and listed the expressions ![image](https://user-images.githubusercontent.com/12819544/98040246-da457c00-1e17-11

[GitHub] [spark] planga82 commented on a change in pull request #29837: [SPARK-32463][SQL][DOCS] Add "Type Conversion" section in "Supported Data Types" of SQL docs

planga82 commented on a change in pull request #29837: URL: https://github.com/apache/spark/pull/29837#discussion_r516958046 ## File path: docs/sql-ref-datatypes.md ## @@ -314,3 +314,200 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2; |3| Infinity| +-+-+

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30210: [SPARK-33259][SS] Disable streaming query with possible correctness issue by default

HeartSaVioR edited a comment on pull request #30210: URL: https://github.com/apache/spark/pull/30210#issuecomment-721373759 That's the chicken and egg problem, you know. dev@ list is not so active because all the important discussions aren't passing through the dev list, which is I think b

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30210: [SPARK-33259][SS] Disable streaming query with possible correctness issue by default

HeartSaVioR edited a comment on pull request #30210: URL: https://github.com/apache/spark/pull/30210#issuecomment-721373759 That's the chicken and egg problem, you know. dev@ list is not so active because all the important discussions aren't passing through the dev list, which is I think b

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30235: [SPARK-33293][SQL][FOLLOW-UP] Rename TableWriteExec to TableWriteExecHelper

AmplabJenkins removed a comment on pull request #30235: URL: https://github.com/apache/spark/pull/30235#issuecomment-721374612 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30235: [SPARK-33293][SQL][FOLLOW-UP] Rename TableWriteExec to TableWriteExecHelper

AmplabJenkins commented on pull request #30235: URL: https://github.com/apache/spark/pull/30235#issuecomment-721374612 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #30235: [SPARK-33293][SQL][FOLLOW-UP] Rename TableWriteExec to TableWriteExecHelper

SparkQA removed a comment on pull request #30235: URL: https://github.com/apache/spark/pull/30235#issuecomment-721232635 **[Test build #130569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130569/testReport)** for PR 30235 at commit [`71dd2cb`](https://gi

[GitHub] [spark] HeartSaVioR commented on pull request #30210: [SPARK-33259][SS] Disable streaming query with possible correctness issue by default

HeartSaVioR commented on pull request #30210: URL: https://github.com/apache/spark/pull/30210#issuecomment-721373759 That's the chicken and egg problem, you know. dev@ list is not so active because all the important discussions aren't passing through the dev list, which is I think bad in p

[GitHub] [spark] SparkQA commented on pull request #30235: [SPARK-33293][SQL][FOLLOW-UP] Rename TableWriteExec to TableWriteExecHelper

SparkQA commented on pull request #30235: URL: https://github.com/apache/spark/pull/30235#issuecomment-721373720 **[Test build #130569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130569/testReport)** for PR 30235 at commit [`71dd2cb`](https://github.co

[GitHub] [spark] venkata91 commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

venkata91 commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516948596 ## File path: common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java ## @@ -363,4 +363,39 @@ public boolean useOldFetchPr

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721367618 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging parti

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721367618 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721367609 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35176/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

AmplabJenkins removed a comment on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-721360032 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

AmplabJenkins commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-721360032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

SparkQA removed a comment on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-721280219 **[Test build #130571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130571/testReport)** for PR 29247 at commit [`32a55a6`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

SparkQA commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-721358865 **[Test build #130571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130571/testReport)** for PR 29247 at commit [`32a55a6`](https://github.co

[GitHub] [spark] HeartSaVioR commented on a change in pull request #30203: [SPARK-33303][SQL] Deduplicate deterministic PythonUDF calls

HeartSaVioR commented on a change in pull request #30203: URL: https://github.com/apache/spark/pull/30203#discussion_r516936056 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala ## @@ -218,13 +218,22 @@ object ExtractPythonUDFs

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721357313 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35176/ -

[GitHub] [spark] davidrabinowitz commented on pull request #30071: [SPARK-33172][SQL] Adding support for UserDefinedType for Spark SQL Code generator

davidrabinowitz commented on pull request #30071: URL: https://github.com/apache/spark/pull/30071#issuecomment-721355772 @HyukjinKwon Should I create another PR aimed at master? In order to test it first you need to create a table in BigQuery in the following manner: ```

[GitHub] [spark] otterc commented on a change in pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging

otterc commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r516929549 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1927,4 +1927,34 @@ package object config { .version("3.0.1")

[GitHub] [spark] xkrogen commented on a change in pull request #30096: [SPARK-33185][YARN][WIP] Set up yarn.Client to print direct links to driver stdout/stderr

xkrogen commented on a change in pull request #30096: URL: https://github.com/apache/spark/pull/30096#discussion_r516925641 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ## @@ -1172,6 +1181,31 @@ private[spark] class Client(

[GitHub] [spark] xkrogen commented on a change in pull request #30096: [SPARK-33185][YARN][WIP] Set up yarn.Client to print direct links to driver stdout/stderr

xkrogen commented on a change in pull request #30096: URL: https://github.com/apache/spark/pull/30096#discussion_r516925449 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ## @@ -1172,6 +1181,31 @@ private[spark] class Client(

[GitHub] [spark] xkrogen commented on a change in pull request #30096: [SPARK-33185][YARN][WIP] Set up yarn.Client to print direct links to driver stdout/stderr

xkrogen commented on a change in pull request #30096: URL: https://github.com/apache/spark/pull/30096#discussion_r516923948 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ## @@ -1172,6 +1181,31 @@ private[spark] class Client(

[GitHub] [spark] xkrogen commented on pull request #30096: [SPARK-33185][YARN][WIP] Set up yarn.Client to print direct links to driver stdout/stderr

xkrogen commented on pull request #30096: URL: https://github.com/apache/spark/pull/30096#issuecomment-721346209 > Is it still WIP @xkrogen ? Yes, just because I haven't looked at how to test it in an automated fashion (if that makes sense at all). Wanted to gather some feedback on t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins removed a comment on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721343606 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins removed a comment on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721343616 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins commented on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins commented on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721343606 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30229: [SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier

SparkQA commented on pull request #30229: URL: https://github.com/apache/spark/pull/30229#issuecomment-721343576 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35175/ ---

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r516912542 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -0,0 +1,966 @@ +/* + * Licensed to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-721338244 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721338687 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130

[GitHub] [spark] SparkQA removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging par

SparkQA removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721334930 **[Test build #130575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130575/testReport)** for PR 30164 at commit [`3a6219f`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721338676 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721338654 **[Test build #130575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130575/testReport)** for PR 30164 at commit [`3a6219f`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging parti

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-721338676 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-721338233 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

SparkQA commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-721338208 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35174/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-721338233 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

<    1   2   3   4   5   6   7   >