[GitHub] [spark] AngersZhuuuu commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
AngersZh commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902481790 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [spark] cxzl25 opened a new pull request #33796: [SPARK-36550][SQL] Propagation cause when UDF reflection fails

2021-08-20 Thread GitBox
cxzl25 opened a new pull request #33796: URL: https://github.com/apache/spark/pull/33796 ### What changes were proposed in this pull request? When the exception is InvocationTargetException, get cause. ### Why are the changes needed? Now when UDF reflection fails, InvocationTarg

[GitHub] [spark] cxzl25 commented on pull request #33796: [SPARK-36550][SQL] Propagation cause when UDF reflection fails

2021-08-20 Thread GitBox
cxzl25 commented on pull request #33796: URL: https://github.com/apache/spark/pull/33796#issuecomment-902490299 Manual test ``` Error in query: No handler for Hive UDF 'XXX': java.lang.UnsatisfiedLinkError: at java.lang.ClassLoader$NativeLibrary.load(Native Method)

[GitHub] [spark] SparkQA commented on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shutdown h

2021-08-20 Thread GitBox
SparkQA commented on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902492673 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47166/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33796: [SPARK-36550][SQL] Propagation cause when UDF reflection fails

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33796: URL: https://github.com/apache/spark/pull/33796#issuecomment-902497701 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
SparkQA commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902500329 **[Test build #142666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142666/testReport)** for PR 30057 at commit [`979e314`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shutdown h

2021-08-20 Thread GitBox
SparkQA commented on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902511086 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47166/ -- This is an automated message from the A

[GitHub] [spark] MaxGekk commented on a change in pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
MaxGekk commented on a change in pull request #33794: URL: https://github.com/apache/spark/pull/33794#discussion_r692766129 ## File path: docs/job-scheduling.md ## @@ -254,10 +254,13 @@ properties: The pool properties can be set by creating an XML file, similar to `conf/fai

[GitHub] [spark] SparkQA commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
SparkQA commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902528934 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47167/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shut

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902531071 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47166/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid execu

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902531071 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47166/

[GitHub] [spark] SparkQA commented on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shutdown h

2021-08-20 Thread GitBox
SparkQA commented on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902540140 **[Test build #142665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142665/testReport)** for PR 33795 at commit [`3ccc0f8`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor sh

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902469200 **[Test build #142665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142665/testReport)** for PR 33795 at commit [`3ccc0f8`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shut

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902541260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142665/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid execu

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902541260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142665/ -

[GitHub] [spark] SparkQA commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
SparkQA commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902552264 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47167/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902556496 **[Test build #142662 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142662/testReport)** for PR 32816 at commit [`b54e9c2`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902559282 **[Test build #142663 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142663/testReport)** for PR 32816 at commit [`f5ad40e`](https://github.co

[GitHub] [spark] gengliangwang opened a new pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release docker script

2021-08-20 Thread GitBox
gengliangwang opened a new pull request #33797: URL: https://github.com/apache/spark/pull/33797 ### What changes were proposed in this pull request? After https://github.com/apache/spark/pull/32726, Python doc build requires `sphinx-plotly-directive`. This PR is to install

[GitHub] [spark] SparkQA removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902424243 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902563139 **[Test build #142664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142664/testReport)** for PR 32816 at commit [`8058fe9`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902426885 **[Test build #142664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142664/testReport)** for PR 32816 at commit [`8058fe9`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902566875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902566881 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47167/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902566881 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47167/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-902566875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] SparkQA commented on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
SparkQA commented on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902567863 **[Test build #142667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142667/testReport)** for PR 33797 at commit [`b5153b0`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
SparkQA commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902567981 **[Test build #142668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142668/testReport)** for PR 33794 at commit [`bc2cc92`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
SparkQA commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902574750 **[Test build #142668 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142668/testReport)** for PR 33794 at commit [`bc2cc92`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902567981 **[Test build #142668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142668/testReport)** for PR 33794 at commit [`bc2cc92`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902574938 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142668/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902574938 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142668/ -- This

[GitHub] [spark] SparkQA commented on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
SparkQA commented on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902593624 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47168/ -- This

[GitHub] [spark] SparkQA commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
SparkQA commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902596097 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47169/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902600240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47168/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902600240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47168/

[GitHub] [spark] SparkQA commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
SparkQA commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902622997 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47169/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902629832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47169/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902629832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47169/

[GitHub] [spark] gengliangwang commented on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
gengliangwang commented on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902642118 Merging to master/3.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] gengliangwang closed pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
gengliangwang closed pull request #33797: URL: https://github.com/apache/spark/pull/33797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] cloud-fan commented on pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shutdown

2021-08-20 Thread GitBox
cloud-fan commented on pull request #33795: URL: https://github.com/apache/spark/pull/33795#issuecomment-902643445 thanks, merging to 3.1/3.0! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan closed pull request #33795: [SPARK-36532][CORE][3.1] Fix deadlock in CoarseGrainedExecutorBackend.onDisconnected to avoid executorsconnected to avoid executor shutdown hang

2021-08-20 Thread GitBox
cloud-fan closed pull request #33795: URL: https://github.com/apache/spark/pull/33795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] SparkQA commented on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
SparkQA commented on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902645091 **[Test build #142667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142667/testReport)** for PR 33797 at commit [`b5153b0`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #33673: [SPARK-36448][SQL] Exceptions in NoSuchItemException.scala have to be case classes

2021-08-20 Thread GitBox
cloud-fan commented on pull request #33673: URL: https://github.com/apache/spark/pull/33673#issuecomment-902649756 The GA failure is unrelated, I'm merging it to master, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] cloud-fan closed pull request #33673: [SPARK-36448][SQL] Exceptions in NoSuchItemException.scala have to be case classes

2021-08-20 Thread GitBox
cloud-fan closed pull request #33673: URL: https://github.com/apache/spark/pull/33673 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] SparkQA commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
SparkQA commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902652756 **[Test build #142666 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142666/testReport)** for PR 30057 at commit [`979e314`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902500329 **[Test build #142666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142666/testReport)** for PR 30057 at commit [`979e314`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902567863 **[Test build #142667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142667/testReport)** for PR 33797 at commit [`b5153b0`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902659750 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142666/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902659775 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142667/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33797: [SPARK-36551][BUILD] Add sphinx-plotly-directive in Spark release Dockerfile

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33797: URL: https://github.com/apache/spark/pull/33797#issuecomment-902659775 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142667/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-902659750 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142666/ -

[GitHub] [spark] cloud-fan commented on pull request #33536: [SPARK-36101][CORE] Grouping exception in core/api

2021-08-20 Thread GitBox
cloud-fan commented on pull request #33536: URL: https://github.com/apache/spark/pull/33536#issuecomment-902684147 centralizing user-facing errors is a good direction (easy for auditing, support different languages), but we need to make sure they are user-facing errors. There are a

[GitHub] [spark] cloud-fan commented on a change in pull request #33536: [SPARK-36101][CORE] Grouping exception in core/api

2021-08-20 Thread GitBox
cloud-fan commented on a change in pull request #33536: URL: https://github.com/apache/spark/pull/33536#discussion_r692936350 ## File path: core/src/main/scala/org/apache/spark/api/java/JavaUtils.scala ## @@ -89,7 +91,7 @@ private[spark] object JavaUtils { mm

[GitHub] [spark] srowen commented on pull request #33790: Updates AuthEngine to pass the correct SecretKeySpec format

2021-08-20 Thread GitBox
srowen commented on pull request #33790: URL: https://github.com/apache/spark/pull/33790#issuecomment-902694982 (This is a follow-up to https://github.com/apache/spark/commit/3b0dd14f1c5dd033ad0a6295baa288eda9dfe10a ) -- This is an automated message from the Apache Git Service. To respo

[GitHub] [spark] srowen closed pull request #33790: Updates AuthEngine to pass the correct SecretKeySpec format

2021-08-20 Thread GitBox
srowen closed pull request #33790: URL: https://github.com/apache/spark/pull/33790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] srowen commented on pull request #33790: Updates AuthEngine to pass the correct SecretKeySpec format

2021-08-20 Thread GitBox
srowen commented on pull request #33790: URL: https://github.com/apache/spark/pull/33790#issuecomment-902695917 Merged to master/3.2/3.1/3.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] yaooqinn closed pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
yaooqinn closed pull request #33794: URL: https://github.com/apache/spark/pull/33794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsub

[GitHub] [spark] yaooqinn commented on pull request #33794: [SPARK-35083][CORE][FOLLLOWUP] Improve docs and migration guide

2021-08-20 Thread GitBox
yaooqinn commented on pull request #33794: URL: https://github.com/apache/spark/pull/33794#issuecomment-902696656 thanks, merged to master and 3.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902700015 **[Test build #142669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142669/testReport)** for PR 33706 at commit [`ec9`](https://github.com

[GitHub] [spark] yaooqinn opened a new pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
yaooqinn opened a new pull request #33798: URL: https://github.com/apache/spark/pull/33798 ### What changes were proposed in this pull request? For the hive table, the actual write path and the schema handling are inconsistent when `spark.sql.legacy.charVarcharAsString` i

[GitHub] [spark] yaooqinn commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
yaooqinn commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902703405 cc @cloud-fan @HyukjinKwon @maropu thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
SparkQA commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902705245 **[Test build #142670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142670/testReport)** for PR 33798 at commit [`f33ca39`](https://github.com

[GitHub] [spark] yaooqinn commented on a change in pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
yaooqinn commented on a change in pull request #33706: URL: https://github.com/apache/spark/pull/33706#discussion_r692963820 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala ## @@ -68,7 +70,8 @@ private[sql] class JsonInferSche

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902710616 **[Test build #142671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142671/testReport)** for PR 33706 at commit [`9e8b227`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902731653 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47170/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902732991 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47171/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
SparkQA commented on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902738032 **[Test build #142672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142672/testReport)** for PR 33650 at commit [`7156958`](https://github.com

[GitHub] [spark] normalscene commented on pull request #5708: [SPARK-7155] [CORE] Allow newAPIHadoopFile to support comma-separated list of files as input

2021-08-20 Thread GitBox
normalscene commented on pull request #5708: URL: https://github.com/apache/spark/pull/5708#issuecomment-902743967 Hello @yongtang , I am troubleshooting slow wholeTextFiles reading issue, and while digging through the function code, I came across this ticket number. I have a questio

[GitHub] [spark] SparkQA commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
SparkQA commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902744141 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47172/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902748035 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47173/ -- This is an automated message from the Apache

[GitHub] [spark] yongtang commented on pull request #5708: [SPARK-7155] [CORE] Allow newAPIHadoopFile to support comma-separated list of files as input

2021-08-20 Thread GitBox
yongtang commented on pull request #5708: URL: https://github.com/apache/spark/pull/5708#issuecomment-902766960 @normalscene I haven't been very active in the community recently but I think you can create an issue in https://issues.apache.org/jira/projects/SPARK/issues/SPARK-33807?filter=

[GitHub] [spark] SparkQA commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
SparkQA commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902771523 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47172/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902771988 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47170/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902772240 **[Test build #142669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142669/testReport)** for PR 33706 at commit [`ec9`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902700015 **[Test build #142669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142669/testReport)** for PR 33706 at commit [`ec9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902773313 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47171/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
SparkQA commented on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902777802 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47174/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902779131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902779134 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47172/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902779131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902779134 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47172/

[GitHub] [spark] AmplabJenkins commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902787298 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47173/ -- T

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902787254 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47173/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall handle CharConversionException/MalformedInputException

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-902787298 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47173/

[GitHub] [spark] SparkQA commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
SparkQA commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902789094 **[Test build #142670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142670/testReport)** for PR 33798 at commit [`f33ca39`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
SparkQA removed a comment on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902705245 **[Test build #142670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142670/testReport)** for PR 33798 at commit [`f33ca39`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902790094 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142670/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33798: [SPARK-36552][SQL] Fix different behavior for writing char/varchar to hive and datasource table

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33798: URL: https://github.com/apache/spark/pull/33798#issuecomment-902790094 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142670/ -

[GitHub] [spark] SparkQA commented on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
SparkQA commented on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902812123 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47174/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
AmplabJenkins commented on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902815942 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47174/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
AmplabJenkins removed a comment on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902815942 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47174/

[GitHub] [spark] holdenk commented on pull request #33508: [SPARK-36058][K8S] Add support for statefulset APIs in K8s

2021-08-20 Thread GitBox
holdenk commented on pull request #33508: URL: https://github.com/apache/spark/pull/33508#issuecomment-902820557 Jenkins retest this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] SparkQA commented on pull request #33508: [SPARK-36058][K8S] Add support for statefulset APIs in K8s

2021-08-20 Thread GitBox
SparkQA commented on pull request #33508: URL: https://github.com/apache/spark/pull/33508#issuecomment-902821437 **[Test build #142673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142673/testReport)** for PR 33508 at commit [`49ab072`](https://github.com

[GitHub] [spark] xinrong-databricks commented on a change in pull request #33756: [SPARK-36470][PYTHON] Implement `CategoricalIndex.map` and `DatetimeIndex.map`

2021-08-20 Thread GitBox
xinrong-databricks commented on a change in pull request #33756: URL: https://github.com/apache/spark/pull/33756#discussion_r693103627 ## File path: python/pyspark/pandas/tests/indexes/test_base.py ## @@ -516,19 +515,6 @@ def test_missing(self): getattr(psdf.se

[GitHub] [spark] SparkQA commented on pull request #33508: [SPARK-36058][K8S] Add support for statefulset APIs in K8s

2021-08-20 Thread GitBox
SparkQA commented on pull request #33508: URL: https://github.com/apache/spark/pull/33508#issuecomment-902848441 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47175/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33756: [SPARK-36470][PYTHON] Implement `CategoricalIndex.map` and `DatetimeIndex.map`

2021-08-20 Thread GitBox
SparkQA commented on pull request #33756: URL: https://github.com/apache/spark/pull/33756#issuecomment-902850839 **[Test build #142674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142674/testReport)** for PR 33756 at commit [`080b746`](https://github.com

[GitHub] [spark] viirya commented on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
viirya commented on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902856349 Some description looks ambiguous. E.g., > add pushFilters in file source v2. In this method... As you are adding a method to a class. It is better to use the exact cl

[GitHub] [spark] viirya commented on pull request #33650: [SPARK-36351][SQL] Refactor filter push down in file source v2

2021-08-20 Thread GitBox
viirya commented on pull request #33650: URL: https://github.com/apache/spark/pull/33650#issuecomment-902858157 `pushDataFilters` is not mentioned in the description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

  1   2   >