[GitHub] [spark] SparkQA commented on pull request #32729: [SPARK-35539] Restore to_koalas to keep the backward compatibility

SparkQA commented on pull request #32729: URL: https://github.com/apache/spark/pull/32729#issuecomment-851874598 **[Test build #139144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139144/testReport)** for PR 32729 at commit [`6dcda50`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

AmplabJenkins removed a comment on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851874181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43660/

[GitHub] [spark] gengliangwang commented on a change in pull request #32725: [SPARK-33933][FOLLOW-UP][SQL] Fix a flaky test case in AdaptiveQueryExecSuite

gengliangwang commented on a change in pull request #32725: URL: https://github.com/apache/spark/pull/32725#discussion_r642828871 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1605,7 +1605,7 @@ class AdaptiveQ

[GitHub] [spark] SparkQA commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

SparkQA commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851874162 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43660/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

AmplabJenkins commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851874181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43660/ -- T

[GitHub] [spark] SparkQA commented on pull request #32726: [WIP][SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851873710 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43659/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32725: [SPARK-33933][FOLLOW-UP][SQL] Fix a flaky test case in AdaptiveQueryExecSuite

SparkQA commented on pull request #32725: URL: https://github.com/apache/spark/pull/32725#issuecomment-851873543 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43656/ -- This is an automated message from the Apache

[GitHub] [spark] itholic opened a new pull request #32729: [SPARK-35539] Restore to_koalas to keep the backward compatibility

itholic opened a new pull request #32729: URL: https://github.com/apache/spark/pull/32729 ### What changes were proposed in this pull request? This PR proposes restoring `to_koalas` to keep the backward compatibility, with throwing deprecated warning. ### Why are the changes n

[GitHub] [spark] gengliangwang commented on a change in pull request #32725: [SPARK-33933][FOLLOW-UP][SQL] Fix a flaky test case in AdaptiveQueryExecSuite

gengliangwang commented on a change in pull request #32725: URL: https://github.com/apache/spark/pull/32725#discussion_r642828871 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1605,7 +1605,7 @@ class AdaptiveQ

[GitHub] [spark] HeartSaVioR commented on a change in pull request #32653: [SPARK-35312][SS] Introduce new Option in Kafka source to specify minimum number of records to read per trigger

HeartSaVioR commented on a change in pull request #32653: URL: https://github.com/apache/spark/pull/32653#discussion_r642793811 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/streaming/CompositeReadLimit.java ## @@ -0,0 +1,59 @@ +/* + * Licensed t

[GitHub] [spark] SparkQA commented on pull request #32727: [SPARK-35589][CORE] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

SparkQA commented on pull request #32727: URL: https://github.com/apache/spark/pull/32727#issuecomment-851872617 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43658/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

SparkQA commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851872159 **[Test build #139142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139142/testReport)** for PR 32723 at commit [`f453070`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32726: [WIP][SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851872107 **[Test build #139141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139141/testReport)** for PR 32726 at commit [`262c5e1`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851872197 **[Test build #139143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139143/testReport)** for PR 32721 at commit [`64220b7`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32722: [SPARK-35586][[K8S][TESTS] Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests

AmplabJenkins removed a comment on pull request #32722: URL: https://github.com/apache/spark/pull/32722#issuecomment-851871648 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43652/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

AmplabJenkins removed a comment on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851871643 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43654/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32718: [SPARK-21957][SQL] Support current_user and session_user functions

AmplabJenkins removed a comment on pull request #32718: URL: https://github.com/apache/spark/pull/32718#issuecomment-851871645 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139127/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32658: [SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page

AmplabJenkins removed a comment on pull request #32658: URL: https://github.com/apache/spark/pull/32658#issuecomment-851871644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139126/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32726: [WIP][SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

AmplabJenkins removed a comment on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851871640 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139136/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

AmplabJenkins removed a comment on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851871642 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43655/

[GitHub] [spark] AmplabJenkins commented on pull request #32726: [WIP][SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

AmplabJenkins commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851871640 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139136/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32722: [SPARK-35586][[K8S][TESTS] Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests

AmplabJenkins commented on pull request #32722: URL: https://github.com/apache/spark/pull/32722#issuecomment-851871648 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43652/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32658: [SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page

AmplabJenkins commented on pull request #32658: URL: https://github.com/apache/spark/pull/32658#issuecomment-851871644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139126/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

AmplabJenkins commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851871643 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43654/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

AmplabJenkins commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851871642 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43655/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32718: [SPARK-21957][SQL] Support current_user and session_user functions

AmplabJenkins commented on pull request #32718: URL: https://github.com/apache/spark/pull/32718#issuecomment-851871645 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139127/ -- This

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851867667 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43655/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

SparkQA commented on pull request #32724: URL: https://github.com/apache/spark/pull/32724#issuecomment-851867518 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43657/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #32718: [SPARK-21957][SQL] Support current_user and session_user functions

SparkQA removed a comment on pull request #32718: URL: https://github.com/apache/spark/pull/32718#issuecomment-851767500 **[Test build #139127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139127/testReport)** for PR 32718 at commit [`ae337c1`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #32658: [SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page

SparkQA removed a comment on pull request #32658: URL: https://github.com/apache/spark/pull/32658#issuecomment-851749314 **[Test build #139126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139126/testReport)** for PR 32658 at commit [`f55a2fa`](https://gi

[GitHub] [spark] itholic commented on a change in pull request #32726: [WIP][SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

itholic commented on a change in pull request #32726: URL: https://github.com/apache/spark/pull/32726#discussion_r642820228 ## File path: python/docs/source/development/ps_contributing.rst ## @@ -0,0 +1,192 @@ +== +Contributing Guide +== + +.. co

[GitHub] [spark] itholic commented on a change in pull request #32726: [WIP][SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

itholic commented on a change in pull request #32726: URL: https://github.com/apache/spark/pull/32726#discussion_r642820228 ## File path: python/docs/source/development/ps_contributing.rst ## @@ -0,0 +1,192 @@ +== +Contributing Guide +== + +.. co

[GitHub] [spark] SparkQA commented on pull request #32718: [SPARK-21957][SQL] Support current_user and session_user functions

SparkQA commented on pull request #32718: URL: https://github.com/apache/spark/pull/32718#issuecomment-851863297 **[Test build #139127 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139127/testReport)** for PR 32718 at commit [`ae337c1`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32658: [SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page

SparkQA commented on pull request #32658: URL: https://github.com/apache/spark/pull/32658#issuecomment-851862974 **[Test build #139126 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139126/testReport)** for PR 32658 at commit [`f55a2fa`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

SparkQA commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851858720 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43654/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

HyukjinKwon commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851858113 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

HyukjinKwon commented on a change in pull request #32726: URL: https://github.com/apache/spark/pull/32726#discussion_r642815768 ## File path: python/docs/source/development/ps_contributing.rst ## @@ -0,0 +1,192 @@ +== +Contributing Guide +== + +.

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32727: [SPARK-35589][CORE] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

dongjoon-hyun commented on a change in pull request #32727: URL: https://github.com/apache/spark/pull/32727#discussion_r642814533 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala ## @@ -574,8 +574,9 @@ class BlockManagerMasterEndpoint

[GitHub] [spark] HyukjinKwon closed pull request #32728: [SPARK-35582][PYTHON][DOCS] Remove # noqa in Python API documents.

HyukjinKwon closed pull request #32728: URL: https://github.com/apache/spark/pull/32728 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [spark] HyukjinKwon commented on pull request #32728: [SPARK-35582][PYTHON][DOCS] Remove # noqa in Python API documents.

HyukjinKwon commented on pull request #32728: URL: https://github.com/apache/spark/pull/32728#issuecomment-851854693 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] cloud-fan commented on a change in pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

cloud-fan commented on a change in pull request #32724: URL: https://github.com/apache/spark/pull/32724#discussion_r642812433 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1373,6 +1380,31 @@ class AdaptiveQuer

[GitHub] [spark] cloud-fan commented on a change in pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

cloud-fan commented on a change in pull request #32724: URL: https://github.com/apache/spark/pull/32724#discussion_r642811199 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEOptimizer.scala ## @@ -27,9 +28,15 @@ import org.apache.spark.util.Uti

[GitHub] [spark] cloud-fan commented on a change in pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

cloud-fan commented on a change in pull request #32724: URL: https://github.com/apache/spark/pull/32724#discussion_r642811083 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEOptimizer.scala ## @@ -63,3 +70,8 @@ class AQEOptimizer(conf: SQLConf)

[GitHub] [spark] itholic commented on pull request #32728: [SPARK-35582][PYTHON][DOCS] Remove # noqa in Python API documents.

itholic commented on pull request #32728: URL: https://github.com/apache/spark/pull/32728#issuecomment-851852612 There is only one case remaining in the `schema_of_json` since other cases are all resolved in the previous PR. Please refer to: https://github.com/apache/spark/pull/3265

[GitHub] [spark] SparkQA commented on pull request #32722: [SPARK-35586][[K8S][TESTS] Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests

SparkQA commented on pull request #32722: URL: https://github.com/apache/spark/pull/32722#issuecomment-851852621 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43652/ -- This is an automated message from the A

[GitHub] [spark] itholic opened a new pull request #32728: [SPARK-35582][PYTHON][DOCS] Remove # noqa in Python API documents.

itholic opened a new pull request #32728: URL: https://github.com/apache/spark/pull/32728 ### What changes were proposed in this pull request? This PR aims to move `# noqa` in the Python docstring to the proper place so that hide them from the official documents. ### Why a

[GitHub] [spark] SparkQA removed a comment on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

SparkQA removed a comment on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851838979 **[Test build #139136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139136/testReport)** for PR 32726 at commit [`262c5e1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851851779 **[Test build #139136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139136/testReport)** for PR 32726 at commit [`262c5e1`](https://github.co

[GitHub] [spark] gengliangwang closed pull request #32720: [SPARK-35576][SQL][3.1] Redact the sensitive info in the result of Set command

gengliangwang closed pull request #32720: URL: https://github.com/apache/spark/pull/32720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] gengliangwang commented on pull request #32720: [SPARK-35576][SQL][3.1] Redact the sensitive info in the result of Set command

gengliangwang commented on pull request #32720: URL: https://github.com/apache/spark/pull/32720#issuecomment-851849727 Merging to branch-3.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] itholic commented on a change in pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

itholic commented on a change in pull request #32726: URL: https://github.com/apache/spark/pull/32726#discussion_r642807235 ## File path: python/docs/source/development/ps_contributing.rst ## @@ -0,0 +1,192 @@ +== +Contributing Guide +== + +.. co

[GitHub] [spark] dongjoon-hyun commented on pull request #32727: [SPARK-35589][CORE] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

dongjoon-hyun commented on pull request #32727: URL: https://github.com/apache/spark/pull/32727#issuecomment-851848696 cc @holdenk and @attilapiros -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] itholic commented on a change in pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

itholic commented on a change in pull request #32726: URL: https://github.com/apache/spark/pull/32726#discussion_r642807235 ## File path: python/docs/source/development/ps_contributing.rst ## @@ -0,0 +1,192 @@ +== +Contributing Guide +== + +.. co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32693: [SPARK-35556][SQL][TESTS] Avoid log NoSuchMethodError when HiveClientImpl.state close

AmplabJenkins removed a comment on pull request #32693: URL: https://github.com/apache/spark/pull/32693#issuecomment-851847341 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139130/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32693: [SPARK-35556][SQL][TESTS] Avoid log NoSuchMethodError when HiveClientImpl.state close

AmplabJenkins commented on pull request #32693: URL: https://github.com/apache/spark/pull/32693#issuecomment-851847341 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139130/ -- This

[GitHub] [spark] Yikun commented on pull request #32719: [SPARK-35584][TESTS] Increase the timeout in FallbackStorageSuite

Yikun commented on pull request #32719: URL: https://github.com/apache/spark/pull/32719#issuecomment-851847169 @HyukjinKwon retriggered -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851846824 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43655/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

HyukjinKwon commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-851846833 Just to double check, are these: ``` { "value": "foo" }

[GitHub] [spark] SparkQA removed a comment on pull request #32693: [SPARK-35556][SQL][TESTS] Avoid log NoSuchMethodError when HiveClientImpl.state close

SparkQA removed a comment on pull request #32693: URL: https://github.com/apache/spark/pull/32693#issuecomment-851785773 **[Test build #139130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139130/testReport)** for PR 32693 at commit [`698bea5`](https://gi

[GitHub] [spark] MaxGekk commented on a change in pull request #32725: [SPARK-33933][FOLLOW-UP][SQL] Fix a flaky test case in AdaptiveQueryExecSuite

MaxGekk commented on a change in pull request #32725: URL: https://github.com/apache/spark/pull/32725#discussion_r642805850 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1605,7 +1605,7 @@ class AdaptiveQueryEx

[GitHub] [spark] SparkQA commented on pull request #32693: [SPARK-35556][SQL][TESTS] Avoid log NoSuchMethodError when HiveClientImpl.state close

SparkQA commented on pull request #32693: URL: https://github.com/apache/spark/pull/32693#issuecomment-851846208 **[Test build #139130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139130/testReport)** for PR 32693 at commit [`698bea5`](https://github.co

[GitHub] [spark] HyukjinKwon commented on pull request #32719: [SPARK-35584][TESTS] Increase the timeout in FallbackStorageSuite

HyukjinKwon commented on pull request #32719: URL: https://github.com/apache/spark/pull/32719#issuecomment-851844383 can you retrigger the test https://github.com/Yikun/spark/actions/runs/894584787 please, @Yikun? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

HyukjinKwon commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851843868 cc @maropu FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [spark] SparkQA commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

SparkQA commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851843774 **[Test build #139140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139140/testReport)** for PR 32723 at commit [`2126a67`](https://github.com

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

HyukjinKwon commented on a change in pull request #32723: URL: https://github.com/apache/spark/pull/32723#discussion_r642803769 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -754,6 +746,8 @@ final class DataFrameWriter[T] private[sql](ds

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

HyukjinKwon commented on a change in pull request #32723: URL: https://github.com/apache/spark/pull/32723#discussion_r642803647 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -301,23 +306,22 @@ class DataFrameReader private[sql](sparkSess

[GitHub] [spark] SparkQA commented on pull request #32727: [SPARK-35589][CORE] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

SparkQA commented on pull request #32727: URL: https://github.com/apache/spark/pull/32727#issuecomment-851841549 **[Test build #139139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139139/testReport)** for PR 32727 at commit [`fef3e75`](https://github.com

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32725: [SPARK-33933][FOLLOW-UP][SQL] Fix a flaky test case in AdaptiveQueryExecSuite

HyukjinKwon commented on a change in pull request #32725: URL: https://github.com/apache/spark/pull/32725#discussion_r642801775 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1605,7 +1605,7 @@ class AdaptiveQue

[GitHub] [spark] dongjoon-hyun opened a new pull request #32727: [SPARK-35589][CORE] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

dongjoon-hyun opened a new pull request #32727: URL: https://github.com/apache/spark/pull/32727 … ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change?

[GitHub] [spark] SparkQA commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

SparkQA commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851839313 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43654/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32725: [SPARK-33933][FollowUp][SQL] Fix a flaky test case in AdaptiveQueryExecSuite

SparkQA commented on pull request #32725: URL: https://github.com/apache/spark/pull/32725#issuecomment-851839028 **[Test build #139137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139137/testReport)** for PR 32725 at commit [`f890295`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851838979 **[Test build #139136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139136/testReport)** for PR 32726 at commit [`262c5e1`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

SparkQA commented on pull request #32724: URL: https://github.com/apache/spark/pull/32724#issuecomment-851839022 **[Test build #139138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139138/testReport)** for PR 32724 at commit [`42a217e`](https://github.com

[GitHub] [spark] HyukjinKwon commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

HyukjinKwon commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-851837787 cc @ueshin @xinrong-databricks @itholic FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon opened a new pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

HyukjinKwon opened a new pull request #32726: URL: https://github.com/apache/spark/pull/32726 ### What changes were proposed in this pull request? This PR proposes to port Koalas documentation to PySpark documentation as its initial step. It ports almost as is except that the impo

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

AmplabJenkins removed a comment on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851836879 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43653/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32719: [SPARK-35584][TESTS] Increase the timeout in FallbackStorageSuite

AmplabJenkins removed a comment on pull request #32719: URL: https://github.com/apache/spark/pull/32719#issuecomment-851836878 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139129/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32719: [SPARK-35584][TESTS] Increase the timeout in FallbackStorageSuite

AmplabJenkins commented on pull request #32719: URL: https://github.com/apache/spark/pull/32719#issuecomment-851836878 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139129/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

AmplabJenkins commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851836879 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43653/ -- T

[GitHub] [spark] gengliangwang opened a new pull request #32725: [SPARK-33933][FollowUp][SQL]

gengliangwang opened a new pull request #32725: URL: https://github.com/apache/spark/pull/32725 ### What changes were proposed in this pull request? Fix a flaky test case in AdaptiveQueryExecSuite ### Why are the changes needed? The test case becomes flaky since

[GitHub] [spark] MaxGekk commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

MaxGekk commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-851835967 cc @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [spark] ulysses-you commented on pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

ulysses-you commented on pull request #32724: URL: https://github.com/apache/spark/pull/32724#issuecomment-851834150 cc @maropu @cloud-fan @yaooqinn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] ulysses-you commented on a change in pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

ulysses-you commented on a change in pull request #32724: URL: https://github.com/apache/spark/pull/32724#discussion_r642797088 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1711,7 +1711,7 @@ object DecimalAggregates e

[GitHub] [spark] SparkQA commented on pull request #32722: [SPARK-35586][[K8S][TESTS] Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests

SparkQA commented on pull request #32722: URL: https://github.com/apache/spark/pull/32722#issuecomment-851832519 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43652/ -- This is an automated message from the Apache

[GitHub] [spark] yaooqinn commented on a change in pull request #32718: [SPARK-21957][SQL] Support current_user and session_user functions

yaooqinn commented on a change in pull request #32718: URL: https://github.com/apache/spark/pull/32718#discussion_r642796052 ## File path: sql/core/src/test/scala/org/apache/spark/sql/MiscFunctionsSuite.scala ## @@ -40,6 +40,12 @@ class MiscFunctionsSuite extends QueryTest with

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851830047 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43653/ -- This

[GitHub] [spark] MaxGekk commented on a change in pull request #32714: [SPARK-35581][SQL] Support special datetime values in typed literals only

MaxGekk commented on a change in pull request #32714: URL: https://github.com/apache/spark/pull/32714#discussion_r642795083 ## File path: docs/sql-migration-guide.md ## @@ -91,6 +91,8 @@ license: | - In Spark 3.2, `CREATE TABLE AS SELECT` with non-empty `LOCATION` will th

[GitHub] [spark] ulysses-you opened a new pull request #32724: [SPARK-35585][SQL] Support propagate empty relation through project/filter

ulysses-you opened a new pull request #32724: URL: https://github.com/apache/spark/pull/32724 ### What changes were proposed in this pull request? Add rule `ConvertToLocalRelation` into AQE Optimizer. ### Why are the changes needed? Support propagate empty local

[GitHub] [spark] HeartSaVioR commented on a change in pull request #32653: [SPARK-35312][SS] Introduce new Option in Kafka source to specify minimum number of records to read per trigger

HeartSaVioR commented on a change in pull request #32653: URL: https://github.com/apache/spark/pull/32653#discussion_r642792749 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/streaming/CompositeReadLimit.java ## @@ -0,0 +1,61 @@ +/* + * Licensed t

[GitHub] [spark] SparkQA removed a comment on pull request #32719: [SPARK-35584][TESTS] Increase the timeout in FallbackStorageSuite

SparkQA removed a comment on pull request #32719: URL: https://github.com/apache/spark/pull/32719#issuecomment-851769807 **[Test build #139129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139129/testReport)** for PR 32719 at commit [`941ee9c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32719: [SPARK-35584][TESTS] Increase the timeout in FallbackStorageSuite

SparkQA commented on pull request #32719: URL: https://github.com/apache/spark/pull/32719#issuecomment-851821942 **[Test build #139129 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139129/testReport)** for PR 32719 at commit [`941ee9c`](https://github.co

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

HyukjinKwon commented on a change in pull request #32723: URL: https://github.com/apache/spark/pull/32723#discussion_r642788618 ## File path: python/pyspark/sql/readwriter.py ## @@ -627,8 +627,6 @@ def jdbc(self, url, table, column=None, lowerBound=None, upperBound=None, numPa

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851806715 **[Test build #139135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139135/testReport)** for PR 32721 at commit [`864ee6f`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA removed a comment on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851803742 **[Test build #139133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139133/testReport)** for PR 32721 at commit [`d95d332`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

AmplabJenkins removed a comment on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851806052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139133/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

AmplabJenkins commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851806052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139133/ -- This

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851806030 **[Test build #139133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139133/testReport)** for PR 32721 at commit [`d95d332`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

SparkQA commented on pull request #32723: URL: https://github.com/apache/spark/pull/32723#issuecomment-851805108 **[Test build #139134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139134/testReport)** for PR 32723 at commit [`5853560`](https://github.com

[GitHub] [spark] itholic opened a new pull request #32723: [SPARK-35583][DOCS] Move JDBC data source options from Python and Scala into a single page

itholic opened a new pull request #32723: URL: https://github.com/apache/spark/pull/32723 ### What changes were proposed in this pull request? This PR proposes move missing JDBC data source options from Python, Scala and Java into a single page. ### Why are the changes needed?

[GitHub] [spark] SparkQA commented on pull request #32721: [WIP][SPARK-35077][SQL] Migrate to transformWithPruning for leftover optimizer rules

SparkQA commented on pull request #32721: URL: https://github.com/apache/spark/pull/32721#issuecomment-851803742 **[Test build #139133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139133/testReport)** for PR 32721 at commit [`d95d332`](https://github.com

  1   2   3   4   5   6   >