[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748045155 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -598,6 +606,15 @@ abstract class CastBase extends U

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966898566 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49619/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

SparkQA commented on pull request #34561: URL: https://github.com/apache/spark/pull/34561#issuecomment-966897783 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49615/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966891480 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49617/ -- This is an automated message from the A

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748037204 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -81,9 +82,13 @@ object Cast { case (StringType

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748037204 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -81,9 +82,13 @@ object Cast { case (StringType

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966890385 **[Test build #145148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145148/testReport)** for PR 34559 at commit [`aafb6a1`](https://github.co

[GitHub] [spark] ulysses-you commented on a change in pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

ulysses-you commented on a change in pull request #34568: URL: https://github.com/apache/spark/pull/34568#discussion_r748036217 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SortExec.scala ## @@ -206,3 +179,43 @@ case class SortExec( override protected

[GitHub] [spark] HyukjinKwon closed pull request #34566: [SPARK-36825][FOLLOWUP] Move the test code from `ParquetIOSuite` to `ParquetFileFormatSuite`

HyukjinKwon closed pull request #34566: URL: https://github.com/apache/spark/pull/34566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] ulysses-you opened a new pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

ulysses-you opened a new pull request #34568: URL: https://github.com/apache/spark/pull/34568 ### What changes were proposed in this pull request? - Add a new trait `V1Write` to hold some sort infos of v1 write. e.g., partition columns, bucket spec. - Then let the foll

[GitHub] [spark] HyukjinKwon commented on pull request #34566: [SPARK-36825][FOLLOWUP] Move the test code from `ParquetIOSuite` to `ParquetFileFormatSuite`

HyukjinKwon commented on pull request #34566: URL: https://github.com/apache/spark/pull/34566#issuecomment-966888698 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] cloud-fan commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

cloud-fan commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748032282 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -81,9 +82,13 @@ object Cast { case (StringTyp

[GitHub] [spark] cloud-fan commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

cloud-fan commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748031987 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -81,9 +82,13 @@ object Cast { case (StringTyp

[GitHub] [spark] mohamadrezarostami commented on a change in pull request #34331: [SPARK-37060][CORE] Handle driver status response from backup masters

mohamadrezarostami commented on a change in pull request #34331: URL: https://github.com/apache/spark/pull/34331#discussion_r748029609 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -190,13 +190,13 @@ private class ClientEndpoint(

[GitHub] [spark] cloud-fan commented on a change in pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

cloud-fan commented on a change in pull request #34546: URL: https://github.com/apache/spark/pull/34546#discussion_r748028549 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -204,11 +214,8 @@ class Analyzer(override val cat

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34497: [SPARK-37219][SQL] Add AS OF syntax support

AmplabJenkins removed a comment on pull request #34497: URL: https://github.com/apache/spark/pull/34497#issuecomment-966877953 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145130/ -

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966878738 **[Test build #145148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145148/testReport)** for PR 34559 at commit [`aafb6a1`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #34497: [SPARK-37219][SQL] Add AS OF syntax support

AmplabJenkins commented on pull request #34497: URL: https://github.com/apache/spark/pull/34497#issuecomment-966877953 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145130/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34497: [SPARK-37219][SQL] Add AS OF syntax support

SparkQA removed a comment on pull request #34497: URL: https://github.com/apache/spark/pull/34497#issuecomment-966768304 **[Test build #145130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145130/testReport)** for PR 34497 at commit [`f60bed6`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34497: [SPARK-37219][SQL] Add AS OF syntax support

SparkQA commented on pull request #34497: URL: https://github.com/apache/spark/pull/34497#issuecomment-966876886 **[Test build #145130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145130/testReport)** for PR 34497 at commit [`f60bed6`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34566: [SPARK-36825][FOLLOWUP] Move the test code from `ParquetIOSuite` to `ParquetFileFormatSuite`

AmplabJenkins removed a comment on pull request #34566: URL: https://github.com/apache/spark/pull/34566#issuecomment-966875659 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49613/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

AmplabJenkins removed a comment on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966875654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins removed a comment on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966875655 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145146/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tamper

AmplabJenkins removed a comment on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966875658 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145126/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

AmplabJenkins commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966875657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #34566: [SPARK-36825][FOLLOWUP] Move the test code from `ParquetIOSuite` to `ParquetFileFormatSuite`

AmplabJenkins commented on pull request #34566: URL: https://github.com/apache/spark/pull/34566#issuecomment-966875659 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49613/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966875655 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145146/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

AmplabJenkins commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966875658 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145126/ -- This

[GitHub] [spark] SparkQA commented on pull request #34567: [SPARK-37298][SQL] Use unique exprIds in RewriteAsOfJoin

SparkQA commented on pull request #34567: URL: https://github.com/apache/spark/pull/34567#issuecomment-966874373 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49618/ -- This is an automated message from the Apache

[GitHub] [spark] Ngone51 commented on a change in pull request #34331: [SPARK-37060][CORE] Handle driver status response from backup masters

Ngone51 commented on a change in pull request #34331: URL: https://github.com/apache/spark/pull/34331#discussion_r748022121 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -190,13 +190,13 @@ private class ClientEndpoint( logDebug(s"

[GitHub] [spark] mohamadrezarostami commented on a change in pull request #34331: [SPARK-37060][CORE] Handle driver status response from backup masters

mohamadrezarostami commented on a change in pull request #34331: URL: https://github.com/apache/spark/pull/34331#discussion_r748021561 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -190,13 +190,13 @@ private class ClientEndpoint(

[GitHub] [spark] Ngone51 commented on a change in pull request #34331: [SPARK-37060][CORE] Handle driver status response from backup masters

Ngone51 commented on a change in pull request #34331: URL: https://github.com/apache/spark/pull/34331#discussion_r748019724 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -190,13 +190,13 @@ private class ClientEndpoint( logDebug(s"

[GitHub] [spark] SparkQA removed a comment on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA removed a comment on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966853542 **[Test build #145146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145146/testReport)** for PR 34401 at commit [`ba81f80`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

SparkQA removed a comment on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966741491 **[Test build #145126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145126/testReport)** for PR 34543 at commit [`81f0ba1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966870614 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49617/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

SparkQA commented on pull request #34561: URL: https://github.com/apache/spark/pull/34561#issuecomment-966869420 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49615/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966868828 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49611/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966867110 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49614/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966865692 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49612/ -- This is an automated message from the A

[GitHub] [spark] GuoPhilipse commented on pull request #34549: [SPARK-37274][SQL]When the value of this parameter is greater than the maximum value of int type, the value will be thrown out of bounds.

GuoPhilipse commented on pull request #34549: URL: https://github.com/apache/spark/pull/34549#issuecomment-966864436 Changing the type to be long seems a good idea, or we may suffer this bounds issue in furture.:) if we simply want avoid this error ,we can set the value to the max int v

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966864259 **[Test build #145146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145146/testReport)** for PR 34401 at commit [`ba81f80`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34566: [SPARK-36825][FOLLOWUP] Move the test code from `ParquetIOSuite` to `ParquetFileFormatSuite`

SparkQA commented on pull request #34566: URL: https://github.com/apache/spark/pull/34566#issuecomment-966863698 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49613/ -- This is an automated message from the A

[GitHub] [spark] Peng-Lei commented on a change in pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

Peng-Lei commented on a change in pull request #34561: URL: https://github.com/apache/spark/pull/34561#discussion_r748009418 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -2915,6 +2915,39 @@ class DataSourceV2SQLSuite

[GitHub] [spark] Peng-Lei commented on a change in pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

Peng-Lei commented on a change in pull request #34561: URL: https://github.com/apache/spark/pull/34561#discussion_r748009418 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -2915,6 +2915,39 @@ class DataSourceV2SQLSuite

[GitHub] [spark] Peng-Lei commented on a change in pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

Peng-Lei commented on a change in pull request #34561: URL: https://github.com/apache/spark/pull/34561#discussion_r748009418 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -2915,6 +2915,39 @@ class DataSourceV2SQLSuite

[GitHub] [spark] SparkQA commented on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

SparkQA commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966859374 **[Test build #145126 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145126/testReport)** for PR 34543 at commit [`81f0ba1`](https://github.co

[GitHub] [spark] viirya commented on a change in pull request #34557: [SPARK-37292][SQL] Removes outer join if it only has DISTINCT on streamed side with alias

viirya commented on a change in pull request #34557: URL: https://github.com/apache/spark/pull/34557#discussion_r748005299 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala ## @@ -173,16 +173,18 @@ object EliminateOuterJoin extends R

[GitHub] [spark] SparkQA commented on pull request #34567: [SPARK-37298][SQL] Use unique exprIds in RewriteAsOfJoin

SparkQA commented on pull request #34567: URL: https://github.com/apache/spark/pull/34567#issuecomment-966856329 **[Test build #145147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145147/testReport)** for PR 34567 at commit [`48aaa4a`](https://github.com

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748005095 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -598,6 +606,15 @@ abstract class CastBase extends U

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

AmplabJenkins removed a comment on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966855446 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145145/ -

[GitHub] [spark] SparkQA removed a comment on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

SparkQA removed a comment on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966853389 **[Test build #145145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145145/testReport)** for PR 34546 at commit [`44ce71c`](https://gi

[GitHub] [spark] allisonwang-db opened a new pull request #34567: [SPARK-37298][SQL] Use unique exprIds in RewriteAsOfJoin

allisonwang-db opened a new pull request #34567: URL: https://github.com/apache/spark/pull/34567 ### What changes were proposed in this pull request? This PR updates the `RewriteAsOfJoin` rule to use new exprIds instead of reusing old exprIds. ### Why are the changes needed? To

[GitHub] [spark] SparkQA commented on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

SparkQA commented on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966855425 **[Test build #145145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145145/testReport)** for PR 34546 at commit [`44ce71c`](https://github.co

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r748004567 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -598,6 +606,15 @@ abstract class CastBase extends U

[GitHub] [spark] AmplabJenkins commented on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

AmplabJenkins commented on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966855446 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145145/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

AmplabJenkins removed a comment on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966854977 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49616/

[GitHub] [spark] SparkQA commented on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

SparkQA commented on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966854963 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49616/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

AmplabJenkins commented on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966854977 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49616/ -- T

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966853542 **[Test build #145146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145146/testReport)** for PR 34401 at commit [`ba81f80`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34546: [SPARK-37702][SQL] Use AnalysisContext to track referred temp functions

SparkQA commented on pull request #34546: URL: https://github.com/apache/spark/pull/34546#issuecomment-966853389 **[Test build #145145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145145/testReport)** for PR 34546 at commit [`44ce71c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

SparkQA commented on pull request #34561: URL: https://github.com/apache/spark/pull/34561#issuecomment-966853362 **[Test build #145144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145144/testReport)** for PR 34561 at commit [`a3d06d4`](https://github.com

[GitHub] [spark] Peng-Lei commented on pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

Peng-Lei commented on pull request #34561: URL: https://github.com/apache/spark/pull/34561#issuecomment-966853179 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tamper

AmplabJenkins removed a comment on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966851504 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49608/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

AmplabJenkins removed a comment on pull request #34565: URL: https://github.com/apache/spark/pull/34565#issuecomment-966851501 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49610/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

AmplabJenkins removed a comment on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966851498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins removed a comment on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966851506 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49609/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34505: [SPARK-37228][SQL][PYTHON] Implement DataFrame.mapInArrow in Python

AmplabJenkins removed a comment on pull request #34505: URL: https://github.com/apache/spark/pull/34505#issuecomment-966851479 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145133/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

AmplabJenkins commented on pull request #34565: URL: https://github.com/apache/spark/pull/34565#issuecomment-966851501 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49610/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966851506 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49609/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34505: [SPARK-37228][SQL][PYTHON] Implement DataFrame.mapInArrow in Python

AmplabJenkins commented on pull request #34505: URL: https://github.com/apache/spark/pull/34505#issuecomment-966851479 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145133/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

AmplabJenkins commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966851498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

AmplabJenkins commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966851504 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49608/ -- T

[GitHub] [spark] dh20 commented on pull request #34549: [SPARK-37274][SQL]When the value of this parameter is greater than the maximum value of int type, the value will be thrown out of bounds. The do

dh20 commented on pull request #34549: URL: https://github.com/apache/spark/pull/34549#issuecomment-966851332 > Your original claim is to note about the type of the property for clarification isn't it? But your next claim is to change the type of the property to expand the maximum batch si

[GitHub] [spark] dongjoon-hyun commented on pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

dongjoon-hyun commented on pull request #34565: URL: https://github.com/apache/spark/pull/34565#issuecomment-966850655 😃 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966850630 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49611/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34505: [SPARK-37228][SQL][PYTHON] Implement DataFrame.mapInArrow in Python

SparkQA removed a comment on pull request #34505: URL: https://github.com/apache/spark/pull/34505#issuecomment-966785601 **[Test build #145133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145133/testReport)** for PR 34505 at commit [`857fc77`](https://gi

[GitHub] [spark] dongjoon-hyun commented on pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

dongjoon-hyun commented on pull request #34565: URL: https://github.com/apache/spark/pull/34565#issuecomment-966850496 Oops. I merged it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [spark] SparkQA commented on pull request #34505: [SPARK-37228][SQL][PYTHON] Implement DataFrame.mapInArrow in Python

SparkQA commented on pull request #34505: URL: https://github.com/apache/spark/pull/34505#issuecomment-966850423 **[Test build #145133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145133/testReport)** for PR 34505 at commit [`857fc77`](https://github.co

[GitHub] [spark] HyukjinKwon commented on pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

HyukjinKwon commented on pull request #34565: URL: https://github.com/apache/spark/pull/34565#issuecomment-966850262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [spark] dongjoon-hyun closed pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

dongjoon-hyun closed pull request #34565: URL: https://github.com/apache/spark/pull/34565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] dongjoon-hyun commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

dongjoon-hyun commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966849936 Thank you for updates, @AngersZh . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] cloud-fan commented on a change in pull request #34497: [SPARK-37219][SQL] Add AS OF syntax support

cloud-fan commented on a change in pull request #34497: URL: https://github.com/apache/spark/pull/34497#discussion_r747999585 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsCatalogOptions.java ## @@ -52,4 +52,15 @@ default String ext

[GitHub] [spark] AngersZhuuuu commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

AngersZh commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966849062 @dongjoon-hyun tested a lot since not familiar with python code. Now I think can be reviewed -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966849011 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49614/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-966848699 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49609/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966848235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[GitHub] [spark] SparkQA removed a comment on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA removed a comment on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966825955 **[Test build #145143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145143/testReport)** for PR 34559 at commit [`7f1e668`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34566: [SPARK-36825][FOLLOWUP] Move the test code from `ParquetIOSuite` to `ParquetFileFormatSuite`

SparkQA commented on pull request #34566: URL: https://github.com/apache/spark/pull/34566#issuecomment-966847150 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49613/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34565: [MINOR][PYTHON][DOCS] Remove the reference of typeName as DDL format in documentation

SparkQA commented on pull request #34565: URL: https://github.com/apache/spark/pull/34565#issuecomment-966840168 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49610/ -- This is an automated message from the A

[GitHub] [spark] xuechendi commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

xuechendi commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r747993176 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkPla

[GitHub] [spark] SparkQA commented on pull request #34559: [SPARK-37291][SQL][PYSPARK] PySpark init SparkSession should copy conf to sharedState

SparkQA commented on pull request #34559: URL: https://github.com/apache/spark/pull/34559#issuecomment-966837260 **[Test build #145143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145143/testReport)** for PR 34559 at commit [`7f1e668`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

SparkQA commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-966835393 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49608/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

HyukjinKwon commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r747990678 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkP

[GitHub] [spark] dongjoon-hyun commented on pull request #34557: [SPARK-37292][SQL] Removes outer join if it only has DISTINCT on streamed side with alias

dongjoon-hyun commented on pull request #34557: URL: https://github.com/apache/spark/pull/34557#issuecomment-966834893 Merged to master for Apache Spark 3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect

cloud-fan commented on a change in pull request #34554: URL: https://github.com/apache/spark/pull/34554#discussion_r747991324 ## File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ## @@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializab

[GitHub] [spark] dongjoon-hyun closed pull request #34557: [SPARK-37292][SQL] Removes outer join if it only has DISTINCT on streamed side with alias

dongjoon-hyun closed pull request #34557: URL: https://github.com/apache/spark/pull/34557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34561: [SPARK-37294][SQL][TESTS] Check inserting of ANSI intervals into a table partitioned by the interval columns

dongjoon-hyun commented on a change in pull request #34561: URL: https://github.com/apache/spark/pull/34561#discussion_r747990988 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -2915,6 +2915,40 @@ class DataSourceV2SQLSuit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

HyukjinKwon commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r747990678 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkP

[GitHub] [spark] xuechendi commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

xuechendi commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r747988830 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkPla

[GitHub] [spark] cloud-fan commented on a change in pull request #34543: [SPARK-37266][SQL] Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tamper

cloud-fan commented on a change in pull request #34543: URL: https://github.com/apache/spark/pull/34543#discussion_r747990201 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewTestSuite.scala ## @@ -530,4 +530,23 @@ class PersistedViewTestSuite extend

  1   2   3   4   5   6   7   >