[GitHub] [spark] huaxingao commented on a change in pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

huaxingao commented on a change in pull request #34754: URL: https://github.com/apache/spark/pull/34754#discussion_r760355203 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -586,19 +586,13 @@ final class DataFrameWriter[T]

[GitHub] [spark] huaxingao commented on a change in pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

huaxingao commented on a change in pull request #34754: URL: https://github.com/apache/spark/pull/34754#discussion_r760355475 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2Exec.scala ## @@ -196,11 +204,19 @@ case class

[GitHub] [spark] SparkQA commented on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

SparkQA commented on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983836128 **[Test build #145814 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145814/testReport)** for PR 34769 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983605763 **[Test build #145815 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145815/testReport)** for PR 34741 at commit

[GitHub] [spark] SparkQA commented on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

SparkQA commented on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983941647 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50296/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

SparkQA commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-983942079 **[Test build #145823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145823/testReport)** for PR 32875 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

AmplabJenkins removed a comment on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983941692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50296/

[GitHub] [spark] AmplabJenkins commented on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

AmplabJenkins commented on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983941692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50296/ --

[GitHub] [spark] SparkQA removed a comment on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

SparkQA removed a comment on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983825614 **[Test build #145821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145821/testReport)** for PR 34607 at commit

[GitHub] [spark] SparkQA commented on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

SparkQA commented on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983953987 **[Test build #145821 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145821/testReport)** for PR 34607 at commit

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760524332 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java ## @@ -39,12 +42,13 @@

[GitHub] [spark] SparkQA commented on pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

SparkQA commented on pull request #34754: URL: https://github.com/apache/spark/pull/34754#issuecomment-983814722 **[Test build #145820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145820/testReport)** for PR 34754 at commit

[GitHub] [spark] SparkQA commented on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

SparkQA commented on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983825614 **[Test build #145821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145821/testReport)** for PR 34607 at commit

[GitHub] [spark] attilapiros commented on pull request #34672: [SPARK-37394][CORE] Skip registering with ESS if a customized shuffle manager is configured

attilapiros commented on pull request #34672: URL: https://github.com/apache/spark/pull/34672#issuecomment-983826971 @tgravescs I agree with you and I would be happy to work on the ideal solution. This is why I tried to push SPARK-31801 by copying Matthew Cheah's PR as

[GitHub] [spark] dongjoon-hyun closed pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun closed pull request #34770: URL: https://github.com/apache/spark/pull/34770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

SparkQA commented on pull request #34771: URL: https://github.com/apache/spark/pull/34771#issuecomment-983857410 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50294/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32298: [SPARK-34079][SQL] Merge non-correlated scalar subqueries

SparkQA commented on pull request #32298: URL: https://github.com/apache/spark/pull/32298#issuecomment-983937212 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50297/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32298: [SPARK-34079][SQL] Merge non-correlated scalar subqueries

AmplabJenkins removed a comment on pull request #32298: URL: https://github.com/apache/spark/pull/32298#issuecomment-983983324 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50297/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

AmplabJenkins removed a comment on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983983439 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145821/

[GitHub] [spark] AmplabJenkins commented on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

AmplabJenkins commented on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983984304 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145817/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

AmplabJenkins commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983810244 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145813/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

AmplabJenkins removed a comment on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983810243 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50292/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

AmplabJenkins removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983810244 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145813/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

AmplabJenkins removed a comment on pull request #34599: URL: https://github.com/apache/spark/pull/34599#issuecomment-983810370 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50291/

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983852618 **[Test build #145815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145815/testReport)** for PR 34741 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

AmplabJenkins removed a comment on pull request #34754: URL: https://github.com/apache/spark/pull/34754#issuecomment-983938196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50295/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

AmplabJenkins removed a comment on pull request #34771: URL: https://github.com/apache/spark/pull/34771#issuecomment-983938195 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50294/

[GitHub] [spark] mridulm commented on a change in pull request #34760: [SPARK-37506][CORE][SQL][DSTREAM][GRAPHX][ML][MLLIB][SS][EXAMPLES] Change the never changed 'var' to 'val'

mridulm commented on a change in pull request #34760: URL: https://github.com/apache/spark/pull/34760#discussion_r760479319 ## File path: examples/src/main/scala/org/apache/spark/examples/MiniReadWriteTest.scala ## @@ -59,7 +59,7 @@ object MiniReadWriteTest {

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760514700 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetColumnVector.java ## @@ -0,0 +1,321 @@ +/* + * Licensed

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760525243 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java ## @@ -303,55 +313,88 @@

[GitHub] [spark] SparkQA commented on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

SparkQA commented on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983572753 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50285/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983580955 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50284/ -- This is an automated message from the

[GitHub] [spark] weixiuli opened a new pull request #34768: [SPARK-11150][SQL][FOLLOWUP] We should drop all tables after testing dynamic partition pruning.

weixiuli opened a new pull request #34768: URL: https://github.com/apache/spark/pull/34768 ### What changes were proposed in this pull request? Drop all tables after testing dynamic partition pruning. ### Why are the changes needed? We should drop all

[GitHub] [spark] beliefer opened a new pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

beliefer opened a new pull request #34769: URL: https://github.com/apache/spark/pull/34769 ### What changes were proposed in this pull request? This PR used to fix the issue https://github.com/apache/spark/pull/33588#issuecomment-978719988 The root cause is Orc write/read

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983585767 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50287/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

SparkQA removed a comment on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983370498 **[Test build #145802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145802/testReport)** for PR 34758 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

AmplabJenkins removed a comment on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-983599243 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145801/

[GitHub] [spark] SparkQA removed a comment on pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

SparkQA removed a comment on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-983403563 **[Test build #145806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145806/testReport)** for PR 34673 at commit

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983715649 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50290/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

SparkQA commented on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983750385 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50292/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

AmplabJenkins commented on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983810243 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50292/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983864272 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145815/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

AmplabJenkins commented on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983864270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145814/ -- This

[GitHub] [spark] dongjoon-hyun closed pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

dongjoon-hyun closed pull request #34771: URL: https://github.com/apache/spark/pull/34771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] dongjoon-hyun commented on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

dongjoon-hyun commented on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983914739 +1, LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

SparkQA commented on pull request #34754: URL: https://github.com/apache/spark/pull/34754#issuecomment-983936753 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50295/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

SparkQA commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-983948021 **[Test build #145824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145824/testReport)** for PR 32875 at commit

[GitHub] [spark] sunchao commented on a change in pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

sunchao commented on a change in pull request #32875: URL: https://github.com/apache/spark/pull/32875#discussion_r760479294 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala ## @@ -70,61 +70,98 @@ case class

[GitHub] [spark] sunchao commented on pull request #34656: [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

sunchao commented on pull request #34656: URL: https://github.com/apache/spark/pull/34656#issuecomment-983955492 Thanks all! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

SparkQA commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-983978516 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50298/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

AmplabJenkins removed a comment on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983984304 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145817/

[GitHub] [spark] SparkQA commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

SparkQA commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-983985398 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50299/ -- This is an automated message from the Apache

[GitHub] [spark] mridulm commented on a change in pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

mridulm commented on a change in pull request #34767: URL: https://github.com/apache/spark/pull/34767#discussion_r760512680 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala ## @@ -74,7 +74,7 @@ private[spark]

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760522527 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetColumnVector.java ## @@ -0,0 +1,321 @@ +/* + * Licensed

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760528848 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java ## @@ -303,55 +313,88 @@

[GitHub] [spark] beliefer closed pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

beliefer closed pull request #34712: URL: https://github.com/apache/spark/pull/34712 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] beliefer edited a comment on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

beliefer edited a comment on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983586891 Because my mistake rebase not correctly, I create this PR to replace https://github.com/apache/spark/pull/34712 ping @cloud-fan @bersprockets This PR can fix all

[GitHub] [spark] AmplabJenkins commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983594755 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145802/ -- This

[GitHub] [spark] SparkQA commented on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

SparkQA commented on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983594523 **[Test build #145814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145814/testReport)** for PR 34769 at commit

[GitHub] [spark] SparkQA commented on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

SparkQA commented on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983629826 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50289/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

SparkQA commented on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-983629803 **[Test build #145806 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145806/testReport)** for PR 34673 at commit

[GitHub] [spark] zero323 commented on a change in pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py

zero323 commented on a change in pull request #34363: URL: https://github.com/apache/spark/pull/34363#discussion_r760182927 ## File path: python/pyspark/_typing.pyi ## @@ -21,11 +21,14 @@ from typing_extensions import Protocol F = TypeVar("F", bound=Callable) T =

[GitHub] [spark] cloud-fan edited a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

cloud-fan edited a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983660633 @bersprockets After reading more ORC code, I feel the timestamp implementation is quite messy in ORC. Not only the reader side, but also the writer side shifts the

[GitHub] [spark] Yikun commented on pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

Yikun commented on pull request #34599: URL: https://github.com/apache/spark/pull/34599#issuecomment-983685340 > In theory, this feature looks unsafe because there is a chance to leak the pre-populated resources because they have no owner yet. Yes, that's a good point, and the

[GitHub] [spark] SparkQA commented on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

SparkQA commented on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983706656 **[Test build #145817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145817/testReport)** for PR 34738 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

AmplabJenkins commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983559882 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145809/ -- This

[GitHub] [spark] cloud-fan commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

cloud-fan commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r760148616 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownTopN.java ## @@ -0,0 +1,37 @@ +/* + * Licensed to the

[GitHub] [spark] beliefer commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

beliefer commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r760174691 ## File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ## @@ -370,6 +370,8 @@ abstract class JdbcDialect extends

[GitHub] [spark] cloud-fan commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

cloud-fan commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983660633 @bersprockets After reading more ORC code, I feel the timestamp implementation is quite messy in ORC. Not only the reader side, but also the writer side shifts the timestamp

[GitHub] [spark] cloud-fan commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

cloud-fan commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r760225314 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala ## @@ -1012,6 +1012,196 @@ abstract class CSVSuite

[GitHub] [spark] SparkQA commented on pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

SparkQA commented on pull request #34599: URL: https://github.com/apache/spark/pull/34599#issuecomment-983695873 **[Test build #145816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145816/testReport)** for PR 34599 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983751502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

SparkQA commented on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983752675 **[Test build #145818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145818/testReport)** for PR 34770 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

AmplabJenkins commented on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983766929 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145818/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

SparkQA removed a comment on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983752675 **[Test build #145818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145818/testReport)** for PR 34770 at commit

[GitHub] [spark] SparkQA commented on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

SparkQA commented on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983766486 **[Test build #145818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145818/testReport)** for PR 34770 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

AmplabJenkins removed a comment on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983766929 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145818/

[GitHub] [spark] sarutak commented on pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

sarutak commented on pull request #34771: URL: https://github.com/apache/spark/pull/34771#issuecomment-983774542 cc: @sadikovi @MaxGekk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

AmplabJenkins removed a comment on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983559882 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145809/

[GitHub] [spark] SparkQA removed a comment on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA removed a comment on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983450312 **[Test build #145809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145809/testReport)** for PR 34753 at commit

[GitHub] [spark] zero323 commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

zero323 commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r760125119 ## File path: R/pkg/_pkgdown.yml ## @@ -0,0 +1,293 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

AmplabJenkins removed a comment on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983592141 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50285/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

AmplabJenkins removed a comment on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983592144 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50283/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983592142 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50284/

[GitHub] [spark] AmplabJenkins commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

AmplabJenkins commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983592144 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50283/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

AmplabJenkins commented on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983592141 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50285/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983592142 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50284/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

AmplabJenkins commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983655989 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50288/ --

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983655923 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50288/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

cloud-fan commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r760222777 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TimestampFormatter.scala ## @@ -66,10 +68,23 @@ sealed trait

[GitHub] [spark] cloud-fan commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

cloud-fan commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r760223753 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TimestampFormatter.scala ## @@ -66,10 +68,23 @@ sealed trait

[GitHub] [spark] Yikun commented on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

Yikun commented on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983717852 As @dongjoon-hyun suggestion: https://github.com/apache/spark/pull/34734#issuecomment-983400375 , backport to branch-3.2. -- This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983559573 **[Test build #145809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145809/testReport)** for PR 34753 at commit

[GitHub] [spark] cloud-fan closed pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

cloud-fan closed pull request #34718: URL: https://github.com/apache/spark/pull/34718 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan edited a comment on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

cloud-fan edited a comment on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983583155 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins removed a comment on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983594755 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145802/

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983602523 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50288/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

cloud-fan commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r760145878 ## File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ## @@ -370,6 +370,8 @@ abstract class JdbcDialect extends

[GitHub] [spark] AmplabJenkins commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983652224 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145804/ -- This

[GitHub] [spark] cloud-fan commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

cloud-fan commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983672127 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

<    1   2   3   4   5   6   7   >