[GitHub] [spark] cloud-fan edited a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

cloud-fan edited a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872054287 cc @ulysses-you @yaooqinn @viirya @JkSelf @wangyum -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] cloud-fan commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

cloud-fan commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872054287 cc @ulysses-you @yaooqinn @viirya @JkSelf -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] cloud-fan commented on a change in pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

cloud-fan commented on a change in pull request #33172: URL: https://github.com/apache/spark/pull/33172#discussion_r662102426 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ShufflePartitionsUtilSuite.scala ## @@ -459,9 +466,36 @@ class

[GitHub] [spark] cloud-fan opened a new pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

cloud-fan opened a new pull request #33172: URL: https://github.com/apache/spark/pull/33172 ### What changes were proposed in this pull request? By default, AQE will set `COALESCE_PARTITIONS_MIN_PARTITION_NUM` to the spark default parallelism, which is usually big. This is

[GitHub] [spark] SparkQA commented on pull request #32959: [WIP][SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-872052443 **[Test build #140505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140505/testReport)** for PR 32959 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

AmplabJenkins removed a comment on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-872051587 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140480/

[GitHub] [spark] SparkQA commented on pull request #33171: [SPARK-35969][K8S] Make the pod prefix more readable and tallied with K8S DNS Label Names

SparkQA commented on pull request #33171: URL: https://github.com/apache/spark/pull/33171#issuecomment-872052214 **[Test build #140504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140504/testReport)** for PR 33171 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

AmplabJenkins removed a comment on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872051852 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45007/

[GitHub] [spark] AmplabJenkins commented on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

AmplabJenkins commented on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872051852 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45007/ --

[GitHub] [spark] SparkQA commented on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

SparkQA commented on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872051823 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45007/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

AmplabJenkins commented on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-872051587 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140480/ -- This

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-872050821 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45004/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

SparkQA removed a comment on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-871889080 **[Test build #140480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140480/testReport)** for PR 33142 at commit

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872050269 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45006/ -- This is an automated message from the Apache

[GitHub] [spark] yaooqinn opened a new pull request #33171: [SPARK-35969][K8S] Make the pod prefix more readable and tallied with K8S DNS Label Names

yaooqinn opened a new pull request #33171: URL: https://github.com/apache/spark/pull/33171 ### What changes were proposed in this pull request? By default, the executor pod prefix is generated by the app name. It handles characters that match [^a-z0-9\\-]

[GitHub] [spark] ulysses-you commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

ulysses-you commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-872049667 cc @cloud-fan @yaooqinn @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

SparkQA commented on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-872049775 **[Test build #140480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140480/testReport)** for PR 33142 at commit

[GitHub] [spark] SparkQA commented on pull request #33170: [SPARK-35967][SQL] Update nullability based on column statistics

SparkQA commented on pull request #33170: URL: https://github.com/apache/spark/pull/33170#issuecomment-872049198 **[Test build #140502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140502/testReport)** for PR 33170 at commit

[GitHub] [spark] SparkQA commented on pull request #33058: [SPARK-35874][SQL] AQE Shuffle should wait for its subqueries to finish before materializing

SparkQA commented on pull request #33058: URL: https://github.com/apache/spark/pull/33058#issuecomment-872049267 **[Test build #140503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140503/testReport)** for PR 33058 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

AmplabJenkins removed a comment on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872047053 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140496/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

AmplabJenkins removed a comment on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872047052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140483/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33169: [SPARK-35966][SQL] Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings

AmplabJenkins removed a comment on pull request #33169: URL: https://github.com/apache/spark/pull/33169#issuecomment-872047047 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140495/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

AmplabJenkins removed a comment on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872047043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872047048 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45000/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33165: [SPARK-35961][SQL] Only use local shuffle reader when REBALANCE_PARTITIONS_BY_NONE without CustomShuffleReaderExec

AmplabJenkins removed a comment on pull request #33165: URL: https://github.com/apache/spark/pull/33165#issuecomment-872047046 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140477/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

AmplabJenkins removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872047044 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140482/

[GitHub] [spark] AmplabJenkins commented on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

AmplabJenkins commented on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872047052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140483/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

AmplabJenkins commented on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872047043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33169: [SPARK-35966][SQL] Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings

AmplabJenkins commented on pull request #33169: URL: https://github.com/apache/spark/pull/33169#issuecomment-872047047 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140495/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872047048 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45000/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

AmplabJenkins commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872047044 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140482/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

AmplabJenkins commented on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872047053 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140496/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33165: [SPARK-35961][SQL] Only use local shuffle reader when REBALANCE_PARTITIONS_BY_NONE without CustomShuffleReaderExec

AmplabJenkins commented on pull request #33165: URL: https://github.com/apache/spark/pull/33165#issuecomment-872047046 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140477/ -- This

[GitHub] [spark] xuanyuanking commented on a change in pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

xuanyuanking commented on a change in pull request #33038: URL: https://github.com/apache/spark/pull/33038#discussion_r662094639 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala ## @@ -252,6 +238,12 @@

[GitHub] [spark] SparkQA commented on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

SparkQA commented on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872045569 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45002/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33165: [SPARK-35961][SQL] Only use local shuffle reader when REBALANCE_PARTITIONS_BY_NONE without CustomShuffleReaderExec

SparkQA removed a comment on pull request #33165: URL: https://github.com/apache/spark/pull/33165#issuecomment-871889009 **[Test build #140477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140477/testReport)** for PR 33165 at commit

[GitHub] [spark] SparkQA commented on pull request #33165: [SPARK-35961][SQL] Only use local shuffle reader when REBALANCE_PARTITIONS_BY_NONE without CustomShuffleReaderExec

SparkQA commented on pull request #33165: URL: https://github.com/apache/spark/pull/33165#issuecomment-872042014 **[Test build #140477 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140477/testReport)** for PR 33165 at commit

[GitHub] [spark] gengliangwang commented on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

gengliangwang commented on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872041956 > Just to highlight, you are going to introduce an alternative name TIMESTMAP_NTZ for the SQL standard TIMESTAMP WITOUT TIME ZONE in SQL, correct? Like `CAST(... AS

[GitHub] [spark] MaxGekk edited a comment on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

MaxGekk edited a comment on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872037451 > We will introduce a new SQL configuration spark.sql.timestampType for the default timestamp type. The configuration values can be "TIMESTMAP_NTZ" or "TIMESTMAP_LTZ"

[GitHub] [spark] SparkQA removed a comment on pull request #33169: [SPARK-35966][SQL] Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings

SparkQA removed a comment on pull request #33169: URL: https://github.com/apache/spark/pull/33169#issuecomment-872005609 **[Test build #140495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140495/testReport)** for PR 33169 at commit

[GitHub] [spark] SparkQA commented on pull request #33169: [SPARK-35966][SQL] Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings

SparkQA commented on pull request #33169: URL: https://github.com/apache/spark/pull/33169#issuecomment-872040569 **[Test build #140495 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140495/testReport)** for PR 33169 at commit

[GitHub] [spark] MaxGekk edited a comment on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

MaxGekk edited a comment on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872037451 > We will introduce a new SQL configuration spark.sql.timestampType for the default timestamp type. The configuration values can be "TIMESTMAP_NTZ" or "TIMESTMAP_LTZ"

[GitHub] [spark] MaxGekk commented on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

MaxGekk commented on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872037451 > We will introduce a new SQL configuration spark.sql.timestampType for the default timestamp type. The configuration values can be "TIMESTMAP_NTZ" or "TIMESTMAP_LTZ" for

[GitHub] [spark] wangyum commented on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

wangyum commented on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-872036939 Thank you all. Close it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] wangyum closed pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

wangyum closed pull request #33105: URL: https://github.com/apache/spark/pull/33105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] wangyum commented on pull request #33165: [SPARK-35961][SQL] Only use local shuffle reader when REBALANCE_PARTITIONS_BY_NONE without CustomShuffleReaderExec

wangyum commented on pull request #33165: URL: https://github.com/apache/spark/pull/33165#issuecomment-872034698 Late LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] beliefer commented on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

beliefer commented on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-872033879 cc @allisonwang-db Please check the naming reasonable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872028154 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45000/ -- This is an automated message from the

[GitHub] [spark] sunpe commented on pull request #33154: [SPARK-35949][CORE]Fixes bug for sparkContext stopped on client mode

sunpe commented on pull request #33154: URL: https://github.com/apache/spark/pull/33154#issuecomment-872026868 hello @kotlovs , @mridulm . Issue [SPARK-34674](https://issues.apache.org/jira/browse/SPARK-34674) said the spark context could not close on k8s. But this

[GitHub] [spark] SparkQA commented on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

SparkQA commented on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872025927 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44998/ -- This is an automated message from the

[GitHub] [spark] wangyum opened a new pull request #33170: [SPARK-35967][SQL] Update nullability based on column statistics

wangyum opened a new pull request #33170: URL: https://github.com/apache/spark/pull/33170 ### What changes were proposed in this pull request? 1. Update column nullability based on column statistics. 2. Change the nullable of `First`, `Last`, `Max` and `Min` to child.nullable.

[GitHub] [spark] SparkQA removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

SparkQA removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-871936763 **[Test build #140482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140482/testReport)** for PR 33164 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

SparkQA removed a comment on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-871936855 **[Test build #140483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140483/testReport)** for PR 33157 at commit

[GitHub] [spark] SparkQA commented on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

SparkQA commented on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872020991 **[Test build #140483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140483/testReport)** for PR 33157 at commit

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872021121 **[Test build #140482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140482/testReport)** for PR 33164 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

SparkQA removed a comment on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872005730 **[Test build #140496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140496/testReport)** for PR 33168 at commit

[GitHub] [spark] cloud-fan edited a comment on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

cloud-fan edited a comment on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872017814 Snowflake uses this short name: https://docs.snowflake.com/en/sql-reference/data-types-datetime.html#timestamp-ltz-timestamp-ntz-timestamp-tz Flink also uses

[GitHub] [spark] cloud-fan commented on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

cloud-fan commented on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872017814 Snowflake uses this short name: https://docs.snowflake.com/en/sql-reference/data-types-datetime.html#timestamp-ltz-timestamp-ntz-timestamp-tz Flink also uses part of

[GitHub] [spark] SparkQA commented on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

SparkQA commented on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872016425 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45002/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

SparkQA commented on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872016142 **[Test build #140496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140496/testReport)** for PR 33168 at commit

[GitHub] [spark] viirya commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

viirya commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-872012494 I'll take another look tomorrow. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

SparkQA commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-872006587 **[Test build #140501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140501/testReport)** for PR 29326 at commit

[GitHub] [spark] MaxGekk commented on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

MaxGekk commented on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872006264 > I don't get your point here. @cloud-fan My point is that you going to introduce some confusions for users - two different names for the same. Or you think the mapping

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871971040 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140488/

[GitHub] [spark] SparkQA commented on pull request #32831: [SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

SparkQA commented on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-872005990 **[Test build #140500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140500/testReport)** for PR 32831 at commit

[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

SparkQA commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-872005977 **[Test build #140499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140499/testReport)** for PR 32832 at commit

[GitHub] [spark] SparkQA commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

SparkQA commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-872005862 **[Test build #140498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140498/testReport)** for PR 32933 at commit

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872005821 **[Test build #140497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140497/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

SparkQA commented on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872005730 **[Test build #140496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140496/testReport)** for PR 33168 at commit

[GitHub] [spark] SparkQA commented on pull request #33169: [SPARK-35966][SQL] Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings

SparkQA commented on pull request #33169: URL: https://github.com/apache/spark/pull/33169#issuecomment-872005609 **[Test build #140495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140495/testReport)** for PR 33169 at commit

[GitHub] [spark] yaooqinn opened a new pull request #33169: [SPARK-35966][SQL] Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings

yaooqinn opened a new pull request #33169: URL: https://github.com/apache/spark/pull/33169 ### What changes were proposed in this pull request? Port HIVE-17952: Fix license headers to avoid dangling javadoc warnings ### Why are the changes needed?

[GitHub] [spark] SparkQA removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

SparkQA removed a comment on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-871937014 **[Test build #140484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140484/testReport)** for PR 32832 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

AmplabJenkins removed a comment on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-872004123 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140484/

[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

AmplabJenkins commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-872004123 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140484/ -- This

[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

SparkQA commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-872003906 **[Test build #140484 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140484/testReport)** for PR 32832 at commit

[GitHub] [spark] cloud-fan commented on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

cloud-fan commented on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872003787 @MaxGekk The SQL standard does not specify how a DB should name its internal classes, I don't get your point here. `CAST(... AS TIMESTAMP WITHOUT TIME ZONE)` is still

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

AmplabJenkins removed a comment on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871971844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for older Hadoop 3.x versions

AmplabJenkins removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872003264 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44999/

[GitHub] [spark] viirya commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

viirya commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-872003647 @HeartSaVioR thanks for ping. I will look into this in next days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

AmplabJenkins removed a comment on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-872003261 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44996/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

AmplabJenkins removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872003265 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44994/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-872003267 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140481/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

AmplabJenkins removed a comment on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872003263 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140490/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

AmplabJenkins removed a comment on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872003262 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44995/

[GitHub] [spark] AmplabJenkins commented on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

AmplabJenkins commented on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-872003263 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140490/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

AmplabJenkins commented on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872003262 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44995/ --

[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

AmplabJenkins commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-872003260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45001/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

AmplabJenkins commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872003265 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44994/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for older Hadoop 3.x versions

AmplabJenkins commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872003264 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44999/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

AmplabJenkins commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-872003261 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44996/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-872003267 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140481/ -- This

[GitHub] [spark] xuanyuanking commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

xuanyuanking commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-872002977 @HeartSaVioR Addressed done, the redundant code should be introduced by my old commits. Thanks for reminding me! -- This is an automated message from the Apache Git

[GitHub] [spark] viirya commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

viirya commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872002816 > This WIP PR has a working prototype for SPARK-35779 per design doc. This is not WIP PR anymore. @aokolnychyi Could you update the description? Thanks. -- This is an

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for older Hadoop 3.x versions

SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872002748 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44999/ --

[GitHub] [spark] SparkQA commented on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

SparkQA commented on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872002472 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44995/ -- This is an automated message from the

[GitHub] [spark] MaxGekk commented on pull request #33167: [SPARK-35963][SQL] Rename TimestampWithoutTZType to TimestampNTZType

MaxGekk commented on pull request #33167: URL: https://github.com/apache/spark/pull/33167#issuecomment-872001851 I am -0 for that. I would prefer to either: 1. follow to the SQL standard. What we do now. 2. or follow popular DBMSs -- This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872001706 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45000/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

SparkQA commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-872001701 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45001/ --

[GitHub] [spark] SparkQA commented on pull request #33166: [SPARK-35962][DOCS] Deprecate old Java 8 versions prior to 8u201

SparkQA commented on pull request #33166: URL: https://github.com/apache/spark/pull/33166#issuecomment-871999844 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44998/ -- This is an automated message from the Apache

[GitHub] [spark] linhongliu-db commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view

linhongliu-db commented on a change in pull request #32832: URL: https://github.com/apache/spark/pull/32832#discussion_r662049294 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -910,4 +910,45 @@ abstract class SQLViewSuite

<    4   5   6   7   8   9   10   >