[GitHub] [spark] AmplabJenkins removed a comment on pull request #30467: [SPARK-32002][SQL]Support ExtractValue from nested ArrayStruct
AmplabJenkins removed a comment on pull request #30467: URL: https://github.com/apache/spark/pull/30467#issuecomment-732841775 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30467: [SPARK-32002][SQL]Support ExtractValue from nested ArrayStruct
AmplabJenkins commented on pull request #30467: URL: https://github.com/apache/spark/pull/30467#issuecomment-732841775 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
beliefer commented on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732840939 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30412: [SPARK-33480][SQL] Support char/varchar type
AmplabJenkins removed a comment on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-732838910 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30412: [SPARK-33480][SQL] Support char/varchar type
SparkQA removed a comment on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-732756465 **[Test build #131637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131637/testReport)** for PR 30412 at commit [`f46e32f`](https://github.com/apache/spark/commit/f46e32fb1649023eed0ddab4cb23ca4a97b14a0f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type
AmplabJenkins commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-732838910 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type
SparkQA commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-732838546 **[Test build #131637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131637/testReport)** for PR 30412 at commit [`f46e32f`](https://github.com/apache/spark/commit/f46e32fb1649023eed0ddab4cb23ca4a97b14a0f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
AmplabJenkins removed a comment on pull request #30481: URL: https://github.com/apache/spark/pull/30481#issuecomment-732835381 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
AmplabJenkins commented on pull request #30481: URL: https://github.com/apache/spark/pull/30481#issuecomment-732835381 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
SparkQA removed a comment on pull request #30481: URL: https://github.com/apache/spark/pull/30481#issuecomment-732790797 **[Test build #131642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131642/testReport)** for PR 30481 at commit [`ba382a3`](https://github.com/apache/spark/commit/ba382a350751291042b177abce67a50e5aa20540). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
SparkQA commented on pull request #30481: URL: https://github.com/apache/spark/pull/30481#issuecomment-732834680 **[Test build #131642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131642/testReport)** for PR 30481 at commit [`ba382a3`](https://github.com/apache/spark/commit/ba382a350751291042b177abce67a50e5aa20540). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #30457: [SPARK-33514][SQL] Migrate TRUNCATE TABLE command to use UnresolvedTable to resolve the identifier
cloud-fan closed pull request #30457: URL: https://github.com/apache/spark/pull/30457 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #30457: [SPARK-33514][SQL] Migrate TRUNCATE TABLE command to use UnresolvedTable to resolve the identifier
cloud-fan commented on pull request #30457: URL: https://github.com/apache/spark/pull/30457#issuecomment-732833005 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast
gengliangwang commented on a change in pull request #30440:
URL: https://github.com/apache/spark/pull/30440#discussion_r529459190
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
##
@@ -98,6 +98,19 @@ object Cast {
case _ => false
}
+ def typeCheckFailureMessage(from: DataType, to: DataType): String = (from,
to) match {
+case (_: NumericType, TimestampType) =>
+ // scalastyle:off line.size.limit
+ s"""
+ | cannot cast ${from.catalogString} to ${to.catalogString},
+ | you can enable the casting by setting
${SQLConf.LEGACY_ALLOW_CAST_NUMERIC_TO_TIMESTAMP.key}
Review comment:
+1, we can have another PR for it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value
SparkQA commented on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-732827738 **[Test build #131650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131650/testReport)** for PR 30421 at commit [`05f1962`](https://github.com/apache/spark/commit/05f196254fb2054aa3926e830db8889b60f4fbac). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30427: [SPARK-33224][SS] Add watermark gap information into SS UI page
SparkQA commented on pull request #30427: URL: https://github.com/apache/spark/pull/30427#issuecomment-732827618 **[Test build #131649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131649/testReport)** for PR 30427 at commit [`d19fd10`](https://github.com/apache/spark/commit/d19fd10dab7c4fc28d1c4a893a2db74405d4ff9f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
SparkQA commented on pull request #30482: URL: https://github.com/apache/spark/pull/30482#issuecomment-732827355 **[Test build #131647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131647/testReport)** for PR 30482 at commit [`26b83a1`](https://github.com/apache/spark/commit/26b83a101f39c8d0878c96b6ffd228e040169c8b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30470: [SPARK-33495][BUILD] Remove commons-logging.jar's dependency
SparkQA commented on pull request #30470: URL: https://github.com/apache/spark/pull/30470#issuecomment-732827379 **[Test build #131648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131648/testReport)** for PR 30470 at commit [`bc3cb8b`](https://github.com/apache/spark/commit/bc3cb8b419bb985cdf98aaf172b20c900d40e806). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30382: [SPARK-33457][PYTHON] Adjust mypy configuration
AmplabJenkins removed a comment on pull request #30382: URL: https://github.com/apache/spark/pull/30382#issuecomment-732823899 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
AmplabJenkins removed a comment on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732823896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30427: [SPARK-33224][SS] Add watermark gap information into SS UI page
AmplabJenkins removed a comment on pull request #30427: URL: https://github.com/apache/spark/pull/30427#issuecomment-732823908 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30432: [SPARK-33494][SQL][AQE] Do not use local shuffle reader for repartition
AmplabJenkins removed a comment on pull request #30432: URL: https://github.com/apache/spark/pull/30432#issuecomment-732823898 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
AmplabJenkins removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732823924 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30427: [SPARK-33224][SS] Add watermark gap information into SS UI page
AmplabJenkins commented on pull request #30427: URL: https://github.com/apache/spark/pull/30427#issuecomment-732823908 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
AmplabJenkins commented on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732823896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30382: [SPARK-33457][PYTHON] Adjust mypy configuration
AmplabJenkins commented on pull request #30382: URL: https://github.com/apache/spark/pull/30382#issuecomment-732823899 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30432: [SPARK-33494][SQL][AQE] Do not use local shuffle reader for repartition
AmplabJenkins commented on pull request #30432: URL: https://github.com/apache/spark/pull/30432#issuecomment-732823898 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
AmplabJenkins commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732823901 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30470: [SPARK-33495][BUILD] Remove commons-logging.jar's dependency
HyukjinKwon commented on pull request #30470: URL: https://github.com/apache/spark/pull/30470#issuecomment-732823644 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on pull request #30427: [SPARK-33224][SS] Add watermark gap information into SS UI page
gaborgsomogyi commented on pull request #30427: URL: https://github.com/apache/spark/pull/30427#issuecomment-732819837 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
SparkQA removed a comment on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732764182 **[Test build #131640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131640/testReport)** for PR 30465 at commit [`8280665`](https://github.com/apache/spark/commit/82806650d9498a51c78d824f451b031a6af2e055). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
SparkQA commented on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732818260 **[Test build #131640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131640/testReport)** for PR 30465 at commit [`8280665`](https://github.com/apache/spark/commit/82806650d9498a51c78d824f451b031a6af2e055). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30432: [SPARK-33494][SQL][AQE] Do not use local shuffle reader for repartition
SparkQA removed a comment on pull request #30432: URL: https://github.com/apache/spark/pull/30432#issuecomment-732756403 **[Test build #131636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131636/testReport)** for PR 30432 at commit [`1ed28f2`](https://github.com/apache/spark/commit/1ed28f2a6db980d7c9231318cc45f15718a4804c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30432: [SPARK-33494][SQL][AQE] Do not use local shuffle reader for repartition
SparkQA commented on pull request #30432: URL: https://github.com/apache/spark/pull/30432#issuecomment-732816771 **[Test build #131636 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131636/testReport)** for PR 30432 at commit [`1ed28f2`](https://github.com/apache/spark/commit/1ed28f2a6db980d7c9231318cc45f15718a4804c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30427: [SPARK-33224][SS] Add watermark gap information into SS UI page
SparkQA removed a comment on pull request #30427: URL: https://github.com/apache/spark/pull/30427#issuecomment-732725370 **[Test build #131630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131630/testReport)** for PR 30427 at commit [`d19fd10`](https://github.com/apache/spark/commit/d19fd10dab7c4fc28d1c4a893a2db74405d4ff9f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30427: [SPARK-33224][SS] Add watermark gap information into SS UI page
SparkQA commented on pull request #30427: URL: https://github.com/apache/spark/pull/30427#issuecomment-732815872 **[Test build #131630 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131630/testReport)** for PR 30427 at commit [`d19fd10`](https://github.com/apache/spark/commit/d19fd10dab7c4fc28d1c4a893a2db74405d4ff9f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value
AngersZh commented on a change in pull request #30421:
URL: https://github.com/apache/spark/pull/30421#discussion_r529403802
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -503,13 +503,32 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef] with
SQLConfHelper with Logg
}
}
+ def convertTypeConstructedLiteralToString(literal: Literal): String =
literal match {
Review comment:
Done
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##
@@ -2584,6 +2584,14 @@ abstract class SQLQuerySuiteBase extends QueryTest with
SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-33474: Support TypeConstructed partition spec value") {
+withTable("t") {
+ sql("CREATE TABLE t(name STRING) PARTITIONED BY (part DATE) STORED AS
ORC")
Review comment:
> btw, why did you use `stored as orc` explicitly for this test?
removed
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##
@@ -2584,6 +2584,14 @@ abstract class SQLQuerySuiteBase extends QueryTest with
SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-33474: Support TypeConstructed partition spec value") {
+withTable("t") {
+ sql("CREATE TABLE t(name STRING) PARTITIONED BY (part DATE) STORED AS
ORC")
Review comment:
> Please add more tests for the other types and add tests in
`DDLParserSuite`.
Both updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode
AmplabJenkins commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-732803500 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path
AmplabJenkins commented on pull request #29966: URL: https://github.com/apache/spark/pull/29966#issuecomment-732803609 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path
SparkQA removed a comment on pull request #29966: URL: https://github.com/apache/spark/pull/29966#issuecomment-732725669 **[Test build #131632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131632/testReport)** for PR 29966 at commit [`63e877b`](https://github.com/apache/spark/commit/63e877b64f7a61888ea59a2c632ef5e9373e087e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode
SparkQA removed a comment on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-732730489 **[Test build #131634 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131634/testReport)** for PR 29000 at commit [`45f8ea5`](https://github.com/apache/spark/commit/45f8ea5c47c84de94ed64ddf6fb7668ac6af8ff1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value
AngersZh commented on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-732802953 > > Yes, user can use TypeConstruct value as partition spec value such as > > `` > > Is the description above incomplete? Btw, could you put an example query that this PR intends to support in the PR description? Yea... seem I forgot to save it， updated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path
SparkQA commented on pull request #29966: URL: https://github.com/apache/spark/pull/29966#issuecomment-732802966 **[Test build #131632 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131632/testReport)** for PR 29966 at commit [`63e877b`](https://github.com/apache/spark/commit/63e877b64f7a61888ea59a2c632ef5e9373e087e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode
SparkQA commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-732802576 **[Test build #131634 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131634/testReport)** for PR 29000 at commit [`45f8ea5`](https://github.com/apache/spark/commit/45f8ea5c47c84de94ed64ddf6fb7668ac6af8ff1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value
SparkQA commented on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-732802365 **[Test build #131646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131646/testReport)** for PR 30421 at commit [`6adefa7`](https://github.com/apache/spark/commit/6adefa76eb31c82dc44e826014a16e656da5a3b1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid
AmplabJenkins removed a comment on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-732799105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30479: [WIP][SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases
SparkQA commented on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-732799386 **[Test build #131645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131645/testReport)** for PR 30479 at commit [`338d28b`](https://github.com/apache/spark/commit/338d28b03cc9d5882ec807c432449b81a579b7c8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid
AmplabJenkins commented on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-732799105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid
SparkQA removed a comment on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-732756404 **[Test build #131635 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131635/testReport)** for PR 30442 at commit [`0417977`](https://github.com/apache/spark/commit/0417977c5cffa824e0e684f94ef62a2179018f14). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid
SparkQA commented on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-732798784 **[Test build #131635 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131635/testReport)** for PR 30442 at commit [`0417977`](https://github.com/apache/spark/commit/0417977c5cffa824e0e684f94ef62a2179018f14). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30476: [SPARK-33523][SQL][TEST] Add predicate related benchmark to SubExprEliminationBenchmark
AmplabJenkins removed a comment on pull request #30476: URL: https://github.com/apache/spark/pull/30476#issuecomment-732798130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30450: [SPARK-33185][YARN][FOLLOW-ON] Leverage RM's RPC API instead of REST to fetch driver log links in yarn.Client
AmplabJenkins removed a comment on pull request #30450: URL: https://github.com/apache/spark/pull/30450#issuecomment-732797906 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30476: [SPARK-33523][SQL][TEST] Add predicate related benchmark to SubExprEliminationBenchmark
AmplabJenkins commented on pull request #30476: URL: https://github.com/apache/spark/pull/30476#issuecomment-732798130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
AmplabJenkins removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732797564 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30450: [SPARK-33185][YARN][FOLLOW-ON] Leverage RM's RPC API instead of REST to fetch driver log links in yarn.Client
AmplabJenkins commented on pull request #30450: URL: https://github.com/apache/spark/pull/30450#issuecomment-732797906 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
AmplabJenkins commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732797564 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
SparkQA removed a comment on pull request #30482: URL: https://github.com/apache/spark/pull/30482#issuecomment-732790773 **[Test build #131641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131641/testReport)** for PR 30482 at commit [`4ad95c5`](https://github.com/apache/spark/commit/4ad95c53ab89b990d495bd82c10a55854086225c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
AmplabJenkins removed a comment on pull request #30482: URL: https://github.com/apache/spark/pull/30482#issuecomment-732794192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
SparkQA commented on pull request #30482: URL: https://github.com/apache/spark/pull/30482#issuecomment-732794164 **[Test build #131641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131641/testReport)** for PR 30482 at commit [`4ad95c5`](https://github.com/apache/spark/commit/4ad95c53ab89b990d495bd82c10a55854086225c). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
AmplabJenkins commented on pull request #30482: URL: https://github.com/apache/spark/pull/30482#issuecomment-732794192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-732791895 **[Test build #131644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131644/testReport)** for PR 28647 at commit [`c45489a`](https://github.com/apache/spark/commit/c45489ad5b8ddd53d5e81fbba4cd08c0b4fd9850). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
SparkQA commented on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732790908 **[Test build #131643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131643/testReport)** for PR 30465 at commit [`985352e`](https://github.com/apache/spark/commit/985352ef5d8bc878c5ff07a1a24576a1ac77dfed). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30479: [WIP][SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases
SparkQA removed a comment on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-732725257 **[Test build #131628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131628/testReport)** for PR 30479 at commit [`cd81faf`](https://github.com/apache/spark/commit/cd81faf6ea86a34bb79514566bf08e6cb22cd7a7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30479: [WIP][SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases
AmplabJenkins removed a comment on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-732790562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
SparkQA commented on pull request #30481: URL: https://github.com/apache/spark/pull/30481#issuecomment-732790797 **[Test build #131642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131642/testReport)** for PR 30481 at commit [`ba382a3`](https://github.com/apache/spark/commit/ba382a350751291042b177abce67a50e5aa20540). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
SparkQA commented on pull request #30482: URL: https://github.com/apache/spark/pull/30482#issuecomment-732790773 **[Test build #131641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131641/testReport)** for PR 30482 at commit [`4ad95c5`](https://github.com/apache/spark/commit/4ad95c53ab89b990d495bd82c10a55854086225c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30479: [WIP][SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases
AmplabJenkins commented on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-732790562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30479: [WIP][SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases
SparkQA commented on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-732790339 **[Test build #131628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131628/testReport)** for PR 30479 at commit [`cd81faf`](https://github.com/apache/spark/commit/cd81faf6ea86a34bb79514566bf08e6cb22cd7a7). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Decode(fakeChild: Expression, params: Seq[Expression]) extends RuntimeReplaceable ` * `case class StringDecode(bin: Expression, charset: Expression)` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30476: [SPARK-33523][SQL][TEST] Add predicate related benchmark to SubExprEliminationBenchmark
AmplabJenkins removed a comment on pull request #30476: URL: https://github.com/apache/spark/pull/30476#issuecomment-732789718 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30476: [SPARK-33523][SQL][TEST] Add predicate related benchmark to SubExprEliminationBenchmark
AmplabJenkins commented on pull request #30476: URL: https://github.com/apache/spark/pull/30476#issuecomment-732790036 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30476: [SPARK-33523][SQL][TEST] Add predicate related benchmark to SubExprEliminationBenchmark
AmplabJenkins commented on pull request #30476: URL: https://github.com/apache/spark/pull/30476#issuecomment-732789718 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30475: [SPARK-33522][SQL] Improve exception messages while handling UnresolvedTableOrView
AmplabJenkins removed a comment on pull request #30475: URL: https://github.com/apache/spark/pull/30475#issuecomment-732789363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
AmplabJenkins removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732789364 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30475: [SPARK-33522][SQL] Improve exception messages while handling UnresolvedTableOrView
AmplabJenkins commented on pull request #30475: URL: https://github.com/apache/spark/pull/30475#issuecomment-732789363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
AmplabJenkins commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732789364 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #30408: [SPARK-33477][SQL] Hive Metastore should support filter by date type
maropu commented on pull request #30408: URL: https://github.com/apache/spark/pull/30408#issuecomment-732788459 @wangyum How about asking it in the spark-dev thread so that Shane could notice it quickly?http://apache-spark-developers-list.1001551.n3.nabble.com/jenkins-downtime-tomorrow-evening-weekend-tt30405.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
MaxGekk commented on a change in pull request #30482:
URL: https://github.com/apache/spark/pull/30482#discussion_r529377394
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTablePartitionV2SQLSuite.scala
##
@@ -243,4 +243,22 @@ class AlterTablePartitionV2SQLSuite extends
DatasourceV2SQLBase {
assert(!partTable.partitionExists(expectedPartition))
}
}
+
+ test("SPARK-33529: handle __HIVE_DEFAULT_PARTITION__") {
+val t = "testpart.ns1.ns2.tbl"
+withTable(t) {
+ sql(s"CREATE TABLE $t (part0 string) USING foo PARTITIONED BY (part0)")
+ val partTable = catalog("testpart")
+.asTableCatalog
+.loadTable(Identifier.of(Array("ns1", "ns2"), "tbl"))
+.asPartitionable
+ val expectedPartition = InternalRow.fromSeq(Seq[Any](null))
Review comment:
`'__HIVE_DEFAULT_PARTITION__'` should be handled as `null`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
MaxGekk commented on a change in pull request #30482:
URL: https://github.com/apache/spark/pull/30482#discussion_r529377394
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTablePartitionV2SQLSuite.scala
##
@@ -243,4 +243,22 @@ class AlterTablePartitionV2SQLSuite extends
DatasourceV2SQLBase {
assert(!partTable.partitionExists(expectedPartition))
}
}
+
+ test("SPARK-33529: handle __HIVE_DEFAULT_PARTITION__") {
+val t = "testpart.ns1.ns2.tbl"
+withTable(t) {
+ sql(s"CREATE TABLE $t (part0 string) USING foo PARTITIONED BY (part0)")
+ val partTable = catalog("testpart")
+.asTableCatalog
+.loadTable(Identifier.of(Array("ns1", "ns2"), "tbl"))
+.asPartitionable
+ val expectedPartition = InternalRow.fromSeq(Seq[Any](null))
Review comment:
'__HIVE_DEFAULT_PARTITION__' should be handled as `null`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #30474: [SPARK-33521][SQL] Universal type conversion in resolving V2 partition specs
MaxGekk commented on a change in pull request #30474:
URL: https://github.com/apache/spark/pull/30474#discussion_r529375116
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolvePartitionSpec.scala
##
@@ -65,31 +65,8 @@ object ResolvePartitionSpec extends Rule[LogicalPlan] {
conf.resolver)
val partValues = partSchema.map { part =>
- val partValue = normalizedSpec.get(part.name).orNull
- if (partValue == null) {
-null
- } else {
-// TODO: Support other datatypes, such as DateType
-part.dataType match {
- case _: ByteType =>
-partValue.toByte
- case _: ShortType =>
-partValue.toShort
- case _: IntegerType =>
-partValue.toInt
- case _: LongType =>
-partValue.toLong
- case _: FloatType =>
-partValue.toFloat
- case _: DoubleType =>
-partValue.toDouble
- case _: StringType =>
-partValue
- case _ =>
-throw new AnalysisException(
- s"Type ${part.dataType.typeName} is not supported for
partition.")
-}
- }
+ val raw = normalizedSpec.get(part.name).orNull
+ Cast(Literal.create(raw, StringType), part.dataType,
Some(conf.sessionLocalTimeZone)).eval()
Review comment:
I reused the code from DSv1 https://github.com/apache/spark/pull/30482
and fixed an issue. @HyukjinKwon Please, review it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #30474: [SPARK-33521][SQL] Universal type conversion in resolving V2 partition specs
MaxGekk commented on a change in pull request #30474:
URL: https://github.com/apache/spark/pull/30474#discussion_r529375116
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolvePartitionSpec.scala
##
@@ -65,31 +65,8 @@ object ResolvePartitionSpec extends Rule[LogicalPlan] {
conf.resolver)
val partValues = partSchema.map { part =>
- val partValue = normalizedSpec.get(part.name).orNull
- if (partValue == null) {
-null
- } else {
-// TODO: Support other datatypes, such as DateType
-part.dataType match {
- case _: ByteType =>
-partValue.toByte
- case _: ShortType =>
-partValue.toShort
- case _: IntegerType =>
-partValue.toInt
- case _: LongType =>
-partValue.toLong
- case _: FloatType =>
-partValue.toFloat
- case _: DoubleType =>
-partValue.toDouble
- case _: StringType =>
-partValue
- case _ =>
-throw new AnalysisException(
- s"Type ${part.dataType.typeName} is not supported for
partition.")
-}
- }
+ val raw = normalizedSpec.get(part.name).orNull
+ Cast(Literal.create(raw, StringType), part.dataType,
Some(conf.sessionLocalTimeZone)).eval()
Review comment:
I reused the code from DSv1 https://github.com/apache/spark/pull/30482
and fixed an issue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
MaxGekk commented on a change in pull request #30482:
URL: https://github.com/apache/spark/pull/30482#discussion_r529373882
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/util/PartitioningUtils.scala
##
@@ -18,9 +18,15 @@
package org.apache.spark.sql.util
import org.apache.spark.sql.AnalysisException
+import org.apache.spark.sql.catalyst.InternalRow
import org.apache.spark.sql.catalyst.analysis.Resolver
+import org.apache.spark.sql.catalyst.catalog.CatalogTypes.TablePartitionSpec
+import org.apache.spark.sql.catalyst.catalog.ExternalCatalogUtils
+import org.apache.spark.sql.catalyst.expressions.{Cast, Literal}
+import org.apache.spark.sql.catalyst.util.{CaseInsensitiveMap, DateTimeUtils}
+import org.apache.spark.sql.types.StructType
-object PartitioningUtils {
+private[sql] object PartitioningUtils {
Review comment:
Addressed @cloud-fan 's comment
https://github.com/apache/spark/pull/30454#discussion_r528549153
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk opened a new pull request #30482: [SPARK-33529][SQL] Handle '__HIVE_DEFAULT_PARTITION__' while resolving V2 partition specs
MaxGekk opened a new pull request #30482: URL: https://github.com/apache/spark/pull/30482 ### What changes were proposed in this pull request? 1. Extract the code for partition values casting from DSv1 to the common place `sql.util.PartitioningUtils` - the method `castPartitionValues()`. 2. Re-use `castPartitionValues()` from DSv2 resolver of partition specs - `ResolvePartitionSpec`. ### Why are the changes needed? To have the same behavior as DSv1 which interprets `__HIVE_DEFAULT_PARTITION__` as `NULL`: ```sql spark-sql> CREATE TABLE tbl11 (id int, part0 string) USING parquet PARTITIONED BY (part0); spark-sql> ALTER TABLE tbl11 ADD PARTITION (part0 = '__HIVE_DEFAULT_PARTITION__'); spark-sql> INSERT INTO tbl11 PARTITION (part0='__HIVE_DEFAULT_PARTITION__') SELECT 1; spark-sql> SELECT * FROM tbl11; 1NULL ``` ### Does this PR introduce _any_ user-facing change? Yes ### How was this patch tested? Add new test to `AlterTablePartitionV2SQLSuite`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
SparkQA removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732757294 **[Test build #131639 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131639/testReport)** for PR 28026 at commit [`a471f33`](https://github.com/apache/spark/commit/a471f332cf956c89e5bbfbc18635c6aa8ed3f7f1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax
SparkQA commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-732781939 **[Test build #131639 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131639/testReport)** for PR 28026 at commit [`a471f33`](https://github.com/apache/spark/commit/a471f332cf956c89e5bbfbc18635c6aa8ed3f7f1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30408: [SPARK-33477][SQL] Hive Metastore should support filter by date type
wangyum commented on pull request #30408: URL: https://github.com/apache/spark/pull/30408#issuecomment-732769319 @shaneknapp Did you set :`export LANG=en_US.UTF-8`? ``` org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /home/jenkins/workspace/SparkPullRequestBuilder@3/sql/hive/target/tmp/hive_execution_test_group/warehouse-1355e680-268f-4224-b549-eaddcadcf136/DaTaBaSe_I.db/tab_ı); ``` This issue should be fixed if we set `export LANG=en_US.UTF-8`, more details:https://issues.apache.org/jira/browse/SPARK-27177 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ulysses-you commented on a change in pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
ulysses-you commented on a change in pull request #30481:
URL: https://github.com/apache/spark/pull/30481#discussion_r529336316
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerWithSparkContextSuite.scala
##
@@ -79,6 +84,45 @@ trait ThriftServerWithSparkContextSuite extends
SharedThriftServer {
"java.lang.NumberFormatException: invalid input syntax for type
numeric: 1.2"))
}
}
+
+ test("SPARK-33526: Add config to control if cancel invoke interrupt task on
thriftserver") {
+withJdbcStatement { statement =>
+ val forceCancel = new AtomicBoolean(false)
+ val listener = new SparkListener {
+override def onTaskEnd(taskEnd: SparkListenerTaskEnd): Unit = {
+ taskEnd.reason match {
+case _: TaskKilled =>
+ if (forceCancel.get()) {
+assert(System.currentTimeMillis() -
taskEnd.taskInfo.launchTime < 1000)
+ } else {
+assert(System.currentTimeMillis() -
taskEnd.taskInfo.launchTime >= 2900)
Review comment:
avoid the sleep accuracy so check 2.9s instead of 3s.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
beliefer commented on a change in pull request #30465:
URL: https://github.com/apache/spark/pull/30465#discussion_r529334200
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1404,11 +1404,24 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with SQLConfHelper with Logg
case SqlBaseParser.LIKE =>
Option(ctx.quantifier).map(_.getType) match {
case Some(SqlBaseParser.ANY) | Some(SqlBaseParser.SOME) =>
-getLikeQuantifierExprs(ctx.expression).reduceLeft(Or)
+validate(!ctx.expression.isEmpty, "Expected something between '('
and ')'.", ctx)
Review comment:
Let's remove `getLikeQuantifierExprs`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables
SparkQA removed a comment on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-732727595 **[Test build #131633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131633/testReport)** for PR 30403 at commit [`a5923ab`](https://github.com/apache/spark/commit/a5923ab0e5c525a998f3e0f630fb3de0bf460071). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables
AmplabJenkins removed a comment on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-732766504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables
AmplabJenkins commented on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-732766504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ulysses-you opened a new pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver
ulysses-you opened a new pull request #30481: URL: https://github.com/apache/spark/pull/30481 ### What changes were proposed in this pull request? This PR add a new config `spark.sql.thriftServer.forceCancel` to give user a way to interrupt task when cancel statement. ### Why are the changes needed? After [#29933](https://github.com/apache/spark/pull/29933), we support cancel query if timeout, but the default behavior of `SparkContext.cancelJobGroups` won't interrupt task and just let task finish by itself. In some case it's dangerous, e.g., data skew or exists a heavily shuffle. A task will hold in a long time after do cancel and the resource will not release. ### Does this PR introduce _any_ user-facing change? Yes, a new config. ### How was this patch tested? Add test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables
SparkQA commented on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-732766111 **[Test build #131633 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131633/testReport)** for PR 30403 at commit [`a5923ab`](https://github.com/apache/spark/commit/a5923ab0e5c525a998f3e0f630fb3de0bf460071). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode
AmplabJenkins removed a comment on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-732765233 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode
AmplabJenkins commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-732765233 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
SparkQA commented on pull request #30465: URL: https://github.com/apache/spark/pull/30465#issuecomment-732764182 **[Test build #131640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131640/testReport)** for PR 30465 at commit [`8280665`](https://github.com/apache/spark/commit/82806650d9498a51c78d824f451b031a6af2e055). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments
AmplabJenkins removed a comment on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-732763225 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments
AmplabJenkins commented on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-732763225 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30468: [SPARK-33518][ML][WIP] Improve performance of ML ALS recommendForAll by GEMV
SparkQA removed a comment on pull request #30468: URL: https://github.com/apache/spark/pull/30468#issuecomment-732725287 **[Test build #131629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131629/testReport)** for PR 30468 at commit [`8ca7d56`](https://github.com/apache/spark/commit/8ca7d562c20812062e11e8f6961034157cc08ea8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.
beliefer commented on a change in pull request #30465:
URL: https://github.com/apache/spark/pull/30465#discussion_r529323100
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -216,16 +216,16 @@ object SQLConf {
"for using switch statements in InSet must be non-negative and less
than or equal to 600")
.createWithDefault(400)
- val OPTIMIZER_LIKE_ALL_CONVERSION_THRESHOLD =
-buildConf("spark.sql.optimizer.likeAllConversionThreshold")
+ val OPTIMIZER_MULTI_LIKE_CONVERSION_THRESHOLD =
+buildConf("spark.sql.optimizer.multiLikeConversionThreshold")
.internal()
- .doc("Configure the maximum size of the pattern sequence in like all.
Spark will convert " +
-"the logical combination of like to avoid StackOverflowError. 200 is
an empirical value " +
-"that will not cause StackOverflowError.")
+ .doc("Configure the maximum size of the pattern sequence in like
[all|any|some]. Spark " +
+"will convert the logical combination of like to avoid
StackOverflowError. 200 is an " +
Review comment:
OK
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -216,16 +216,16 @@ object SQLConf {
"for using switch statements in InSet must be non-negative and less
than or equal to 600")
.createWithDefault(400)
- val OPTIMIZER_LIKE_ALL_CONVERSION_THRESHOLD =
-buildConf("spark.sql.optimizer.likeAllConversionThreshold")
+ val OPTIMIZER_MULTI_LIKE_CONVERSION_THRESHOLD =
+buildConf("spark.sql.optimizer.multiLikeConversionThreshold")
.internal()
- .doc("Configure the maximum size of the pattern sequence in like all.
Spark will convert " +
-"the logical combination of like to avoid StackOverflowError. 200 is
an empirical value " +
-"that will not cause StackOverflowError.")
+ .doc("Configure the maximum size of the pattern sequence in like
[all|any|some]. Spark " +
+"will convert the logical combination of like to avoid
StackOverflowError. 200 is an " +
+"empirical value that will not cause StackOverflowError.")
.version("3.1.0")
.intConf
.checkValue(threshold => threshold >= 0, "The maximum size of pattern
sequence " +
-"in like all must be non-negative")
+"in like [all|any|some] must be non-negative")
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30468: [SPARK-33518][ML][WIP] Improve performance of ML ALS recommendForAll by GEMV
AmplabJenkins removed a comment on pull request #30468: URL: https://github.com/apache/spark/pull/30468#issuecomment-732761870 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
