[GitHub] [spark] fuwhu commented on a change in pull request #26805: [SPARK-15616][SQL] Add optimizer rule PruneHiveTablePartitions
fuwhu commented on a change in pull request #26805: [SPARK-15616][SQL] Add
optimizer rule PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/26805#discussion_r376695382
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/PruneHiveTablePartitions.scala
##
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.execution
+
+import org.apache.hadoop.hive.common.StatsSetupConst
+
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.catalyst.analysis.CastSupport
+import org.apache.spark.sql.catalyst.catalog.{CatalogStatistics, CatalogTable,
CatalogTablePartition, ExternalCatalogUtils, HiveTableRelation}
+import org.apache.spark.sql.catalyst.expressions.{And, AttributeSet,
Expression, ExpressionSet, SubqueryExpression}
+import org.apache.spark.sql.catalyst.planning.PhysicalOperation
+import org.apache.spark.sql.catalyst.plans.logical.{Filter, LogicalPlan,
Project}
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.datasources.DataSourceStrategy
+import org.apache.spark.sql.internal.SQLConf
+
+/**
+ * TODO: merge this with PruneFileSourcePartitions after we completely make
hive as a data source.
Review comment:
sure, so you mean just add class description in
PruneHiveTablePartitions.scala and PruneFileSourcePartitions.scala file ? Or
need to add comment in some doc ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583713491 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583713493 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118059/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583713493 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118059/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
Eric5553 commented on a change in pull request #26977: [SPARK-30326][SQL] Raise
exception if analyzer exceed max iterations
URL: https://github.com/apache/spark/pull/26977#discussion_r376695166
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala
##
@@ -156,7 +163,7 @@ abstract class RuleExecutor[TreeType <: TreeNode[_]]
extends Logging {
// Only log if this is a rule that is supposed to run more than once.
if (iteration != 2) {
val message = s"Max iterations (${iteration - 1}) reached for
batch ${batch.name}"
-if (Utils.isTesting) {
+if (Utils.isTesting || batch.strategy.errorOnExceed) {
throw new TreeNodeException(curPlan, message, null)
Review comment:
I see, thanks for the clarification. :-)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583713491 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
SparkQA removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705639 **[Test build #118059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118059/testReport)** for PR 27492 at commit [`274af0e`](https://github.com/apache/spark/commit/274af0e6562cd9a9622369402b67810527774f8c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
SparkQA commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583713414 **[Test build #118059 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118059/testReport)** for PR 27492 at commit [`274af0e`](https://github.com/apache/spark/commit/274af0e6562cd9a9622369402b67810527774f8c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27368: [SPARK-30651][SQL] Add detailed information for Aggregate operators in EXPLAIN FORMATTED
Eric5553 commented on a change in pull request #27368: [SPARK-30651][SQL] Add
detailed information for Aggregate operators in EXPLAIN FORMATTED
URL: https://github.com/apache/spark/pull/27368#discussion_r376695100
##
File path: sql/core/src/test/resources/sql-tests/results/explain.sql.out
##
@@ -786,6 +870,144 @@ Output: []
(4) Project
+-- !query
+EXPLAIN FORMATTED
+ SELECT
+COUNT(val) + SUM(key) as TOTAL,
+COUNT(key) FILTER (WHERE val > 1)
+ FROM explain_temp1
+-- !query schema
+struct
+-- !query output
+== Physical Plan ==
+* HashAggregate (5)
++- Exchange (4)
+ +- HashAggregate (3)
+ +- * ColumnarToRow (2)
+ +- Scan parquet default.explain_temp1 (1)
+
+
+(1) Scan parquet default.explain_temp1
+Output: [key#x, val#x]
+Batched: true
+Location [not included in comparison]/{warehouse_dir}/explain_temp1]
+ReadSchema: struct
+
+(2) ColumnarToRow [codegen id : 1]
+Input: [key#x, val#x]
+
+(3) HashAggregate
+Input: [key#x, val#x]
+Keys: []
+Functions: [partial_count(val#x), partial_sum(cast(key#x as bigint)),
partial_count(key#x) FILTER (WHERE (val#x > 1))]
+Aggregate Attributes: [count#xL, sum#xL, count#xL]
+Results: [count#xL, sum#xL, count#xL]
+
+(4) Exchange
+Input: [count#xL, sum#xL, count#xL]
+
+(5) HashAggregate [codegen id : 2]
+Input: [count#xL, sum#xL, count#xL]
+Keys: []
+Functions: [count(val#x), sum(cast(key#x as bigint)), count(key#x)]
+Aggregate Attributes: [count(val#x)#xL, sum(cast(key#x as bigint))#xL,
count(key#x)#xL]
+Results: [(count(val#x)#xL + sum(cast(key#x as bigint))#xL) AS TOTAL#xL,
count(key#x)#xL AS count(key) FILTER (WHERE (val > 1))#xL]
+
+
+-- !query
+EXPLAIN FORMATTED
+ SELECT key, sort_array(collect_set(val))[0]
+ FROM explain_temp4
+ GROUP BY key
+-- !query schema
+struct
+-- !query output
+== Physical Plan ==
+ObjectHashAggregate (5)
++- Exchange (4)
+ +- ObjectHashAggregate (3)
+ +- * ColumnarToRow (2)
+ +- Scan parquet default.explain_temp4 (1)
+
+
+(1) Scan parquet default.explain_temp4
+Output: [key#x, val#x]
+Batched: true
+Location [not included in comparison]/{warehouse_dir}/explain_temp4]
+ReadSchema: struct
+
+(2) ColumnarToRow [codegen id : 1]
+Input: [key#x, val#x]
+
+(3) ObjectHashAggregate
+Input: [key#x, val#x]
+Keys: [key#x]
+Functions: [partial_collect_set(val#x, 0, 0)]
+Aggregate Attributes: [buf#x]
+Results: [key#x, buf#x]
+
+(4) Exchange
+Input: [key#x, buf#x]
+
+(5) ObjectHashAggregate
+Input: [key#x, buf#x]
+Keys: [key#x]
+Functions: [collect_set(val#x, 0, 0)]
+Aggregate Attributes: [collect_set(val#x, 0, 0)#x]
+Results: [key#x, sort_array(collect_set(val#x, 0, 0)#x, true)[0] AS
sort_array(collect_set(val), true)[0]#x]
Review comment:
Thanks for review! I'll follow up with another PR for these observations.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
gatorsmile commented on a change in pull request #26977: [SPARK-30326][SQL]
Raise exception if analyzer exceed max iterations
URL: https://github.com/apache/spark/pull/26977#discussion_r376694885
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala
##
@@ -156,7 +163,7 @@ abstract class RuleExecutor[TreeType <: TreeNode[_]]
extends Logging {
// Only log if this is a rule that is supposed to run more than once.
if (iteration != 2) {
val message = s"Max iterations (${iteration - 1}) reached for
batch ${batch.name}"
-if (Utils.isTesting) {
+if (Utils.isTesting || batch.strategy.errorOnExceed) {
throw new TreeNodeException(curPlan, message, null)
Review comment:
We need a clear message for end users. Usability is more important.
Regarding the implementation, I think we should avoid over-engineering. Just
make it simple. Adding the message postfix in abstract class Strategy looks
fine to me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #27368: [SPARK-30651][SQL] Add detailed information for Aggregate operators in EXPLAIN FORMATTED
gatorsmile commented on a change in pull request #27368: [SPARK-30651][SQL] Add
detailed information for Aggregate operators in EXPLAIN FORMATTED
URL: https://github.com/apache/spark/pull/27368#discussion_r376694900
##
File path: sql/core/src/test/resources/sql-tests/results/explain.sql.out
##
@@ -786,6 +870,144 @@ Output: []
(4) Project
+-- !query
+EXPLAIN FORMATTED
+ SELECT
+COUNT(val) + SUM(key) as TOTAL,
+COUNT(key) FILTER (WHERE val > 1)
+ FROM explain_temp1
+-- !query schema
+struct
+-- !query output
+== Physical Plan ==
+* HashAggregate (5)
++- Exchange (4)
+ +- HashAggregate (3)
+ +- * ColumnarToRow (2)
+ +- Scan parquet default.explain_temp1 (1)
+
+
+(1) Scan parquet default.explain_temp1
+Output: [key#x, val#x]
+Batched: true
+Location [not included in comparison]/{warehouse_dir}/explain_temp1]
+ReadSchema: struct
+
+(2) ColumnarToRow [codegen id : 1]
+Input: [key#x, val#x]
+
+(3) HashAggregate
+Input: [key#x, val#x]
+Keys: []
+Functions: [partial_count(val#x), partial_sum(cast(key#x as bigint)),
partial_count(key#x) FILTER (WHERE (val#x > 1))]
+Aggregate Attributes: [count#xL, sum#xL, count#xL]
+Results: [count#xL, sum#xL, count#xL]
+
+(4) Exchange
+Input: [count#xL, sum#xL, count#xL]
+
+(5) HashAggregate [codegen id : 2]
+Input: [count#xL, sum#xL, count#xL]
+Keys: []
+Functions: [count(val#x), sum(cast(key#x as bigint)), count(key#x)]
+Aggregate Attributes: [count(val#x)#xL, sum(cast(key#x as bigint))#xL,
count(key#x)#xL]
+Results: [(count(val#x)#xL + sum(cast(key#x as bigint))#xL) AS TOTAL#xL,
count(key#x)#xL AS count(key) FILTER (WHERE (val > 1))#xL]
+
+
+-- !query
+EXPLAIN FORMATTED
+ SELECT key, sort_array(collect_set(val))[0]
+ FROM explain_temp4
+ GROUP BY key
+-- !query schema
+struct
+-- !query output
+== Physical Plan ==
+ObjectHashAggregate (5)
++- Exchange (4)
+ +- ObjectHashAggregate (3)
+ +- * ColumnarToRow (2)
+ +- Scan parquet default.explain_temp4 (1)
+
+
+(1) Scan parquet default.explain_temp4
+Output: [key#x, val#x]
+Batched: true
+Location [not included in comparison]/{warehouse_dir}/explain_temp4]
+ReadSchema: struct
+
+(2) ColumnarToRow [codegen id : 1]
+Input: [key#x, val#x]
+
+(3) ObjectHashAggregate
+Input: [key#x, val#x]
+Keys: [key#x]
+Functions: [partial_collect_set(val#x, 0, 0)]
+Aggregate Attributes: [buf#x]
+Results: [key#x, buf#x]
+
+(4) Exchange
+Input: [key#x, buf#x]
+
+(5) ObjectHashAggregate
+Input: [key#x, buf#x]
+Keys: [key#x]
+Functions: [collect_set(val#x, 0, 0)]
+Aggregate Attributes: [collect_set(val#x, 0, 0)#x]
+Results: [key#x, sort_array(collect_set(val#x, 0, 0)#x, true)[0] AS
sort_array(collect_set(val), true)[0]#x]
Review comment:
This can be a separate PR if this is a general issue for all the other
operator. We should make all of them consistent.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #27368: [SPARK-30651][SQL] Add detailed information for Aggregate operators in EXPLAIN FORMATTED
gatorsmile commented on a change in pull request #27368: [SPARK-30651][SQL] Add
detailed information for Aggregate operators in EXPLAIN FORMATTED
URL: https://github.com/apache/spark/pull/27368#discussion_r376694496
##
File path: sql/core/src/test/resources/sql-tests/results/explain.sql.out
##
@@ -786,6 +870,144 @@ Output: []
(4) Project
+-- !query
+EXPLAIN FORMATTED
+ SELECT
+COUNT(val) + SUM(key) as TOTAL,
+COUNT(key) FILTER (WHERE val > 1)
+ FROM explain_temp1
+-- !query schema
+struct
+-- !query output
+== Physical Plan ==
+* HashAggregate (5)
++- Exchange (4)
+ +- HashAggregate (3)
+ +- * ColumnarToRow (2)
+ +- Scan parquet default.explain_temp1 (1)
+
+
+(1) Scan parquet default.explain_temp1
+Output: [key#x, val#x]
+Batched: true
+Location [not included in comparison]/{warehouse_dir}/explain_temp1]
+ReadSchema: struct
+
+(2) ColumnarToRow [codegen id : 1]
+Input: [key#x, val#x]
+
+(3) HashAggregate
+Input: [key#x, val#x]
+Keys: []
+Functions: [partial_count(val#x), partial_sum(cast(key#x as bigint)),
partial_count(key#x) FILTER (WHERE (val#x > 1))]
+Aggregate Attributes: [count#xL, sum#xL, count#xL]
+Results: [count#xL, sum#xL, count#xL]
+
+(4) Exchange
+Input: [count#xL, sum#xL, count#xL]
+
+(5) HashAggregate [codegen id : 2]
+Input: [count#xL, sum#xL, count#xL]
+Keys: []
+Functions: [count(val#x), sum(cast(key#x as bigint)), count(key#x)]
+Aggregate Attributes: [count(val#x)#xL, sum(cast(key#x as bigint))#xL,
count(key#x)#xL]
+Results: [(count(val#x)#xL + sum(cast(key#x as bigint))#xL) AS TOTAL#xL,
count(key#x)#xL AS count(key) FILTER (WHERE (val > 1))#xL]
+
+
+-- !query
+EXPLAIN FORMATTED
+ SELECT key, sort_array(collect_set(val))[0]
+ FROM explain_temp4
+ GROUP BY key
+-- !query schema
+struct
+-- !query output
+== Physical Plan ==
+ObjectHashAggregate (5)
++- Exchange (4)
+ +- ObjectHashAggregate (3)
+ +- * ColumnarToRow (2)
+ +- Scan parquet default.explain_temp4 (1)
+
+
+(1) Scan parquet default.explain_temp4
+Output: [key#x, val#x]
+Batched: true
+Location [not included in comparison]/{warehouse_dir}/explain_temp4]
+ReadSchema: struct
+
+(2) ColumnarToRow [codegen id : 1]
+Input: [key#x, val#x]
+
+(3) ObjectHashAggregate
+Input: [key#x, val#x]
+Keys: [key#x]
+Functions: [partial_collect_set(val#x, 0, 0)]
+Aggregate Attributes: [buf#x]
+Results: [key#x, buf#x]
+
+(4) Exchange
+Input: [key#x, buf#x]
+
+(5) ObjectHashAggregate
+Input: [key#x, buf#x]
+Keys: [key#x]
+Functions: [collect_set(val#x, 0, 0)]
+Aggregate Attributes: [collect_set(val#x, 0, 0)#x]
+Results: [key#x, sort_array(collect_set(val#x, 0, 0)#x, true)[0] AS
sort_array(collect_set(val), true)[0]#x]
Review comment:
Since the attribute names are automatically generated, it is hard to tell it
is a name or an expression. A few observations:
- Using comma as the separator is not clear, especially commas are used
inside the expressions too.
- Show the column counts first? For example, `Results [4]: ... `
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
Eric5553 commented on a change in pull request #26977: [SPARK-30326][SQL] Raise
exception if analyzer exceed max iterations
URL: https://github.com/apache/spark/pull/26977#discussion_r376694303
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala
##
@@ -156,7 +163,7 @@ abstract class RuleExecutor[TreeType <: TreeNode[_]]
extends Logging {
// Only log if this is a rule that is supposed to run more than once.
if (iteration != 2) {
val message = s"Max iterations (${iteration - 1}) reached for
batch ${batch.name}"
-if (Utils.isTesting) {
+if (Utils.isTesting || batch.strategy.errorOnExceed) {
throw new TreeNodeException(curPlan, message, null)
Review comment:
IMO, here is the common logic in `RuleExecutor` to handle `errorOnExceed`
strategy, which should not include specific knowledge `ANALYZER_MAX_ITERATIONS`
of inheritance class `analyzer`. This will lead to potential bug when we
introduce more MAX_ITERATION setting?
Maybe we can use a common phrase here like 'increasing the value of
corresponding SQLConf setting'. Or an alternative way of adding a string hint
field in `Strategy`, and set it in `Analyzer` together with `errorOnExceed`
flag?
What do you think ? @gatorsmile Thanks a lot!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583711860 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #27368: [SPARK-30651][SQL] Add detailed information for Aggregate operators in EXPLAIN FORMATTED
gatorsmile commented on a change in pull request #27368: [SPARK-30651][SQL] Add detailed information for Aggregate operators in EXPLAIN FORMATTED URL: https://github.com/apache/spark/pull/27368#discussion_r376694045 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectHashAggregateExec.scala ## @@ -67,7 +67,7 @@ case class ObjectHashAggregateExec( initialInputBufferOffset: Int, resultExpressions: Seq[NamedExpression], Review comment: We need to add override modifier; otherwise the compiler will not capture it if we rename it in the future. Add `private override val` to all the four variables defined in BaseAggregateExec? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583711862 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118054/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583711860 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583711862 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118054/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-583711745 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22826/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
SparkQA removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583697652 **[Test build #118054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118054/testReport)** for PR 27495 at commit [`464cd03`](https://github.com/apache/spark/commit/464cd03a546ccb72e1a2efe70900df03872e3fb9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-583711744 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-583711745 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22826/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-583711744 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
SparkQA commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583711770 **[Test build #118054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118054/testReport)** for PR 27495 at commit [`464cd03`](https://github.com/apache/spark/commit/464cd03a546ccb72e1a2efe70900df03872e3fb9). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
SparkQA commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-583711659 **[Test build #118060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118060/testReport)** for PR 27499 at commit [`8aafa57`](https://github.com/apache/spark/commit/8aafa573a9cda815fdfbfe5c8864c8595c4e1f93). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya opened a new pull request #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
viirya opened a new pull request #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499 ### What changes were proposed in this pull request? This patch proposes to throw analysis exception if untyped `Dataset.select` takes typed column expression. This patch also proposes to make `Dataset.selectUntyped` as public API for selecting multiple typed column expressions. ### Why are the changes needed? `Dataset` provides few typed `select` helper functions to select typed column expressions. The maximum number of typed columns supported is 5. If wanting to select more than 5 typed columns, it silently calls untyped `Dataset.select` can causes weird unresolved error. We should explicitly let users know that they are incorrectly calling untyped `select` with typed columns. Because typed `Dataset.select` cannot be used to select more than 5 typed columns, this also opens `selectUntyped`. ### Does this PR introduce any user-facing change? Yes. If users intentionally or unintentionally call `Dataset.select` API with typed column, an analysis exception will be thrown. Users can use `selectUntyped` API to select multiple typed columns. ### How was this patch tested? Unit tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #26805: [SPARK-15616][SQL] Add optimizer rule PruneHiveTablePartitions
gatorsmile commented on a change in pull request #26805: [SPARK-15616][SQL] Add
optimizer rule PruneHiveTablePartitions
URL: https://github.com/apache/spark/pull/26805#discussion_r376692200
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/PruneHiveTablePartitions.scala
##
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.execution
+
+import org.apache.hadoop.hive.common.StatsSetupConst
+
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.catalyst.analysis.CastSupport
+import org.apache.spark.sql.catalyst.catalog.{CatalogStatistics, CatalogTable,
CatalogTablePartition, ExternalCatalogUtils, HiveTableRelation}
+import org.apache.spark.sql.catalyst.expressions.{And, AttributeSet,
Expression, ExpressionSet, SubqueryExpression}
+import org.apache.spark.sql.catalyst.planning.PhysicalOperation
+import org.apache.spark.sql.catalyst.plans.logical.{Filter, LogicalPlan,
Project}
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.datasources.DataSourceStrategy
+import org.apache.spark.sql.internal.SQLConf
+
+/**
+ * TODO: merge this with PruneFileSourcePartitions after we completely make
hive as a data source.
Review comment:
@fuwhu We need a description about the rule. Could you submit a follow-up PR
to add the descriptions to both PruneHiveTablePartitions and
PruneFileSourcePartitions?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583707159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118057/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583707158 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583707159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118057/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
SparkQA removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583703535 **[Test build #118057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118057/testReport)** for PR 27461 at commit [`a62203d`](https://github.com/apache/spark/commit/a62203d64413a63bc13f6324aff2950d82dcb521). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583707158 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
SparkQA commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583707098 **[Test build #118057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118057/testReport)** for PR 27461 at commit [`a62203d`](https://github.com/apache/spark/commit/a62203d64413a63bc13f6324aff2950d82dcb521). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter
AmplabJenkins commented on issue #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter URL: https://github.com/apache/spark/pull/27498#issuecomment-583706526 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter
AmplabJenkins removed a comment on issue #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter URL: https://github.com/apache/spark/pull/27498#issuecomment-583706450 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter
AmplabJenkins commented on issue #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter URL: https://github.com/apache/spark/pull/27498#issuecomment-583706450 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] javierivanov opened a new pull request #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter
javierivanov opened a new pull request #27498: [SPARK-30688][SQL] Week based
dates not being parsed with TimestampFormatter
URL: https://github.com/apache/spark/pull/27498
### What changes were proposed in this pull request?
When executing the following query using a week based pattern the result is
null.
```
scala> spark.sql("select unix_timestamp('2020-10', '-ww')").show
++
|unix_timestamp(2020-10, -ww)|
++
|null|
++
```
You can replicate this behavior with:
```
import java.time.temporal._
import java.time.format._
import java.util.Locale
import java.time.chrono.IsoChronology
val pattern = "-ww"
val locale = Locale.US
val s = "2020-10"
val formatter = new DateTimeFormatterBuilder().appendPattern(pattern)
.parseDefaulting(ChronoField.ERA, 1)
.parseDefaulting(ChronoField.MONTH_OF_YEAR, 1)
.parseDefaulting(ChronoField.DAY_OF_MONTH, 1)
.parseDefaulting(ChronoField.MINUTE_OF_HOUR, 0)
.parseDefaulting(ChronoField.SECOND_OF_MINUTE, 0)
.toFormatter(locale)
.withChronology(IsoChronology.INSTANCE)
.withResolverStyle(ResolverStyle.STRICT)
val result = formatter.parse(s)
result.query(TemporalQueries.localDate)
```
The date is parsed, but not resolved.
This is caused by the default temporal fields that are conflicting with the
week based values:
```
.parseDefaulting(ChronoField.ERA, 1)
.parseDefaulting(ChronoField.MONTH_OF_YEAR, 1)
.parseDefaulting(ChronoField.DAY_OF_MONTH, 1)
```
To avoid this conflict, I propose to check if the pattern is a week/year
based and change the defaults as follows:
```
.parseDefaulting(ChronoField.ERA, 1)
.parseDefaulting(ChronoField.DAY_OF_WEEK, 1)
```
### Why are the changes needed?
Week/Year based dates should be supported since it is part of ISO8601. Also,
as seen in [SPARK-30688](https://issues.apache.org/jira/browse/SPARK-30688),
this issue is failing silently.
### Does this PR introduce any user-facing change?
- No
### How was this patch tested?
- Passing current tests
- Added test case for different timezones in `DateExpressionsSuite.scala`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause
AmplabJenkins removed a comment on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause URL: https://github.com/apache/spark/pull/27454#issuecomment-583706063 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause
AmplabJenkins removed a comment on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause URL: https://github.com/apache/spark/pull/27454#issuecomment-583706064 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118053/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause
AmplabJenkins commented on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause URL: https://github.com/apache/spark/pull/27454#issuecomment-583706064 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118053/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause
AmplabJenkins commented on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause URL: https://github.com/apache/spark/pull/27454#issuecomment-583706063 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior
AmplabJenkins removed a comment on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior URL: https://github.com/apache/spark/pull/27478#issuecomment-583705939 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior
AmplabJenkins removed a comment on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior URL: https://github.com/apache/spark/pull/27478#issuecomment-583705941 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118052/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause
SparkQA removed a comment on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause URL: https://github.com/apache/spark/pull/27454#issuecomment-583686663 **[Test build #118053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118053/testReport)** for PR 27454 at commit [`ebd337b`](https://github.com/apache/spark/commit/ebd337b9250b9a1426232fe16d5eda92e16b23c8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior
AmplabJenkins commented on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior URL: https://github.com/apache/spark/pull/27478#issuecomment-583705939 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior
AmplabJenkins commented on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior URL: https://github.com/apache/spark/pull/27478#issuecomment-583705941 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118052/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause
SparkQA commented on issue #27454: [SPARK-28228][SQL] Change the default behavior for name conflict in nested WITH clause URL: https://github.com/apache/spark/pull/27454#issuecomment-583705968 **[Test build #118053 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118053/testReport)** for PR 27454 at commit [`ebd337b`](https://github.com/apache/spark/commit/ebd337b9250b9a1426232fe16d5eda92e16b23c8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior
SparkQA removed a comment on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior URL: https://github.com/apache/spark/pull/27478#issuecomment-583685270 **[Test build #118052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118052/testReport)** for PR 27478 at commit [`66dc51f`](https://github.com/apache/spark/commit/66dc51f0bc0fb45f7c44ef72f26e1918e01d0212). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior
SparkQA commented on issue #27478: [SPARK-25829][SQL] Add config `spark.sql.deduplicateMapKey.lastWinsPolicy.enabled` and change the default behavior URL: https://github.com/apache/spark/pull/27478#issuecomment-583705844 **[Test build #118052 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118052/testReport)** for PR 27478 at commit [`66dc51f`](https://github.com/apache/spark/commit/66dc51f0bc0fb45f7c44ef72f26e1918e01d0212). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
SparkQA commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705639 **[Test build #118059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118059/testReport)** for PR 27492 at commit [`274af0e`](https://github.com/apache/spark/commit/274af0e6562cd9a9622369402b67810527774f8c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705334 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22825/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins removed a comment on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705331 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705334 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22825/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
AmplabJenkins commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705331 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3
wangyum commented on issue #27492: [SPARK-30755][SQL][test-hive1.2] Support Hive 1.2.1's Serde after making built-in Hive to 2.3 URL: https://github.com/apache/spark/pull/27492#issuecomment-583705263 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #27477: [SPARK-30724][SQL] Support 'LIKE ANY' and 'LIKE ALL' operators
beliefer commented on a change in pull request #27477: [SPARK-30724][SQL]
Support 'LIKE ANY' and 'LIKE ALL' operators
URL: https://github.com/apache/spark/pull/27477#discussion_r376688951
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1375,6 +1375,14 @@ class AstBuilder(conf: SQLConf) extends
SqlBaseBaseVisitor[AnyRef] with Logging
case other => Seq(other)
}
+def getLikeQuantifierExps(expressions: java.util.List[ExpressionContext]):
Seq[Expression] = {
+ if (expressions.isEmpty) {
+throw new ParseException("Syntax error: expected something between '('
and ')'.", ctx)
Review comment:
I think should remove `Syntax error: `, because `ParseException` could
replace it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed
AmplabJenkins removed a comment on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed URL: https://github.com/apache/spark/pull/27497#issuecomment-583704100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22824/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed
AmplabJenkins removed a comment on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed URL: https://github.com/apache/spark/pull/27497#issuecomment-583704097 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed
SparkQA commented on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed URL: https://github.com/apache/spark/pull/27497#issuecomment-583704454 **[Test build #118058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118058/testReport)** for PR 27497 at commit [`b43a871`](https://github.com/apache/spark/commit/b43a871e8ce3536a682936998539bd9ba00ee2ff). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #21985: [SPARK-24884][SQL] add regexp_extract_all support
beliefer commented on issue #21985: [SPARK-24884][SQL] add regexp_extract_all support URL: https://github.com/apache/spark/pull/21985#issuecomment-583704236 @gatorsmile OK. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed
AmplabJenkins commented on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed URL: https://github.com/apache/spark/pull/27497#issuecomment-583704097 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
beliefer commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583704122 @gatorsmile Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed
AmplabJenkins commented on issue #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed URL: https://github.com/apache/spark/pull/27497#issuecomment-583704100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22824/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer opened a new pull request #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed
beliefer opened a new pull request #27497: [SPARK-30245][SQL][FOLLOWUP] Improve regex expression when pattern not changed URL: https://github.com/apache/spark/pull/27497 ### What changes were proposed in this pull request? This PR follows up https://github.com/apache/spark/pull/26875. ### Why are the changes needed? When pattern is not static, we should avoid compile pattern every time if some pattern is same. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Exists UT. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-583703568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118050/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-583703566 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
SparkQA commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583703535 **[Test build #118057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118057/testReport)** for PR 27461 at commit [`a62203d`](https://github.com/apache/spark/commit/a62203d64413a63bc13f6324aff2950d82dcb521). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-583703568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118050/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-583703566 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
SparkQA removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-583677519 **[Test build #118050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118050/testReport)** for PR 27019 at commit [`2796407`](https://github.com/apache/spark/commit/2796407bfe9ce10d6c3587ea22f48ed1fd4d18eb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-583703452 **[Test build #118050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118050/testReport)** for PR 27019 at commit [`2796407`](https://github.com/apache/spark/commit/2796407bfe9ce10d6c3587ea22f48ed1fd4d18eb). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait GeneratePredicateHelper extends PredicateHelper ` * `case class FilterExec(condition: Expression, child: SparkPlan)` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #21985: [SPARK-24884][SQL] add regexp_extract_all support
gatorsmile commented on issue #21985: [SPARK-24884][SQL] add regexp_extract_all support URL: https://github.com/apache/spark/pull/21985#issuecomment-583703247 cc @beliefer Could you take this over? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583703069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22823/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583703067 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-583703030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118051/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583703067 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-583703069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22823/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-583703028 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-583703028 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-583703030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118051/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
SparkQA removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-583677553 **[Test build #118051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118051/testReport)** for PR 26918 at commit [`8cded00`](https://github.com/apache/spark/commit/8cded00b31ed5522aa5ed81ee36381e7b9593584). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
SparkQA commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-583702903 **[Test build #118051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118051/testReport)** for PR 26918 at commit [`8cded00`](https://github.com/apache/spark/commit/8cded00b31ed5522aa5ed81ee36381e7b9593584). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
AmplabJenkins removed a comment on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496#issuecomment-583702257 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
AmplabJenkins removed a comment on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496#issuecomment-583702260 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22821/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
SparkQA commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496#issuecomment-583702586 **[Test build #118056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118056/testReport)** for PR 27496 at commit [`e5634b0`](https://github.com/apache/spark/commit/e5634b07910e38b2ad9655e1fa95f92c027a389a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
gatorsmile commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-583702527 cc @maropu @gengliangwang @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
gatorsmile commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702344 cc @cloud-fan @maryannxue This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
AmplabJenkins removed a comment on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702269 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
AmplabJenkins removed a comment on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702270 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22822/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
gatorsmile commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702337 LGTM except a comment about the message This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
AmplabJenkins commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496#issuecomment-583702260 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22821/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
AmplabJenkins commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496#issuecomment-583702257 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
AmplabJenkins commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702269 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
AmplabJenkins commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702270 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22822/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
gatorsmile commented on a change in pull request #26977: [SPARK-30326][SQL]
Raise exception if analyzer exceed max iterations
URL: https://github.com/apache/spark/pull/26977#discussion_r376687022
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala
##
@@ -156,7 +163,7 @@ abstract class RuleExecutor[TreeType <: TreeNode[_]]
extends Logging {
// Only log if this is a rule that is supposed to run more than once.
if (iteration != 2) {
val message = s"Max iterations (${iteration - 1}) reached for
batch ${batch.name}"
-if (Utils.isTesting) {
+if (Utils.isTesting || batch.strategy.errorOnExceed) {
throw new TreeNodeException(curPlan, message, null)
Review comment:
In the message, we need to explicitly tell users how to resolve this
exception. Increasing the value of SQLConf.ANALYZER_MAX_ITERATIONS.key
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
imback82 commented on issue #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496#issuecomment-583702256 cc @cloud-fan @rdblue This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 opened a new pull request #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior
imback82 opened a new pull request #27496: [SPARK-30757][SQL][DOC] Update the doc on TableCatalog.alterTable's behavior URL: https://github.com/apache/spark/pull/27496 ### What changes were proposed in this pull request? This PR updates the documentation on `TableCatalog.alterTable`s behavior on the order by which the requested changes are applied. It now explicitly mentions that the changes are applied in the order given. ### Why are the changes needed? The current documentation on `TableCatalog.alterTable` doesn't mention which order the requested changes are applied. It will be useful to explicitly document this behavior so that the user can expect the behavior. For example, `REPLACE COLUMNS` needs to delete columns before adding new columns, and if the order is guaranteed by `alterTable`, it's much easier to work with the catalog API. ### Does this PR introduce any user-facing change? Yes, document change. ### How was this patch tested? Not added (doc changes). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
SparkQA commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702180 **[Test build #118055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118055/testReport)** for PR 26977 at commit [`259d12d`](https://github.com/apache/spark/commit/259d12dc2a3b3e15d2865a6045ee6bb111b97b46). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations
gatorsmile commented on issue #26977: [SPARK-30326][SQL] Raise exception if analyzer exceed max iterations URL: https://github.com/apache/spark/pull/26977#issuecomment-583702054 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #27487: Revert "[SPARK-30642][SPARK-30659][SPARK-30660][SPARK-30662]"
zhengruifeng commented on issue #27487: Revert "[SPARK-30642][SPARK-30659][SPARK-30660][SPARK-30662]" URL: https://github.com/apache/spark/pull/27487#issuecomment-583699847 @srowen Never mind, I should have maken more perfermance tests. Maybe we should do such refactors only when we have enough time for testing. When working on these PRs, it maybe too close to the code freeze. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
