pan3793 commented on code in PR #58851:
URL: https://github.com/apache/spark/pull/58851#discussion_r4032572802
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/connector/PartitionPredicateImpl.scala:
##########
@@ -26,10 +27,23 @@ import
org.apache.spark.sql.connector.expressions.filter.PartitionPredicate
/**
* An implementation for [[PartitionPredicate]] that wraps a Catalyst
Expression representing a
* partition filter.
+ *
+ * @param catalystExpr the partition filter this predicate evaluates.
+ * @param partitionFields one entry per transform of `Table.partitioning()`,
in that order, so a
+ * bound ordinal matches the partition key a connector
passes to [[eval]].
+ * @param failOpen what [[eval]] does when it cannot evaluate the expression
for a partition.
Review Comment:
Renamed to `keepOnEvalFailure` in 9b8d884. Kept it shorter than the
suggestion since the class is already `PartitionPredicateImpl`, so "Partition"
reads as redundant.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/connector/PartitionPredicateImpl.scala:
##########
@@ -47,6 +61,12 @@ class PartitionPredicateImpl private (
override def eval(partitionValues: InternalRow): Boolean = {
if (partitionValues.numFields != partitionFields.length) {
+ if (!failOpen) {
+ throw SparkException.internalError(
Review Comment:
Applied in 9b8d884, as suggested. The exception text is unchanged, so the
assertion that pins it still matches.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/connector/PartitionPredicateImpl.scala:
##########
@@ -58,7 +78,8 @@ class PartitionPredicateImpl private (
try {
boundPredicate(partitionValues)
} catch {
- case e: Exception =>
+ // Propagated when this predicate is the only evaluator: see `failOpen`.
Review Comment:
Dropped it in 9b8d884. You were right that the rename carries it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
