aokolnychyi commented on code in PR #6077:
URL: https://github.com/apache/iceberg/pull/6077#discussion_r1008755707
##########
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/dynamicpruning/RowLevelCommandDynamicPruning.scala:
##########
@@ -69,27 +73,34 @@ case class RowLevelCommandDynamicPruning(spark:
SparkSession) extends Rule[Logic
// use reference equality to find exactly the required scan relations
val newRewritePlan = rewritePlan transformUp {
case r: DataSourceV2ScanRelation if r.scan eq scan =>
- val pruningKeys = ExtendedV2ExpressionUtils.resolveRefs[Attribute](
- ArraySeq.unsafeWrapArray(scan.filterAttributes), r)
- val dynamicPruningCond = buildDynamicPruningCondition(r, command,
pruningKeys)
- val filter = Filter(dynamicPruningCond, r)
- // always optimize dynamic filtering subqueries for row-level
commands as it is important
- // to rewrite introduced predicates as joins because Spark recently
stopped optimizing
- // dynamic subqueries to facilitate broadcast reuse
- optimizeSubquery(filter)
+ // use the original table instance that was loaded for this
row-level operation
Review Comment:
I adapted this place to match what we did in Spark (including comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
