jayzhan211 commented on code in PR #26056:
URL: https://github.com/apache/datafusion/pull/26056#discussion_r4356833330
##########
datafusion/physical-expr/src/expressions/dynamic_filters/mod.rs:
##########
@@ -343,6 +344,31 @@ impl DynamicFilterPhysicalExpr {
Ok(())
}
+ /// Reset this dynamic filter so its producer can publish a new expression
+ /// during a later execution of the same plan.
+ pub fn reset(&self) -> Result<()> {
+ let new_expr = Self::remap_children(
+ &self.children,
+ self.remapped_children.as_ref(),
+ lit(true),
+ )?;
+
+ let mut current = self.inner.write();
+ let new_generation = current.generation + 1;
+ *current = Inner {
Review Comment:
It isn't obvious, so a comment would help. The generation has to keep
increasing across a reset. `current()` caches the remapped expression by
generation, so if `reset()` went back to 1 (or 0), later updates would reach
the cached generation again and `current()` would return the previous run's
expression. I tried it: with the generation reset to 1, the first update after
the reset returns the old expression.
@mikamikasuki could you add a comment like:
```rust
// Keep the generation increasing: `current()` caches by generation, so going
// back would let a later update hit the previous run's cached expression.
```
##########
datafusion/physical-expr/src/expressions/dynamic_filters/mod.rs:
##########
@@ -343,6 +344,31 @@ impl DynamicFilterPhysicalExpr {
Ok(())
}
+ /// Reset this dynamic filter so its producer can publish a new expression
+ /// during a later execution of the same plan.
+ pub fn reset(&self) -> Result<()> {
+ let new_expr = Self::remap_children(
+ &self.children,
+ self.remapped_children.as_ref(),
+ lit(true),
+ )?;
+
+ let mut current = self.inner.write();
+ let new_generation = current.generation + 1;
+ *current = Inner {
+ expression_id: current.expression_id,
+ generation: new_generation,
+ expr: new_expr,
+ is_complete: false,
+ };
+ drop(current);
+
+ let _ = self.state_watch.send(FilterState::InProgress {
+ generation: new_generation,
+ });
Review Comment:
The watch channel still holds `Complete` from the previous run. Without this
send, `wait_complete()` sees `Complete` and returns right away, before this
run's filter is ready. I checked this with a subscriber held across the reset.
It sends `new_generation` so the channel matches `inner`.
@mikamikasuki and a comment here like:
```rust
// The channel still holds `Complete` from the previous run; publish
// `InProgress` so `wait_complete()` waits for this run's filter.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
