nuno-faria opened a new issue, #16998:
URL: https://github.com/apache/datafusion/issues/16998
### Describe the bug
The `SortExec` operator will share the same `DynamicFilterPhysicalExpr`
across multiple invocations of `with_new_children` (e.g., from
`reset_plan_states`), causing queries that repeatedly execute it to fail.
### To Reproduce
For instance, this query returns just 2 rows, while it should return 5:
```sql
> with recursive r as (
select 0 as k, 0 as v
union all
(
select *
from r
order by v
limit 1
)
)
select *
from r
limit 5;
+---+---+
| k | v |
+---+---+
| 0 | 0 |
| 0 | 0 |
+---+---+
2 row(s) fetched.
```
### Expected behavior
The query above should return:
```
+---+---+
| k | v |
+---+---+
| 0 | 0 |
| 0 | 0 |
| 0 | 0 |
| 0 | 0 |
| 0 | 0 |
+---+---+
```
### Additional context
When comparing the physical plan, we can see that the second one (after one
execution) is different than the initial one in the `DynamicFilterPhysicalExpr`
expression:
```diff
@@ -1,24 +1,30 @@
DynamicFilterPhysicalExpr {
children: [
Column {
name: "v", index: 1
}
]
, remapped_children: None, inner: RwLock {
data: Inner {
- generation: 1, expr: Literal {
- value: Boolean(true), field: Field {
- name: "lit", data_type: Boolean, nullable: false, dict_id: 0,
dict_is_ordered: false, metadata: {
+ generation: 2, expr: BinaryExpr {
+ left: Column {
+ name: "v", index: 1
+ }
+ , op: Lt, right: Literal {
+ value: Int64(0), field: Field {
+ name: "lit", data_type: Int64, nullable: false, dict_id: 0,
dict_is_ordered: false, metadata: {
+ }
}
}
+ , fail_on_overflow: false
}
}
, poisoned: false, ..
}
, data_type: RwLock {
data: None, poisoned: false, ..
}
, nullable: RwLock {
data: None, poisoned: false, ..
}
}
```
Updating `SortExec::with_new_children` to fully clone the `filter` makes the
recursive query run correctly, but I don't know if there are any side effects
that I'm not aware.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
