comphead opened a new issue, #9144:
URL: https://github.com/apache/arrow-datafusion/issues/9144
### Is your feature request related to a problem or challenge?
it was found that the planner calling schema transform functions excessively.
Experiment, lets take a simple query
```
let sql = "select a, a + 1 b from (select 1 a union all select 2 a)
x";
```
And increase rows in the x subquery by adding more and more rows, and we can
see that schema function grows with every record
rows | new_with_metadata | merge
-- | -- | --
2 | 58 | 35
3 | 131 | 83
4 | 165 | 109
IMHO That is not expected, once the plan has built the schema calls should
not increase with every new record in the dataset
### Describe the solution you'd like
Ideally resolve excessive, more real is to reduce such calls
### Describe alternatives you've considered
_No response_
### Additional context
Follow up on https://github.com/apache/arrow-datafusion/pull/9104
investigations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
