hareshkh opened a new issue, #17122: URL: https://github.com/apache/datafusion/issues/17122
### Describe the bug When computing partition_statistics during evalutaion - flamegraph shows a lot of time spend in bounds_check() which happens as part of a Column::data_type() call. <img width="1728" height="995" alt="Image" src="https://github.com/user-attachments/assets/d6a7e75c-a1ad-4bfc-b7b3-32c918a7eced"; /> Almost all of the time in bounds_check() is also spend in fmt() which suggests that this goes into the error branch: ```rust impl Column { fn bounds_check(&self, input_schema: &Schema) -> Result<()> { if self.index < input_schema.fields.len() { Ok(()) } else { internal_err!( "PhysicalExpr Column references column '{}' at index {} (zero-based) but input schema only has {} columns: {:?}", self.name, self.index, input_schema.fields.len(), input_schema.fields().iter().map(|f| f.name()).collect::<Vec<_>>() ) } } } ``` All occurrences that I hand checked from my example were originating from `ProjectionExec::partition_statistics()` ### To Reproduce Run with RUST_BACKTRACE enabled. ### Expected behavior data_type() method should not trigger bounds_check() to go to an error path for the column. ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
