toutane opened a new issue, #2359: URL: https://github.com/apache/iceberg-rust/issues/2359
### Is your feature request related to a problem or challenge? When DataFusion pushes a `LIMIT` down to `IcebergTableProvider::scan`, the limit value is forwarded into `IcebergTableScan` (see `crates/integrations/datafusion/src/physical_plan/scan.rs`) and applied at stream time. However, the current `DisplayAs` implementation only prints projection and predicate: `IcebergTableScan projection:[id,name] predicate:[]` This makes it impossible to tell from `EXPLAIN` output whether a `LIMIT` actually reached the scan or not. For users debugging query plans - in particular to verify that limit pushdown is working as expected on large Iceberg tables - this is a real observability gap. It's also inconsistent with DataFusion's own data source execs (`DataSourceExec`, `ParquetExec`, …) which surface `limit=N` in their `EXPLAIN` output. ### Describe the solution you'd like Extend `IcebergTableScan`'s `DisplayAs::fmt_as` so that, when `self.limit` is `Some(n)`, the rendered plan line includes a `limit:[n]` suffix. When there is no pushdown limit, the output should be left unchanged (no empty `limit:[]` marker), so this is a purely additive change that does not break existing snapshots of downstream users that match on the current format. **Example:** - no limit pushed down - identical to today `IcebergTableScan projection:[id,name] predicate:[name STARTS WITH "Al"]` - limit pushed down - new behavior `IcebergTableScan projection:[id,name,score,category] predicate:[] limit:[3]` Coverage for the populated case would be added as a new `EXPLAIN ... LIMIT N` assertion in `crates/sqllogictest/testdata/slts/df_test/basic_queries.slt`, reusing the existing `query_test_table` fixture. **Out of scope for this request (but worth noting)** The same ergonomic issue exists today for predicate: when no filter is pushed down, the plan still renders a noisy empty `predicate:[]`: `IcebergTableScan projection:[id,name] predicate:[]` Applying the same "omit when None" treatment would be a natural follow-up and would make the scan's `Display` uniformly additive. I'm deliberately not bundling it here - it would change every existing `EXPLAIN` snapshot in the `sqllogictest` suite, which deserves its own focused discussion and review. Happy to open a separate issue/PR for it if maintainers agree it's desirable. ### Willingness to contribute I can contribute to this feature independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
