gruuya commented on code in PR #880:
URL: https://github.com/apache/iceberg-rust/pull/880#discussion_r1907874949
##########
crates/integrations/datafusion/src/table/mod.rs:
##########
@@ -41,16 +42,21 @@ pub struct IcebergTableProvider {
table: Table,
/// Table snapshot id that will be queried via this provider.
snapshot_id: Option<i64>,
+ /// Statistics for the table; row count and null count/min-max values per
column.
+ /// If not present defaults to `None`.
+ statistics: Option<Statistics>,
/// A reference-counted arrow `Schema`.
schema: ArrowSchemaRef,
}
impl IcebergTableProvider {
- pub(crate) fn new(table: Table, schema: ArrowSchemaRef) -> Self {
+ pub(crate) async fn new(table: Table, schema: ArrowSchemaRef) -> Self {
+ let statistics = compute_statistics(&table, None).await.ok();
Review Comment:
Maybe some people don't care about stats (e.g. they don't really do a lot of
joins). I assume a friendlier API in that case would be making the stats
computation opt-in via something like
```rust
--- a/crates/integrations/datafusion/src/table/mod.rs
+++ b/crates/integrations/datafusion/src/table/mod.rs
@@ -119,6 +119,12 @@ impl IcebergTableProvider {
statistics,
})
}
+
+ pub async fn with_computed_statistics(mut self) -> Self {
+ let statistics = compute_statistics(&self.table,
self.snapshot_id).await.ok();
+ self.statistics = statistics;
+ self
+ }
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
