ZENOTME commented on issue #242:
URL: https://github.com/apache/iceberg-rust/issues/242#issuecomment-2012508866
Thanks for raising this discussion @marvinlanhenke! The basic idea for the
integration is to provide the wrap struct using type in iceberg-rust so that
users can use them to connect with datafusion directly.
## Implementation outline
### 1. Implement trait for managing the iceberg table.
The datafusion provides the following trait to manage the table:
- CatalogProviderList
- CatalogProvider
- SchemaProvider
- TableProvider
We can map them into the type in iceberg-rs
- CatalogProviderList: Maybe we don't need to implement this
- CatalogProvider: Catalog in iceberg-rust
- SchemaProvider: Namespace in iceberg-rust
- TableProvider: Table in iceberg-rust
We can implement them by wrapping using type in iceberg-rs internally.
Like
```
struct IcebergCatalogProvider {
inner: iceberg_rs::Catalog
}
impl CatalogProvider for IcebergCatalogProvider {
...
}
```
### 2. Implement the trait for scanning the table.
And we also need to implement an ExecutionPlan for [scan in
TableProvider](https://docs.rs/datafusion/latest/datafusion/datasource/provider/trait.TableProvider.html#tymethod.scan).
This part we can rely on [TableScan in
iceberg-rs](https://github.com/apache/iceberg-rust/blob/39aafdd2ea69968213e94534b5864fd595a6034a/crates/iceberg/src/scan.rs#L186)
Feel free to any suggestions and if something can be improved. Please let me
know if there is something confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
