NoahKusaba opened a new issue, #28: URL: https://github.com/apache/datafusion-iceberg/issues/28
### Is your feature request related to a problem? Distributed engines such as Ballista send physical plans to other processes through a datafusion-proto `PhysicalExtensionCodec`. To serialize a node, the codec names its type, reads what it was built from, and rebuilds an equivalent node on the other side from those parts alone. None of that is possible for the Iceberg plan nodes from outside this crate: - `IcebergCommitExec` and `IcebergWriteExec` are `pub(crate)`, and the modules of all four nodes are private. - None of the nodes expose the table, catalog or provider they hold. - `IcebergTableScan` can only be built from DataFusion filters, which can't be recovered from the Iceberg `Predicate` it pushes down. - `IcebergTableProvider::try_new` and `IcebergMetadataTableProvider`'s fields are crate-private. ### Describe the solution you'd like Export the plan nodes, add read-only accessors for what each holds, and give each a public constructor that takes only what its accessors return, so a codec can rebuild any node from its accessors alone. This was first outlined on apache/iceberg-rust#2862. Part of #23. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
