CTTY commented on code in PR #1972:
URL: https://github.com/apache/iceberg-rust/pull/1972#discussion_r2684168560
##########
crates/integrations/datafusion/src/schema.rs:
##########
@@ -127,7 +139,61 @@ impl SchemaProvider for IcebergSchemaProvider {
Ok(self
.tables
.get(name)
- .cloned()
- .map(|t| t as Arc<dyn TableProvider>))
+ .map(|entry| entry.value().clone() as Arc<dyn TableProvider>))
+ }
+
+ fn register_table(
+ &self,
+ name: String,
+ table: Arc<dyn TableProvider>,
+ ) -> DFResult<Option<Arc<dyn TableProvider>>> {
+ // Convert DataFusion schema to Iceberg schema
+ // DataFusion schemas don't have field IDs, so we use the function
that assigns them automatically
+ let df_schema = table.schema();
+ let iceberg_schema =
arrow_schema_to_schema_auto_assign_ids(df_schema.as_ref())
+ .map_err(to_datafusion_error)?;
+
+ // Create the table in the Iceberg catalog
+ let table_creation = TableCreation::builder()
+ .name(name.clone())
+ .schema(iceberg_schema)
+ .build();
+
+ let catalog = self.catalog.clone();
+ let namespace = self.namespace.clone();
+ let tables = self.tables.clone();
+ let name_clone = name.clone();
+
+ // Use tokio's spawn_blocking to handle the async work on a blocking
thread pool
+ let result = tokio::task::spawn_blocking(move || {
Review Comment:
I think the reason is that we don't know if the context is sync/async. even
though `register_table` is a sync func, another async func may call
`register_table`. `spawn_blocking` is to ensure we execute block_on in a sync
context.
I was referring to this thread:
https://users.rust-lang.org/t/calling-async-from-sync-code/67767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
