NoahKusaba commented on code in PR #2784:
URL: 
https://github.com/apache/datafusion-ballista/pull/2784#discussion_r4648079387


##########
integrations/iceberg/src/catalog.rs:
##########
@@ -109,18 +125,20 @@ impl SchemaProvider for PinningSchema {
         let Some(table) = self.0.table(name).await? else {
             return Ok(None);
         };
-        let Some(provider) = 
table.downcast_ref::<IcebergCatalogTableProvider>() else {
-            return Ok(Some(table));
+        if let Some(provider) = 
table.downcast_ref::<IcebergCatalogTableProvider>() {
+            let table = load(provider).await;

Review Comment:
   Agreed, that's the common case. #2885 wouldn't reach it yet, though. 
ballista-iceberg pins ballista-core at a DataFusion 55 commit until 
datafusion-iceberg supports 56, and the composed codec would still flatten the 
cause.
   
   So register_iceberg now adds an analyzer rule, RequireLoadedTables. It fails 
a query that reads a table its catalog couldn't load, with the load error, 
while the query is planned. Direct reads, view reads and subqueries, with or 
without a filter, now all fail before anything is encoded:
   
   require_loaded_iceberg_tables
   caused by
   IcebergError(TableNotFound)
   caused by
   Execution error: TableNotFound => No such table: …
   
   DataFusion doesn't analyze DDL or DESCRIBE, so dropping, replacing and 
describing still work while a table is gone, and #2882 stays fixed. 
   
   The rule is a stopgap with a TODO to remove it once #2881 lands, since a 
table that can't be loaded will then fail the query where it's loaded. 
should_fail_to_read_a_table_that_is_gone now reads t directly, through a view 
and in a subquery, each with and without a filter.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to