pankajastro commented on code in PR #74228:
URL: https://github.com/apache/airflow/pull/74228#discussion_r4207231009
##########
providers/common/sql/src/airflow/providers/common/sql/datafusion/engine.py:
##########
@@ -80,9 +80,13 @@ def _register_object_store(
object_store = storage_provider.create_object_store(
datasource_config.uri, connection_config=connection_config
)
- schema = storage_provider.get_scheme()
- self.session_context.register_object_store(schema=schema,
store=object_store)
- self.log.info("Registered object store for schema: %s", schema)
+ schema = storage_provider.get_scheme(datasource_config.uri)
+ # DataFusion's object-store registry keys on (schema, host);
omitting host only
+ # matches URIs with an empty authority (e.g. file:///path), so a
bucket/container
+ # URI's netloc must be passed explicitly or lookup fails at query
time.
+ host = urlsplit(datasource_config.uri).netloc
Review Comment:
Done — container alone may not be enough: two different accounts sharing a
container name would then collide. Folds in account too:
az://<account>.<container>/<path>.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
