djouallah opened a new pull request, #1357: URL: https://github.com/apache/datafusion-python/pull/1357
## Summary This PR adds support for the `use_fabric_endpoint` parameter to the `MicrosoftAzure` object store class, enabling connections to Microsoft Fabric OneLake storage. Fixes #1356 ## Problem The Python `MicrosoftAzure` class was missing the `use_fabric_endpoint` parameter that exists in the underlying Rust `object_store` crate. This prevented users from connecting to Microsoft Fabric OneLake storage, which requires Data Lake Storage Gen2 endpoints instead of Azure Blob Storage endpoints. **Current behavior**: Defaults to `https://onelake.blob.core.windows.net/...` **Required behavior**: Support `https://onelake.dfs.fabric.microsoft.com/...` ## Changes Added the `use_fabric_endpoint` parameter to the `MicrosoftAzure` class in `src/store.rs`: 1. Added parameter to PyO3 signature macro (line 79) 2. Added parameter to function signature as `Option<bool>` (line 92) 3. Added conditional builder chain call to `with_use_fabric_endpoint()` (lines 130-132) This implementation follows the exact same pattern as existing boolean parameters (`use_emulator`, `allow_http`). ## Usage Example ```python from datafusion.object_store import MicrosoftAzure from datafusion import SessionContext import os # Create OneLake store with Fabric endpoint onelake_store = MicrosoftAzure( container_name="tpch", account='onelake', bearer_token=os.environ["AZURE_STORAGE_TOKEN"], use_fabric_endpoint=True # NEW PARAMETER ) # Register and use ctx = SessionContext() ctx.register_object_store("abfss://[email protected]/storage.Lakehouse", onelake_store, None) ``` ## Testing Note Comprehensive end-to-end testing of this feature requires: - An active Microsoft Fabric workspace - OneLake storage with data - Valid authentication tokens for OneLake access I have manually verified that: - The parameter is accepted without errors - The object store can be successfully registered with a SessionContext - Queries against OneLake storage work correctly with `use_fabric_endpoint=True` The code change is minimal and follows existing patterns exactly, making it low-risk for existing functionality. ## Checklist - [x] Follows existing code patterns - [x] Minimal, focused change - [x] Manually tested with OneLake - [x] No new dependencies required -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
