pankajastro opened a new pull request, #73370: URL: https://github.com/apache/airflow/pull/73370
Part of AFL-214's second defect (DataFusion credential resolution only handled AWS, so GCS/Azure got nothing even though DataFusion's storage layer supports them). This PR covers GCS; Azure is left for a separate follow-up. `DataFusionEngine._get_credentials` only had a `case "aws":` arm, so a GCS-backed `DataSourceConfig` failed with `Unknown connection type google_cloud_platform` at registration, even though nothing prevented constructing one (`gs://` was never recognized by `_extract_storage_type` either). Adds: - `StorageType.GCS` and `gs://` URI recognition in `DataSourceConfig` - `GCSObjectStorageProvider` using DataFusion's `GoogleCloud` object store binding - A `google_cloud_platform` branch in `_get_credentials` resolving the connection's `key_path` field, falling back to Application Default Credentials when unset (DataFusion's `GoogleCloud` binding only accepts a `service_account_path`, not inline JSON credentials, so `keyfile_dict`-based connections aren't covered — ADC is the fallback for those) - `google` extra in `pyproject.toml`, docs updated to mention GCS alongside S3 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Sonnet 5) Generated-by: Claude Code (Sonnet 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
