aaron-y-chen opened a new pull request, #70504: URL: https://github.com/apache/airflow/pull/70504
closes: #70266 related: #70265 related: #67056 ## Summary Migrate the `gs` remote logging scheme to the provider dispatch introduced by #67056: - add `GCSRemoteLogIO.from_config()`, preserving the legacy GCS config mapping and IO kwargs; - register `gs` in the Google provider metadata generated from `provider.yaml`. Core and the Task SDK can now resolve GCS logging through `ProvidersManager`. The existing legacy path remains as a compatibility fallback. ## Validation Tested against a real GCS bucket from Breeze using forwarded user ADC and: ```ini remote_logging = True remote_base_log_folder = gs://airflow-70266-<random>/airflow-70266 delete_local_logs = True ``` No `remote_log_conn_id` or `google_key_path` was set, so the handler used ADC. - `gs` resolved to `airflow.providers.google.cloud.log.gcs_task_handler.GCSRemoteLogIO`; - the task emitted `AIRFLOW_70266_GCS_REMOTE_LOG_E2E_OK`, and `gcloud storage cat` returned the same marker from `attempt=1.log`; - the corresponding local log no longer existed after upload; - the Airflow UI reported the `gs://.../attempt=1.log` source and displayed the marker. Airflow UI reading the task log from GCS after local cleanup: <img width="1431" height="533" alt="70266_1" src="https://github.com/user-attachments/assets/4dbd0bc6-1d49-4c29-aa53-ed0e7e3084eb"; /> The uploaded `attempt=1.log` in the real GCS bucket: <img width="1431" height="591" alt="70266_2" src="https://github.com/user-attachments/assets/bdcaba81-e3e7-4662-9cb7-ea9e028e4629"; /> --- ##### Was generative AI tooling used to co-author this PR? <!-- If generative AI tooling has been used in the process of authoring this PR, please change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by". --> - [x] Yes (please specify the tool below) Generated-by: [GPT 5.6 Sol] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
