GitHub user n-badtke-cg created a discussion: WASB remote logging - upload error - error logging and retry mechanism
Hi, I have a concern regarding remote logging. ## The Setup: In our case, we are using Azure Blob Storage (WASB) for the remote logging destination. Some details regarding our setup: - OS: - Azure SKU OS: Ubuntu 22.04 / Kubernetes 1.32.4 / Airflow Container: Debian 12 (Bookworm) - Airflow Version 3.1.7 - Provider Version: apache-airflow-providers-microsoft-azure==12.10.3 ## My Concern: Currently, when the upload to the WASB is unsuccessful, the error log is not really helpful and the upload will not be retried. See [providers/microsoft/azure/src/airflow/providers/microsoft/azure/log/wasb_task_handler.py#L170](https://github.com/apache/airflow/blob/baa8c72036920f67d8b7479685dadc2c73da0739/providers/microsoft/azure/src/airflow/providers/microsoft/azure/log/wasb_task_handler.py#L170). In my opinion, when the upload to the WASB is unsuccessful, the error log should log the specific error message and the upload should be retried at a given time. I don't know if this is only a provider-specific behavior. GitHub link: https://github.com/apache/airflow/discussions/62009 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
