ahipp13 opened a new issue, #54413: URL: https://github.com/apache/airflow/issues/54413
### Apache Airflow Provider(s) microsoft-azure ### Versions of Apache Airflow Providers apache-airflow @ [file:///home/build/dist/apache_airflow-3.0.4-py3-none-any.whl](file://home/build/dist/apache_airflow-3.0.4-py3-none-any.whl) apache-airflow-core==3.0.4 apache-airflow-providers-amazon @ [file:///home/build/providers/amazon/dist/apache_airflow_providers_amazon-9.7.0-py3-none-any.whl](file://home/build/providers/amazon/dist/apache_airflow_providers_amazon-9.7.0-py3-none-any.whl) apache-airflow-providers-apache-kafka==1.10.2 apache-airflow-providers-celery @ [file:///home/build/providers/celery/dist/apache_airflow_providers_celery-3.10.6-py3-none-any.whl](file://home/build/providers/celery/dist/apache_airflow_providers_celery-3.10.6-py3-none-any.whl) apache-airflow-providers-cncf-kubernetes @ [file:///home/build/providers/cncf/kubernetes/dist/apache_airflow_providers_cncf_kubernetes-10.4.2-py3-none-any.whl](file://home/build/providers/cncf/kubernetes/dist/apache_airflow_providers_cncf_kubernetes-10.4.2-py3-none-any.whl) apache-airflow-providers-common-compat==1.7.3 apache-airflow-providers-common-io==1.6.2 apache-airflow-providers-common-sql==1.27.5 apache-airflow-providers-docker @ [file:///home/build/providers/docker/dist/apache_airflow_providers_docker-4.3.1-py3-none-any.whl](file://home/build/providers/docker/dist/apache_airflow_providers_docker-4.3.1-py3-none-any.whl) apache-airflow-providers-elasticsearch @ [file:///home/build/providers/elasticsearch/dist/apache_airflow_providers_elasticsearch-6.2.2-py3-none-any.whl](file://home/build/providers/elasticsearch/dist/apache_airflow_providers_elasticsearch-6.2.2-py3-none-any.whl) apache-airflow-providers-fab @ [file:///home/build/providers/fab/dist/apache_airflow_providers_fab-2.2.0-py3-none-any.whl](file://home/build/providers/fab/dist/apache_airflow_providers_fab-2.2.0-py3-none-any.whl) apache-airflow-providers-ftp @ [file:///home/build/providers/ftp/dist/apache_airflow_providers_ftp-3.12.3-py3-none-any.whl](file://home/build/providers/ftp/dist/apache_airflow_providers_ftp-3.12.3-py3-none-any.whl) apache-airflow-providers-google @ [file:///home/build/providers/google/dist/apache_airflow_providers_google-15.1.0-py3-none-any.whl](file://home/build/providers/google/dist/apache_airflow_providers_google-15.1.0-py3-none-any.whl) apache-airflow-providers-grpc @ [file:///home/build/providers/grpc/dist/apache_airflow_providers_grpc-3.7.3-py3-none-any.whl](file://home/build/providers/grpc/dist/apache_airflow_providers_grpc-3.7.3-py3-none-any.whl) apache-airflow-providers-hashicorp @ [file:///home/build/providers/hashicorp/dist/apache_airflow_providers_hashicorp-4.1.1-py3-none-any.whl](file://home/build/providers/hashicorp/dist/apache_airflow_providers_hashicorp-4.1.1-py3-none-any.whl) apache-airflow-providers-http @ [file:///home/build/providers/http/dist/apache_airflow_providers_http-5.2.2-py3-none-any.whl](file://home/build/providers/http/dist/apache_airflow_providers_http-5.2.2-py3-none-any.whl) apache-airflow-providers-imap @ [file:///home/build/providers/imap/dist/apache_airflow_providers_imap-3.8.3-py3-none-any.whl](file://home/build/providers/imap/dist/apache_airflow_providers_imap-3.8.3-py3-none-any.whl) apache-airflow-providers-microsoft-azure @ [file:///home/build/providers/microsoft/azure/dist/apache_airflow_providers_microsoft_azure-12.3.1-py3-none-any.whl](file://home/build/providers/microsoft/azure/dist/apache_airflow_providers_microsoft_azure-12.3.1-py3-none-any.whl) apache-airflow-providers-mysql @ [file:///home/build/providers/mysql/dist/apache_airflow_providers_mysql-6.2.1-py3-none-any.whl](file://home/build/providers/mysql/dist/apache_airflow_providers_mysql-6.2.1-py3-none-any.whl) apache-airflow-providers-odbc @ [file:///home/build/providers/odbc/dist/apache_airflow_providers_odbc-4.9.2-py3-none-any.whl](file://home/build/providers/odbc/dist/apache_airflow_providers_odbc-4.9.2-py3-none-any.whl) apache-airflow-providers-openlineage @ [file:///home/build/providers/openlineage/dist/apache_airflow_providers_openlineage-2.1.3-py3-none-any.whl](file://home/build/providers/openlineage/dist/apache_airflow_providers_openlineage-2.1.3-py3-none-any.whl) apache-airflow-providers-postgres @ [file:///home/build/providers/postgres/dist/apache_airflow_providers_postgres-6.1.3-py3-none-any.whl](file://home/build/providers/postgres/dist/apache_airflow_providers_postgres-6.1.3-py3-none-any.whl) apache-airflow-providers-redis @ [file:///home/build/providers/redis/dist/apache_airflow_providers_redis-4.0.2-py3-none-any.whl](file://home/build/providers/redis/dist/apache_airflow_providers_redis-4.0.2-py3-none-any.whl) apache-airflow-providers-sendgrid @ [file:///home/build/providers/sendgrid/dist/apache_airflow_providers_sendgrid-4.0.1-py3-none-any.whl](file://home/build/providers/sendgrid/dist/apache_airflow_providers_sendgrid-4.0.1-py3-none-any.whl) apache-airflow-providers-sftp @ [file:///home/build/providers/sftp/dist/apache_airflow_providers_sftp-5.2.0-py3-none-any.whl](file://home/build/providers/sftp/dist/apache_airflow_providers_sftp-5.2.0-py3-none-any.whl) apache-airflow-providers-slack @ [file:///home/build/providers/slack/dist/apache_airflow_providers_slack-9.0.4-py3-none-any.whl](file://home/build/providers/slack/dist/apache_airflow_providers_slack-9.0.4-py3-none-any.whl) apache-airflow-providers-smtp==2.2.0 apache-airflow-providers-snowflake==6.5.2 apache-airflow-providers-sqlite @ [file:///home/build/providers/sqlite/dist/apache_airflow_providers_sqlite-4.0.2-py3-none-any.whl](file://home/build/providers/sqlite/dist/apache_airflow_providers_sqlite-4.0.2-py3-none-any.whl) apache-airflow-providers-ssh @ [file:///home/build/providers/ssh/dist/apache_airflow_providers_ssh-4.0.1-py3-none-any.whl](file://home/build/providers/ssh/dist/apache_airflow_providers_ssh-4.0.1-py3-none-any.whl) apache-airflow-providers-standard==1.5.0 apache-airflow-task-sdk==1.0.4 google-cloud-orchestration-airflow==1.17.5 ### Apache Airflow version 3.0.4 ### Operating System bash-5.3$ cat /etc/os-release ID=chainguard NAME="Chainguard" PRETTY_NAME="Chainguard" VERSION_ID="20230214" HOME_URL="https://chainguard.dev/"; ### Deployment Official Apache Airflow Helm Chart ### Deployment details Deploying helm chart on to Kubernetes version v1.32.4-gke.200. For remote logging I have these environment variables set: ``` # Remote Logging - name: "AIRFLOW__LOGGING__REMOTE_LOGGING" value: "True" - name: "AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER" value: "wasb://airflow-logs@ACCOUNT_NAME.blob.core.windows.net/wasb-airflow-logs" - name: "AIRFLOW__AZURE_REMOTE_LOGGING__REMOTE_WASB_LOG_CONTAINER" value: "airflow-logs" - name: "AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID" value: "edw_az_conn" - name: "AIRFLOW__LOGGING__DELETE_LOCAL_LOGS" value: "True" ``` And I have this connection created: <img width="867" height="779" alt="Image" src="https://github.com/user-attachments/assets/4bbda1a1-5445-46a0-bb42-84294480c1f2"; /> <img width="858" height="758" alt="Image" src="https://github.com/user-attachments/assets/526a3a6b-d3c0-480d-9d15-689738634dd2"; /> ### What happened Whenever I run one of my DAGs, it will complete successfully, but then when it tries to write the logs it will give an error that says "The conn_id `edw_az_conn` isn't defined" when it clearly is as you can see from the screenshots above. Full log below ``` {"event":"Could not write logs to wasb-airflow-logs/dag_id=assettracking_sand_meili_namespaces_dag/run_id=manual__2025-08-12T13:54:52.960501+00:00/task_id=process_namespa │ │ ces_sand/attempt=1.log","level":"error","logger":"airflow.providers.microsoft.azure.log.wasb_task_handler.WasbRemoteLogIO","timestamp":"2025-08-12T13:55:53.482145Z","exce │ │ ption":[{"exc_type":"AirflowNotFoundException","exc_value":"The conn_id `edw_az_conn` isn't defined","exc_notes":[],"syntax_error":null,"is_cause":false,"frames":[{"filen │ │ ame":"/opt/airflow/lib/python3.12/site-packages/airflow/providers/microsoft/azure/log/wasb_task_handler.py","lineno":170,"name":"write"},{"filename":"/opt/airflow/lib/pyt │ │ hon3.12/site-packages/airflow/providers/microsoft/azure/hooks/wasb.py","lineno":359,"name":"load_string"},{"filename":"/opt/airflow/lib/python3.12/site-packages/airflow/p │ │ roviders/microsoft/azure/hooks/wasb.py","lineno":416,"name":"upload"},{"filename":"/opt/airflow/lib/python3.12/site-packages/airflow/providers/microsoft/azure/hooks/wasb. │ │ py","lineno":231,"name":"_get_blob_client"},{"filename":"/usr/lib/python3.12/functools.py","lineno":998,"name":"__get__"},{"filename":"/opt/airflow/lib/python3.12/site-pa │ │ ckages/airflow/providers/microsoft/azure/hooks/wasb.py","lineno":156,"name":"blob_service_client"},{"filename":"/opt/airflow/lib/python3.12/site-packages/airflow/provider │ │ s/microsoft/azure/hooks/wasb.py","lineno":160,"name":"get_conn"},{"filename":"/opt/airflow/lib/python3.12/site-packages/airflow/hooks/base.py","lineno":64,"name":"get_con │ │ nection"},{"filename":"/opt/airflow/lib/python3.12/site-packages/airflow/models/connection.py","lineno":512,"name":"get_connection_from_secrets"}],"is_group":false,"excep │ │ tions":[]}]} ``` ### What you think should happen instead It should log to the blob storage like normal. I have this same configuration running in Airflow 2.11 ### How to reproduce Would need to set up Azure remote logging with a storage account and use the airflow provider. ### Anything else I brought up this issue here: https://github.com/apache/airflow/issues/54192 I was told upgrading to 3.0.4 would fix it, but it is still occurring to me. Another note, I know it is not the connection that is broken, because I can read logs in the blob storage before I upgraded the environment... ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
