Kenny1217 commented on issue #52501:
URL: https://github.com/apache/airflow/issues/52501#issuecomment-3070999712
I think I am having a bit of the same issue with remote logging to Minio S3.
The tasks are completing successfully but aren't saving the logs to S3. We are
running Airflow in k8s using the KubernetesExecutor. I have an aws connection
named Minio for our on prem S3 instance. I have the connection using the extra
fields json section since we have to point to Minio endpoint and not AWS. This
same setup works fine in 2.9.3 instance of Airflow but doesn't in 3.0.2. What
is weird is that if I create a task that uses S3Hook it has no issue finding
the Minio connection when looking at the logs on the k8s pods. The pod also
then gives the same error about not the connection id.
```
{"timestamp":"2025-07-14T19:56:48.737458","level":"info","event":"Filling up
the DagBag from
/usr/local/airflow/dags/test_dag.py","logger":"airflow.models.dagbag.DagBag"}
{"timestamp":"2025-07-14T19:56:49.741806Z","level":"info","event":"hello
world!","chan":"stdout","logger":"supervisor"}
{"timestamp":"2025-07-14T19:56:49.741424","level":"info","event":"Connection
Retrieved 'Minio'","logger":"airflow.hooks.base"}
{"timestamp":"2025-07-14T19:56:49.741805","level":"info","event":"AWS
Connection (conn_id='Minio', conn_type='aws') credentials retrieved from login
and
password.","logger":"airflow.providers.amazon.aws.utils.connection_wrapper.AwsConnectionWrapper"}
{"timestamp":"2025-07-14T19:56:50.676600Z","level":"info","event":"['test/test.csv']","chan":"stdout","logger":"supervisor"}
{"timestamp":"2025-07-14T19:56:50.678866","level":"info","event":"Done.
Returned value was:
None","logger":"airflow.task.operators.airflow.providers.standard.operators.python.PythonOperator"}
{"event":"Unable to find AWS Connection ID 'Minio', switching to
empty.","level":"warning","logger":"airflow.task.hooks.airflow.providers.amazon.aws.hooks.s3.S3Hook","timestamp":"2025-07-14T19:56:50.733131Z"}
{"event":"No connection ID provided. Fallback on boto3 credential strategy
(region_name=None). See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html","level":"info","logger":"airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory","timestamp":"2025-07-14T19:56:50.733774Z"}
{"event":"Could not verify previous log to
append","level":"error","logger":"airflow.providers.amazon.aws.log.s3_task_handler.S3RemoteLogIO","timestamp":"2025-07-14T19:56:53.629692Z","exception":[{"exc_type":"NoCredentialsError","exc_value":"Unable
to locate
credentials","exc_notes":[],"syntax_error":null,"is_cause":false,"frames":[{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/log/s3_task_handler.py","lineno":120,"name":"write"},{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/log/s3_task_handler.py","lineno":81,"name":"s3_log_exists"},{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/hooks/s3.py","lineno":153,"name":"wrapper"},{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/hooks/s3.py","lineno":126,"name":"wrapper"},{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/hooks/s3.py","lineno":953,"name":"check_for_key"},
{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/hooks/s3.py","lineno":153,"name":"wrapper"},{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/hooks/s3.py","lineno":126,"name":"wrapper"},{"filename":"/usr/local/lib/python3.12/site-packages/airflow/providers/amazon/aws/hooks/s3.py","lineno":934,"name":"head_object"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/client.py","lineno":569,"name":"_api_call"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/client.py","lineno":1005,"name":"_make_api_call"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/client.py","lineno":1029,"name":"_make_request"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/endpoint.py","lineno":119,"name":"make_request"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/endpoint.py","lineno":196,"name":"_send_request"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/e
ndpoint.py","lineno":132,"name":"create_request"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/hooks.py","lineno":412,"name":"emit"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/hooks.py","lineno":256,"name":"emit"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/hooks.py","lineno":239,"name":"_emit"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/signers.py","lineno":106,"name":"handler"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/signers.py","lineno":198,"name":"sign"},{"filename":"/usr/local/lib/python3.12/site-packages/botocore/auth.py","lineno":424,"name":"add_auth"}],"is_group":false,"exceptions":[]}]}
{"timestamp":"2025-07-14T19:56:53.630904Z","level":"info","event":"Task
finished","exit_code":0,"duration":6.419133752002381,"final_state":"success","logger":"supervisor"}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
