GitHub user cheeyeelim edited a comment on the discussion: Issue
troubleshooting an s3 connection for remote logging to Min.io
Other S3-compatible storage like Digital Ocean Spaces works with a similar
configuration as well. Below is the configuration I used, all set as
environment variables (for ease of k8s deployment).
`
AIRFLOW__LOGGING__REMOTE_LOGGING=True
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER='s3://<BUCKET_NAME>/logs'
AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID='s3_do_conn'
AIRFLOW_CONN_S3_DO_CONN='{
"conn_type": "aws",
"login": "<ACCESS_KEY>",
"password": "<SECRET_ACCESS_KEY>",
"extra": {
"region_name": "<REGION_NAME>",
"endpoint_url": "https://<REGION_NAME>.digitaloceanspaces.com"
}
}'
`
It took me a few hours to figure this out. Similarly I was stumped by constant
testing errors from `airflow connections test`. Getting errors from the test
does not mean that the connection will not work for remote logging.
GitHub link:
https://github.com/apache/airflow/discussions/26979#discussioncomment-13765204
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
