alanbrent edited a comment on issue #8212:
URL: https://github.com/apache/airflow/issues/8212#issuecomment-655216002
Can confirm that using a threaded `gunicorn` worker (`gevent` in our case)
breaks the web component's ability to show task logs that are in S3. Moving
back to `sync` works for us, and since we're running on Kubernetes a few more
web pods to absorb the ALB/ELB health checks isn't (so far) a performance
concern.
Would love to see this fixed, however.
<details><summary>Using Airflow default <tt>aws_default</tt> for the S3
connection. Relevant configuration shared among web, scheduler, and worker pods
below the fold</summary>
```
- AIRFLOW__CORE__REMOTE_LOGGING=True
- AIRFLOW__CORE__REMOTE_LOG_CONN_ID=aws_default
- AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER=s3://${OUR-BUCKET}/logs
- AIRFLOW__WEBSERVER__WORKER_CLASS=sync
- AIRFLOW__WEBSERVER__WORKER_REFRESH_INTERVAL=3600 # the default 30s
worker lifetime is *way* too short
- AWS_DEFAULT_REGION=us-east-1
-
AWS_ROLE_ARN=arn:aws:iam::${OUR_AWS_ACCOUNT_ID}:role/${OUR_AIRFLOW_APP_ROLE}
-
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
# This is how EKS does IAM<>Pod stuff
```
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
