NicoMovh opened a new issue, #51775: URL: https://github.com/apache/airflow/issues/51775
### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.10.5 ### What happened? We are experiencing slowness and timeouts of some when fetching streaming job's logs from S3 through the WebUI on our production instance. The issue is intermittant, sometimes it works correctly, sometimes it's slow and sometimes it fails with an Internal Server Error, but there's no error logs making thoubleshooting difficult. It seems to occurs more often when the logs are large but it can still affect smaller logs. The issue does not occur at all on our development airflow instance, which is otherwise identical, it's possibly due to smaller logs size. ### What you think should happen instead? Regardless of their size, logs should be fetched consistenly from S3 without significant delays. And if any issues with the logs fetching occurs, it should be captured by the Webserver. ### How to reproduce 1)Set up Airflow 2.10.5 with S3 storage for logs 2)Create a DAG tasks that generate large logs 3)Try to fetch logs through the Web UI. 4) Issue is intermittent, sometimes logs fetch fine, and other time there's slowness/timeout or internal error. ### Operating System Kubernetes-based deployment based on airflow:slim ### Versions of Apache Airflow Providers _No response_ ### Deployment Official Apache Airflow Helm Chart ### Deployment details _No response_ ### Anything else? There's no issue with network conectivity or S3 permissions. Pods have more than sufficiant CPU and memory allocation. Timeout configuration have been increased, but it did now show significative improvement. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
