jscheffl commented on issue #44753: URL: https://github.com/apache/airflow/issues/44753#issuecomment-2525071649
I have also espcially seen that the webserver gets OOM even if you download because the FileTaskHandler tries to sort+merge different log sources. One important thing is not only the download but also the FileTaskHandler must 1:1 stream the logs from the backend, else it will go OOM as well. This included when I took a look last time... why actually log sorting and merging is implemented in general I doubt the usefulness because I also had situations where log sorting by timestamp gave "strange" results. In my view logs should not be modified/sorted/merged when served from webserver. This would also remove the OOM problem in general... whereas I have seen situations as well where the browser/client "dies" in rendering. So a limit in logs provided to browser might be a important feature as well (but please not like Github is doing it) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
