vincbeck commented on issue #33634: URL: https://github.com/apache/airflow/issues/33634#issuecomment-1691801084
As explained in #33231: > Fetching logs from AWS CloudWatch can take a lot of times or even fail when the log stream is old and no time boundary is specified when querying CloudWatch. > > Example: if you try to look at the logs of an old task on the Airflow UI, it can be very slow or even fail doing so. > > By setting start_time and end_time (end_time is the most important), it improves drastically the latency. CloudWatch team recommended adding end time for performance to restrict the search space. I tested it with a 10 days old task and I could experience an improvement from 5 seconds to 1 second when fetching task logs from the UI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
