killua1zoldyck commented on issue #31105: URL: https://github.com/apache/airflow/issues/31105#issuecomment-1604974487
@potiuk The current log-streaming logic pulls the whole file into memory before moving to the specific log_pos based on the metadata after combining all the logs and sorting them based on timestamp. https://github.com/apache/airflow/blob/24e3d6ce57eae1784066ed5678369e61637285a4/airflow/utils/log/file_task_handler.py#L334-L350 So, I don't think we can change the logic to make it streaming. Is my thinking correct? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
