GitHub user chyzouakia edited a comment on the discussion: S3TaskHandler doesn't seem to run the close method and doesn't push the logs to S3
I was able to push the logs from the worker and from local manually. Everything is explained in the description above, I could run the code that pushes the logs manually. I was not able to push the logs to S3 using the S3TaskHandler (with s3 logging configured). I tried checking the logs and couldn't find any error. It feels like you didn't check the `what happened `section above and just dismissed the issue 🙂 All my tasks when trying to read in S3 the logs return `No logs found on s3` : <img width="690" height="95" alt="image" src="https://github.com/user-attachments/assets/bd9023bb-7ac3-49ec-85c2-fa75a35a9500"; /> I don't have any logs pushed into S3 automatically which lead me to believe there is an issue with the `close()` method in S3TaskHandler because : * When I connect to any worker in my cluster, I am able to push logs to the S3 manually using the service account and the S3 remote logging connection in Airflow (The code is in the `what happened` section) * When I push the logs manually to S3 the task is able to read the logs in S3 : <img width="909" height="325" alt="image" src="https://github.com/user-attachments/assets/edc5dfcd-34bd-45b9-8d70-85ac471c1298"; /> GitHub link: https://github.com/apache/airflow/discussions/55402#discussioncomment-14359099 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
