spoutin commented on PR #41531: URL: https://github.com/apache/airflow/pull/41531#issuecomment-2408945693
unless I'm missing something, the docker sdk function to `get_archive` is only available on the the workers that hosted the docker container since it uses the container ID. If you have a multi node setup how are you able to retrieve the logs if the airflow worker that executed the docker container is not on the same docker swarm worker node? i also noticed that you're calling the `inspect_container`, this has the same limitation; you can only inspect containers that are on the same worker. The docker "tasks" APIs are what we should be using to access Docker Services tasks not the "container" APIs. What's confusing me is that this was tested but I'm not sure how. Unless I'm confused about the Docker Swarm APIs or there is some form of a Docker API gateway / proxy that is routing the docker APIs to the correct workers I'm not sure how this is working. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
