hashar added a comment.
I was watching integration-agent-docker-1004 which had 3 builds involving Quibble. I have opened pages using the Jenkins monitoring interface that lets one list processes and java threads. As soon as one showed the delay I refreshed the two pages. On the list of processes at https://integration.wikimedia.org/ci/monitoring/nodes/integration-agent-docker-1004?part=processes I see: - three `docker run` corresponding to the three builds. - a bunch of old `docker run --rm --init -v /srv/jenkins/workspace/workspace:/workspaces docker-registry.wikimedia.org/releng/ci-stretch find /workspaces -mindepth 1 -maxdepth 1 -type d -exec du -s {} ;`. They come from a maintenance job written in Groovy used to collect the disk space used by workspaces. - containerd only has TWO containerd-shim So the containerized process has completed, it is no more present in the process table but somehow containerd hasn't noticed it, docker run did not receive the notification or ignored it. Which well ... sounds like a Docker bug? We use Debian Stretch with: apt-cache madison docker-ce containerd.io | docker-ce | 5:19.03.5~3-0~debian-stretch | http://apt.wikimedia.org/wikimedia stretch-wikimedia/thirdparty/ci amd64 Packages | | containerd.io | 1.2.10-3 | http://apt.wikimedia.org/wikimedia stretch-wikimedia/thirdparty/ci amd64 Packages | | TASK DETAIL https://phabricator.wikimedia.org/T265615 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hashar Cc: thcipriani, hashar, awight, kostajh, Lucas_Werkmeister_WMDE, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Liudvikas, Scott_WUaS, Wikidata-bugs, aude, Jdforrester-WMF, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
