1.5 hours of runtime: looks good so far (`5.4.0-132-generic #148+epollcascardo1-Ubuntu`).
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1996678 Title: containerd sporadic timeouts Status in containerd package in Ubuntu: Confirmed Status in linux package in Ubuntu: Invalid Status in containerd source package in Focal: New Status in linux source package in Focal: In Progress Bug description: This morning I routinely upgraded security upgrades on number of machines. Containerd has upgraded from `1.5.9-0ubuntu1~20.04.4` for me `1.5.9-0ubuntu1~20.04.5`. What happened next: at some random time on machines with new containerd something happens with containerd tasks and/or cgroups. This is how it's seen in syslog: containerd[710]: time="2022-11-16T03:21:21.004153517Z" level=error msg="get state for 2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a" error="context deadline exceeded: unknown" And some ctr commands: # ctr --namespace k8s.io task ls|grep 2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a 2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a 0 UNKNOWN See that the status of the task is UNKNOWN (!!!) # ctr --namespace k8s.io container ls|grep 2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a 2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a k8s.gcr.io/pause:3.5 io.containerd.runc.v2 Cgroups: ├─kubepods-besteffort-pod3fdc0061_bbf6_47d1_97f5_b1f271b46e23.slice │ ├─cri-containerd-5f78e0cb957de97fd8465cc42c842bdd764d981ca7a903a2515bbc6bb06796a9.scope … │ │ └─2677 /csi-node-driver-registrar --v=0 --csi-address=/csi/csi.sock --k… │ ├─cri-containerd-af070f16c1f0ff22eb16661e787e85db3810727909abd23d69a6a43578c1dced.scope … │ │ └─3264 /usr/local/bin/cephcsi --type=liveness --endpoint=unix:///csi/cs… │ ├─cri-containerd-4e063ef0c8f768dbf34cf7a179bca5cc98a04fa7e00b29d20c17d3031d409f86.scope … │ │ └─2960 /usr/local/bin/cephcsi --nodeid=nas-decent-bobcat.dev-k8s-1.hq.w… │ └─cri-containerd-2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a.scope … │ └─2414 /pause # ps auxf|grep 2414 -B 2 root 2279 0.1 0.0 114100 4956 ? Sl Nov15 0:42 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id 2f5a8376b476809b1696b140ca87f91422113bb16b27a8174437cc63b48e259a -address /run/containerd/containerd.sock 65535 2414 0.0 0.0 964 4 ? Ss Nov15 0:00 \_ /pause It happens not immediately - but after some random time. Sometimes it's several minutes, sometimes it's around an hour. But nonetheless - all machines with the new package get into this weird state. As long as I revert package - it all returns to run as expected. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1996678/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
