[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2024-03-25 Thread Adam Saponara
Following up, `Acquire::Retries=0` seems to be avoiding the bug in prod, which strengthens my suspicion that it's related to retries. As for the actual bug, when apt is hanging, I notice that the work queue has items in it, but they have not been delegated to a worker. There is a condition that

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2024-03-21 Thread Adam Saponara
Hello, also affected by this. I'm able to reproduce the bug using Walter's mock server. I suspect it may be related to the retry code that kicks in when a transient error is encountered (like a 503). Retries are enabled by default since apt 2.3.2, which seems to fit the regression window people

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-10-13 Thread anarcat
We're seeing a similar issue here. At first we thought it was an issue specific to a prometheus collector (https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=1028212 / https://github.com/prometheus- community/node-exporter-textfile-collector-scripts/issues/179) but now that I see this bug report,

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-10-04 Thread Jay D.R.
FYI, ran into this same issue with Debian bookworm's apt-get (2.6.1) hanging indefinitely talking to an apt-cacher-ng 3.7.x instance, easily reproducible by having ansible run an `apt-get update` on ~20-25 machines simultaneously. Problem disappears when switching back to apt- cacher-ng 3.6.x, so

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-07-21 Thread Alexey Gavrilov
I suggest a temporary solution for cron . wrap apt update in timeout -k 60 1200 or add to systemd unit add JobTimeoutSec=1200 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu.

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-05-02 Thread Walter
Hi Julian, > [There] isn't much we can do otherwise, this highly depends on network specifics like latency and mtus and is never reliably reproducible. So, with my bogus python PoC server I mentioned in the comments, it is quite easily reproducible. If I can assist you in getting it up and

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-04-24 Thread Ken Sharp
** Tags added: jammy -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/2003851 Title: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04 Status in apt

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-02-23 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: apt (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu.

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-02-23 Thread Walter
Relevant bug in apt-cacher-ng: https://bugs.launchpad.net/ubuntu/+source/apt-cacher-ng/+bug/1983856 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/2003851 Title: occasional

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-02-23 Thread Walter
Got some more pcaps. I can also reproduce by sending a 503 after a 200, without aborting during/before body transfer (*). server.py logs 13:29:55 [3666956] client said: b'GET /ubuntu/dists/jammy/InRelease HTTP/1.1\r\nHost:

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-02-21 Thread Walter
Okay. I got some pcaps now. And apparently our apt-cacher-ng proxy sometimes serves: - an InRelease - with a 200 OK - and a non-zero Content-Length - BUT, no content, but instead a TCP-FIN. Under the right circumstances, this causes apt to stall forever. The right circumstances seem to include:

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-02-10 Thread Walter
Checking in. I've had an `apt-get update` running in a while loop since Feb 6 on a machine: no luck getting it to stall. Halfway, I added a `iptables -A INPUT -m statistic --mode random --probability 0.05 -j DROP` but this did not affect anything. But, in the mean time, on production I have had

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-01-25 Thread Walter
Thanks for the reply. Julian. Let's assume that the problem is indeed latency/dropped packets/whatever on our side. IMO, an (occasionally) broken network should not cause apt- get to hang indefinitely. Do you think it should? Also, it doesn't address that the behaviour seems recent. We have not

[Touch-packages] [Bug 2003851] Re: occasional hanging 'apt-get update' from daily cronjob since Jammy 22.04

2023-01-25 Thread Julian Andres Klode
Patches are welcome but unfortunately there isn't much we can do otherwise, this highly depends on network specifics like latency and mtus and is never reliably reproducible. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to