[Bug 1836329] Re: Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS
Uploaded to cosmic and bionic proposed queues, unapproved. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to apache2 in Ubuntu. https://bugs.launchpad.net/bugs/1836329 Title: Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836329/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1836898] [NEW] DEP8 run-test-suite not really running
Public bug reported: Up until 2.4.37-1, the d/t/run-test-suite test isn't running because it's called as root: (...) make[1]: Leaving directory '/tmp/autopkgtest.wAqieG/autopkgtest_tmp/perl-framework/c-modules/eat_post' [ error] Apache cannot spawn child processes as root, therefore the test suite must be run as a non-privileged user. + rm apache2.conf.debian + run_tests mpm_worker (...) This was fixed in 2.4.37-1: * Fix test suite to actually run by creating a test user. It turns out the test suite refuses to run as root but returns true even in that case. It seems this has been broken since 2.4.27-4, where the test suite had been updated and the debci test duration dropped from 15min to 3min. Also, don't rely on the exit status anymore but parse the test output. We should prepare this for bionic, so the next time a bionic SRU comes, it can be incorporated. Note that a patch is introducing a failure in that suite: # Test 2 got: "68" (t/apache/mmn.t at line 42) # Expected: "75" t/apache/mmn.t .. Failed 1/2 subtests The patch that introduces this failure is debian/patches/balance-member-long-hostname-part2.patch, because it adds this comment: --- 2.4.x/include/ap_mmn.h 2018/02/16 14:58:32 1824503 +++ 2.4.x/include/ap_mmn.h 2018/02/16 15:04:41 1824504 @@ -496,6 +496,7 @@ * to ap_[r]getline() * 20120211.68 (2.4.26-dev) Add ap_get_basic_auth_components() and deprecate * ap_get_basic_auth_pw() + * 20120211.75 (2.4.30-dev) Add hostname_ex to proxy_worker_shared */ The test compares the version in the comment to the actual #define that happens later on. Since that #define wasn't changed, the test errors. ** Affects: apache2 (Ubuntu) Importance: Low Status: Triaged ** Summary changed: - test-suite DEP8 not really running + DEP8 run-test-suite not really running ** Changed in: apache2 (Ubuntu) Status: New => Triaged ** Changed in: apache2 (Ubuntu) Importance: Undecided => Low -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to apache2 in Ubuntu. https://bugs.launchpad.net/bugs/1836898 Title: DEP8 run-test-suite not really running To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836898/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1836898] Re: DEP8 run-test-suite not really running
patched run-test-suite so that the suite actually runs, and this is the output ** Attachment added: "log" https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836898/+attachment/5277637/+files/log -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to apache2 in Ubuntu. https://bugs.launchpad.net/bugs/1836898 Title: DEP8 run-test-suite not really running To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836898/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1836329] Re: Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS
** Description changed: [Impact] With latest apache 2.4.29-1ubuntu4.7 published to 18.04 LTS bionic, when running ssllabs.com/ssltest against it to verify the configuration it leaves 2 apache processes using 100% indefinitely. Downgrading to 2.4.29-1ubuntu4.6 make it not reproducible anymore. [Test Case] We didn't find a reproducer that didn't involve https://ssllabs.com/ssltest, so the test case needs a publicly facing server with a DNS record. On a test system that has a public IP and is reachable via https on a hostname (not just IP): sudo apt update sudo apt install apache2 sudo a2enmod ssl sudo a2ensite default-ssl.conf sudo service apache2 restart In a terminal, monitor the apache2 processes CPU usage with top. Go to https://www.ssllabs.com/ssltest/ and input the url to your test server, using https. After a few seconds, the site will ask you if it should ignore the certificate error, confirm, and let it continue the test. After a few minutes, the test will finish and you will get a report. Go back to the terminal where top is running, and the apache2 processes will be spinning and using CPU, even though there isn't anymore traffic. With the fixed packages, the apache processes will remain idle. [Regression Potential] This upload is already fixing a regression which fixed a previous regression (#1833039), which shows that the situation is tricky. The fix here (clear-retry-flags-before-abort.patch) is at least not changing anything in the previous patch from bug #1833039, so that fix was correct. The second patch, for http/2 errors with openssl 1.1.1, unfortunately has no test case, and deals with error status and is specific to openssl 1.1.1. It's been applied upstream (and backported to the 2.4.x branch) for many months now. The trunk commit at http://svn.apache.org/viewvc?view=revision&revision=1843954 has a more elaborate explanation about behavior changes this does, and doesn't, introduce. We do have a DEP8 test that covers HTTP/2 SSL downloads, and it passes. But it also passed before this patch. I also manually tried such downloads of varying sizes (up to 10Mbytes) with no failures. [Other Info] While investigating this issue, another fix for an openssl 1.1.1 issue was found in the apache upstream git repo which involves http2 and how the code handles SSL_read() return values: https://github.com/apache/httpd/commit/644cff9977efa322fe6c0ae3357a5b8cb1eeec11 No upstream bug was found, nor could I come up with a reproducer case, but it seemed sensible to include that patch in this SRU, which was, after all, triggered by the openssl 1.1.1 upgrade in bionic. + The d/t/run-test-suite DEP8 test is falsely returning success, but it's not running due to being called as root, and it doesn't fail either. I filed bug #1836898 about this, and ran it manually for both cosmic and bionic. There is one test failure, but it's a silly one, introduced by a patch that added a comment. The test actually parses C comments in that particular header file. The bug has the details. [Original Description] With latest apache 2.4.29-1ubuntu4.7 published to 18.04 LTS bionic, when running ssllabs.com/ssltest against it to verify the configuration it leaves 2 apache processes using 100% indefinitely. Downgrading to 2.4.29-1ubuntu4.6 make it not reproducible anymore. So far i do not know if it is easy/likely to hit this case in normal https usage or only triggered by that testing site. But given that this is backported to LTS and allows easy DoS maybe the 4.7 should be backed out? So likely regression in the update to 4.7 having only single fix: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1833039 Extra info observed when that ssltest is over but processes are still there using up cpu: /server-status shows both processes 25234,25235 here in 'Reading' state: Srv PID Acc M CPU SS Req ConnChild Slot Client ProtocolVHost Request 0-0 25234 0/0/0 W 0.000 0 0.0 0.000.00 127.0.0.1 http/1.1ip-172-30-1-107.eu-west-1.compu GET /server-status HTTP/1.1 0-0 25234 0/0/0 R 0.00641 0 0.0 0.000.00 64.41.200.107 http/1.1 1-0 25235 0/1/1 _ 0.00505 2 0.0 0.000.00 64.41.200.107 http/1.1 1-0 25235 0/1/1 _ 0.00501 0 0.0 0.000.00 64.41.200.107 http/1.1 1-0 25235 0/1/1 _ 0.00500 0 0.0 0.000.00 64.41.200.107 http/1.1 1-0 25235 0/1/1 _ 0.00494 0 0.0 0.000.00 64.41.200.107 http/1.1 1-0 25235 0/1/1 _ 0.00604 0 0.0 0.000.00 64.41.200.106 http/1.1 1-0 25235 0/1/1 _ 0.00604 0 0.0 0.000.00 64.41.200.107
[Bug 1836329] Re: Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS
Cosmic verification First, reproducing the bug with: *** 2.4.34-1ubuntu2.2 500 500 http://us.archive.ubuntu.com/ubuntu cosmic-updates/main amd64 Packages Then I enabled the required modules, restarted apache, kept monitoring cpu usage per process using top, and asked ssllabs for the test run After the test finished, I still had apache processes using as much cpu as they could: PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 4709 www-data 20 0 759644 7616 5292 S 48.5 0.8 1:46.23 apache2 4710 www-data 20 0 759448 5068 2988 S 48.5 0.5 1:55.00 apache2 Installing the packages from proposed now: *** 2.4.34-1ubuntu2.3 500 500 http://us.archive.ubuntu.com/ubuntu cosmic-proposed/main amd64 Packages 100 /var/lib/dpkg/status 2.4.34-1ubuntu2.2 500 500 http://us.archive.ubuntu.com/ubuntu cosmic-updates/main amd64 Packages Hitting "clear cache" in the ssl labs report page triggers a re-run. Test finished, and no sign of apache processes near the top: PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 4818 root 20 0 13808 4016 2848 S 1.0 0.4 0:05.14 tmux -u -2 -f /usr/share/byobu/profiles/tmuxrc new-session -n - /usr/bin/byobu-shell 1 root 20 0 123356 9388 6752 S 0.0 0.9 0:06.27 /sbin/init Cosmic verification succeeded. ** Tags removed: verification-needed-cosmic ** Tags added: verification-done-cosmic -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to apache2 in Ubuntu. https://bugs.launchpad.net/bugs/1836329 Title: Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836329/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1787225] Re: systemctl disable apache2 does nothing
Thanks for filing this bug in Ubuntu. I just repeated your steps on a xenial lxd, using apache2 2.4.18-2ubuntu3.10, and after rebooting the container, apache2 was not started. I got the same warnings that you did, however. Furthermore, status also shows it's inactive after the reboot: $ systemctl status apache2 ● apache2.service - LSB: Apache2 web server Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled) Drop-In: /lib/systemd/system/apache2.service.d └─apache2-systemd.conf Active: inactive (dead) Docs: man:systemd-sysv-generator(8) Could you please try the above status command after you reboot, see what it says? Maybe you have something else starting apache, or requiring it perhaps? ** Changed in: apache2 (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to apache2 in Ubuntu. https://bugs.launchpad.net/bugs/1787225 Title: systemctl disable apache2 does nothing To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1787225/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1787225] Re: systemctl disable apache2 does nothing
You can also check "sudo systemctl cat apache2.service", that will show if any overrides are in place. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to apache2 in Ubuntu. https://bugs.launchpad.net/bugs/1787225 Title: systemctl disable apache2 does nothing To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1787225/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1828496] Re: service haproxy reload sometimes fails to pick up new TLS certificates
Going over the details from comment #7 This is the state before the reload: ubuntu@foo:~$ ps auxfwww | grep haproxy root 1346 0.0 0.0 4356 684 ?Ss May22 0:00 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid haproxy 2210 0.0 0.2 42644 10520 ?SMay22 0:00 \_ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 1378 haproxy 2215 2.7 0.8 68576 36308 ?Ss May22 84:46 \_ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 1378 -sf means to send the finish signal (which is SIGTTOU and SIGUSR1 according to haproxy(1)) to the pids listed after startup, which is pid 1378 in this case. There is no haproxy 1378 in this list, so I wonder if the "before" state was already a bit borked and what haproxy does if the pids listed after -sf do not exist. After reload, we have: ubuntu@foo:~$ ps auxfwww | grep haproxy root 1346 0.0 0.0 4356 724 ?Ss May22 0:00 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid haproxy 2210 0.0 0.2 42644 10520 ?SMay22 0:00 \_ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 1378 haproxy 2215 2.7 0.8 68496 36228 ?Ss May22 84:47 | \_ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 1378 haproxy 8151 0.0 0.2 42644 10456 ?S07:36 0:00 \_ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 2215 haproxy 8152 2.0 0.2 43048 10568 ?Ss 07:36 0:00 \_ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 2215 ubuntu@foo:~$ Here we can see new haproxy processes with -sf pointing at the previous 2215 one. The ones with -sf 1378 are still there, and will remain there until a full restart probably. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to haproxy in Ubuntu. https://bugs.launchpad.net/bugs/1828496 Title: service haproxy reload sometimes fails to pick up new TLS certificates To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/1828496/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
In other words, we are waiting for the SRU team to go through the queue. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to ruby2.3 in Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1833713] Re: Metadata is broken with dpdk bonding, jumbo frames and metadata from qdhcp
No reply yet in the ML. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to dpdk in Ubuntu. https://bugs.launchpad.net/bugs/1833713 Title: Metadata is broken with dpdk bonding, jumbo frames and metadata from qdhcp To manage notifications about this bug go to: https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1833713/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1836180] Re: TLS1.2 and newer not available in dovecot
With or without this line in /etc/dovecot/conf.d/10-ssl.conf, openssl s_client -connect localhost:993 uses TLSv1.3: ssl_protocols = !SSLv2 !SSLv3 Could you perhaps "grep ssl -r /etc/dovecot" and see if it's being changed elsewhere? And perhaps paste this if you can (in terms of sanitization): # cat conf.d/10-ssl.conf |grep -vE "^(#|$)" ssl = yes ssl_cert = https://bugs.launchpad.net/bugs/1836180 Title: TLS1.2 and newer not available in dovecot To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1836180/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
