[Sts-sponsors] [Bug 1960863] Re: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances
Performing verification for openssl on Focal. An affected user performed the verification, due to c7g instance types being in "Preview" state on Amazon AWS, and not generally accessible. The user started a c7g instance, and checked they had openssl 1.1.1f-1ubuntu2.10 from -updates. They attempted to use the poly1035 MAC downloading the file from the testcase: $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0Segmentation fault (core dumped) They can reproduce the issue. They then enabled -proposed from ports.ubuntu.com mirror, and installed openssl 1.1.1f-1ubuntu2.11. They again tried downloading the file: $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 (note the file doesn't actually download due to curl not automatically following 301 redirects): $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin --verbose ... * SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305 ... < HTTP/1.1 301 Moved Permanently < Location: https://downloads.gradle-dn.com/distributions/gradle-7.2-bin.zip ... curl does not segfault, and exits successfully. The package in -proposed fixes the issue. Happy to mark as verified. ** Tags removed: sts-sponsor verification-needed verification-needed-focal ** Tags added: verification-done verification-done-focal -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1960863 Title: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Focal: Fix Committed Bug description: [Impact] Support for hardware pointer authentication for armv8 systems was merged in openssl 1.1.1f, but it contains a bug in the implementation for poly1305 message authenticated code routines, which causes the calling program to fail pointer authentication, which causes the program to crash with a segmentation fault. You can easily test it by accessing any website that uses poly1305. There is no workaround except use a different MAC. [Testcase] This bug applies to armv8 systems which support pointer authentication. Start an armv8 instance, such as a c7g graviton 3 instance on AWS, and make sure the paca flag is present in lscpu: $ grep paca /proc/cpuinfo Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng Next, attempt to connect to any website that uses poly1305 MAC. $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Segmentation fault (core dumped) There is a test package available in the following ppa: https://launchpad.net/~mruffell/+archive/ubuntu/sf327917-test Install it, and poly1305 operations will no longer segfault. [Where problems could occur] The patch changes the order of operations for loading the SP and checking the AUTIASP against it, from checking the AUTIASP against nothing then loading the correct SP to check with, to the correct loading the SP and then checking the AUTIASP against the SP. This only changes one code path for armv8 systems, and other architectures are not affected. This is also only limited to poly1305 MAC. If a regression were to occur, it would only affect users of poly1035 MAC on armv8 with pacs support. [Other info] The fix landed upstream in openssl 1.1.1i with the following commit: commit 5795acffd8706e1cb584284ee5bb3a30986d0e75 Author: Ard Biesheuvel Date: Tue Oct 27 18:02:40 2020 +0100 Subject: crypto/poly1305/asm: fix armv8 pointer authentication Link: https://github.com/openssl/openssl/commit/5795acffd8706e1cb584284ee5bb3a30986d0e75 This commit is already present in Impish onward. Only Focal needs the fix. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1960863/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe :
[Sts-sponsors] [Bug 1960863] Autopkgtest regression report (openssl/1.1.1f-1ubuntu2.11)
All autopkgtests for the newly accepted openssl (1.1.1f-1ubuntu2.11) for focal have finished running. The following regressions have been reported in tests triggered by the package: python3.8/3.8.10-0ubuntu1~20.04.2 (armhf, amd64) linux-hwe-5.11/5.11.0-60.60 (amd64) linux-azure-5.11/5.11.0-1029.32~20.04.2 (amd64) mysql-8.0/8.0.28-0ubuntu0.20.04.3 (i386) linux-gcp-5.13/5.13.0-1015.18~20.04.1 (amd64) linux-oem-5.13/5.13.0-1029.36 (amd64) kopanocore/8.7.0-7ubuntu1 (amd64) python3.9/3.9.5-3ubuntu0~20.04.1 (armhf) Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1]. https://people.canonical.com/~ubuntu-archive/proposed- migration/focal/update_excuses.html#openssl [1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions Thank you! -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1960863 Title: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Focal: Fix Committed Bug description: [Impact] Support for hardware pointer authentication for armv8 systems was merged in openssl 1.1.1f, but it contains a bug in the implementation for poly1305 message authenticated code routines, which causes the calling program to fail pointer authentication, which causes the program to crash with a segmentation fault. You can easily test it by accessing any website that uses poly1305. There is no workaround except use a different MAC. [Testcase] This bug applies to armv8 systems which support pointer authentication. Start an armv8 instance, such as a c7g graviton 3 instance on AWS, and make sure the paca flag is present in lscpu: $ grep paca /proc/cpuinfo Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng Next, attempt to connect to any website that uses poly1305 MAC. $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Segmentation fault (core dumped) There is a test package available in the following ppa: https://launchpad.net/~mruffell/+archive/ubuntu/sf327917-test Install it, and poly1305 operations will no longer segfault. [Where problems could occur] The patch changes the order of operations for loading the SP and checking the AUTIASP against it, from checking the AUTIASP against nothing then loading the correct SP to check with, to the correct loading the SP and then checking the AUTIASP against the SP. This only changes one code path for armv8 systems, and other architectures are not affected. This is also only limited to poly1305 MAC. If a regression were to occur, it would only affect users of poly1035 MAC on armv8 with pacs support. [Other info] The fix landed upstream in openssl 1.1.1i with the following commit: commit 5795acffd8706e1cb584284ee5bb3a30986d0e75 Author: Ard Biesheuvel Date: Tue Oct 27 18:02:40 2020 +0100 Subject: crypto/poly1305/asm: fix armv8 pointer authentication Link: https://github.com/openssl/openssl/commit/5795acffd8706e1cb584284ee5bb3a30986d0e75 This commit is already present in Impish onward. Only Focal needs the fix. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1960863/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1903776] Re: Changed ubuntu-keyring paths breaks upgrade to focal.
** Changed in: landscape-client (Ubuntu Bionic) Assignee: Simon Poirier (simpoir) => (unassigned) ** Changed in: landscape-client (Ubuntu Focal) Assignee: Simon Poirier (simpoir) => (unassigned) ** Changed in: landscape-client (Ubuntu Groovy) Assignee: Simon Poirier (simpoir) => (unassigned) ** Changed in: landscape-client (Ubuntu Hirsute) Assignee: Simon Poirier (simpoir) => (unassigned) ** Changed in: landscape-client (Ubuntu Jammy) Assignee: Simon Poirier (simpoir) => (unassigned) -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1903776 Title: Changed ubuntu-keyring paths breaks upgrade to focal. Status in Landscape Client: Fix Committed Status in landscape-client package in Ubuntu: In Progress Status in landscape-client source package in Bionic: In Progress Status in landscape-client source package in Focal: In Progress Status in landscape-client source package in Groovy: Won't Fix Status in landscape-client source package in Hirsute: In Progress Status in landscape-client source package in Impish: New Status in landscape-client source package in Jammy: In Progress Bug description: [Impact] * When launching an Ubuntu release-upgrade through landscape-client, the upgrade-tool fails GPG verification due to trusted apt key having changed location as of 18.04 LTS. * The proposed patch extends gpg lookup path to include all /etc/apt/trusted.gpg.d/*.gpg files in addition to /etc/apt/trusted.gpg when verifying the upgrade-tool signature. [Test Case] * Install and register the landscape-client against a landscape-server on a series supporting an upgrade. * Wait for it to sync up packages. * On the computer packages page, there is a link at the bottom to request a release upgrade of that machine, if a supported version is available. * The upgrade fails and /var/log/landscape/release-upgrader.log will indicate a failed gpg verification. [Where problems could occur] * One thing which has been considered in this fix is how someone could have worked around the issue by re-creating the old key path. The fix covers such a case by still reading the deprecated trusted.gpg file. * Although some care has been taken to only load valid gpg keys from apt trusted keychain, there could be unforeseen scenarios where invalid data gets read from the keychain. In such a case, the strict nature of gpg would reject the signature verification, thus being no worse than without the fix. * The affected callsite is used for verifying the release-upgrader code prior to running it. One bad thing which we could imagine with this code path is falsely accepting an invalid file signature, which may create a security issue. This would likely take shape of injecting a gpg key, without having root access, in the search path. [Other Info] * There is no way to directly verify this issue on 20.10 Groovy and later (without faking a release) due to the lack of upgrade path to a supported LTS. The ubuntu-keyring package having the same file layout, the same validation failure is however to be expected if left unpatched. [Original description] Since bionic, ubuntu-keyring removed `/etc/apt/trusted.gpg` in favor of `/etc/apt/trusted.gpg.d/` This breaks signature verification for the upgrade-tool. Trying to release-upgrade through landscape yields a failure on signature check: 2020-11-10 15:47:51,019 WARNING [MainThread] Invalid signature for upgrade-tool tarball: /usr/bin/gpg failed (out='', err='gpg: keybox '/etc/apt/trusted.gpg' created gpg: Signature made Fri Oct 16 03:28:09 2020 UTC gpg:using RSA key 3B4FE6ACC0B21F32 gpg: Can't check signature: No public key To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/1903776/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1960863] Re: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances
This was a perfectly researched and written up bug and patch and made SRU review easy. Thank you! -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1960863 Title: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Focal: Fix Committed Bug description: [Impact] Support for hardware pointer authentication for armv8 systems was merged in openssl 1.1.1f, but it contains a bug in the implementation for poly1305 message authenticated code routines, which causes the calling program to fail pointer authentication, which causes the program to crash with a segmentation fault. You can easily test it by accessing any website that uses poly1305. There is no workaround except use a different MAC. [Testcase] This bug applies to armv8 systems which support pointer authentication. Start an armv8 instance, such as a c7g graviton 3 instance on AWS, and make sure the paca flag is present in lscpu: $ grep paca /proc/cpuinfo Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng Next, attempt to connect to any website that uses poly1305 MAC. $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Segmentation fault (core dumped) There is a test package available in the following ppa: https://launchpad.net/~mruffell/+archive/ubuntu/sf327917-test Install it, and poly1305 operations will no longer segfault. [Where problems could occur] The patch changes the order of operations for loading the SP and checking the AUTIASP against it, from checking the AUTIASP against nothing then loading the correct SP to check with, to the correct loading the SP and then checking the AUTIASP against the SP. This only changes one code path for armv8 systems, and other architectures are not affected. This is also only limited to poly1305 MAC. If a regression were to occur, it would only affect users of poly1035 MAC on armv8 with pacs support. [Other info] The fix landed upstream in openssl 1.1.1i with the following commit: commit 5795acffd8706e1cb584284ee5bb3a30986d0e75 Author: Ard Biesheuvel Date: Tue Oct 27 18:02:40 2020 +0100 Subject: crypto/poly1305/asm: fix armv8 pointer authentication Link: https://github.com/openssl/openssl/commit/5795acffd8706e1cb584284ee5bb3a30986d0e75 This commit is already present in Impish onward. Only Focal needs the fix. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1960863/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1960863] Re: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances
Hello Matthew, or anyone else affected, Accepted openssl into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl/1.1.1f-1ubuntu2.11 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-focal. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: openssl (Ubuntu Focal) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-focal -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1960863 Title: armv8 paca: poly1305 users see segfaults when pointer authentication in use on AWS Graviton 3 instances Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Focal: Fix Committed Bug description: [Impact] Support for hardware pointer authentication for armv8 systems was merged in openssl 1.1.1f, but it contains a bug in the implementation for poly1305 message authenticated code routines, which causes the calling program to fail pointer authentication, which causes the program to crash with a segmentation fault. You can easily test it by accessing any website that uses poly1305. There is no workaround except use a different MAC. [Testcase] This bug applies to armv8 systems which support pointer authentication. Start an armv8 instance, such as a c7g graviton 3 instance on AWS, and make sure the paca flag is present in lscpu: $ grep paca /proc/cpuinfo Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng Next, attempt to connect to any website that uses poly1305 MAC. $ curl https://services.gradle.org/distributions/gradle-7.2-bin.zip --output gradle-7.2.bin % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Segmentation fault (core dumped) There is a test package available in the following ppa: https://launchpad.net/~mruffell/+archive/ubuntu/sf327917-test Install it, and poly1305 operations will no longer segfault. [Where problems could occur] The patch changes the order of operations for loading the SP and checking the AUTIASP against it, from checking the AUTIASP against nothing then loading the correct SP to check with, to the correct loading the SP and then checking the AUTIASP against the SP. This only changes one code path for armv8 systems, and other architectures are not affected. This is also only limited to poly1305 MAC. If a regression were to occur, it would only affect users of poly1035 MAC on armv8 with pacs support. [Other info] The fix landed upstream in openssl 1.1.1i with the following commit: commit 5795acffd8706e1cb584284ee5bb3a30986d0e75 Author: Ard Biesheuvel Date: Tue Oct 27 18:02:40 2020 +0100 Subject: crypto/poly1305/asm: fix armv8 pointer authentication Link: https://github.com/openssl/openssl/commit/5795acffd8706e1cb584284ee5bb3a30986d0e75 This commit is already present in Impish onward. Only Focal needs the fix. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1960863/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1947099] Re: ipconfig does not honour user-requested timeouts in some cases
> However, Focal (which uses klibc 2.0.7) uses dhclient for networking
initialization instead of ipconfig...
If no users in Focal could be affected by the bug then I'm fine with
accepting the fix to Bionic only. But could you please expand on the
above statement? Under what conditions does Focal use dhclient instead
of ipconfig? Presumably through initramfs-tools and ip=/ip6=? Are there
any other situations where users could reasonably be using
klibc/ipconfig on Focal? Is it possible for them to be doing this but
not using initramfs-tools?
It would help me understand if you could present a complete user story
to explain the user impact. Right now the bug description doesn't
explain the full scenario under which they would be impacted by this
bug.
> [Where problems could occur]
I don't think you've really answered this question. What different
scenarios, from a user perspective (ie. user stories), can reasonably be
predicted that will hit this ipconfig DHCP-related timeout case?
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1947099
Title:
ipconfig does not honour user-requested timeouts in some cases
Status in klibc package in Ubuntu:
New
Status in klibc source package in Bionic:
In Progress
Bug description:
[Impact]
In some cases, ipconfig can take a longer time than the user-specified
timeouts, causing unexpected delays.
[Test Plan]
Any situation where ipconfig encounters an error sending the DHCP packet, it
will automatically set a delay of 10 seconds, which could be longer than the
user-specified timeout. It can be reproduced by creating a dummy interface and
attempting to run ipconfig on it with a timeout value of less than 10:
# ip link add eth1 type dummy
# date; /usr/lib/klibc/bin/ipconfig -t 2 eth1; date
Thu Nov 18 04:46:13 EST 2021
IP-Config: eth1 hardware address ae:e0:f5:9d:7e:00 mtu 1500 DHCP RARP
IP-Config: no response after 2 secs - giving up
Thu Nov 18 04:46:23 EST 2021
^ Notice above, ipconfig thinks that it waited 2 seconds, but the
timestamps show an actual delay of 10 seconds.
[Where problems could occur]
Please see reproduction steps above. We are seeing this in production too
(see comment #2).
[Other Info]
A patch to fix the issue is being proposed here. It is a safe fix - it only
checks before going into sleep that the timeout never exceeds the
user-requested value.
[Original Description]
In some cases, ipconfig can take longer than the user-specified
timeouts, causing unexpected delays.
in main.c, in function loop(), the process can go into
process_timeout_event() (or process_receive_event() ) and if it
encounters an error situation, will set an attempt to "try again
later" at time equal now + 10 seconds by setting
s->expire = now + 10;
This can happen at any time during the main event loop, which can end
up extending the user-specified timeout if "now + 10" is greater than
"start_time + user-specified-timeout".
I believe a patch like the following is needed to avoid this problem:
--- a/usr/kinit/ipconfig/main.c
+++ b/usr/kinit/ipconfig/main.c
@@ -437,6 +437,13 @@ static int loop(void)
if (timeout > s->expire - now.tv_sec)
timeout = s->expire - now.tv_sec;
+
+ /* Compensate for already-lost time */
+ gettimeofday(, NULL);
+ if (now.tv_sec + timeout > start + loop_timeout) {
+ timeout = loop_timeout - (now.tv_sec - start);
+ printf("Lowered timeout to match user request
= (%d s) \n", timeout);
+ }
}
I believe the current behaviour is buggy. This is confirmed when the
following line is executed:
if (loop_timeout >= 0 &&
now.tv_sec - start >= loop_timeout) {
printf("IP-Config: no response after %d "
"secs - giving up\n", loop_timeout);
rc = -1;
goto bail;
}
'loop_timeout' is the user-specified time-out. With a value of 2, in
case of error, this line prints:
IP-Config: no response after 2 secs - giving up
So it thinks that it waited 2 seconds - however, in reality it had
actually waited for 10 seconds.
The suggested code-change ensures that the timeout that is actually
used never exceeds the user-specified timeout.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/klibc/+bug/1947099/+subscriptions
--
Mailing list: https://launchpad.net/~sts-sponsors
Post to : sts-sponsors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~sts-sponsors
More help :
