Bug#1113774: Debian Technical Committee: Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Marc, * Marc Haber [2026-01-04 06:59]: Of course, people now demand that fix in Trixie as well (see #1004894). I expected this to happen. People are never satisfied with what they get. What does the ctte think about that? As far as I am concerned, you have final authority here. If you feel particularly generous, you can apply the fix, but I don't see the need if the only affected hardware does not satisfy the trixie baseline requirements anyway. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Bug#1113774: Debian Technical Committee: Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Marc, These are my own opinions and not a formal statement of the CTTE. On Sun, Jan 04, 2026 at 06:59:12AM +0100, Marc Haber wrote: > Of course, people now demand that fix in Trixie as well (see #1004894). I > expected this to happen. People are never satisfied with what they get. What > does the ctte think about that? Our release notes have been very explicit about unsupporting i386. The discussion seems to have reached a point where it is clear that Debian does not support the configuration in question. That said, making software work outside the specification that we consider supported can be valuable. It then becomes a question of trading costs (risks) and effort against benefit. A sudo update is not free of charge. You lowered the severity of the bug to minor and given the unsupported configuration that feels appropriate to me. If the submitter is providing an improvement (bearing part of the cost) and the cost of including such an improvement is acceptable, then by all means do it. What is acceptable is the interesting question here. Asking the release team to review another stable update would exceed the cost in my opinion (not considering your own effort here). In contrast, piggy-backing the change onto an update of sudo for other reasons would seem ok-ish to me, because the risks are now very well understood. You may evaluate the costs differently than I do and reach the conclusion that the update is not worth the effort (as you closed the bug wontfix). Saying "no" is part of a maintainers duties and in this instance I would have difficulties challenging your "no". Indeed, one reason that made me recommend fixing bookworm was that it was the last release to support i386 as a standalone architecture, so people would likely end up being stuck on bookworm. Helmut
Bug#1113774: Debian Technical Committee: Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Sun, Jan 04, 2026 at 06:59:12AM +0100, Marc Haber wrote: Of course, people now demand that fix in Trixie as well (see #1004894). I expected this to happen. People are never satisfied with what they get. What does the ctte think about that? My 2c here is that that bug self-resolved to a sensible place. I agree with zeha's read on-thread. Just to put it out there -- If someone can find a system which is: 1. An amd64 processor that meets our ISA baseline 2. Running a clean trixie amd64 install (not a i386 franken-install) 3. Which produces an invalid opcode error when running sudo:i386 I'd be interested in learning more about that CPU and what may be going on with the host. I do not believe this happens on systems that meet our baseline for trixie. I would be interested in being proven wrong. That being said: I suspect our calculus (this compiler flag change as discussed is a security noop in the way we talked about) still applies to trixie. This flag change seems to be to be "fine" to backport to trixie, technically. *HOWEVER*, I'd be interested in use-cases for why people would install sudo:i386 within trixie amd64 or later at all, and rational on why we even ship it for a port that's no longer bootable. My personal instinct as a package maintainer would be to RM sudo [i386], although, perhaps I don't understand something important. I'd be interested in learning what that misunderstanding is. paultag -- ⢀⣴⠾⠻⢶⣦⠀ Paul Tagliamonte ⣾⠁⢠⠒⠀⣿⡁ https://people.debian.org/~paultag | https://pault.ag/ ⢿⡄⠘⠷⠚⠋Debian, the universal operating system. ⠈⠳⣄⠀⠀ 4096R / FEF2 EB20 16E6 A856 B98C E820 2DCD 6B5D E858 ADF3 signature.asc Description: PGP signature
Bug#1113774: Debian Technical Committee: Bug#1113774: Disabling -fcf-protection in sudo for bookworm
unarchive 1113774 thanks On Thu, Dec 04, 2025 at 04:39:43PM +0100, Marc Haber wrote: I would also like information whether I could expect the binaries for non-i386 architectures to be identical even after the patch was applied. They are. Thank you, reproducible builds. That was a huge help. I won't have time before christmas to dive into the technical discussion as deeply as I would. Of course, people now demand that fix in Trixie as well (see #1004894). I expected this to happen. People are never satisfied with what they get. What does the ctte think about that? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, 05 Dec 2025 at 15:26:50 +, James Addison wrote: It seems unlikely to me that [an amd64 system with an i386 foreign architecture] would have both sudo:amd64 and sudo:i386 installed This isn't possible. sudo is not Multi-Arch: same, so you can only have one instance of it at a time. (Each architecture's instance of sudo contains a different /usr/bin/sudo, so it is impossible for more than one to coexist.) hopefully even less likely that only sudo:i386 is installed This is possible. Realistically, this would only happen if the user specifically asked for it, or if they did an incomplete crossgrade from i386, or if they installed an i386 package that Depends or Recommends on sudo while not already having sudo:amd64 installed. The first two of those options are at least arguably user error, and the last seems unlikely to me: the typical use-cases for an i386 foreign architecture on an amd64 system (Wine, Steam, cross-compilers) don't have a reason to depend on sudo. smcv
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Dec 5, 2025, 17:06 Christoph Berg wrote: > Re: James Addison > > Even so: does this mean that we should be careful about disabling > > fcf-protection=branch on a broader/default basis? > > The question was entirely only about bookworm. In bookworm, none of > the fcf-protection bits are enabled by default. It was solely active > in sudo because upstream enabled it. > > I believe the question of disabling fcf-protection is not relevant for > any other bookworm package. The "unstable" part of that question > should be discussed on -devel, not in this bug. > > Christoph > Acknowledged - thanks, Christoph. >
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Re: James Addison > Even so: does this mean that we should be careful about disabling > fcf-protection=branch on a broader/default basis? The question was entirely only about bookworm. In bookworm, none of the fcf-protection bits are enabled by default. It was solely active in sudo because upstream enabled it. I believe the question of disabling fcf-protection is not relevant for any other bookworm package. The "unstable" part of that question should be discussed on -devel, not in this bug. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Dec 5, 2025, 14:30 James Addison wrote:
> On Fri, Dec 5, 2025, 14:05 Helmut Grohne wrote:
>
>> Hi James,
>>
>> On Fri, Dec 05, 2025 at 12:38:59PM +, James Addison wrote:
>> > My reading of the thread is that fcf-protection=return can be
>> > security-effective on 32-bit x86 processors, has no effect on binary
>> > size, and does not introduce the compatibility issues that
>> > fcf-protection=branch does.
>>
>> In order for -fcf-protection=return to provide any benefit, a shadow
>> stack is required. That support has not trickled down yet. It is only
>> since trixie that 64bit enables CONFIG_X86_USER_SHADOW_STACK, so no
>> Debian i386 kernel ever enabled that. I also doubt that 32bit hardware
>> supports this in any way.
>>
>> > I think this is what Helmut was pointing out -- the two halves of the
>> > flag's behaviour.
>>
>> My clarification was that we're disabling both features, but doing so is
>> ok, because neither has any practical benefit on i386.
>>
>> Helmut
>>
>
> Thank you, Helmut - that completely resolves my question.
>
> And indeed, as I should have checked myself before emailing, the GCC
> Instrumentation Options documentation[1] confirms that enabling either the
> branch or return instrumentation via fcf-protection will cause a __CET__
> macro to be emitted -- and it also mentions that hardware-level CET is
> *compatible* ('works') with i686 onwards (I think the subtext there is that
> the instructions are no-ops on i686 -- but as you mention, effective only
> on suitable x86-64 processors -- since those are the only processor models
> that handle the instruction other than as a no-o).
>
> Regards,
> James
>
> [1] - https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html
>
Sorry, one more question came to mind after all: I realise that the advice
in this thread only relates to the sudo:i386 package; however I believe it
is the case that an x86-64 system can install and run i386 binaries
alongside amd64 ones.
It seems unlikely to me that such a system would have both sudo:amd64 and
sudo:i386 installed -- and hopefully even less likely that only sudo:i386
is installed (or preferred, e.g. by PATH).
Even so: does this mean that we should be careful about disabling
fcf-protection=branch on a broader/default basis?
>
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Dec 5, 2025, 14:31 Paul Tagliamonte wrote: > On Fri, Dec 05, 2025 at 12:38:59PM +, James Addison wrote: > >My reading of the thread is that fcf-protection=return can be > >security-effective on 32-bit x86 processors, has no effect on binary > >size, and does not introduce the compatibility issues that > >fcf-protection=branch does. > > [snip] > > >So to reformulate that as a question: why is the advice to remove the > >flag completely, instead of reducing it to fcf-protection=return? > > This requires kernel support to be effective - and Bookworm does not > have a kernel with that flag turned on. I understand there to be no > difference between disabling fcf-protection entirely vs return in i386 > for Bookworm. > [ ... snip ... ] Thanks, Paul. I briefly wondered about people who could be running custom kernels (e.g. with support enabled) in combination with the Debian sudo (and potentially other) binaries, or that Debian might choose to enable it at the kernel config level in futute -- but, given my understanding is that the patch will only affect i386 packages, and that the CET instructions are no-ops on that platform, I think that that consideration is moot. >
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Dec 05, 2025 at 12:38:59PM +, James Addison wrote: My reading of the thread is that fcf-protection=return can be security-effective on 32-bit x86 processors, has no effect on binary size, and does not introduce the compatibility issues that fcf-protection=branch does. [snip] So to reformulate that as a question: why is the advice to remove the flag completely, instead of reducing it to fcf-protection=return? This requires kernel support to be effective - and Bookworm does not have a kernel with that flag turned on. I understand there to be no difference between disabling fcf-protection entirely vs return in i386 for Bookworm. The two (related) flags, as Marcos points out earlier in reply to my misunderstanding here too, the related toggle here is CONFIG_X86_USER_SHADOW_STACK, which is required for fcf-protection=return FWIW; these flags were set specifically in sudo upstream -- not Debian's cross-distro default flags. Upstream has since removed them, for the same reason(s) as we resolved here. If we wanted to rebuild the distro to take advantage of the (new!) enablement of the CONFIG_X86_USER_SHADOW_STACK=y in the sid x86_64 kernel running i386 binaries (or even amd64 binaries), I reckon we'd need to do some work across the archive. In that case we'd want to use fcf-protection=return, rather than fcf-protection=full, as I understand it, since we can't take meaningful advantage of the IBT -- since the CONFIG_X86_KERNEL_IBT flag is for kernelspace not userspace, and there's no reason to turn that on for userspace programs. FWIW, I stand by the advice; it's good. There is no difference between disabling fcf-protection entirely and setting return, since no bookworm kernels will do anything different with =return. I agree with Helmut's reading on his ctte vote, and I share it completely. paultag -- ⢀⣴⠾⠻⢶⣦⠀ Paul Tagliamonte ⣾⠁⢠⠒⠀⣿⡁ https://people.debian.org/~paultag | https://pault.ag/ ⢿⡄⠘⠷⠚⠋Debian, the universal operating system. ⠈⠳⣄⠀⠀ 4096R / FEF2 EB20 16E6 A856 B98C E820 2DCD 6B5D E858 ADF3 signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Dec 5, 2025, 14:05 Helmut Grohne wrote:
> Hi James,
>
> On Fri, Dec 05, 2025 at 12:38:59PM +, James Addison wrote:
> > My reading of the thread is that fcf-protection=return can be
> > security-effective on 32-bit x86 processors, has no effect on binary
> > size, and does not introduce the compatibility issues that
> > fcf-protection=branch does.
>
> In order for -fcf-protection=return to provide any benefit, a shadow
> stack is required. That support has not trickled down yet. It is only
> since trixie that 64bit enables CONFIG_X86_USER_SHADOW_STACK, so no
> Debian i386 kernel ever enabled that. I also doubt that 32bit hardware
> supports this in any way.
>
> > I think this is what Helmut was pointing out -- the two halves of the
> > flag's behaviour.
>
> My clarification was that we're disabling both features, but doing so is
> ok, because neither has any practical benefit on i386.
>
> Helmut
>
Thank you, Helmut - that completely resolves my question.
And indeed, as I should have checked myself before emailing, the GCC
Instrumentation Options documentation[1] confirms that enabling either the
branch or return instrumentation via fcf-protection will cause a __CET__
macro to be emitted -- and it also mentions that hardware-level CET is
*compatible* ('works') with i686 onwards (I think the subtext there is that
the instructions are no-ops on i686 -- but as you mention, effective only
on suitable x86-64 processors -- since those are the only processor models
that handle the instruction other than as a no-o).
Regards,
James
[1] - https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html
>
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi James, On Fri, Dec 05, 2025 at 12:38:59PM +, James Addison wrote: > My reading of the thread is that fcf-protection=return can be > security-effective on 32-bit x86 processors, has no effect on binary > size, and does not introduce the compatibility issues that > fcf-protection=branch does. In order for -fcf-protection=return to provide any benefit, a shadow stack is required. That support has not trickled down yet. It is only since trixie that 64bit enables CONFIG_X86_USER_SHADOW_STACK, so no Debian i386 kernel ever enabled that. I also doubt that 32bit hardware supports this in any way. > I think this is what Helmut was pointing out -- the two halves of the > flag's behaviour. My clarification was that we're disabling both features, but doing so is ok, because neither has any practical benefit on i386. Helmut
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hello, I'd like to ask about a point in this CTTE advice for Marc: On Sun, 23 Nov 2025 23:03:49 +0100, Christoph wrote: > On Thu, 20 Nov 2025 10:09:25 +0100, Helmut wrote: > > A minor aspect missing in the > > summary is that -fcf-protection is actually controlling two distinct > > features with one flag, one of which poses the problem we've been > > discussing. The other feature likewise does not apply to i386. > > Therefore, this addition does not affect the conclusion. > > Thanks, I should have mentioned that in the summary. I left it out > from the ballot because only half-disabling the feature would likely > not make the clean, "obviously correct" patch that Marc wanted. My reading of the thread is that fcf-protection=return can be security-effective on 32-bit x86 processors, has no effect on binary size, and does not introduce the compatibility issues that fcf-protection=branch does. I think this is what Helmut was pointing out -- the two halves of the flag's behaviour. My uncertainty/concern is why the CTTE decision seems to be to remove the flag entirely, because I worry that that would reduce security, something that I understand Marc wants to avoid. So to reformulate that as a question: why is the advice to remove the flag completely, instead of reducing it to fcf-protection=return? Thanks, James
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Re: Helmut Grohne > A minor aspect missing in the > summary is that -fcf-protection is actually controlling two distinct > features with one flag, one of which poses the problem we've been > discussing. The other feature likewise does not apply to i386. > Therefore, this addition does not affect the conclusion. Thanks, I should have mentioned that in the summary. I left it out from the ballot because only half-disabling the feature would likely not make the clean, "obviously correct" patch that Marc wanted. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Christoph, On Wed, Nov 19, 2025 at 08:31:28PM +0100, Christoph Berg wrote: > The TC has been discussing the issue with all involved parties and Marc, the > sudo maintainer has agreed to accept advice, so we will just do that instead > of > overruling him. Thank you for the summary. With the exception of the interaction with Ben (which I appear to have missed), it accurately matches my understanding of the matter at hand. A minor aspect missing in the summary is that -fcf-protection is actually controlling two distinct features with one flag, one of which poses the problem we've been discussing. The other feature likewise does not apply to i386. Therefore, this addition does not affect the conclusion. > I am calling for votes on this ballot: > > [A] The TC advises the sudo maintainer to update the sudo package in > bookworm > such that on the i386 architecture, the `-fcf-protection` compiler flag is > no > longer used. > > [F] Further discussion. I vote A > F. Helmut signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Christoph (2025.11.19_20:31:28_+0100) I am calling for votes on this ballot: [A] The TC advises the sudo maintainer to update the sudo package in bookworm such that on the i386 architecture, the `-fcf-protection` compiler flag is no longer used. [F] Further discussion. I vote: A > F -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272 signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi, On Wed, 19 Nov 2025 19:31:28 +, Christoph Berg wrote: > I am calling for votes on this ballot: > > [A] The TC advises the sudo maintainer to update the sudo package in > bookworm > such that on the i386 architecture, the `-fcf-protection` compiler flag is > no > longer used. > > [F] Further discussion. I vote: A > F Thanks, Matthew pgp7mNA_Sesva.pgp Description: OpenPGP Digital Signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi, On Wed, 19 Nov 2025 20:31:28 +0100 Christoph Berg wrote: I am calling for votes on this ballot: [A] The TC advises the sudo maintainer to update the sudo package in bookworm such that on the i386 architecture, the `-fcf-protection` compiler flag is no longer used. [F] Further discussion. I vote A > F. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Re: To [email protected] > I am calling for votes on this ballot: > > [A] The TC advises the sudo maintainer to update the sudo package in > bookworm > such that on the i386 architecture, the `-fcf-protection` compiler flag is > no > longer used. > > [F] Further discussion. I vote A > F. Christoph signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Wed, Nov 19, 2025 at 08:31:28PM +0100, Christoph Berg wrote: I am calling for votes on this ballot: [A] The TC advises the sudo maintainer to update the sudo package in bookworm such that on the i386 architecture, the `-fcf-protection` compiler flag is no longer used. [F] Further discussion. I'm incredibly grateful to patient explinations from Marcos; it took me a second to catch up, but I understand it now thanks to your explanations and refs. This bug winds up with me feeling pretty good about Debian, all in all. Marc is doing an exceptional job maintaining sudo, and the thought that has gone into his cautious approach to changes from upstream is not lost on me. I don't see any technical reason why this isn't a safe and Debian policy-aligned change to those running sudo on i386. In addition, upstream has accepted a similar patch. Last note: At the end of the day, what we do with bookworm is ultimately up to the (old?)stable release managers -- bookworm (albeit the last full "i386" [read: i686] release), is still now oldstable. While it's not officially EOL until June 2026, there is still work to be done to socalize this change further in order to actually update bookworm. Thank you both very much. I vote A > F -- ⢀⣴⠾⠻⢶⣦⠀ Paul Tagliamonte ⣾⠁⢠⠒⠀⣿⡁ https://people.debian.org/~paultag | https://pault.ag/ ⢿⡄⠘⠷⠚⠋Debian, the universal operating system. ⠈⠳⣄⠀⠀ 4096R / FEF2 EB20 16E6 A856 B98C E820 2DCD 6B5D E858 ADF3 signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
In #1113774, Marcos Del Sol Vives is asking the committee about the compiler flags used for sudo in bookworm on the i386 architecture. The sudo version there is enabling `-fcf-protection` when supported by the compiler: https://sources.debian.org/src/sudo/1.9.13p3-1%2Bdeb12u2/m4/hardening.m4#L108-L114 The problem is, that on his machine, a Vortex86DX3, the generated ENDBR instructions, which live in an opcode region declared as NOPs in earlier architecture specs, are not ignored, but raise exceptions and cause sudo to abort. There is a lot of evidence that Control-flow Enforcement Technology (CET or cf-protection) is only meant to be enabled on 64-bit binaries and is ineffective elsewhere: * https://docs.kernel.org/next/x86/shstk.html * https://lkml.org/lkml/2025/9/1/1704 One part of the thread was discussing the usefulness of this feature even in 64-bit environments (the kernel only half-supports it in userland) which has led to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1113864 being filed on dpkg-dev, but this is not relevant to the TC question. In fact, dpkg-dev is only emitting -fcf-protection on amd64 and not on i386. A large part of the thread assumed the default bookworm compiler flags had that problem, but it's actually upstream sudo adding -fcf-protection. Around the time of the discussion, upstream sudo included a change that limits -fcf-protection to x86_64: https://github.com/sudo-project/sudo/pull/468 The question if Vortex86DX3 is part of bookworm's i386 architecture baseline was raised. In https://lists.debian.org/debian-devel/2023/10/msg00120.html Ben Hutchings confirms that ENDBR32 should be ignored by i686-conformant processors, and that i686 is required for bookworm. (He corrects himself in the next mail saying this would apply to trixie only, but again corrects himself saying this applies to bookworm indeed.) This seems to indicate that Vortex86DX3 is not i686-conformant. The submitter claims the CPU is conformant, citing https://psc.informatik.uni-jena.de/hw/p-pro-3.pdf page 417 as saying ENDBR32 was "reserved". https://www.debian.org/releases/bookworm/i386/release-notes/ch-information.en.html#i386-is-i686 Debian trixie bumps the compiler baseline for i386 such that this CPU is definitely no longer supported so this issue is solely about bookworm. The TL;DR summary of the problem is: in Debian bookworm, the sudo package is using -fcf-protection on i386 (where it should be a no-op), but this breaks sudo on this Vortex86DX3 CPU (that should ignore ENDBR32 but does not). The TC has been discussing the issue with all involved parties and Marc, the sudo maintainer has agreed to accept advice, so we will just do that instead of overruling him. I am calling for votes on this ballot: [A] The TC advises the sudo maintainer to update the sudo package in bookworm such that on the i386 architecture, the `-fcf-protection` compiler flag is no longer used. [F] Further discussion. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Sep 26, 2025 at 04:10:46PM +0200, Helmut Grohne wrote: You can do that, but you need to use the exact same debian/changelog. Otherwise your SOURCE_DATE_EPOCH will differ and that'll likely change the binary. Cross building will not help you, because the cross toolchain presently imposes build-id differences. You may use a porterbox to build the package twice (with and without the patch but without changing d/changelog). Alternatively, debusine.debian.net (I'm being compensated for working on that) also has native arm builders and you might upload packages there for comparison. I have both native amd64 and arm64 hosts available, but I thought of using a porterbox. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Marc, On Fri, Sep 26, 2025 at 02:23:01PM +0200, Marc Haber wrote: > I would consider a failed built a non-identical result. Am I being naive > here? You can do that, but you need to use the exact same debian/changelog. Otherwise your SOURCE_DATE_EPOCH will differ and that'll likely change the binary. Cross building will not help you, because the cross toolchain presently imposes build-id differences. You may use a porterbox to build the package twice (with and without the patch but without changing d/changelog). Alternatively, debusine.debian.net (I'm being compensated for working on that) also has native arm builders and you might upload packages there for comparison. Helmut
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Fri, Sep 26, 2025 at 11:11:08AM +0200, Helmut Grohne wrote: On Thu, Sep 25, 2025 at 01:56:21PM +0200, Marc Haber wrote: I reaffirm that. Should the TC decline to give formal advice (which I would be fine with), I would go ahead to disable -fcf-protection for i386 builds (and verify that the amd64 and arm64 binary stay identical) and build packages for trixie and bookworm, submit both of them for the next point release. Please bear in mind that these flags are architecture-specific. The arm64 compiler does not understand -fcf-protection at all (and this is a recurring problem for cross builds when people mix build/host compiler flags with host/build compilers). For arm64 you should be seeing -mbranch-protection=standard since trixie. Likewise, an amd64 compiler will fail on encountering -mbranch-protection=standard. I would consider a failed built a non-identical result. Am I being naive here? Anyway, thanks for the hint, appreciated. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Marc, On Thu, Sep 25, 2025 at 01:56:21PM +0200, Marc Haber wrote: > I reaffirm that. Should the TC decline to give formal advice (which I would > be fine with), I would go ahead to disable -fcf-protection for i386 builds > (and verify that the amd64 and arm64 binary stay identical) and build > packages for trixie and bookworm, submit both of them for the next point > release. Please bear in mind that these flags are architecture-specific. The arm64 compiler does not understand -fcf-protection at all (and this is a recurring problem for cross builds when people mix build/host compiler flags with host/build compilers). For arm64 you should be seeing -mbranch-protection=standard since trixie. Likewise, an amd64 compiler will fail on encountering -mbranch-protection=standard. Helmut
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
I will try to summarize this TC bug. Sorry for the delay, I was just too busy lately. Marcos Del Sol Vives is asking the committee about the compiler flags used for sudo in bookworm on the i386 architecture. The sudo version there is enabling `-fcf-protection` when supported by the compiler: https://sources.debian.org/src/sudo/1.9.13p3-1%2Bdeb12u2/m4/hardening.m4#L108-L114 The problem is, that on his machine, a Vortex86DX3, the generated ENDBR instructions, which live in an opcode region declared as NOPs in earlier architecture specs, are not ignored, but raise exceptions and cause sudo to abort. There is a lot of evidence that Control-flow Enforcement Technology (CET or cf-protection) is only meant to be enabled on 64-bit binaries and is ineffective elsewhere: * https://docs.kernel.org/next/x86/shstk.html * https://lkml.org/lkml/2025/9/1/1704 One part of the thread was discussing the usefulness of this feature even in 64-bit environments (the kernel only half-supports it in userland) which has led to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1113864 being filed on dpkg-dev, but this is not relevant to the TC question. In fact, dpkg-dev is only emitting -fcf-protection on amd64 and not on i386. A large part of the thread assumed the default bookworm compiler flags had that problem, but it's actually upstream sudo adding -fcf-protection. Around the time of the discussion, upstream sudo included a change that limits -fcf-protection to x86_64: https://github.com/sudo-project/sudo/pull/468 The question if Vortex86DX3 is part of bookworm's i386 architecture baseline was raised. In https://lists.debian.org/debian-devel/2023/10/msg00120.html Ben Hutchings confirms that ENDBR32 should be ignored by i686-conformant processors, and that i686 is required for bookworm. (He corrects himself in the next mail saying this would apply to trixie only, but again corrects himself saying this applies to bookworm indeed.) This seems to indicate that Vortex86DX3 is not i686-conformant. The submitter claims the CPU is conformant, citing https://psc.informatik.uni-jena.de/hw/p-pro-3.pdf page 417 as saying ENDBR32 was "reserved". (The URL doesn't load for me now so I can't validate.) https://www.debian.org/releases/bookworm/i386/release-notes/ch-information.en.html#i386-is-i686 Another sub-question was how this relates to trixie, but this is out of scope of the TC question. So, if I got all parts about right, the problem here is that using -fcf-protection on i386 (where it should be a no-op and hence should not be used in the first place) actually breaks sudo on bookworm on this CPU (that possibly should ignore ENDBR32 but does not). Possible TC rulings are: * agree with the submitter. -fcf-protection is no-op on i386; the sudo package should be updated. * reject the request; changing sudo for a very small number of users is too risky (FWIW, the patch from https://github.com/sudo-project/sudo/pull/468 applies trivially to sudo 1.9.13p3-1+deb12u2) * reject the request; bookworm is already oldstable (if it's reaching us only now, it's not that important) * reject the request; the CPU in question is not part of the baseline Marc has indicated that he would accept advice on this issue so we might go with issuing that instead of formally overriding him. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Thu, Sep 25, 2025 at 04:58:58PM +0200, Christoph Berg wrote: (and verify that the amd64 and arm64 binary stay identical) and build packages for trixie and bookworm, submit both of them for the next point release. Marc, do you have all tools to do that verification? I would apply the patch, build, back out the patch, build again and see whether the amd64 and arm64 binaries /usr/bin/sudo are identical. I am naive to assume that? I really appreciate the work you did on this. Thank you for being careful with this important package! *bow* Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Marcos, Marc, thanks for the prompt feedback. The TC was meeting this morning and the gist of the discussion was "the definition of what i686 exactly is is blurry, and independently from whether we think the CPU in question is conforming to the specs, the problematic part of -fcf-protection is a no-op on i386 and reverting that in sudo on that architecture seems low risk and easy, so we'll recommend that". A formal write-up of that will follow. Minutes: https://meetbot.debian.net/debian-ctte/2025/debian-ctte.2025-09-25-08.00.html Re: Marcos Del Sol > So, according to Intel's own manual, these instructions must not be used > on 2010 32-bit processors - and certainly not on 1995 Pentium III (i686) > processors that Bookworm targets. If anything, I'd say Vortex86 processor > is actually here in the right for raising UD# on a instruction that the > manual states is reserved and invalid! Yeah there's the Intel manual, and there's Intel processors which might behave a bit differently, and some security fix was placed right into that gap... But anyway, see above. > > * reject the request; bookworm is already oldstable (if it's reaching us > > only now, it's not that important) > > Just a tiny note: the first time this was reported was back in 2023-10-17 > by Justin on the debian-devel list (which you've linked) - four months > Bookworm was declared stable. > > The amount of affected people certainly isn't going to be high (these are > not super popular processors after all), but I bet many affected would > have found that bug report (I did) and assumed it was not to be fixed, so > there'll certainly be underreporting. Ack, that is true. Re: Marc Haber > > Around the time of the discussion, upstream sudo included a change that > > limits > > -fcf-protection to x86_64: https://github.com/sudo-project/sudo/pull/468 > > The problem that I have with this change is that it was suggested by the > same individual who wants us to do this change in Debian. Only sudo upstream > didn't push back as hard as I did. I guess that's kind of in line with the underreporting - for everyone else, it's enough if one person stepped up to have it changed. > I reaffirm that. Should the TC decline to give formal advice (which I would > be fine with), I would go ahead to disable -fcf-protection for i386 builds Marcos, since -fcf-protection has two parts "branch" and "return", and only one of then is the problem, would you be willing to offer an additional patch next to https://salsa.debian.org/sudo-team/sudo/-/merge_requests/24/diffs which only enables the good one on i386 (I believe that's -fcf-protection=return but I might well have lost track) in sudo/bookworm? Our idea was to have both variants on the table for Marc to choose from: 1) https://salsa.debian.org/sudo-team/sudo/-/merge_requests/24/diffs is the minimal source change and disables the flag completely on i386. 2) -fcf-protection=return would be the minimal binary change in that it only removes one part of the protection (however effective they might be). > (and verify that the amd64 and arm64 binary stay identical) and build > packages for trixie and bookworm, submit both of them for the next point > release. Marc, do you have all tools to do that verification? > Given that upstream went ahead with that change, I don't plan doing extra > work for sid and forky, that'll happen in due time when I package the next > upstream release. Makes sense. > Sadly, it will be at least mid october until I will have time to do that. So > the TC can take the time to decide whether to go forward or not. Ack. > I really appreciate the work you did on this. Thank you for being careful with this important package! Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Wed, Sep 24, 2025 at 07:19:37PM +0200, Christoph Berg wrote: Around the time of the discussion, upstream sudo included a change that limits -fcf-protection to x86_64: https://github.com/sudo-project/sudo/pull/468 The problem that I have with this change is that it was suggested by the same individual who wants us to do this change in Debian. Only sudo upstream didn't push back as hard as I did. Upstream mentioned in the upstream issue: "https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html explicitly mentions x86_64 for when to use `-fcf-protection`." I planned to talk to upstream about this, but it looks like I didnt get a round tuit quickly enough to actually do that. I apologize. Given your summary [snipped] I would like to thank you for that research. I don't have anything to add to your sound reasoning. It was important to me to hear that from a source I trust. That is the case now. Possible TC rulings are: * agree with the submitter. -fcf-protection is no-op on i386; the sudo package should be updated. * reject the request; changing sudo for a very small number of users is too risky (FWIW, the patch from https://github.com/sudo-project/sudo/pull/468 applies trivially to sudo 1.9.13p3-1+deb12u2) * reject the request; bookworm is already oldstable (if it's reaching us only now, it's not that important) * reject the request; the CPU in question is not part of the baseline Marc has indicated that he would accept advice on this issue so we might go with issuing that instead of formally overriding him. I reaffirm that. Should the TC decline to give formal advice (which I would be fine with), I would go ahead to disable -fcf-protection for i386 builds (and verify that the amd64 and arm64 binary stay identical) and build packages for trixie and bookworm, submit both of them for the next point release. Given that upstream went ahead with that change, I don't plan doing extra work for sid and forky, that'll happen in due time when I package the next upstream release. Sadly, it will be at least mid october until I will have time to do that. So the TC can take the time to decide whether to go forward or not. I really appreciate the work you did on this. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Christoph, > The submitter claims the CPU is conformant, > citing https://psc.informatik.uni-jena.de/hw/p-pro-3.pdf page 417 as saying > ENDBR32 was "reserved". (The URL doesn't load for me now so I can't validate.) Unfortunately Intel does not seem have archived versions of their manuals available for download from their website. However I found an even newer version from 2010 here (Core Nehalem era), which contains opcode maps: https://ee209-2019-spring.github.io/references/253667.pdf Page 661 states: "All blanks in opcode maps are reserved and must not be used. Do not depend on the operation of undefined or blank opcodes." ENDBR32 is F3 0F 1E FB, where: - F3 is the REP prefix - 0F 1E is the opcode - FB the ModR/M. Looking at page 673 where 0F XX opcodes are, NOPL (0F 1F) and PREFETCH (0F 18) both appear, but the space that would belong to 0F 1E is empty, hence "reserved". So, according to Intel's own manual, these instructions must not be used on 2010 32-bit processors - and certainly not on 1995 Pentium III (i686) processors that Bookworm targets. If anything, I'd say Vortex86 processor is actually here in the right for raising UD# on a instruction that the manual states is reserved and invalid! > * reject the request; bookworm is already oldstable (if it's reaching us > only now, it's not that important) Just a tiny note: the first time this was reported was back in 2023-10-17 by Justin on the debian-devel list (which you've linked) - four months Bookworm was declared stable. The amount of affected people certainly isn't going to be high (these are not super popular processors after all), but I bet many affected would have found that bug report (I did) and assumed it was not to be fixed, so there'll certainly be underreporting. Greetings, Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 03/09/2025 a las 13:42, Helmut Grohne escribió: > It also is enabled in forky/sid. While we somewhat disagree on the > importance of old i386 hardware on this matter, would you mind > additionally questioning the usefulness of -fcf-protection (=full) as > opposed to -fcf-protection=return to the project? I suggest that you > report a wishlist bug against dpkg-dev (which contains our default build > flags) and X-Debbugs-Cc: [email protected] to try to change > this for unstable. Hello Helmut, I've opened it as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1113864 Greetings, Marcos (not Macros! :D)
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Macros, On Wed, Sep 03, 2025 at 12:38:28PM +0200, Marcos Del Sol Vives wrote: > I am surprised that bare "-fcf-protection" was enabled by default on Trixie, > as that enables both shadow stacks (supported in user mode, actually > protecting users with zero size overhead) and IBT (not supported in > user-mode, doing nothing but increasing thus the size of all binaries) It also is enabled in forky/sid. While we somewhat disagree on the importance of old i386 hardware on this matter, would you mind additionally questioning the usefulness of -fcf-protection (=full) as opposed to -fcf-protection=return to the project? I suggest that you report a wishlist bug against dpkg-dev (which contains our default build flags) and X-Debbugs-Cc: [email protected] to try to change this for unstable. Let me also note that Ubuntu sets -fcf-protection=none on amd64. The original bug adding -fcf-protection is #1021292. According to Wookey, RedHat enterprise sets -fcf-protection since 2018. > Enabling "-fcf-protection=return" for Trixie which compiles with only > shadow stacks would have resulted in smaller binaries with the same level > of protection (and also would fix the issue with "sudo" for these i686). This feels like a rather convincing case for changing our distribution default from -fcf-protection (with implied value "full") to -fcf-protection=return. One of Marc's complaints is that removing the flag could lower security. Now you indicate that removing "half" of the flag would be sufficient for your cause and that the other half could still have a positive effect. Beware that we will also take the affected user base and timeline into account. Even if the TC ends up agreeing with the technical presentation given, we may still favour not changing sudo in bookworm given an expectation of this affecting too few relevant machines and users. Helmut
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 03/09/2025 a las 13:11, Marc Haber escribió: > What I don't understand is why those people running those rather exotic > machines don't just run another privilege escalation tool like runas or > compile their own sudo package. I am using a self-built version of "sudo" for that particular machine with the fix applied, but it'd be probably nice for all other people that need/want/like to use these machines that it'd all worked out of the box, as most people using a x86 machine would expect. El 03/09/2025 a las 13:20, Marc Haber escribió: > So you're saying this compiled option could be disabled in sudo for i386 > without any loss of security regardless on what CPU the program runs and > regardless whether it runs under a 32bit or a 64bit kernel, right? Yes. IMHO two things could be done: - Apply only for x86-64, as upstream is doing now. - Specify "-fcf-protection=return" instead of just "-fcf-protection". That disables IBT (unsupported anyway) but keeps shadow stacks enabled for both 32-bit and 64-bit (as mentioned, only would protect users on amd64, but unlike IBT will not hurt compatibility on i386) > Why isnt -fcf-protection a no-op in the toolchain on i386 then? I am not sure because I am not a toolchain developer, but I assume it's not the task of the toolchain to check if they are gonna have an effect on your target or not. As I already said, you could compile with -msse2 even if your CPU does not support SSE2 either. >> Note the kernel .config flag for one is "CONFIG_X86_USER_SHADOW_STACK" >> vs "CONFIG_X86_KERNEL_IBT", as one works only in user mode, and the other >> only in kernel mode. > > Both settings are =Y in the x86_64 kernel in Debian unstable. My point was that the name actually indicates whether they are supported in user mode (_USER_) and kernel (_KERNEL_). IBT is, as indicated by the config name, kernel-only. You don't have to trust me on that - I send a couple minutes earlier a program demonstrating that enabling IBT for user programs results in no security protection whatsoever, not even in amd64: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1113774#96 The test was actually conducted by someone that told me knows you, as my main AMD 8745H desktop only supports shadow stack. Feel free to ask that person directly. Have a nice day, Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Tue, Sep 02, 2025 at 04:13:46PM +0200, Marcos Del Sol Vives wrote: El 02/09/2025 a las 16:06, Paul Tagliamonte escribió: On Tue, Sep 02, 2025 at 03:59:03PM +0200, Christoph Berg wrote: Yeah I think you were right in rejecting this. I would need to read more to "get smart" here, but I think another factor to consider is the number of "true" i686 processers running this release vs the number of x86_64 processors running this release under an x86_64 kernel. So you're saying this compiled option could be disabled in sudo for i386 without any loss of security regardless on what CPU the program runs and regardless whether it runs under a 32bit or a 64bit kernel, right? CET's shadow stacks is not compatible with 32-bit user-mode binaries, neither in native 32-bit nor in a 64-bit kernel running 32-bit binaries. Keeping that enabled would do no harm, though no good either. And CET's IBT, the feature that is introducing this incompatibility, is not enabled with user mode applications at all. Why isnt -fcf-protection a no-op in the toolchain on i386 then? Note the kernel .config flag for one is "CONFIG_X86_USER_SHADOW_STACK" vs "CONFIG_X86_KERNEL_IBT", as one works only in user mode, and the other only in kernel mode. Both settings are =Y in the x86_64 kernel in Debian unstable. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi,
On Tue, Sep 02, 2025 at 11:28:22AM -0400, Paul Tagliamonte wrote:
The natural outcome here seems to be:
a) do nothing as-is, some fraction of supported but non-intel CPUs will
get runtime failures, since we've altered the ISA baseline and
never realized it due to popularity
b) remove this flag from sudo specifically, fixing sudo specifically
in bookworm (oldstable)
c) change all i386 package flags for bookworm specifically
(oldstable) and binNMU the whole archive, FTBFS and all
d) declare bookworm i386 retroactively always was a different ISA baseline
("We have always been at war with Eastasia")
What I don't understand is why those people running those rather exotic
machines don't just run another privilege escalation tool like runas or
compile their own sudo package.
I would be willing to provide a signed and patched sudo outside the
debian archive (we still do have people.debian.org, right?) to get this
settled if we can't find consensus that the change is indeed harmless.
Greetings
Marc
--
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Tue, Sep 02, 2025 at 05:46:27PM +0200, Andrea Pappacoda wrote: I personally find the argument of whether Marcos' CPU is supported not really persuasive, since, if I got this correct, that compiler option is doing nothing good and just causing issues to a subset of our users. I think that this is what the entire thing boils down. I am unwilling to disable that compiler option if there is a feather of a possibility that doing so would decrease security for systems that do support the opcode in question. If we (that means Debian, the TC or some other part that I have trust in) come to the consensus that it all our release architectures are well served with full security even if -fcf-protection is just set for x86_64, I am fine with doing that changes and providing an appropriately patched version for bookworm (and trixie). I am not close enough to this level of systems programming to have my own informed knowledge about this matter, but I need that advice coming from a body that I trust. We had the OpenSSL random generator desaster from 2008 originating from not well given upstream advice, and I don't want to repeat this. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hello, On Tue, Sep 02, 2025 at 06:28:12PM +0200, Helmut Grohne wrote: Marc, in https://lore.kernel.org/all/[email protected]/ you say that you require a TC maintainer override to implement the change whereas in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1113774#15 you suggest that TC advice would be sufficient to you. Can you clarify which procedural level you require here? I would be fine with everything. Advice would be good for the beginning. Maybe I end up convinced and then would not need to be overridden. I don't know enough about the issue to have a firm opinion yet. Your expertise is appreciated. From a technical point of view, I note that -fcf-protection is not enabled for i386 at the toolchain level for any Debian release. It was added to the default flags for amd64 in trixie. This wasn't fully evident from the discussion to me. It really is sudo that is adding this flag. https://sources.debian.org/src/sudo/1.9.13p3-1%2Bdeb12u1/m4/hardening.m4#L108 sudo upstream did that back in 2021. The submitter of this TC bug report convinced Upstream to only enable this on x86_64 recently. I don't know whether this makes sense; upstream accepted the patch. Who am I to argue with upstream? This however will never apply to oldstable with the submitter wants changed. There seem to be two major arguments involved both of which I have not yet verified in depth. 1. The -fcf-protection flag bears no benefit in 32bit user applications. 2. The ENDBR32 instruction inserted by -fcf-protection is not supported in some CPUs that were considered supported by bookworm's baseline. In principle, this is a baseline violation and would usually be considered a release-critical bug. In sudo? In the toolchain? in whatever provides -fcf-protection? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 18:28, Helmut Grohne escribió:> From a technical point of
view, I note that -fcf-protection is not
> enabled for i386 at the toolchain level for any Debian release. It was
> added to the default flags for amd64 in trixie. This wasn't fully
> evident from the discussion to me. It really is sudo that is adding this
> flag.
> https://sources.debian.org/src/sudo/1.9.13p3-1%2Bdeb12u1/m4/hardening.m4#L108
>
> There seem to be two major arguments involved both of which I have not
> yet verified in depth.
>
> 1. The -fcf-protection flag bears no benefit in 32bit user applications.
> 2. The ENDBR32 instruction inserted by -fcf-protection is not supported
>in some CPUs that were considered supported by bookworm's baseline.
Hello Helmut,
I am surprised that bare "-fcf-protection" was enabled by default on Trixie,
as that enables both shadow stacks (supported in user mode, actually
protecting users with zero size overhead) and IBT (not supported in
user-mode, doing nothing but increasing thus the size of all binaries)
I can easily prove that (and also your point 1) with one simple C program:
--
#include
#include
void test_func() {
puts("IBT not working\n");
}
int main() {
void (* func_ptr)();
/*
* ENDBR32 is F3 0F 1E FB
* ENDBR64 is F3 0F 1E FA
*
* Both are 4-bytes long. By adding 4 to the start of the function,
* we skip past the ENDBR that IBT uses as landing instructions.
*/
func_ptr = (void *) ((uint8_t *) test_func + 4);
/*
* Now perform an invalid indirect call.
*
* If IBT actually worked, the program would be immediately halted now
* since the first fetched after an indirect call when IBT is enabled
* MUST be an ENDBR, which is not as we skipped it.
*
* However, as I said, since IBT is not supported in user-mode, this
* will run just fine on both 32-bit and 64-bit, proving IBT at this
* point just increases code size, causes compatibility issues on these
* i686 machines, all while providing no security whatsoever.
*/
func_ptr();
return 0;
}
--
Compiling this with "gcc -fcf-protection ibt.c -o ibt && ./ibt" on
an IBT-capable machine will run without problems. On an Intel N100 a
friend (Manawyrm, which tells me Marc Haber knows about) has:
root@vmhost ~ # gcc -fcf-protection ibt.c -o ibt
root@vmhost ~ # ./ibt
IBT not working
root@vmhost ~ # gcc ibt.c -o ibt
root@vmhost ~ # ./ibt
IBT not working
Speicherzugriffsfehler
root@vmhost ~ # cat /proc/cpuinfo | grep Intel
vendor_id : GenuineIntel
model name : Intel(R) N100
vendor_id : GenuineIntel
model name : Intel(R) N100
vendor_id : GenuineIntel
model name : Intel(R) N100
vendor_id : GenuineIntel
model name : Intel(R) N100
root@vmhost ~ # uname -a
Linux vmhost 6.5.0-0.deb12.4-amd64 #1 SMP PREEMPT_DYNAMIC Debian
6.5.10-1~bpo12+1 (2023-11-23) x86_64 GNU/Linux
root@vmhost ~ # cat /etc/debian_version
12.5
This is on a Bookworm installation but the same should apply for Trixie,
Sid or any other Linux - it's a limitation of the kernel. Not even in the
current Linux's master branch IBT is supported for user-mode.
(Note that the execution without -fcf-protection crashes with
"Speicherzugriffsfehler" - SIGSEGV - because it's jumping past the
first PUSH instruction that stores the return address on the stack,
instead of the ENDBR64 that the former is)
Enabling "-fcf-protection=return" for Trixie which compiles with only
shadow stacks would have resulted in smaller binaries with the same level
of protection (and also would fix the issue with "sudo" for these i686).
Greetings,
Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Tue, Sep 02, 2025 at 03:59:03PM +0200, Christoph Berg wrote: Yeah I think you were right in rejecting this. I would need to read more to "get smart" here, but I think another factor to consider is the number of "true" i686 processers running this release vs the number of x86_64 processors running this release under an x86_64 kernel. My understanding from a quick read of the docs here (although, about 2 minutes worth so i'm very open to being convinced otherwise here) is that disabling this would disable CET for sudo:i386 when running under an amd64 kernel, in order to allow a i586 to run a i686 binary. paultag -- ⢀⣴⠾⠻⢶⣦⠀ Paul Tagliamonte ⣾⠁⢠⠒⠀⣿⡁ https://people.debian.org/~paultag | https://pault.ag/ ⢿⡄⠘⠷⠚⠋Debian, the universal operating system. ⠈⠳⣄⠀⠀ 4096R / FEF2 EB20 16E6 A856 B98C E820 2DCD 6B5D E858 ADF3 signature.asc Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi all! On Tue Sep 2, 2025 at 5:28 PM CEST, Paul Tagliamonte wrote: b) remove this flag from sudo specifically, fixing sudo specifically in bookworm (oldstable) c) change all i386 package flags for bookworm specifically (oldstable) and binNMU the whole archive, FTBFS and all While I too think that option "c" is the most correct one, it seems that nobody really complained much for packages other than sudo. So why not fixing it only there? I personally find the argument of whether Marcos' CPU is supported not really persuasive, since, if I got this correct, that compiler option is doing nothing good and just causing issues to a subset of our users. Hope it makes sense! Bye :)
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Re: Helmut Grohne > Christoph, Paul, Stefano, you've all been replying quickly. Would any of > you have capacity to take the moderation role? I prefer not to at > this time. I can do that. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Paul (2025.09.02_15:28:22_+) c) change all i386 package flags for bookworm specifically (oldstable) and binNMU the whole archive, FTBFS and all I think packages had to specifically request -fcf-protection and not many did. (We have no codesearch for bookworm, of course). So this is not *quite* as insane as it sounds. But... still a non-trivial amount of work to support some old CPUs on our oldstable release. Stefano
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 16:06, Paul Tagliamonte escribió: > On Tue, Sep 02, 2025 at 03:59:03PM +0200, Christoph Berg wrote: >> Yeah I think you were right in rejecting this. > > I would need to read more to "get smart" here, but I think another factor to > consider is the number of "true" i686 processers running this release vs the > number of x86_64 processors running this release under an x86_64 kernel. > > My understanding from a quick read of the docs here (although, about 2 > minutes worth so i'm very open to being convinced otherwise here) is that > disabling this would disable CET for sudo:i386 when running under > an amd64 kernel, in order to allow a i586 to run a i686 binary. > > paultag > Hi Paul, CET's shadow stacks is not compatible with 32-bit user-mode binaries, neither in native 32-bit nor in a 64-bit kernel running 32-bit binaries. Keeping that enabled would do no harm, though no good either. And CET's IBT, the feature that is introducing this incompatibility, is not enabled with user mode applications at all. Note the kernel .config flag for one is "CONFIG_X86_USER_SHADOW_STACK" vs "CONFIG_X86_KERNEL_IBT", as one works only in user mode, and the other only in kernel mode. Greetings, Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Re: Marc Haber > The Geode is an i586 machine that doesn't support the full i686 instruction > set. As far as I know, we stopped supporting i586 iterally decades ago. i586 was unsupported at least since bookworm: https://www.debian.org/releases/bookworm/i386/release-notes/ch-information.en.html#i386-is-i686 https://wiki.debian.org/ArchitectureSpecificsMemo#i386-1 > That's why I said - repeatedly - no to the request. Yeah I think you were right in rejecting this. Marcos: If you want to keep using this i586 machine, you will have to stick with bullseye (which still has LTS support). After that - sorry, the world has moved on. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Control: tags -1 - moreinfo Hi Marcos and Marc, Your request is received. Thanks for providing extensive detail and adding more where questions have been asked. Can we all slow down a bit to avoid getting repetitive? I am removing the moreinfo tag for now as we need a bit of time to digest what's there. Marc, in https://lore.kernel.org/all/[email protected]/ you say that you require a TC maintainer override to implement the change whereas in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1113774#15 you suggest that TC advice would be sufficient to you. Can you clarify which procedural level you require here? >From a technical point of view, I note that -fcf-protection is not enabled for i386 at the toolchain level for any Debian release. It was added to the default flags for amd64 in trixie. This wasn't fully evident from the discussion to me. It really is sudo that is adding this flag. https://sources.debian.org/src/sudo/1.9.13p3-1%2Bdeb12u1/m4/hardening.m4#L108 There seem to be two major arguments involved both of which I have not yet verified in depth. 1. The -fcf-protection flag bears no benefit in 32bit user applications. 2. The ENDBR32 instruction inserted by -fcf-protection is not supported in some CPUs that were considered supported by bookworm's baseline. In principle, this is a baseline violation and would usually be considered a release-critical bug. An argument against this change is that bookworm has been released more than two years ago and that indicates that the number of systems affected by this problem cannot be huge. Christoph, Paul, Stefano, you've all been replying quickly. Would any of you have capacity to take the moderation role? I prefer not to at this time. Helmut
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 17:28, Paul Tagliamonte escribió:
> The natural outcome here seems to be:
>
> a) do nothing as-is, some fraction of supported but non-intel CPUs will
> get runtime failures, since we've altered the ISA baseline and
> never realized it due to popularity
>
> b) remove this flag from sudo specifically, fixing sudo specifically
> in bookworm (oldstable)
>
> c) change all i386 package flags for bookworm specifically (oldstable)
> and binNMU the whole archive, FTBFS and all
>
> d) declare bookworm i386 retroactively always was a different ISA baseline
> ("We have always been at war with Eastasia")
Hello Paul,
B would be in my opinion the solution. So far, I have not found any other
package that has these issues. Not even running a full-fledged desktop
environment (XFCE with X11) has been an issue.
This flag is present on sudo, because the "hardening.m4" file it contains
explicitelly enabled "-fcf-protection".
Considering all reports I've personally seen of illegal instruction on
these kind i686 processors have been limited to "sudo" in Debian, Gentoo
and other BSDs, I assume the amount of packages that I assumed have enabled
"-fcf-protection=full" or "-fcf-protection=branch" will be probably
minimal.
Greetings,
Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 17:14, Christoph Berg escribió: > Re: Stefano Rivera >> It seems that the intention of the new instruction was to be interpreted as >> a NOP on older hardware, but that obviously didn't happen on these non-Intel >> CPUs. > > https://lists.debian.org/debian-devel/2023/10/msg00120.html states > these processors are unsupported. > > Christoph The Pentium Pro manual, page 417, does not define instruction ENDBR32. https://psc.informatik.uni-jena.de/hw/p-pro-3.pdf. They are "reserved". If you are targetting and advertising i686-compatibility, do not use instructions that are not present in the Pentium Pro. What is the exact goal of keeping these instructions, even? I keep seeing you refuse because "it is not supported" but I don't understand what you wanna achieve exactly keeping them when the kernel will not support IBT.
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
On Tue, Sep 02, 2025 at 05:14:27PM +0200, Christoph Berg wrote:
Re: Stefano Rivera
It seems that the intention of the new instruction was to be interpreted as
a NOP on older hardware, but that obviously didn't happen on these non-Intel
CPUs.
https://lists.debian.org/debian-devel/2023/10/msg00120.html states
these processors are unsupported.
It's tough because the link is gone now (since it was /testing/ at the
time that email was sent) -- but in reply, Ben adds[1]
Sorry, the page I linked is for testing ("trixie", which will become
Debian 13). Debian 12 "bookworm" is supposed to still support this
CPU.
I also appreciate the detail about this CPU instruction being disused on
all bin:i386 even if the kernel is amd64.
I've grown a bit more sympathetic to the argument here; but I'm still
not 100% what to think of this.
The natural outcome here seems to be:
a) do nothing as-is, some fraction of supported but non-intel CPUs will
get runtime failures, since we've altered the ISA baseline and
never realized it due to popularity
b) remove this flag from sudo specifically, fixing sudo specifically
in bookworm (oldstable)
c) change all i386 package flags for bookworm specifically (oldstable)
and binNMU the whole archive, FTBFS and all
d) declare bookworm i386 retroactively always was a different ISA baseline
("We have always been at war with Eastasia")
It seems to me that option "c" here is a nonstarter, even though it's
likely the correct way to go about this. If bookworm was still testing
and we found this, I can't imagine we'd do anything *except* that route
(to Marc's point -- which, I think that's right -- why is sudo
special-cased here besides "it's run a lot" and why isn't this
archive-wide if it's truely a noop?)
[1]: https://lists.debian.org/debian-devel/2023/10/msg00128.html
--
⢀⣴⠾⠻⢶⣦⠀ Paul Tagliamonte
⣾⠁⢠⠒⠀⣿⡁ https://people.debian.org/~paultag | https://pault.ag/
⢿⡄⠘⠷⠚⠋Debian, the universal operating system.
⠈⠳⣄⠀⠀ 4096R / FEF2 EB20 16E6 A856 B98C E820 2DCD 6B5D E858 ADF3
signature.asc
Description: PGP signature
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Re: Stefano Rivera > It seems that the intention of the new instruction was to be interpreted as > a NOP on older hardware, but that obviously didn't happen on these non-Intel > CPUs. https://lists.debian.org/debian-devel/2023/10/msg00120.html states these processors are unsupported. Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi Marcos (2025.09.02_14:35:01_+) They do not, however, support ENDBR32 which was introduced in 2020, twenty-five years after the introduction of the original i686 (https://www.intel.com/content/www/us/en/developer/articles/technical/technical-look-control-flow-enforcement-technology.html). It seems that the intention of the new instruction was to be interpreted as a NOP on older hardware, but that obviously didn't happen on these non-Intel CPUs. Stefano -- Stefano Rivera
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 14:48, Christoph Berg escribió: > Control: tag -1 moreinfo > > Re: Marcos Del Sol Vives >> Since bookworm is the last i686 release, I think it'd make sense to fix >> this issue. > > Hi Marcos, > > did you discuss this with the sudo maintainer? Where is the technical > disagreement that led you to escalate to the TC? > > Christoph Hello Christoph, We discussed it in multiple places: - The PR in the Salsa repo: https://salsa.debian.org/sudo-team/sudo/-/merge_requests/24 - The PR I opened for upstream: https://github.com/sudo-project/sudo/pull/468#issuecomment-3244198193 - The LKML: https://lore.kernel.org/all/[email protected]/ He's asked me three times to contact the technical comittee (in the LKML and in the PR, plus another via private email), and directly said that "The change [...] will not happen in Debian unless the technical committee overrides my decision", so I just did as he asked me. Sorry if there were any other procedure to be followed, because I am not familiar with this process. Greetings, Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 15:58, Matthew Vernon escribió: > Hi, > > I found an old d-devel thread about this from a few years ago[0], which led > me to the bookworm release notes[1] which say: > > "Debian's support for 32-bit PC (known as the Debian architecture i386) now > no longer covers any i586 processor. The new minimum requirement is i686. > What this means that the i386 architecture now requires the "long NOP" (NOPL) > instruction, while bullseye still supported some i586 processors without that > instruction (e.g. the "AMD Geode"). " > > As I read it, the conclusion of that d-d thread was that these processors are > not supported in Bookworm, as it only supports i686, which the CPU that the > submitter of this TC bug is using is not completely i686 compatible. > > Regards, > > Matthew > > [0] https://lists.debian.org/debian-devel/2023/10/msg00118.html > [1] > https://www.debian.org/releases/bookworm/i386/release-notes/ch-information.en.html#i386-is-i686 Hello Matthew, Both that VIA C3 Nehalem and this Vortex86DX3 support long NOPs natively, and CMOV which were also introduced with the i686 in 1995. The Vortex86DX3, in fact, supports up to SSE1. They do not, however, support ENDBR32 which was introduced in 2020, twenty-five years after the introduction of the original i686 (https://www.intel.com/content/www/us/en/developer/articles/technical/technical-look-control-flow-enforcement-technology.html). If Debian bookworm advertises itself as i686-compatible, it seems reasonable to me that it does not require instructions that were introduced long after the original i686. Greetings, Marcos
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
El 02/09/2025 a las 15:48, Marc Haber escribió: > The Geode is an i586 machine that doesn't support the full i686 instruction > set. As far as I know, we stopped supporting i586 iterally decades ago. Via Nehalem C3 and Vortex86DX3 are i686. Otherwise the kernel would be completely unbootable. And I know because that is the case for other i586 machines (eg Vortex86MX) If you are targetting a i686, a Pentium Pro, that makes even less reasonable that you are enabling a security feature that was introduced in 2020 and that breaks on i686-era processors. > The OP is suggesting to disable a security feature for i386 so that sudo (and > other software that uses -fcf-protection) can run on their CPU that was never > officially supported in bookworm. They're claiming that this option is a > no-op on i386 anyway, but why is it enabled in our toolchain then? Should > this issue not be addressed in the toolchain? It is enabled in the toolchain for the same reason you can use AVX2 with unsupported processors - it is your duty as a programmer to use compatible flags. The sudo maintainer used the flag because the thought it was supposed to enable all protection on processors that supported it, such as AArch64, which is not the case and why he agreed to enable explicitely on AMD64 only.
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi, I found an old d-devel thread about this from a few years ago[0], which led me to the bookworm release notes[1] which say: "Debian's support for 32-bit PC (known as the Debian architecture i386) now no longer covers any i586 processor. The new minimum requirement is i686. What this means that the i386 architecture now requires the "long NOP" (NOPL) instruction, while bullseye still supported some i586 processors without that instruction (e.g. the "AMD Geode"). " As I read it, the conclusion of that d-d thread was that these processors are not supported in Bookworm, as it only supports i686, which the CPU that the submitter of this TC bug is using is not completely i686 compatible. Regards, Matthew [0] https://lists.debian.org/debian-devel/2023/10/msg00118.html [1] https://www.debian.org/releases/bookworm/i386/release-notes/ch-information.en.html#i386-is-i686
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Hi sudo maintainer here. On Tue, Sep 02, 2025 at 02:48:14PM +0200, Christoph Berg wrote: did you discuss this with the sudo maintainer? They did. I said no. The Geode is an i586 machine that doesn't support the full i686 instruction set. As far as I know, we stopped supporting i586 iterally decades ago. The OP is suggesting to disable a security feature for i386 so that sudo (and other software that uses -fcf-protection) can run on their CPU that was never officially supported in bookworm. They're claiming that this option is a no-op on i386 anyway, but why is it enabled in our toolchain then? Should this issue not be addressed in the toolchain? When I asked for clarification I got screenfuls of technobabble. I am neither a toolchain person nor a kernel person. I am just responsible for a security relevant package. That's why I said - repeatedly - no to the request. I am open to advice from the TC since I know that I can trust you. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Control: tag -1 moreinfo Re: Marcos Del Sol Vives > Since bookworm is the last i686 release, I think it'd make sense to fix > this issue. Hi Marcos, did you discuss this with the sudo maintainer? Where is the technical disagreement that led you to escalate to the TC? Christoph
Bug#1113774: Disabling -fcf-protection in sudo for bookworm
Package: tech-ctte Severity: normal Currently "sudo" in Bookworm is broken on i686 for some i686-like processors such as a Vortex86DX3 I own and VIA processors others have (https://lists.debian.org/debian-devel/2023/10/msg00118.html), causing a SIGILL if you attempt to run on them. The issue is that sudo in bookworm is compiled with "-fcf-protection=full", which causes binaries to contain ENDBR32 instructions. These are part of Indirect Branch Tracking, a mechanism introduced by Intel's CET meant to harden against exploits using return-oriented programming. END32s are part of a formely-reserved chunk of instructions called "hintable NOPs". These, to my knowledge, were only defined in US patent US5701442A but not on the software design manual, where they appeared as "reserved". Thus these processors do what in general reserved instructions should do - raise an exception. Disabling a security mechanism for these uncommon processors would be probably a bad idea, but the thing is that this mechanism is _not_ supported by the Linux kernel in user mode. Thus, the compilation with IBT just increases the size of the binaries at best, and prevents running the binary at worst. For user-mode, and only in 64-bit mode (but not on native 32-bit system, neither when running 32-bit applications on a 64-bit kernel), Linux uses another mechanism that does not require new instructions called shadow stacks. This is documented in the Kernel's own page about CET: https://docs.kernel.org/arch/x86/shstk.html#cet-background There it explicitely says that the protections are only available in 64-bit modes, and further analysis of the kernel's code I've done confirms that being the case: https://lore.kernel.org/all/[email protected]/ I've submitted a patch against upstream sudo that has been accepted (https://github.com/sudo-project/sudo/pull/468) that enables the compilation with -fcf-protection only for 64-bit mode, but the current sudo Debian maintainer has refused to accept a patch for "bookworm" that fixes this issue, being concerned that it could lower the overall security of the binary. As part of an effort to handle these ENDBR32s in the kernel and ignore them, H. Peter Anvin (a major x86 arch maintainer in the Linux kernel) confirmed that 32-bit user-mode applications do not gain any security from using ENDBR32, and that just disabling the IBT protection for them would be the best approach: https://lore.kernel.org/all/[email protected]/ Since bookworm is the last i686 release, I think it'd make sense to fix this issue. Greetings, Marcos
