Re: xz backdoor

[Christoph Anton Mitterer]

Hey.

Seems some of the reverse engineers may have found some more
interesting stuff[0].


As far as I understand it, that would still require a running an
reachable sshd (so we'd still be mostly safe).

But he also thinks[1] that it may allow an interactive session.
(Not that this would change a lot, if the adversary can use system().)


Still, shows that there may be more hidden stuff, and we may not be out
of the woods yet.


Cheers,
Chris.

[0] https://twitter.com/bl4sty/status/1776691497506623562
[1] https://twitter.com/bl4sty/status/1776692874232434932

Re: xz backdoor

[Pierre-Elliott Bécue]

Hello,

Michael Shuler  wrote on 06/04/2024 at 16:31:28+0200:

> On 4/5/24 10:30, Pierre-Elliott Bécue wrote:
>> Pierre-Elliott Bécue  wrote on 31/03/2024 at 14:31:37+0200:
>>> Wookey  wrote on 31/03/2024 at 04:34:00+0200:
>>>
 On 2024-03-30 20:52 +0100, Ansgar  wrote:
> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
> Possibly also TPM modules in computers.
>
> These can usually be used for both OpenPGP and SSH keys.

 Slightly off-topic, but a couple of recent posts have given me the
 same thought:

 Can someone point to good docs on this?  I've had a yubikey for 3/4 of
 a year now but have not yet worked out how I put my GPG key in it. (or
 if it should be another key, or a subkey, or whatever). So I'm not
 actually using it yet.

 PEB also described what sounded like a very sensible way to manage
 keys (using subkeys) in one of these threads but I don't know how to
 do that myself.
>>>
>>> I have started (and never finished) a blog article on how I use my
>>> YubiKey and what config I put in it. I'll definitely try to get it out
>>> before the end of next week. I'll probably extend it to mention the
>>> creation of GPG subkeys etc.
>>>
>>> I would also be happy if it helps my fellow DDs to try making an article
>>> about some basic crypto concepts regarding PGP, RSA et al. But not in
>>> the same piece I guess.
>> Hello,
>> For those interested in: I've published two articles:
>>   1. One on PGP subkeys https://pe.becue.phd/openpgp-subkeys
>>   2. One on the OpenPGP module of YubiKeys:
>>  https://pe.becue.phd/yubikey-workfow-openpgp
>> I'm happy to receive any kind of constructive feedback.
>> 
>
> Thank you so much for working on these. I last-minute cobbled together
> a BOF on GPG Key Best Practices at Columbia in 2010, since the topic
> came up in another talk. I was blown away at how much I did not know,
> the complexity, as well as how many people crammed in that room -
> definitely there are interested people (I think Wookey was there,
> too?). I include myself in each of the things others mentioned, that I
> should have been doing since then, but just never got around to.. At
> least I now have a fist full of Yubikeys to play with, as we use them
> at work, so thanks for your work. I appreciate it, and I'm guessing
> there's a rather large, quiet group of people thinking the same.

I'm very happy if it helps as little as one person.

I do intend to add articles to both series, as I think these topics are
really interesting, and having a good knowledge is a good way to take
educated decisions.

Thanks forn your feedback!

-- 
PEB
Nota: I did make some changes based on some feedback I already received,
thanks for those having spent the time reading.


signature.asc
Description: PGP signature

Re: xz backdoor

[Michael Shuler]

On 4/5/24 10:30, Pierre-Elliott Bécue wrote:

Pierre-Elliott Bécue  wrote on 31/03/2024 at 14:31:37+0200:

Wookey  wrote on 31/03/2024 at 04:34:00+0200:


On 2024-03-30 20:52 +0100, Ansgar  wrote:

Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
Possibly also TPM modules in computers.

These can usually be used for both OpenPGP and SSH keys.


Slightly off-topic, but a couple of recent posts have given me the
same thought:

Can someone point to good docs on this?  I've had a yubikey for 3/4 of
a year now but have not yet worked out how I put my GPG key in it. (or
if it should be another key, or a subkey, or whatever). So I'm not
actually using it yet.

PEB also described what sounded like a very sensible way to manage
keys (using subkeys) in one of these threads but I don't know how to
do that myself.


I have started (and never finished) a blog article on how I use my
YubiKey and what config I put in it. I'll definitely try to get it out
before the end of next week. I'll probably extend it to mention the
creation of GPG subkeys etc.

I would also be happy if it helps my fellow DDs to try making an article
about some basic crypto concepts regarding PGP, RSA et al. But not in
the same piece I guess.


Hello,

For those interested in: I've published two articles:

  1. One on PGP subkeys https://pe.becue.phd/openpgp-subkeys
  2. One on the OpenPGP module of YubiKeys:
 https://pe.becue.phd/yubikey-workfow-openpgp

I'm happy to receive any kind of constructive feedback.



Thank you so much for working on these. I last-minute cobbled together a 
BOF on GPG Key Best Practices at Columbia in 2010, since the topic came 
up in another talk. I was blown away at how much I did not know, the 
complexity, as well as how many people crammed in that room - definitely 
there are interested people (I think Wookey was there, too?). I include 
myself in each of the things others mentioned, that I should have been 
doing since then, but just never got around to.. At least I now have a 
fist full of Yubikeys to play with, as we use them at work, so thanks 
for your work. I appreciate it, and I'm guessing there's a rather large, 
quiet group of people thinking the same.


Kind regards,
Michael

Re: xz backdoor

[Christoph Anton Mitterer]

On Fri, 2024-04-05 at 20:47 +0200, Sirius:
> > If there is a final result, can we as a project share the results on a
> > prominent place? Or at least under d-devel-announce and/or d-security-
> > announce? I was also wondering about what could have been compromised,
> > what data might have been stolen, etc. And there is so many sources to
> > follow right now. So sharing the final results would be great. 
> 
> If you have followed the discussion on Openwall ML, there have been a
> couple of posts that points at both a general overview of what the code
> did, an analysis of how the data was hidden in the 'corrupt' xz archive
> under testing and some analysis of the actual .o which suggested this was
> not just a backdoor but a remote-code-execution portal almost.

I've also tried to follow the various lists and RE efforts on discord.
My understanding is, that this hasn't been completed, yet, and while
people seem to *believe* that it looks like as if the backdoor didn't
do anything else than just waiting for commands sent to an sshd (which
might make all people safe, that haven't had sshd running or at least
not publicly listening) - that's not yet 100% sure, or is it?

And given how much effort these attackers spent in hiding the stuff, it
doesn't seem impossible, that they hid even more.


I'd think that most servers are safe, simply because they typically run
stable.
But I guess many people run their personal computers on some
rolling/unstable release.


So I fully agree with Daniel Leidert, that it would be really nice if
there was - eventually, one the reverse engineering has been finished -
some form of official confirmation, whether and when people that had
the compromised xz-utils installed may fell 100% safe or possibly
pwned.


Especially:
- whether any hidden calling home was found (so far not, but this may
  e.g. happen only under special conditions, like some matching host
  or user names), which would possibly compromise private keys, etc.
- whether any commands could have automatically been pulled from remote
- whether any attack vectors other than via sshd were found
- whether some other forms of infestations (adding new user, keys to
  authorized_keys, etc.) was possible

or whether all that can be ruled out for sure.

And whether that has been confirmed for both versions of the maleware
that were distributed.

In short:
- Can people that had it, but had no sshd running and/or had it only
  running behind some firewall/NAT/etc. feel 100% safe to be not
  further compromised?

And while it wouldn't affect me personally, some have also asked
whether:
- They'd be safe it access to sshd was only restricted via
  hosts.allow/hosts.deny.


Last but not least, it would be nice if Debian had some trustworthy
experts which can actually confirm those findings.
No offence meant against those people doing the reverse engineering,
but in principle anyone on the internet could just claim anything and
make people wrongly feel safe.



Cheers,
Chris.

Re: xz backdoor

[Paul R. Tagliamonte]

There's also a very through exploration at https://github.com/amlweems/xzbot

Including, very interestingly, a discussion of format(s) of the
payload(s), and a mechanism to replace the backdoor key to play with
executing commands against a popped sshd, as well as some code to go
along with it.

  paultag

On Fri, Apr 5, 2024 at 2:19 PM Daniel Leidert  wrote:
>
> Am Freitag, dem 29.03.2024 um 23:20 +0100 schrieb Moritz Mühlenhoff:
> > Russ Allbery  wrote:
> > > I think this question can only be answered with reverse-engineering of the
> > > backdoors, and I personally don't have the skills to do that.
> >
> > In the pre-disclosure discussion permission was asked to share the payload
> > with a company specialising in such reverse engineering. If that went
> > through, I'd expect results to be publicly available in the next days.
>
> If there is a final result, can we as a project share the results on a
> prominent place? Or at least under d-devel-announce and/or d-security-
> announce? I was also wondering about what could have been compromised,
> what data might have been stolen, etc. And there is so many sources to
> follow right now. So sharing the final results would be great.
>
> Regards, Daniel



-- 
:wq

Re: xz backdoor

[Sirius]

In days of yore (Fri, 05 Apr 2024), Daniel Leidert thus quoth: 
> Am Freitag, dem 29.03.2024 um 23:20 +0100 schrieb Moritz Mühlenhoff:
> > Russ Allbery  wrote:
> > > I think this question can only be answered with reverse-engineering of the
> > > backdoors, and I personally don't have the skills to do that.
> > 
> > In the pre-disclosure discussion permission was asked to share the payload
> > with a company specialising in such reverse engineering. If that went
> > through, I'd expect results to be publicly available in the next days.
> 
> If there is a final result, can we as a project share the results on a
> prominent place? Or at least under d-devel-announce and/or d-security-
> announce? I was also wondering about what could have been compromised,
> what data might have been stolen, etc. And there is so many sources to
> follow right now. So sharing the final results would be great. 

If you have followed the discussion on Openwall ML, there have been a
couple of posts that points at both a general overview of what the code
did, an analysis of how the data was hidden in the 'corrupt' xz archive
under testing and some analysis of the actual .o which suggested this was
not just a backdoor but a remote-code-execution portal almost.

It has been interesting reading for sure, and the way they hid it, it does
really not look like your average script-kiddie doing this. I have my own
private suspicions about potential culprits being behind this but I figure
it is wiser to keep that under my hat as it were.

By the looks of things, both here and elsewhere, this was caught just in
the nick of time, meaning it did not make it out into the wild (at least
true for Debian and Fedora) so nothing was compromised. It it eerie the
parallels to Clifford Stoll and The Cuckoo's Egg though. I second the
request for sharing "final results" but I recognise that it may be weeks
still before that may happen.

-- 
Kind regards,

/S

Re: xz backdoor

[Daniel Leidert]

Am Freitag, dem 29.03.2024 um 23:20 +0100 schrieb Moritz Mühlenhoff:
> Russ Allbery  wrote:
> > I think this question can only be answered with reverse-engineering of the
> > backdoors, and I personally don't have the skills to do that.
> 
> In the pre-disclosure discussion permission was asked to share the payload
> with a company specialising in such reverse engineering. If that went
> through, I'd expect results to be publicly available in the next days.

If there is a final result, can we as a project share the results on a
prominent place? Or at least under d-devel-announce and/or d-security-
announce? I was also wondering about what could have been compromised,
what data might have been stolen, etc. And there is so many sources to
follow right now. So sharing the final results would be great. 

Regards, Daniel


signature.asc
Description: This is a digitally signed message part

Re: xz backdoor

[Pierre-Elliott Bécue]

Pierre-Elliott Bécue  wrote on 31/03/2024 at 14:31:37+0200:
> Wookey  wrote on 31/03/2024 at 04:34:00+0200:
>
>> On 2024-03-30 20:52 +0100, Ansgar  wrote:
>>> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
>>> Possibly also TPM modules in computers.
>>> 
>>> These can usually be used for both OpenPGP and SSH keys.
>>
>> Slightly off-topic, but a couple of recent posts have given me the
>> same thought:
>>
>> Can someone point to good docs on this?  I've had a yubikey for 3/4 of
>> a year now but have not yet worked out how I put my GPG key in it. (or
>> if it should be another key, or a subkey, or whatever). So I'm not
>> actually using it yet.
>>
>> PEB also described what sounded like a very sensible way to manage
>> keys (using subkeys) in one of these threads but I don't know how to
>> do that myself.
>
> I have started (and never finished) a blog article on how I use my
> YubiKey and what config I put in it. I'll definitely try to get it out
> before the end of next week. I'll probably extend it to mention the
> creation of GPG subkeys etc.
>
> I would also be happy if it helps my fellow DDs to try making an article
> about some basic crypto concepts regarding PGP, RSA et al. But not in
> the same piece I guess.

Hello,

For those interested in: I've published two articles:

 1. One on PGP subkeys https://pe.becue.phd/openpgp-subkeys
 2. One on the OpenPGP module of YubiKeys:
https://pe.becue.phd/yubikey-workfow-openpgp

I'm happy to receive any kind of constructive feedback.

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Colin Watson]

On Fri, Mar 29, 2024 at 09:09:45PM +0100, Sirius wrote:
> This is quite actively discussed on Fedora lists.
> https://www.openwall.com/lists/oss-security/2024/
> https://www.openwall.com/lists/oss-security/2024/03/29/4
> 
> Worth taking a look if action need to be taken on Debian.

FWIW, just uploaded:

openssh (1:9.7p1-4) unstable; urgency=medium

  * Rework systemd readiness notification and socket activation patches to
not link against libsystemd (the former via an upstream patch).
  * Force -fzero-call-used-regs=used not to be used on ppc64el (it's
unsupported, but configure fails to detect this).

 -- Colin Watson   Wed, 03 Apr 2024 12:06:08 +0100

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: xz backdoor

[Mike Hommey]

On Wed, Apr 03, 2024 at 02:01:23AM -0400, Robert Edmonds wrote:
> This backdoor abused the IFUNC mechanism in the GNU toolchain to hook into
> the sshd process. Looking on my Debian sid workstation with about 1900 library
> packages installed, I see a very small handful of source packages shipping
> libraries with IFUNC symbols, mostly things like gcc, glibc, haskell, some 
> Intel
> GPU stuff, etc. [0]
> 
> I do not know enough about the underlying technology to guess why the attacker
> chose to abuse the IFUNC mechanism versus, say, using the ELF .init_array
> section to introduce an ordinary initialization function into the backdoor
> library. (E.g., putting the equivalent of an __attribute__((constructor))
> function in the compiled binary blob part of the backdoor.) Perhaps what the
> attacker wanted to do was much easier to implement with the IFUNC mechanism to
> the point that it justified the sourceful changes to the upstream repository.

My understanding of the exploit is that the IFUNC initialization
function is run before relocated but read-only sections of the binary in
memory are actually made read-only (that is, it's run before relocation
is finished). That allows the exploit to overwrite GOT pointers.
Arguably, a contructor can still call mprotect and overwrite GOT
pointers, so there isn't a major advantage to use IFUNC over a more
traditional constructor. I guess there's an advantage in not changing
the pattern of syscalls called during library initialization to be more
stealthy, but OTOH, Firefox does two mprotect calls per library it loads
and nobody opened a bug about that behaviour on bugzilla.mozilla.org in
the close to 7 years it's been doing that (and don't worry, that's
expected, but also don't look at what Firefox does in unstable because
it doesn't do it anymore ; upstream builds still do ; if you're curious
about the details, see https://glandium.org/blog/?p=4297).

There's an argument to be done that read-only portions of binaries ought
to stay read-only (aka forbid mprotect).

Mike

Re: xz backdoor

[Robert Edmonds]

This backdoor abused the IFUNC mechanism in the GNU toolchain to hook into
the sshd process. Looking on my Debian sid workstation with about 1900 library
packages installed, I see a very small handful of source packages shipping
libraries with IFUNC symbols, mostly things like gcc, glibc, haskell, some Intel
GPU stuff, etc. [0]

I do not know enough about the underlying technology to guess why the attacker
chose to abuse the IFUNC mechanism versus, say, using the ELF .init_array
section to introduce an ordinary initialization function into the backdoor
library. (E.g., putting the equivalent of an __attribute__((constructor))
function in the compiled binary blob part of the backdoor.) Perhaps what the
attacker wanted to do was much easier to implement with the IFUNC mechanism to
the point that it justified the sourceful changes to the upstream repository.

If IFUNC symbols are sufficiently rare and sufficiently powerful, is it
worth implementing some hygiene around their proliferation in Debian? For
instance, something like a Lintian tag combined with an ftpmaster autoreject,
such as what we do for libraries that set RPATH?


[0] A quick and dirty way to check for IFUNC symbols in the libraries installed
on the system:

for fname in /usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/*.so.*; do
echo "> ${fname} <"
readelf -X -s ${fname} | grep IFUNC
echo
done

-- 
Robert Edmonds
edmo...@debian.org

Re: xz backdoor

[Paul R. Tagliamonte]

On Tue, Apr 2, 2024 at 5:12 PM Pierre-Elliott Bécue  wrote:

> If you have a master key on your laptop, when a yubikey is in, while
> running gpg --edit-key your_main_key, you can use the "addcardkey" to
> create a subkey on the Yubikey directly.
>

Yeah, seconded for sure. This is the configuration my Debian key is in --
it has an offline root key, which is stored on an LVM encrypted external
drive, and when I need to use it (new yubikey, or updating expiry), I use
an offline only box to mount the lvm drive, plug in the yubikey, and update
the key, exporting the public key to load into my daily box.

It's worked well, and this has been my workflow for a few years now (since
2019). It's not the easiest workflow (I've let my key expire twice because
I couldn't get the offline box set up and key ceremony done in time) but
it's worked well for me, and I'm especially sensitive about keeping private
key material off my disks where I can. I'd rather eat the cost of the setup
over exposing the project to additional keying material sitting around my
disk.

-- 
:wq

Re: xz backdoor

[Pierre-Elliott Bécue]

Iustin Pop  wrote on 01/04/2024 at 12:29:59+0200:

> On 2024-03-31 22:23:10, Arto Jantunen wrote:
>> Didier 'OdyX' Raboud  writes:
>> 
>> > Le dimanche, 31 mars 2024, 14.37:08 h CEST Pierre-Elliott Bécue a écrit :
>> >> I would object against creating a PGP key on the HSM itself. Not having
>> >> the proper control on the key is room for disaster as soon as you lose
>> >> it or it dies.
>> >
>> > For subkeys, isn't that a benefit rather than a disadvantage?
>> >
>> > You lose the key, or it gets destroyed / unusable; good, you get a new 
>> > subkey 
>> > instead of reusing the existing one on a different HSM.
>> 
>> For the authentication and signing subkeys this is indeed true. For the
>> encryption subkey significantly less so (as things encrypted against
>> that key then become impossible to decrypt).
>> 
>> Personally I have generated the signing and authentication subkeys on
>> the HSM itself (and thus at least in theory they cannot leave the HSM),
>> and the encryption subkey I have generated on an airgapped system and
>> stored on the HSM after making a couple of backups.
>
> I am really confused now on how all this works. How can you generate
> parts of a key (i.e. subkeys) on the HSM (well, yubikey), and the other
> parts locally?

If you have a master key on your laptop, when a yubikey is in, while
running gpg --edit-key your_main_key, you can use the "addcardkey" to
create a subkey on the Yubikey directly.

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Emanuele Rocca]

Hi,

On 2024-03-30 10:49, Jonathan Carter wrote:
> Another big question for me is whether I should really still
> package/upload/etc from an unstable machine.

I have been using unstable myself on most of my systems for the past
several years. There are many advantages, including being able to
actually test Debian as many have said in this thread. Case in point,
the xz backdoor has been discovered by a Debian unstable user: it would
have likely been found much later had they used stable instead.

Without trying to be overly dramatic though, I consider the xz incident
as some sort of 9/11 of Linux distros. Everyone knew it could have
happened, but now that it has and we see how relatively easy it was I
think it's time to re-evaluate things. I now do not think anymore that
sid is secure enough for high-profile targets such as DDs. All it takes
is one bad upload, and your systems are immediately compromised. Sure
bad stuff can eventually make it to stable as well, but the longer it
takes the more likely for the malicious change to be spotted.

Other than the time aspect, there's the problem of binary uploads too.
How long would it take to spot a well crafted, malicious binary upload
to sid?

Re: xz backdoor

[Andrey Rakhmatullin]

On Tue, Apr 02, 2024 at 11:49:50AM +0200, Francesco P. Lovergine wrote:
> Speaking about that, I'm a simple guy: how can anyone trust
> sources signed by an unsigned-gnupg-key committer (I mean both the
> actors of this tragically ridicolous drama)? In 2024. Really?
As opposed to sources not signed by any key at all?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Francesco P. Lovergine]

On Fri, Mar 29, 2024 at 09:09:45PM +0100, Sirius wrote:

Hi there,

This is quite actively discussed on Fedora lists.
https://www.openwall.com/lists/oss-security/2024/
https://www.openwall.com/lists/oss-security/2024/03/29/4

Worth taking a look if action need to be taken on Debian.



Speaking about that, I'm a simple guy: how can anyone trust
sources signed by an unsigned-gnupg-key committer (I mean both the
actors of this tragically ridicolous drama)? 
In 2024. Really?

Even the unperfect web-of-trust is better than nothing at all.

--
⢀⣴⠾⠻⢶⣦⠀ Francesco Paolo Lovergine
⣾⠁⢠⠒⠀⣿⡁ Debian Developer
⢿⡄⠘⠷⠚⠋⠀ 0579 A97A 2238 EBF9 BE61
⠈⠳⣄ ED02 0F02 A5E1 1636 86A4

Re: xz backdoor

[Francesco P. Lovergine]

On Sun, Mar 31, 2024 at 12:39:55PM +0200, Johannes Schauer Marin Rodrigues 
wrote:

In summary: would running unstable instead of bookworm let me find more bugs
than running bookworm with unstable chroots? For my specific work: yes,
absolutely.  Am I upgrading from bookworm to unstable or at least testing?
Absolutely not. I just don't have the amount of free-time this would require of
me.  Just performing the 12-13 bisection steps to find the offending kernel
commit which makes my system lock up would easily require a day of free time
from me.  I'm afraid I do not have this luxury. Not even remotely close!



+1 


I run stably and alternatively 4-6 different personal boxes during the week, all
with stable. Dedicating time to debugging silly egg-and-chicken breakages due 
to sid
life cycle (anyone nominated 64t saga?) is out of discussion. I did
that years ago, when I had more time an less personal boxes to run.


So I'll continue to not dogfood as hard as I could and run as much from
bookworm as I can. Would it make me a better contributor if I ran unstable?
Certainly! But this thing is just a hobby of mine and I can only allot that
much time to do risky experiments with my only computer. I guess others are in
the same boat?


I would also add that even stable/oldstable needs care, and I need to support
multiple users to have a decent workflow on them. Our main network runs
about a dozen of general purpose stable servers/VMs and that's more than enough.

That said I still run a single sid boxes (no GPG/essential keys there)
and I would add that some bugs can be seen only at stable upgrade time
or during the testing life-cycle, not when one runs sid all time. 


A lot of issues can be found only by running accurate tests on fresh
boxes, not via sid daily use, which is only a part of the whole story.

--
⢀⣴⠾⠻⢶⣦⠀ Francesco Paolo Lovergine
⣾⠁⢠⠒⠀⣿⡁ Debian Developer
⢿⡄⠘⠷⠚⠋⠀ 0579 A97A 2238 EBF9 BE61
⠈⠳⣄ ED02 0F02 A5E1 1636 86A4

Re: xz backdoor

[Theodore Ts'o]

On Mon, Apr 01, 2024 at 04:47:05PM +0100, Colin Watson wrote:
> On Mon, Apr 01, 2024 at 08:13:58AM -0700, Russ Allbery wrote:
> > Bastian Blank  writes:
> > > I don't understand what you are trying to say.  If we add a hard check
> > > to lintian for m4/*, set it to auto-reject, then it is fully irrelevant
> > > if the upload is a tarball or git.
> > 
> > Er, well, there goes every C package for which I'm upstream, all of which
> > have M4 macros in m4/* that do not come from an external source.
> 
> Ditto.  And a bunch of the packages where I'm not upstream too, such as
> that famously enthusiastic adopter of all things GNU, OpenSSH.

For e2fsprogs, almost all the M4 macros come from an external source;
but I had to patch one of the macros so that it would work on *BSD
when using pmake as opposed to GNU make.  And in another case, I
copied the macro from another package's git repo to fix a portability
issue with Mac OS X.

So it's highly likely that if you added a hard check in Lintian, both
of these would trigger for e2fsprogs.

Portability is hard.  Let's go shopping!

- Ted

Re: xz backdoor

[Adrian Bunk]

On Mon, Apr 01, 2024 at 12:02:09PM +0200, Bastian Blank wrote:
> Hi
> 
> On Sun, Mar 31, 2024 at 07:48:35PM +0300, Adrian Bunk wrote:
> > > What we can do unilaterally is to disallow vendoring those files.
> > These files are supposed to be vendored in release tarballs,
> > the sane approach for getting rid of such vendored files would
> > be to discourage tarball uploads to the archive and encourage
> > git uploads instead.
> 
> I don't understand what you are trying to say.  If we add a hard check
> to lintian for m4/*, set it to auto-reject, then it is fully irrelevant
> if the upload is a tarball or git.

xz also has > 600 LOC of legit own m4 code in m4/*,
and that's not unusual for packages using autoconf.

> > > Does it help?  At least in the case of autoconf it removes one common
> > > source of hard to read files.
> > But I doubt every DD would be able to review the 2k LOC non-vendored 
> > autoconf code in xz.
> 
> But at least changes to this code are visible.  In this case the changes
> to the m4 stuff did not exist in the somewhat reviewed repo, but just in
> the unreviewed tarballs.

There are many other ways how these unreviewed tarballs could be manipulated.

The root cause of the problem you want to solve is that the ftp team 
permits uploading such unreviewed tarballs to our archive.

> Bastian

cu
Adrian

Re: xz backdoor

[Colin Watson]

On Mon, Apr 01, 2024 at 08:13:58AM -0700, Russ Allbery wrote:
> Bastian Blank  writes:
> > I don't understand what you are trying to say.  If we add a hard check
> > to lintian for m4/*, set it to auto-reject, then it is fully irrelevant
> > if the upload is a tarball or git.
> 
> Er, well, there goes every C package for which I'm upstream, all of which
> have M4 macros in m4/* that do not come from an external source.

Ditto.  And a bunch of the packages where I'm not upstream too, such as
that famously enthusiastic adopter of all things GNU, OpenSSH.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: xz backdoor

[Russ Allbery]

Bastian Blank  writes:

> I don't understand what you are trying to say.  If we add a hard check
> to lintian for m4/*, set it to auto-reject, then it is fully irrelevant
> if the upload is a tarball or git.

Er, well, there goes every C package for which I'm upstream, all of which
have M4 macros in m4/* that do not come from an external source.

-- 
Russ Allbery (r...@debian.org)  

Re: xz backdoor

[Pierre-Elliott Bécue]

De : Ansgar  
À : Pierre-Elliott Bécue ; Luca Boccassi 
Cc : debian-devel@lists.debian.org
Date : 1 avr. 2024 12:47:52
Objet : Re: xz backdoor

> 
> Hi,
> 
> On Sun, 2024-03-31 at 14:34 +0200, Pierre-Elliott Bécue wrote:
>> The PGP submodule of a Yubikey can host 3 keys, one signing, one
>> authent, and one encrypt. ISTR accessing the signing key is always
>> prompting for the PIN. Same for the encryption key. (I think both can
>> be configured otherwise)
> 
> I think presence confirmation is more useful, that is, interacting
> physically with the device for each signature.  The Yubikey can do that
> also for OpenPGP:
> 
> ```
> $ ykman openpgp keys set-touch --help
> [...]
>   Touch policies:
> 
>   Off (default)   no touch required
>   On  touch required
>   Fixed   touch required, can't be disabled without deleting the 
> private key
>   Cached  touch required, cached for 15s after use
>   Cached-Fixed    touch required, cached for 15s after use, can't be disabled
>   without deleting the private key
> ```
> 
> (The PIN can still be cached.)
> 
> For OpenSSH it might also be more convenient to use Webauthn, that is,
> the keys generated using `ssh-keygen -t ed25519-sk` or `-t ecdsa-sk`.
> 
> Ansgar
>> 
Yes, I did not mention the touch policy because right now I fail to have it 
enforced by the Yubi after having set it.

-- 
Pierre-Elliott Bécue

Re: xz backdoor

[Bastian Blank]

Hi

On Mon, Apr 01, 2024 at 12:40:51PM +0200, Ansgar  wrote:
> For OpenSSH it might also be more convenient to use Webauthn, that is,
> the keys generated using `ssh-keygen -t ed25519-sk` or `-t ecdsa-sk`.

Also those key types allow two different uses.  Persistent or
non-persistent keys differ in where parts of the key is stored and
protected.

Persistent keys store the second part on the hardware itself.  So you
can extract that part if you know the PIN of the hardware.  I for
example have one for access to my emails and irc system.

Non-persistent keys store the second part on the using system, aka your
hard drive.  Those files can optionally be protected with a standard SSH
passphrase.  You can also have many different keys this way.

But please be aware: resetting the fido part of a yubikey is pretty easy
and will immediatelly make all keys unusable.

Bastian

-- 
Punishment becomes ineffective after a certain point.  Men become insensitive.
-- Eneg, "Patterns of Force", stardate 2534.7

Re: xz backdoor

[Ansgar ]

Hi,

On Sun, 2024-03-31 at 14:34 +0200, Pierre-Elliott Bécue wrote:
> The PGP submodule of a Yubikey can host 3 keys, one signing, one
> authent, and one encrypt. ISTR accessing the signing key is always
> prompting for the PIN. Same for the encryption key. (I think both can
> be configured otherwise)

I think presence confirmation is more useful, that is, interacting
physically with the device for each signature.  The Yubikey can do that
also for OpenPGP:

```
$ ykman openpgp keys set-touch --help
[...]
  Touch policies:

  Off (default)   no touch required
  On  touch required
  Fixed   touch required, can't be disabled without deleting the 
private key
  Cached  touch required, cached for 15s after use
  Cached-Fixedtouch required, cached for 15s after use, can't be disabled
  without deleting the private key
```

(The PIN can still be cached.)

For OpenSSH it might also be more convenient to use Webauthn, that is,
the keys generated using `ssh-keygen -t ed25519-sk` or `-t ecdsa-sk`.

Ansgar
> 

Re: xz backdoor

[Iustin Pop]

On 2024-03-31 22:23:10, Arto Jantunen wrote:
> Didier 'OdyX' Raboud  writes:
> 
> > Le dimanche, 31 mars 2024, 14.37:08 h CEST Pierre-Elliott Bécue a écrit :
> >> I would object against creating a PGP key on the HSM itself. Not having
> >> the proper control on the key is room for disaster as soon as you lose
> >> it or it dies.
> >
> > For subkeys, isn't that a benefit rather than a disadvantage?
> >
> > You lose the key, or it gets destroyed / unusable; good, you get a new 
> > subkey 
> > instead of reusing the existing one on a different HSM.
> 
> For the authentication and signing subkeys this is indeed true. For the
> encryption subkey significantly less so (as things encrypted against
> that key then become impossible to decrypt).
> 
> Personally I have generated the signing and authentication subkeys on
> the HSM itself (and thus at least in theory they cannot leave the HSM),
> and the encryption subkey I have generated on an airgapped system and
> stored on the HSM after making a couple of backups.

I am really confused now on how all this works. How can you generate
parts of a key (i.e. subkeys) on the HSM (well, yubikey), and the other
parts locally?

Looking forward to having up-to-date documentation once the dust
settles. I have enough yubikeys which are only used for 2FA.

(Well, and I'd need an airgapped, separate system, which I don't have)

thanks,
iustin

Re: xz backdoor

[Bastian Blank]

Hi

On Sun, Mar 31, 2024 at 07:48:35PM +0300, Adrian Bunk wrote:
> > What we can do unilaterally is to disallow vendoring those files.
> These files are supposed to be vendored in release tarballs,
> the sane approach for getting rid of such vendored files would
> be to discourage tarball uploads to the archive and encourage
> git uploads instead.

I don't understand what you are trying to say.  If we add a hard check
to lintian for m4/*, set it to auto-reject, then it is fully irrelevant
if the upload is a tarball or git.

> > Does it help?  At least in the case of autoconf it removes one common
> > source of hard to read files.
> But I doubt every DD would be able to review the 2k LOC non-vendored 
> autoconf code in xz.

But at least changes to this code are visible.  In this case the changes
to the m4 stuff did not exist in the somewhat reviewed repo, but just in
the unreviewed tarballs.

Bastian

-- 
Each kiss is as the first.
-- Miramanee, Kirk's wife, "The Paradise Syndrome",
   stardate 4842.6

Re: xz backdoor

[Stephan Verbücheln]

On Mon, 2024-04-01 at 10:59 +0200, tho...@goirand.fr wrote:
> Only for the signing operation, one can turn on the "force-sig"
> option so that the key always prompt for a pin. And that is not the
> default.

There are two levels. In the OpenPGP protocol, the smartcard can be
configured to require the PIN for every signature. This works for any
OpenPGP card, it is not specific to Yubikey.

Yubikey has an additional feature where you can require to physically
touch the Yubikey for each signature. This even protects from malware
using the key in some scenarios where the attacker got the PIN
(keylogger etc.). Not all smartcards/readers have that.

There are also smartcard readers with PIN pad, where the PIN is not
sent to the host in the first place.

It is also possible to forward your gpg-agent via SSH. This way you can
sign large files on a server, but all public-key operations and the PIN
remain on your client.

Regards


signature.asc
Description: This is a digitally signed message part

Re: xz backdoor

[thomas]

On Mar 31, 2024 2:37 PM, Pierre-Elliott Bécue  Wrote:

> The PGP submodule of a Yubikey can host 3 keys, one signing, one 

> authent, and one encrypt. ISTR accessing the signing key is always 

> prompting for the PIN. Same for the encryption key. (I think both can be 

> configured otherwise)


Only for the signing operation, one can turn on the "force-sig" option so that 
the key always prompt for a pin. And that is not the default.


Thomas

Re: xz backdoor

[Didier 'OdyX' Raboud]

Le dimanche, 31 mars 2024, 21.23:10 h CEST Arto Jantunen a écrit :
> Didier 'OdyX' Raboud  writes:
> > Le dimanche, 31 mars 2024, 14.37:08 h CEST Pierre-Elliott Bécue a écrit :
> >> I would object against creating a PGP key on the HSM itself. Not having
> >> the proper control on the key is room for disaster as soon as you lose
> >> it or it dies.
> > 
> > For subkeys, isn't that a benefit rather than a disadvantage?
> > 
> > You lose the key, or it gets destroyed / unusable; good, you get a new
> > subkey instead of reusing the existing one on a different HSM.
> 
> For the authentication and signing subkeys this is indeed true. For the
> encryption subkey significantly less so (as things encrypted against
> that key then become impossible to decrypt).

I was missing that perspective; thanks!

-- 
OdyX

Re: xz backdoor

[Joerg Jaspert]

On 17185 March 1977, Andrey Rakhmatullin wrote:

Didn't DKG start something like this some time ago? Or am I
mis-remembering?
I also thought I remember some Debian-specific PGP-related document 
but I

couldn't find it.


You may remember https://keyring.debian.org/ which has a bunch of links 
to other places too, and from there maybe 
https://keyring.debian.org/creating-key.html - but its all not exactly 
what people ask for. Its more a "go this way and get those results", 
which is fine, but different thing.


--
bye, Joerg

Re: xz backdoor

[Arto Jantunen]

Didier 'OdyX' Raboud  writes:

> Le dimanche, 31 mars 2024, 14.37:08 h CEST Pierre-Elliott Bécue a écrit :
>> I would object against creating a PGP key on the HSM itself. Not having
>> the proper control on the key is room for disaster as soon as you lose
>> it or it dies.
>
> For subkeys, isn't that a benefit rather than a disadvantage?
>
> You lose the key, or it gets destroyed / unusable; good, you get a new subkey 
> instead of reusing the existing one on a different HSM.

For the authentication and signing subkeys this is indeed true. For the
encryption subkey significantly less so (as things encrypted against
that key then become impossible to decrypt).

Personally I have generated the signing and authentication subkeys on
the HSM itself (and thus at least in theory they cannot leave the HSM),
and the encryption subkey I have generated on an airgapped system and
stored on the HSM after making a couple of backups.

-- 
Arto Jantunen

Re: xz backdoor

[Didier 'OdyX' Raboud]

Le dimanche, 31 mars 2024, 14.37:08 h CEST Pierre-Elliott Bécue a écrit :
> Hello,
> 
> Iustin Pop  wrote on 31/03/2024 at 13:13:27+0200:
> > Option 2: Generate keys on the yubikey and have them never leave the
> > secure enclave. That means having 2 yubikeys per developer, and ensuring
> > you keep track of _two_ keys, but it does ensure there's a physical
> > binding to the key.
> > 
> > Are there other options? And which option is proposed?
> 
> I would object against creating a PGP key on the HSM itself. Not having
> the proper control on the key is room for disaster as soon as you lose
> it or it dies.

For subkeys, isn't that a benefit rather than a disadvantage?

You lose the key, or it gets destroyed / unusable; good, you get a new subkey 
instead of reusing the existing one on a different HSM.

-- 
OdyX

Re: xz backdoor

[Andrey Rakhmatullin]

On Sun, Mar 31, 2024 at 12:28:35PM -0400, Roberto C. Sánchez wrote:
> On Sun, Mar 31, 2024 at 09:53:06AM -0300, Carlos Henrique Lima Melara wrote:
> > Hi,
> > 
> > On Sun, Mar 31, 2024 at 02:31:37PM +0200, Pierre-Elliott Bécue wrote:
> > 
> > > I would also be happy if it helps my fellow DDs to try making an article
> > > about some basic crypto concepts regarding PGP, RSA et al. But not in
> > > the same piece I guess.
> > 
> > My suggestion is to create a wiki page with these concepts plus a guide
> > on best practices dor the gpg key (subkeys + hsm - yubikey and others).
> > 
> Didn't DKG start something like this some time ago? Or am I
> mis-remembering?
I also thought I remember some Debian-specific PGP-related document but I
couldn't find it.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Adrian Bunk]

On Sun, Mar 31, 2024 at 09:35:09AM +0200, Bastian Blank wrote:
> On Sat, Mar 30, 2024 at 08:15:10PM +, Colin Watson wrote:
> > On Sat, Mar 30, 2024 at 05:12:17PM +0100, Sirius wrote:
> > > I have seen discussion about shifting away from the whole auto(re)conf
> > > tooling to CMake or Meson with there being a reasonable drawback to CMake.
> > > Is that something being discussed within Debian as well?
> > It's not in general something that Debian can unilaterally change.  And
> > in a number of cases switching build system would be pretty non-trivial.
> 
> What we can do unilaterally is to disallow vendoring those files.

These files are supposed to be vendored in release tarballs,
the sane approach for getting rid of such vendored files would
be to discourage tarball uploads to the archive and encourage
git uploads instead.

> Does it help?  At least in the case of autoconf it removes one common
> source of hard to read files.
>...

But I doubt every DD would be able to review the 2k LOC non-vendored 
autoconf code in xz.

The experimental cmake build of xz also has 2700 LOC.

> Bastian

cu
Adrian

Re: xz backdoor

[Russ Allbery]

Sirius  writes:

> Would throwing away these unmodified (?) macros packaged projects may be
> carrying for hysterical raisins in favour of just using the autoconf
> native macros reduce the attack-surface a potential malicious actor
> would have at their disposal, or would it simply be a "putting all eggs
> in one basket" and just make things worse? And by how much vis-a-vis the
> effort to do it?

Most of the macros of this type are not from Autoconf.  They're from
either gnulib or the Autoconf Archive.  In both cases, blindly upgrading
to a newer upstream version may break things, I believe.  I'm not as sure
with gnulib, but the Autoconf Archive is a huge collection of things with
varying quality and does not necessarily have any guarantees about APIs.

> I think that what I am trying to get at is this: is there low-hanging
> fruit that for minimal effort would disproportionately improve things
> from a security perspective. (I have an inkling that this is a question
> that every distribution is wrestling with today.)

I think the right way to think about this is to say that the Autoconf
ecosystem is rife with embedded code copies and, because the normal way of
using this code is to make a copy, is also somewhat lax about making
breaking changes since the expectation is that you only update during your
release process when you can fix up any changes.

(That code is also notoriously hard to read, both because M4 is a language
with fairly noisy syntax and because the only tools assumed to be
available in the output scripts is a very minimal Bourne shell and
standard POSIX shell utilities, so there's a lot of the type of
programming that only shell aficionados can love.  That was the problem
with detecting this backdoor: the sort of chain of tr and eval and
whatnot that injected the backdoor is what, e.g., all of Libtool looks
like, at least on a first superficial glance.)

I know all this adds up to "why are we using this stuff anyway," but the
amount of hard-won portability knowledge that's baked into these tools is
IMMENSE, and while probably 75% of it is now irrelevant because the
systems that needed it are long-dead, no one can agree on what 75% that is
or figure out which useful 25% to extract.  And rewriting it in some other
programming language is daunting and feels like churn rather than
progress.

-- 
Russ Allbery (r...@debian.org)  

Re: xz backdoor

[Roberto C . Sánchez]

On Sun, Mar 31, 2024 at 09:53:06AM -0300, Carlos Henrique Lima Melara wrote:
> Hi,
> 
> On Sun, Mar 31, 2024 at 02:31:37PM +0200, Pierre-Elliott Bécue wrote:
> 
> > I would also be happy if it helps my fellow DDs to try making an article
> > about some basic crypto concepts regarding PGP, RSA et al. But not in
> > the same piece I guess.
> 
> My suggestion is to create a wiki page with these concepts plus a guide
> on best practices dor the gpg key (subkeys + hsm - yubikey and others).
> 
Didn't DKG start something like this some time ago? Or am I
mis-remembering?

Regards,

-Roberto

-- 
Roberto C. Sánchez

Re: xz backdoor

[Lisandro Damián Nicanor Pérez Meyer]

On Sun, 31 Mar 2024 at 07:40, Johannes Schauer Marin Rodrigues
 wrote:
 [snip]
> In summary: would running unstable instead of bookworm let me find more bugs
> than running bookworm with unstable chroots? For my specific work: yes,
> absolutely.  Am I upgrading from bookworm to unstable or at least testing?
> Absolutely not. I just don't have the amount of free-time this would require 
> of
> me.  Just performing the 12-13 bisection steps to find the offending kernel
> commit which makes my system lock up would easily require a day of free time
> from me.  I'm afraid I do not have this luxury. Not even remotely close!
>
> So I'll continue to not dogfood as hard as I could and run as much from
> bookworm as I can. Would it make me a better contributor if I ran unstable?
> Certainly! But this thing is just a hobby of mine and I can only allot that
> much time to do risky experiments with my only computer. I guess others are in
> the same boat?

As long as we are all contributing our free time we just need to go
the way it fits us better. I am an unstable user but I definitely
respect and support your position. And I would hate losing
contributors for things like this.

-- 
Lisandro Damián Nicanor Pérez Meyer
https://perezmeyer.com.ar/

Re: xz backdoor

[Gard Spreemann]

On 31 March 2024 12:39:55 CEST, Johannes Schauer Marin Rodrigues 
 wrote:
>
>Another example is when I wanted to run a GUI program inside an unshared chroot
>environment.  Wayland does not seem to be happy about that  and I didn't find a
>way to test my GUI application successfully. But maybe my container environment
>just fails to set up some things? Does there exist a way to test GUI
>applications inside a container without requiring superuser privileges to run
>the container?



Hi,

At least with (unprivileged) Podman containers, I have success with just 
passing the host's Wayland socket as a volume with the -v switch.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: xz backdoor

[Sirius]

In days of yore (Sun, 31 Mar 2024), Colin Watson thus quoth: 
> On Sun, Mar 31, 2024 at 10:10:42AM +0200, Sirius wrote:
> > Not worth boiling the ocean over, but is there an estimate of how many
> > packaged projects have customisations to their autoconf that is not found
> > in the upstream autoconf project? If that number is low single digit
> > percent, it may motivate those projects to upstream their modifications.
> > If it is double digits percent, it might not be possible to disallow
> > vendoring the files.
> 
> This is difficult to answer because it's comparing apples and oranges to
> some extent: not all autoconf customizations are vendored or would make
> any kind of sense to upstream.  For example,
> https://gitlab.com/man-db/man-db/-/blob/main/m4/man-arg-config-file.m4
> is obviously specific to that project; it's just in a separate file for
> the same reasons that projects past a certain size don't typically put
> all their code in a single file.
> 
> I suspect the question you're aiming for is something like "how many
> packaged projects have copied autoconf macros from elsewhere and
> modified them but kept the same file names, so that a naïve attempt to
> update them would drop those modifications".  My guess is that the
> number here is very low - IME it's much more common in such cases to
> either rename the macro file to be obviously project-specific or to find
> some workaround that doesn't require changing the upstream macro - but
> I've never seen anything resembling a robust analysis of this and I may
> well have a skewed view.

I find this interesting, even if it is a question that simply does not
have a reasonable answer unless one goes and does the audit. Your
rephrasing is actually closer to what I was aiming at - but that opens
another question that may not have a reasonable answer:

Would throwing away these unmodified (?) macros packaged projects may be
carrying for hysterical raisins in favour of just using the autoconf
native macros reduce the attack-surface a potential malicious actor would
have at their disposal, or would it simply be a "putting all eggs in one
basket" and just make things worse? And by how much vis-a-vis the effort
to do it?

I think that what I am trying to get at is this: is there low-hanging
fruit that for minimal effort would disproportionately improve things from
a security perspective. (I have an inkling that this is a question that
every distribution is wrestling with today.)

My apologies if I am asking too many questions. It has been a long time
since I was using Debian (2004), but I am standing up environments and
looking at how I may be able to put some of my sparse spare time to good
use. So I am looking to learn (by doing) how to package things. I am
building my own .deb of mutt (because I want a few things that the
official package does not have) but am looking for "the right way" to do
it. Just because I built it and it is running does not mean I have done it
right. :-)

-- 
Kind regards,

/S

Re: xz backdoor

[Carlos Henrique Lima Melara]

Hi,

On Sun, Mar 31, 2024 at 02:31:37PM +0200, Pierre-Elliott Bécue wrote:
> Wookey  wrote on 31/03/2024 at 04:34:00+0200:
> 
> > On 2024-03-30 20:52 +0100, Ansgar  wrote:
> >> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
> >> Possibly also TPM modules in computers.
> >> 
> >> These can usually be used for both OpenPGP and SSH keys.
> >
> > Slightly off-topic, but a couple of recent posts have given me the
> > same thought:
> >
> > Can someone point to good docs on this?  I've had a yubikey for 3/4 of
> > a year now but have not yet worked out how I put my GPG key in it. (or
> > if it should be another key, or a subkey, or whatever). So I'm not
> > actually using it yet.
> >
> > PEB also described what sounded like a very sensible way to manage
> > keys (using subkeys) in one of these threads but I don't know how to
> > do that myself.
> 
> I have started (and never finished) a blog article on how I use my
> YubiKey and what config I put in it. I'll definitely try to get it out
> before the end of next week. I'll probably extend it to mention the
> creation of GPG subkeys etc.

That would be really helpful! It's not that easy to find this kind of
information as Wookey said.

> I would also be happy if it helps my fellow DDs to try making an article
> about some basic crypto concepts regarding PGP, RSA et al. But not in
> the same piece I guess.

My suggestion is to create a wiki page with these concepts plus a guide
on best practices dor the gpg key (subkeys + hsm - yubikey and others).

Cheers,
Charles


signature.asc
Description: PGP signature

Re: xz backdoor

[Pierre-Elliott Bécue]

Bastian Blank  wrote on 31/03/2024 at 09:11:59+0200:

> On Sat, Mar 30, 2024 at 07:15:28PM -0700, Otto Kekäläinen wrote:
>> I am doing all my builds inside a (Podman) container with the sources
>> loop-mounted.
>
> You do, but Debian itself (aka DSA) does not.  They still prefer to
> trust all 100k packages and run them as root in the init namespace over
> the five people who can login as buildd and potentially trigger
> capability reachable problems in the kernel.  This is what got as in
> part of the situation, as we don't even know if the buildd hosts are
> untampered.

Ok, maybe the current situation is not that good and maybe we (DSA) need
to change our priority focuses.

But FWIW, being passive-agressive in order to question something and
doing finger pointing is just the best way to get a constructive idea
ignored.

So, IDK, pour some water in your wine and try a nicer way of stating
what you find problematic?

We'll try to jump the unshare schroot train.

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Pierre-Elliott Bécue]

Hello,

Iustin Pop  wrote on 31/03/2024 at 13:13:27+0200:

> On 2024-03-31 10:47:57, Luca Boccassi wrote:
>> On Sun, 31 Mar 2024 at 08:39, Bastian Blank  wrote:
>> >
>> > On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
>> > > On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
>> > > > As others have said, the best solution is to relay on HSW for handling
>> > > > the cryptographic material.
>> > > Aren't these answers to different questions?
>> > > Not all attacks are about stealing the key or using it to sign unintended
>> > > things.
>> >
>> > Also a HSM does only allow to control access to the cryptographic
>> > material.  But it asserts no control over what is actually signed.
>> >
>> > So an attacker needs to wait until you ask the HSM it is okay to sign
>> > something.
>> >
>> > Bastian
>> 
>> This is true as in the default configuration you get asked for the
>> yubikey pin only once per "session", and then it's cached
>> transparently and there's no GUI feedback when the token is used (the
>> light on it blinks, but noticing that requires having it in line of
>> sight at all times). However, it's already better than nothing as it
>> means such an attack must be "online", and run in the same "session"
>> as the active user, so perfect should definitely not be the enemy of
>> good here IMHO. Also, iirc this can be configured to always ask for
>> the pin on each signature, although this could get burdensome. But
>> given the very low price of yubikeys (or similar tokens), and how well
>> and seamless they work these days, I think the offer of buying any DD
>> that doesn't have one such a token is one that we should take up and
>> make it happen.
>
> Jumping in late in the HSM thread, but I'm not sure I understand the
> exact setup people propose.
>
> Option 1: Moving keys to one yubikey, while keeping the original key
> material "safe" offline. How do you know the "safe offline" material is
> safe and hasn't been copied?

If your threat model includes NSA, you can't.

I have two backups of my main PGP key on two encrypted devices. One is
always with me, the other is stored "safely" at home. (I put quotes
because while I'm convinced it'd be from hard to impossible to find it,
theoretically when I leave home, a sufficiently motivated agent could
try and find it)

I consider this model safe enough, especially since the key also is
encrypted with a passphrase, meaning one would need to:

 1. Find the backup device
 2. Copy it (I've put some measures that would normally make me
aware that someone touched the device)
 3. Break the cryptographic protection on it
 4. Break the passphrase protection on the private key.

I consider that doing more would be too expensive for a little to no
gain.

> Option 2: Generate keys on the yubikey and have them never leave the
> secure enclave. That means having 2 yubikeys per developer, and ensuring
> you keep track of _two_ keys, but it does ensure there's a physical
> binding to the key.
>
> Are there other options? And which option is proposed?

I would object against creating a PGP key on the HSM itself. Not having
the proper control on the key is room for disaster as soon as you lose
it or it dies.

> I have quite a few yubikeys, but I haven't migrated to use them since
> it's not clear to me what is a good, and recommended, workflow. I'm
> relatively against option 1, since the "safe offline" key material
> somehow doesn't appeal to me.

Any workflow that makes a non-governmental attacker's life a PITA
without forcing you to share their pain is a good workflow.

Security habits are incremental learning and trials, don't expect to
reach top levels in one day.

As stated elsewhere, I started to work on a blog post regarding my
Yubikey configs/habits. I'll try to get it out at some point. I consider
the opportunity to extend further upon the topic and try to write about
some of my security habits, it that can help.

Cheers,

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Pierre-Elliott Bécue]

Luca Boccassi  wrote on 31/03/2024 at 12:47:57+0200:

> On Sun, 31 Mar 2024 at 08:39, Bastian Blank  wrote:
>>
>> On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
>> > On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
>> > > As others have said, the best solution is to relay on HSW for handling
>> > > the cryptographic material.
>> > Aren't these answers to different questions?
>> > Not all attacks are about stealing the key or using it to sign unintended
>> > things.
>>
>> Also a HSM does only allow to control access to the cryptographic
>> material.  But it asserts no control over what is actually signed.
>>
>> So an attacker needs to wait until you ask the HSM it is okay to sign
>> something.
>>
>> Bastian
>
> This is true as in the default configuration you get asked for the
> yubikey pin only once per "session", and then it's cached
> transparently and there's no GUI feedback when the token is used (the
> light on it blinks, but noticing that requires having it in line of
> sight at all times). However, it's already better than nothing as it
> means such an attack must be "online", and run in the same "session"
> as the active user, so perfect should definitely not be the enemy of
> good here IMHO. Also, iirc this can be configured to always ask for
> the pin on each signature, although this could get burdensome. But
> given the very low price of yubikeys (or similar tokens), and how well
> and seamless they work these days, I think the offer of buying any DD
> that doesn't have one such a token is one that we should take up and
> make it happen.

The PGP submodule of a Yubikey can host 3 keys, one signing, one
authent, and one encrypt. ISTR accessing the signing key is always
prompting for the PIN. Same for the encryption key. (I think both can be
configured otherwise)

On the contrary, the authentication subkey is a one-per-session shot
only.

For the signing slot, there's a counter set in the yubi that increments
for each successful access.

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Pierre-Elliott Bécue]

Wookey  wrote on 31/03/2024 at 04:34:00+0200:

> On 2024-03-30 20:52 +0100, Ansgar  wrote:
>> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
>> Possibly also TPM modules in computers.
>> 
>> These can usually be used for both OpenPGP and SSH keys.
>
> Slightly off-topic, but a couple of recent posts have given me the
> same thought:
>
> Can someone point to good docs on this?  I've had a yubikey for 3/4 of
> a year now but have not yet worked out how I put my GPG key in it. (or
> if it should be another key, or a subkey, or whatever). So I'm not
> actually using it yet.
>
> PEB also described what sounded like a very sensible way to manage
> keys (using subkeys) in one of these threads but I don't know how to
> do that myself.

I have started (and never finished) a blog article on how I use my
YubiKey and what config I put in it. I'll definitely try to get it out
before the end of next week. I'll probably extend it to mention the
creation of GPG subkeys etc.

I would also be happy if it helps my fellow DDs to try making an article
about some basic crypto concepts regarding PGP, RSA et al. But not in
the same piece I guess.

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Bastian Blank]

On Sun, Mar 31, 2024 at 11:59:35AM +0100, Colin Watson wrote:
> > What we can do unilaterally is to disallow vendoring those files.
> > Does it help?  At least in the case of autoconf it removes one common
> > source of hard to read files.
> That's doable unilaterally to some extent, using the dh-autoreconf style
> of approach: the files may be vendored in the upstream tarball, but
> we'll throw them away and rebuild them.

No, I don't think that this is adequat.  There are too many ways they
could still be used accidentaly or on purpose.  The only way to make
sure it is never used, is to remove it all.

In other words:
dh-autoreconf fails open: if something happens it will just use the
vendored stuff.
Nuking the files fails close: if something happens it will just not work
at all.

>  I did that recently for the
> packages that use gnulib where I'm upstream (e.g.
> https://salsa.debian.org/debian/libpipeline/-/commit/b596eefb342e93136cf1f479415981fc7f48).
> Frankly, I suspect that it will introduce slightly more FTBFS bugs over
> time due to differences between the packaged version of gnulib and the
> version in use upstream (although things have got better in that regard
> over the last year or so with the introduction of stable branches), but
> it does seem to be worth it for transparency.

Thanks about this first step.

But shipping dead code is not defence in depth.  It will just produce
another completely unaudited pool of hard to read code.

Bastian

-- 
If a man had a child who'd gone anti-social, killed perhaps, he'd still
tend to protect that child.
-- McCoy, "The Ultimate Computer", stardate 4731.3

Re: xz backdoor

[Andrey Rakhmatullin]

On Sun, Mar 31, 2024 at 12:13:30PM +0200, Alexandre Detiste wrote:
> Le dim. 31 mars 2024 à 10:17, Sirius  a écrit :
> > Reduction of complexity is IMHO always worthwhile as it would open the
> > door for more people being able to step up as maintainers (taking into
> > account that volunteers right this minute might not be overly welcome and
> > when they are, they should likely not be near authentication, crypto and
> > compression at least initially).
> 
> It's worse than that: to make the xz MR looks more legit;
> the fake puppet profile "Hans Jansen" also sent _maybe_ legit MR to
> random games repos:
>https://news.ycombinator.com/item?id=39868390
> 
> Here fixing our Salsa tooling could help also making real newcomers
> life more enjoyable by always disabling MR again upstream & pristine-tar tar.
Yeah, those MRs never made sense to time, it's obvious that "run gbp
import-orig and propose the result as MRs" is just not a workflow we
support with the existing tools.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Iustin Pop]

On 2024-03-31 10:47:57, Luca Boccassi wrote:
> On Sun, 31 Mar 2024 at 08:39, Bastian Blank  wrote:
> >
> > On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
> > > On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
> > > > As others have said, the best solution is to relay on HSW for handling
> > > > the cryptographic material.
> > > Aren't these answers to different questions?
> > > Not all attacks are about stealing the key or using it to sign unintended
> > > things.
> >
> > Also a HSM does only allow to control access to the cryptographic
> > material.  But it asserts no control over what is actually signed.
> >
> > So an attacker needs to wait until you ask the HSM it is okay to sign
> > something.
> >
> > Bastian
> 
> This is true as in the default configuration you get asked for the
> yubikey pin only once per "session", and then it's cached
> transparently and there's no GUI feedback when the token is used (the
> light on it blinks, but noticing that requires having it in line of
> sight at all times). However, it's already better than nothing as it
> means such an attack must be "online", and run in the same "session"
> as the active user, so perfect should definitely not be the enemy of
> good here IMHO. Also, iirc this can be configured to always ask for
> the pin on each signature, although this could get burdensome. But
> given the very low price of yubikeys (or similar tokens), and how well
> and seamless they work these days, I think the offer of buying any DD
> that doesn't have one such a token is one that we should take up and
> make it happen.

Jumping in late in the HSM thread, but I'm not sure I understand the
exact setup people propose.

Option 1: Moving keys to one yubikey, while keeping the original key
material "safe" offline. How do you know the "safe offline" material is
safe and hasn't been copied?

Option 2: Generate keys on the yubikey and have them never leave the
secure enclave. That means having 2 yubikeys per developer, and ensuring
you keep track of _two_ keys, but it does ensure there's a physical
binding to the key.

Are there other options? And which option is proposed?

I have quite a few yubikeys, but I haven't migrated to use them since
it's not clear to me what is a good, and recommended, workflow. I'm
relatively against option 1, since the "safe offline" key material
somehow doesn't appeal to me.

regards,
iustin

Re: xz backdoor

[Colin Watson]

On Sun, Mar 31, 2024 at 10:10:42AM +0200, Sirius wrote:
> Not worth boiling the ocean over, but is there an estimate of how many
> packaged projects have customisations to their autoconf that is not found
> in the upstream autoconf project? If that number is low single digit
> percent, it may motivate those projects to upstream their modifications.
> If it is double digits percent, it might not be possible to disallow
> vendoring the files.

This is difficult to answer because it's comparing apples and oranges to
some extent: not all autoconf customizations are vendored or would make
any kind of sense to upstream.  For example,
https://gitlab.com/man-db/man-db/-/blob/main/m4/man-arg-config-file.m4
is obviously specific to that project; it's just in a separate file for
the same reasons that projects past a certain size don't typically put
all their code in a single file.

I suspect the question you're aiming for is something like "how many
packaged projects have copied autoconf macros from elsewhere and
modified them but kept the same file names, so that a naïve attempt to
update them would drop those modifications".  My guess is that the
number here is very low - IME it's much more common in such cases to
either rename the macro file to be obviously project-specific or to find
some workaround that doesn't require changing the upstream macro - but
I've never seen anything resembling a robust analysis of this and I may
well have a skewed view.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: xz backdoor

[Colin Watson]

On Sun, Mar 31, 2024 at 09:35:09AM +0200, Bastian Blank wrote:
> On Sat, Mar 30, 2024 at 08:15:10PM +, Colin Watson wrote:
> > On Sat, Mar 30, 2024 at 05:12:17PM +0100, Sirius wrote:
> > > I have seen discussion about shifting away from the whole auto(re)conf
> > > tooling to CMake or Meson with there being a reasonable drawback to CMake.
> > > Is that something being discussed within Debian as well?
> > It's not in general something that Debian can unilaterally change.  And
> > in a number of cases switching build system would be pretty non-trivial.
> 
> What we can do unilaterally is to disallow vendoring those files.
> 
> Does it help?  At least in the case of autoconf it removes one common
> source of hard to read files.

That's doable unilaterally to some extent, using the dh-autoreconf style
of approach: the files may be vendored in the upstream tarball, but
we'll throw them away and rebuild them.  I did that recently for the
packages that use gnulib where I'm upstream (e.g.
https://salsa.debian.org/debian/libpipeline/-/commit/b596eefb342e93136cf1f479415981fc7f48).
Frankly, I suspect that it will introduce slightly more FTBFS bugs over
time due to differences between the packaged version of gnulib and the
version in use upstream (although things have got better in that regard
over the last year or so with the introduction of stable branches), but
it does seem to be worth it for transparency.

It's going to take some extra work in some cases though.  For example,
when I looked at groff, I quickly ran into needing
https://lists.gnu.org/archive/html/groff/2024-03/msg00211.html.  And in
cases where upstreams have just copied some individual files from gnulib
rather than adopting the bootstrap script, there may be a bigger hill to
climb.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: xz backdoor

[Luca Boccassi]

On Sun, 31 Mar 2024 at 08:39, Bastian Blank  wrote:
>
> On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
> > On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
> > > As others have said, the best solution is to relay on HSW for handling
> > > the cryptographic material.
> > Aren't these answers to different questions?
> > Not all attacks are about stealing the key or using it to sign unintended
> > things.
>
> Also a HSM does only allow to control access to the cryptographic
> material.  But it asserts no control over what is actually signed.
>
> So an attacker needs to wait until you ask the HSM it is okay to sign
> something.
>
> Bastian

This is true as in the default configuration you get asked for the
yubikey pin only once per "session", and then it's cached
transparently and there's no GUI feedback when the token is used (the
light on it blinks, but noticing that requires having it in line of
sight at all times). However, it's already better than nothing as it
means such an attack must be "online", and run in the same "session"
as the active user, so perfect should definitely not be the enemy of
good here IMHO. Also, iirc this can be configured to always ask for
the pin on each signature, although this could get burdensome. But
given the very low price of yubikeys (or similar tokens), and how well
and seamless they work these days, I think the offer of buying any DD
that doesn't have one such a token is one that we should take up and
make it happen.

Re: xz backdoor

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Christian Kastner (2024-03-30 19:49:48)
> On 2024-03-30 17:00, Marco d'Itri wrote:
> > On Mar 30, Jonathan Carter  wrote:
> > 
> >> Another big question for me is whether I should really still
> >> package/upload/etc from an unstable machine. It seems that it may be 
> >> prudent
> > If we do not use unstable for development then who is going to?
> 
> Are you both talking about unstable hosts, or unstable chroots, or...?
> 
> I do all my development on a stable host, within rootless podman
> containers which are trivial to set up. I run final sbuilds and
> autopkgtests in QEMU VMs, also trivial to set up.
> 
> This is both out of convenience (I want my workstation to be based on stable)
> and precisely because of the afforded isolation.

I find it *very* difficult to balance all the pros and cons of running stable
versus unstable/testing on my own computer.

On the one hand, yes, I was able to find and report bugs to packages in
unstable before they migrated to testing just because I run sbuild and
autopkgtest with unstable chroots and that is nice. Doing so also has become
easier not the least thanks to Christian's work on sbuild-qemu, for example.

On the other hand, I am still running bookworm because I only have only one
computer and I really hate the times when I have to tell my family that sorry,
I cannot spend the next few hours with you because some package on unstable
broke some functionality of my system.

So I should be happy with running stable on the host and unstable in my
chroots, right? Well not so fast. The stuff I work on involves many things
which are impractical or just not possible in a chroot, container or virtual
machine. For example, recently there was a regression in mesa which introduced
graphics glitches on my system (panfrost driver). Luckily, I'm backporting mesa
from unstable, so I was able to notice this bug but I could not've noticed this
if I had just used chroots, containers or virtual machines. This had to be
installed on my system. Same with recent regressions in kernel 6.6 which locks
up my system completely after around 10 minutes of uptime. This is specific to
my platform and would not've been observable in qemu. Or take my work on
mmdebstrap and system image building. I am not going to rung qemu inside qemu
on my already comparatively slow system.

Clearly, for many many things, running bookworm and using unstable chroots is
sufficient to find bugs. There are just a number of situations where it isn't.
So while I agree with your recommendation Christian in general, I do not think
it is a silver bullet.

Another example is when I wanted to run a GUI program inside an unshared chroot
environment.  Wayland does not seem to be happy about that  and I didn't find a
way to test my GUI application successfully. But maybe my container environment
just fails to set up some things? Does there exist a way to test GUI
applications inside a container without requiring superuser privileges to run
the container?

In summary: would running unstable instead of bookworm let me find more bugs
than running bookworm with unstable chroots? For my specific work: yes,
absolutely.  Am I upgrading from bookworm to unstable or at least testing?
Absolutely not. I just don't have the amount of free-time this would require of
me.  Just performing the 12-13 bisection steps to find the offending kernel
commit which makes my system lock up would easily require a day of free time
from me.  I'm afraid I do not have this luxury. Not even remotely close!

So I'll continue to not dogfood as hard as I could and run as much from
bookworm as I can. Would it make me a better contributor if I ran unstable?
Certainly! But this thing is just a hobby of mine and I can only allot that
much time to do risky experiments with my only computer. I guess others are in
the same boat?

Thanks!

cheers, josch

signature.asc
Description: signature

Re: xz backdoor

[Alexandre Detiste]

Le dim. 31 mars 2024 à 10:17, Sirius  a écrit :
> Reduction of complexity is IMHO always worthwhile as it would open the
> door for more people being able to step up as maintainers (taking into
> account that volunteers right this minute might not be overly welcome and
> when they are, they should likely not be near authentication, crypto and
> compression at least initially).

It's worse than that: to make the xz MR looks more legit;
the fake puppet profile "Hans Jansen" also sent _maybe_ legit MR to
random games repos:
   https://news.ycombinator.com/item?id=39868390

Here fixing our Salsa tooling could help also making real newcomers
life more enjoyable by always disabling MR again upstream & pristine-tar tar.

I don't see any real good purpose for these unreviewable [*] huge diff;
one could just ping someone with commit access to do
"gbp import-orig --uscan --pristine-tar ; gbp push"
or if absolutely nobody has the time for this maybe a bot could do it ?

BTW it's quite smart to attack Games team:
as we're used to get legit one-off MR from people
never to be seen again later.

[*] https://salsa.debian.org/games-team/endless-sky/-/merge_requests/5

Greetings

Re: xz backdoor

[Christian Kastner]

On 2024-03-31 04:22, Santiago Ruano Rincón wrote: 
> I don't see the real benefit.
> 
> As others have said, the best solution is to relay on HSW for handling
> the cryptographic material.

That's extremely important (which is why I use a HSM) but that "just"
prevents exfiltration of the keys. An attacker could still simply modify
dpkg-buildpackage or any other part of the toolchain to inject malicious
code into one's builds that one then signs.

As to the benefits, containers can do a lot that you probably couldn't
do directly on your host. As an example, setting up/tearing down complex
environments emulating multiple hosts.

A more obvious example is developing for any environment that is not
unstable. With containers, basically all you have to do is swap the name
of the base image.

Best,
Christian

(Santiago, sorry for sending it twice)

Re: xz backdoor

[Adrian Bunk]

On Sun, Mar 31, 2024 at 03:07:53AM +0100, Colin Watson wrote:
> On Sun, Mar 31, 2024 at 04:14:13AM +0300, Adrian Bunk wrote:
> > The timing of the 5.6.0 release might have been to make it into the 
> > upcoming Ubuntu LTS, it didn't miss it by much.
> 
> It didn't miss it at all, even.  Ubuntu has rolled it back and is
> rebuilding everything that was built using it, but it did make it into
> noble-proposed (the current unstable analogue) for some time and noble
> (the current testing analogue) briefly.

It missed being in the actual release, due to being detected by chance
before the release date of noble.

cu
Adrian

Re: xz backdoor

[Simon Josefsson]

Bastian Blank  writes:

> On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
>> On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
>> > As others have said, the best solution is to relay on HSW for handling
>> > the cryptographic material.
>> Aren't these answers to different questions?
>> Not all attacks are about stealing the key or using it to sign unintended
>> things.
>
> Also a HSM does only allow to control access to the cryptographic
> material.  But it asserts no control over what is actually signed.

Transparency techniques are better suited to solve that problem: make
sure that you don't trust a signature before verifying that the
signature was publicly logged together with its artifacts, so that they
can be independently audited and analyzed eventually.  Preferrably even
verify that the package artifacts build reproducible, but that takes
more resources.  Right now Debian trust signatures at face value which
is fragile.  The WebPKI world -- which is populated by untrustworthy
private key signers -- has moved in that direction, and it does improve
things.

/Simon


signature.asc
Description: PGP signature

Re: xz backdoor

[Sirius]

In days of yore (Sun, 31 Mar 2024), Bastian Blank thus quoth: 
> On Sat, Mar 30, 2024 at 08:15:10PM +, Colin Watson wrote:
> > On Sat, Mar 30, 2024 at 05:12:17PM +0100, Sirius wrote:
> > > I have seen discussion about shifting away from the whole auto(re)conf
> > > tooling to CMake or Meson with there being a reasonable drawback to CMake.
> > > Is that something being discussed within Debian as well?
> > It's not in general something that Debian can unilaterally change.  And
> > in a number of cases switching build system would be pretty non-trivial.
> 
> What we can do unilaterally is to disallow vendoring those files.
> 
> Does it help?  At least in the case of autoconf it removes one common
> source of hard to read files.

Reduction of complexity is IMHO always worthwhile as it would open the
door for more people being able to step up as maintainers (taking into
account that volunteers right this minute might not be overly welcome and
when they are, they should likely not be near authentication, crypto and
compression at least initially).

Not worth boiling the ocean over, but is there an estimate of how many
packaged projects have customisations to their autoconf that is not found
in the upstream autoconf project? If that number is low single digit
percent, it may motivate those projects to upstream their modifications.
If it is double digits percent, it might not be possible to disallow
vendoring the files.

-- 
Kind regards,

/S

Re: xz backdoor

[Bastian Blank]

On Sat, Mar 30, 2024 at 08:15:10PM +, Colin Watson wrote:
> On Sat, Mar 30, 2024 at 05:12:17PM +0100, Sirius wrote:
> > I have seen discussion about shifting away from the whole auto(re)conf
> > tooling to CMake or Meson with there being a reasonable drawback to CMake.
> > Is that something being discussed within Debian as well?
> It's not in general something that Debian can unilaterally change.  And
> in a number of cases switching build system would be pretty non-trivial.

What we can do unilaterally is to disallow vendoring those files.

Does it help?  At least in the case of autoconf it removes one common
source of hard to read files.

Bastian

-- 
Vulcans do not approve of violence.
-- Spock, "Journey to Babel", stardate 3842.4

Re: xz backdoor

[Bastian Blank]

On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
> On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
> > As others have said, the best solution is to relay on HSW for handling
> > the cryptographic material.
> Aren't these answers to different questions?
> Not all attacks are about stealing the key or using it to sign unintended
> things.

Also a HSM does only allow to control access to the cryptographic
material.  But it asserts no control over what is actually signed.

So an attacker needs to wait until you ask the HSM it is okay to sign
something.

Bastian

-- 
War isn't a good life, but it's life.
-- Kirk, "A Private Little War", stardate 4211.8

Re: xz backdoor

[Bastian Blank]

On Sat, Mar 30, 2024 at 07:15:28PM -0700, Otto Kekäläinen wrote:
> I am doing all my builds inside a (Podman) container with the sources
> loop-mounted.

You do, but Debian itself (aka DSA) does not.  They still prefer to
trust all 100k packages and run them as root in the init namespace over
the five people who can login as buildd and potentially trigger
capability reachable problems in the kernel.  This is what got as in
part of the situation, as we don't even know if the buildd hosts are
untampered.

Bastian

-- 
Spock: The odds of surviving another attack are 13562190123 to 1, Captain.

Re: xz backdoor

[Andrey Rakhmatullin]

On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
> > I agree that dogfooding is important for discovering quality issues, but
> > I think it's a poor argument for discovering security issues, especially
> > if it concerns a host which is used for building and signing packages.
> > 
> > As I mentioned earlier, I think containers are one good way to have
> > almost the best of both worlds. One can do anything one could do on
> > host, all while being isolated from that host, and with very little
> > overhead but also a ton of useful extra features.
> 
> I don't see the real benefit.
> 
> As others have said, the best solution is to relay on HSW for handling
> the cryptographic material.
Aren't these answers to different questions?
Not all attacks are about stealing the key or using it to sign unintended
things.


-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Todd Zullinger]

Diane Trout wrote:
> On Sun, 2024-03-31 at 03:34 +0100, Wookey wrote:
>> On 2024-03-30 20:52 +0100, Ansgar  wrote:
>>> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
>>> Possibly also TPM modules in computers.
>>> 
>>> These can usually be used for both OpenPGP and SSH keys.
>> 
>> Slightly off-topic, but a couple of recent posts have given me the
>> same thought:
>> 
>> Can someone point to good docs on this?  I've had a
>> yubikey for 3/4 of a year now but have not yet worked out
>> how I put my GPG key in it.  (or if it should be another
>> key, or a subkey, or whatever). So I'm not actually using
>> it yet.
> 
> I've also been thinking I needed to this, and so far this has looked
> like the most detailed write up I've found so far.

Another useful source is the "Protecting code integrity with
PGP" from the Linux Foundation IT folks:

https://github.com/lfit/itpol/blob/master/protecting-code-integrity.md

It's a bit less daunting than the drduh guide, which can be
helpful for folks who aren't subject-matter experts and just
want a reasonable "how do I make this work" guide. :)

> I haven't followed the advice but I've been working on trying to
> understand it.
> 
> https://github.com/drduh/YubiKey-Guide

-- 
Todd


signature.asc
Description: PGP signature

Re: xz backdoor

[Andreas Metzler]

On 2024-03-31 Wookey  wrote:
[...]
> e.g. I remember it took me years to realise that I used _my_ public
> key for signing,
[...]

Good morning,

s/public/private/ - $recipient can then use your public key to verify
the sig.

cu Andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Re: xz backdoor

[Diane Trout]

On Sun, 2024-03-31 at 03:34 +0100, Wookey wrote:
> On 2024-03-30 20:52 +0100, Ansgar  wrote:
> > Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
> > Possibly also TPM modules in computers.
> > 
> > These can usually be used for both OpenPGP and SSH keys.
> 
> Slightly off-topic, but a couple of recent posts have given me the
> same thought:
> 
> Can someone point to good docs on this?  I've had a yubikey for 3/4
> of
> a year now but have not yet worked out how I put my GPG key in it.
> (or
> if it should be another key, or a subkey, or whatever). So I'm not
> actually using it yet.

I've also been thinking I needed to this, and so far this has looked
like the most detailed write up I've found so far.

I haven't followed the advice but I've been working on trying to
understand it.

https://github.com/drduh/YubiKey-Guide

I'd like to know how subkeys interact with signing packages and there's
an two ways of storing ssh keys with yubikeys. This write up describes
one and mentions the other exists.

Diane


signature.asc
Description: This is a digitally signed message part

Re: xz backdoor

[Otto Kekäläinen]

Hi!

On Sat, 30 Mar 2024 at 14:32, Andrey Rakhmatullin  wrote:
> On Sat, Mar 30, 2024 at 10:49:33AM +0200, Jonathan Carter wrote:
> > Another big question for me is whether I should really still
> > package/upload/etc from an unstable machine. It seems that it may be prudent
> > to consider it best practice to work from stable machines where any private
> > keys are involved. For me it's just been so convenient to use unstable
> > because it helps track changes that affect my users by the time it hits
> > stable and also find bugs early that I care about, but perhaps I just need
> > to make that adjustment and find more efficient ways to track unstable
> > (perhaps on additional machines / VMs / etc). Not sure how other DDs think
> > about this, but I'm also curious how they will deal with this, because
> > there's near to no filter between unstable and the outside world, and this
> > is probably not the last time someone will try something like this.
> For me it's simple: if I'm forced to run my tools not on the host but in
> some kind of inconvenient VM/chroot/whatever, I'll just stop contributing.

I am doing all my builds inside a (Podman) container with the sources
loop-mounted. Thus I can use git and visual code editor directly on
sources with full access, but when the build runs, it is fully inside
a container that has no host access nor even network access. To
achieve this I wrote a tool which you might want to check out:
https://salsa.debian.org/otto/debcraft

Re: xz backdoor

[Wookey]

On 2024-03-30 20:52 +0100, Ansgar  wrote:
> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
> Possibly also TPM modules in computers.
> 
> These can usually be used for both OpenPGP and SSH keys.

Slightly off-topic, but a couple of recent posts have given me the
same thought:

Can someone point to good docs on this?  I've had a yubikey for 3/4 of
a year now but have not yet worked out how I put my GPG key in it. (or
if it should be another key, or a subkey, or whatever). So I'm not
actually using it yet.

PEB also described what sounded like a very sensible way to manage
keys (using subkeys) in one of these threads but I don't know how to
do that myself.

Basically reasonably idiot-proof docs for people who don't understand
crypto and have no idea what to type.  And a mental model for what
keys (and files) are going where, and why.

e.g. I remember it took me years to realise that I used _my_ public
key for signing, and someone _else's_ public key for encrypting
messges for them. Things made so much more sense then. But it wasn't
at all clear from the docs for DD's to get and use a GPG key back in
2000, so I couldn't send a crypted message for years (because I was
trying to use the wrong key).

I also discovered about 2 years ago (i.e ~20 years after making a key)
that I can change the password on it - it's not immutable! That's
probably something that I should have found out/been told sooner.

I am now aware that I could use subkeys for signing and it would be
more secure, but I don't know how, so I'm not doing it (and this has
been the state for quite a few years now). Did/do I have to make it
differently in the first place, do I do something to the one I already
have (chop it up and keep the bits in different places? sign other
keys with it? something else?) 

Hopefully info at the right level already exists and I just need
pointing at it, but I have tried a couple of times in the past to
understand both yubikey initialisation/use and subkey generation/use
and have failed to make any progress despite reading wiki pages and
man pages and blogs. I just realised that I didn't understand how it
worked or what the tradeoffs were, so couldn't really make sensible
decisions about what I should do. I suspect I'm not the only one who
is quite vague about all this.

Wookey
-- 
Principal hats:  Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Re: xz backdoor

[Santiago Ruano Rincón]

El 31/03/24 a las 00:53, Christian Kastner escribió:
> On 2024-03-30 22:59, Santiago Ruano Rincón wrote:
> > The backdoor was discovered by someone using the compromised xz-utils *in 
> > their own machines*. So we are lucky we have people eating our own sid 
> > stuff before it becomes part of a stable release.
> 
> The luck was that this particular compromise was discovered, not that it
> happened.

I don't say the opposite.

> 
> I agree that dogfooding is important for discovering quality issues, but
> I think it's a poor argument for discovering security issues, especially
> if it concerns a host which is used for building and signing packages.
> 
> As I mentioned earlier, I think containers are one good way to have
> almost the best of both worlds. One can do anything one could do on
> host, all while being isolated from that host, and with very little
> overhead but also a ton of useful extra features.

I don't see the real benefit.

As others have said, the best solution is to relay on HSW for handling
the cryptographic material.


signature.asc
Description: PGP signature

Re: xz backdoor

[Colin Watson]

On Sun, Mar 31, 2024 at 04:14:13AM +0300, Adrian Bunk wrote:
> The timing of the 5.6.0 release might have been to make it into the 
> upcoming Ubuntu LTS, it didn't miss it by much.

It didn't miss it at all, even.  Ubuntu has rolled it back and is
rebuilding everything that was built using it, but it did make it into
noble-proposed (the current unstable analogue) for some time and noble
(the current testing analogue) briefly.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: xz backdoor

[Adrian Bunk]

On Sat, Mar 30, 2024 at 10:49:33AM +0200, Jonathan Carter wrote:
>...
> On 2024/03/29 23:38, Russ Allbery wrote:
> > I think the big open question we need to ask now is what exactly the
> > backdoor (or, rather, backdoors; we know there were at least two versions
> > over time) did.
> 
> Another big question for me is whether I should really still
> package/upload/etc from an unstable machine. It seems that it may be prudent
> to consider it best practice to work from stable machines where any private
> keys are involved. For me it's just been so convenient to use unstable
> because it helps track changes that affect my users by the time it hits
> stable and also find bugs early that I care about, but perhaps I just need
> to make that adjustment and find more efficient ways to track unstable
> (perhaps on additional machines / VMs / etc). Not sure how other DDs think
> about this, but I'm also curious how they will deal with this, because
> there's near to no filter between unstable and the outside world, and this
> is probably not the last time someone will try something like this.

I don't think it is such a clear case that stable is more secure than 
unstable.

The uncommon part might be that it was detected so early, and only due 
to a minor visible side effect on performance found by pure luck that a 
better implementation of the exploit might have been able to avoid.

The timing of the 5.6.0 release might have been to make it into the 
upcoming Ubuntu LTS, it didn't miss it by much.

And an intentional backdoor is not necessarily much different from
one caused by a bug:

Heartbleed (CVE-2014-0160) in OpenSSL made it into stable.

The Debian-specific bug that broke the OpenSSL RNG resulting in 
predictable keys (CVE-2008-0166) made it into stable.

There have even been cases where an attacker realized that
a non-security bugfix fixed something that can be exploited.
In such cases unstable might get fixed, but stable not.

Perhaps a case can be made that stable is slightly more secure,
but an intentional backdoor that gets detected early is rather
rare so far.

> -Jonathan

cu
Adrian

Re: xz backdoor

[Christian Kastner]

On 2024-03-30 22:59, Santiago Ruano Rincón wrote:
> The backdoor was discovered by someone using the compromised xz-utils *in 
> their own machines*. So we are lucky we have people eating our own sid stuff 
> before it becomes part of a stable release.

The luck was that this particular compromise was discovered, not that it
happened.

I agree that dogfooding is important for discovering quality issues, but
I think it's a poor argument for discovering security issues, especially
if it concerns a host which is used for building and signing packages.

As I mentioned earlier, I think containers are one good way to have
almost the best of both worlds. One can do anything one could do on
host, all while being isolated from that host, and with very little
overhead but also a ton of useful extra features.

Best,
Christian

Re: xz backdoor

[Pierre-Elliott Bécue]

De : Adrian Bunk 
À : Pierre-Elliott Bécue 
Cc : debian-devel@lists.debian.org
Date : 31 mars 2024 00:25:09
Objet : Re: xz backdoor

> On Sat, Mar 30, 2024 at 11:28:07PM +0100, Pierre-Elliott Bécue wrote:
>> ...
>> I'd be happy to have Debian France care about buying and having yubikeys
>> delivered to any DD over the world.
> 
> Including Russia?
> 
> cu
> Adrian
It I leagally can, yes.

-- 
Pierre-Elliott Bécue

Re: xz backdoor

[Roberto C . Sánchez]

On Sun, Mar 31, 2024 at 01:20:39AM +0200, Adrian Bunk wrote:
> On Sat, Mar 30, 2024 at 11:28:07PM +0100, Pierre-Elliott Bécue wrote:
> >...
> > I'd be happy to have Debian France care about buying and having yubikeys
> > delivered to any DD over the world.
> 
> Including Russia?
> 
Supporting DDs in Russia would definitely demonstrate a commitment to
geographic diversity.

Regards,

-Roberto

-- 
Roberto C. Sánchez

Re: xz backdoor

[Adrian Bunk]

On Sat, Mar 30, 2024 at 11:28:07PM +0100, Pierre-Elliott Bécue wrote:
>...
> I'd be happy to have Debian France care about buying and having yubikeys
> delivered to any DD over the world.

Including Russia?

cu
Adrian

Re: xz backdoor

[Leandro Cunha]

Hi,

On Sat, Mar 30, 2024 at 7:00 PM Santiago Ruano Rincón
 wrote:
>
>
>
> Em 30 de março de 2024 13:00:26 GMT-03:00, Marco d'Itri  
> escreveu:
> >On Mar 30, Jonathan Carter  wrote:
> >
> >> Another big question for me is whether I should really still
> >> package/upload/etc from an unstable machine. It seems that it may be 
> >> prudent
> >If we do not use unstable for development then who is going to?
> >I think that the real question is whether we should really still use
> >code-signing keys which are not stored in (some kind of) HSM.
> >
>
> The backdoor was discovered by someone using the compromised xz-utils *in 
> their own machines*. So we are lucky we have people eating our own sid stuff 
> before it becomes part of a stable release.
>

You told the truth, sometimes I think it's better to wait a while to
launch very new versions, doing tests beforehand and looking for
problems. But if the person hadn't sent the backdoored version to
unstable, hadn't worried about issues of seconds of performance, in my
opinion it would be quite possible that this would have been in vain
and the person did a great job in mitigating this (I personally I
found very interesting). But on the other hand it caused inconvenience
because I saw in an email here that the processing of the Debian file
is momentarily turned off until it checks what needs to be done.

-- 
Cheers,
Leandro Cunha
-BEGIN PGP PUBLIC KEY BLOCK-

mQINBF/gQ8gBEADHVKgoWsUWNGVvR6sMhBPUdBUEH+QALpr1QYXhetBfRwaY0HWN
pKgejHdxKO8H+kIhRMoh89CCKg3hAJ9LmOOTXkX7U5/Cya/zRMKk5zBD3rKIaugh
0XYT15Nz1jwL7TIDG25yPSloDtVgVXTep0ZzKsNYJjb4OAqa88cvUEJEhhqrldlR
gpNbkixEh5ituO8pMShEBWqLs3yt4Hr1VFWnTIm4dl/JLBHpexzubDOw/mKCTpNd
A1JGHTvce1wtJ2fMzCVzhEjd5pyjLZV/o8hVw2/ON/yXvpJuz0lV/hiW0M+cDcas
sKftErtsZpRy3wwXdkBcJt6soYuqfCHwgMfL2iC6mPviE8xWAHMOmhdC3wDskZpb
RcLfH5IMYajJAGRO/GCMcKKbq7WkEOeloivtg64xBlYuJf9aOcHKP/8R3EObiNp7
ubQAJtV3pEGD4mx1mhutFxDHB+CfnxE3dWvxZSV9y1n4UOzkDJ3kDx5Ee0MbRvJD
w6aXKc6dhYREgh7hLDcMFz+3LcBiZDLxI3g+SHe3Bl61vdsnPno+0HhCzvB+fL4S
eoy7Myfiunz9BrB2HPN+wNCT0YgV+Kv8QoDGzBwos5H1vUJSY4t59w6xoXAYUsAm
hjAM8s+rUtG40mcUWePd8kZtgE9IV1eQ+Qt8/SNpSdRnUunmIGl3JjHvEwARAQAB
tClMZWFuZHJvIEN1bmhhIDxsZWFuZHJvY3VuaGEwMTZAZ21haWwuY29tPokCTgQT
AQoAOBYhBLT5oBCvKN3HzFEPK8LZ4zKUW9A8BQJf4EPIAhsDBQsJCAcCBhUKCQgL
AgQWAgMBAh4BAheAAAoJEMLZ4zKUW9A8FjAQAKWYqiLpLUD+DLB+NSy3DI3rf9z3
k0vE7TLaEjdEM5CQWN+j4vBqMnAckdcARvSWPndTjp8K+mtFF4PyfhNbS64z/a7L
F3DdhmX73n7LKFG8Ow9NZwcrkmPwH5WcP7mXTh6R+6/+OSL/K85NB8MLlxQTJOni
julVax9JEZjwBaP2HLCu53Zq9gZcvJlXoAoTHyTxKdp8Mh8V+Qit26E78o9c6SQD
Dq9eyMRG8hYCRfreDjKceRkYHjECySlk+VoI1ssVs07Dqvxg6qSyP4RnW+1+W74C
s0yIyuC/eRJpMAf1PBQEOOrVcTfRfpN+go955t21yIAvT58vqotTM5eaqXYIQn/y
sC4lThZai/ZBZHxl5Mbv42WkkYdjisLQOCALIMBpj5nq4oh2C+kvMupcuBKfERgV
dguU51MzfQktKb6d5y777zYnDaFMQDD2IfiD/C7ln5A9LP/L54ixlA3uRmWx/yAx
/m+Zusws98j4Eq/jw5T54XW655m6lMCTE9WXLJkgxrRcEonHSllbgRSsToEmWq0Z
doxcnpagHdcGQzW+cu2VOGi1da73ZFmrn+ptJgc8cW2suO06IeArOi0TzIg7e65j
Xp2DbJCpFrfzEuBb1u71WvB8V2MkAfJZx/uZJPCA936B4HT8YGPEMzlQRIHI2Y9C
+DloyzlBLTS1EMKuuQINBF/gQ8gBEAC47o9u1Wm9jZ6RC+lfxEDEvVS7MmI5VzSy
q04rFttWwbKix13pc65aDlk47LxWrb84N3Gnf1E/OTsLTXqC7u5JZ7YJkC6CsPbo
D1sQkfCiJCFCTgf7dydEVt8ujS/Uu1kz86ufdRwaMRcvBZAORGdB58LEsLB65WN4
hLRYF7xvcxu6t7FGrIYereaxUAWLA2B/ZnCEdOY94w7s0uaPjHdf4lfHebuZ7T08
iG5ACDvKBjgaFArGfdNYWchXJgbOEg14bGj40/8LuBKQMZASiFSqLPZxoporK9FY
xBw+D080dUWWD5g868TZ3pkM3DXO9bdq22IBKqKOep8CnuKgoDpUvA8dTEY/UDCn
sdOlBUK/Y9zTGVmD/90cO/xkvkV78suqiBnwBSddPzVS0EuiWwrLGu8gaY4EyM/X
7khlbTcMgh4njzUCAE6Tq+TbXSxn86wuOybVY5Y+I99LNdsocI5SIn2nDh2IOi00
4dE/iwO2MatWIOLFBC7pw8Xv4UHZY+WIf3Y/6XjExpllhUkeB6BwZpTr1SXk+cug
q5Dj5i4aGn2LrvQJ57terqUWYyDUBFgXTc4SPOzT5og8CavBgHfrQoFwSnRZ2oyX
xtZhEDI5Pk2j1qTbOhXZ29po4rPNWHMq2HQgM0I+BqQndsoVdkPOFzS2wKkdXjCz
bNYcyanusQARAQABiQI2BBgBCgAgFiEEtPmgEK8o3cfMUQ8rwtnjMpRb0DwFAl/g
Q8gCGwwACgkQwtnjMpRb0Dzh6g//ZjXaWSzKmG5ZS6XJa/ZOokkE2hFOFusWX8Qa
hEwLAnTFEy02dLfV54rKwmu2jHPDKLhE+iYtusvytueZAzVRyQahv0RE4BH8Emqw
gQdBwyJ/L+QhUp/lMdJ6Hh/2ZSZmzU29U24vnY+U+haoB1fLnA3lXgOP59kMLGud
lERR2Vluuc7TcpzvcaRWgrQRU2vSrrBBEp6y07iVKbRM/9yhE/aHJahLbhKh2Dk9
WJvHPnhYJY5yU+Y5vTl3BiW5+EuzMBdPUawOWKhqCq9dswn0GL1g/vlt/bdU/6DO
jECQ6fssTAtDjRClXySsS3X0mh8y8qlGvMPB4anfvOy4+4nUV6IESdJftKn2SMGd
CA3MaQ+S7frWn5v7GIWSC9vumCsiu1JTOugLmbVmu5m5nFsyllavm/k9LtOtswuF
fHM/SlXLFuGBWU6XceqaM2dpP8i5jGz0vIGMhqoFNgXWGO1NhwR1rmeU1CMpnM5e
Wue4h/+mJiuEzuZcmzOcwq3HGMUXO0jZDgLEmlnenO9czhrLuGZaMXGdwnIk0G3O
+SqH36v7blnDh96RXpgaa+ifTHd0qKeoVXVwSq/9jNtHSQrI+NJcTpMhu73xtxhX
UFPr/31+IFLWepC5GDwdu/gQm5E6ntGyxE2p2v76pcjz7SGdXjPFZjqekBveEJuW
fNdY6Ns=
=rdCA
-END PGP PUBLIC KEY BLOCK-

Re: xz backdoor

[Pierre-Elliott Bécue]

Santiago Ruano Rincón  wrote on 30/03/2024 at 
22:59:43+0100:

> Em 30 de março de 2024 13:00:26 GMT-03:00, Marco d'Itri  
> escreveu:
>>On Mar 30, Jonathan Carter  wrote:
>>
>>> Another big question for me is whether I should really still
>>> package/upload/etc from an unstable machine. It seems that it may be prudent
>>If we do not use unstable for development then who is going to?
>>I think that the real question is whether we should really still use 
>>code-signing keys which are not stored in (some kind of) HSM.
>>
>
> The backdoor was discovered by someone using the compromised xz-utils *in 
> their own machines*. So we are lucky we have people eating our own sid stuff 
> before it becomes part of a stable release.

+1 and <3

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Pierre-Elliott Bécue]

Ansgar   wrote on 30/03/2024 at 20:52:29+0100:

> Hi,
>
> On Sun, 2024-03-31 at 00:40 +0500, Andrey Rakhmatullin wrote:
>> On Sat, Mar 30, 2024 at 05:00:26PM +0100, Marco d'Itri wrote:
>> 
>> > I think that the real question is whether we should really still
>> > use 
>> > code-signing keys which are not stored in (some kind of) HSM.
>> What are the options for random DDs for that?
>
> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
> Possibly also TPM modules in computers.
>
> These can usually be used for both OpenPGP and SSH keys.
>
> If someone cannot afford them, I think Debian paying for them is a good
> investment; Debian buying tokens for all project members would also be
> nice, but logistics are probably annoying...
>
> A compromised computer alone is then not enough to get a copy of the
> private key: one would also need an exploit for the hardware token.
> (A compromised computer can still give temporary access to the key when
> it is in use and unlocked; some devices can require pushing a button
> for signing, but of course a compromised computer could claim to sign
> something different than what gets signed and just show a "wrong PIN"
> message to have the user try again.)
>
> If you believe the hardware token to have a backdoor, exploiting it
> might still require physical access to the token.

I'd be happy to have Debian France care about buying and having yubikeys
delivered to any DD over the world.

-- 
PEB
Debian France's Treasurer, happy to spend time to make things safer.


signature.asc
Description: PGP signature

Re: xz backdoor

[Santiago Ruano Rincón]

Em 30 de março de 2024 13:00:26 GMT-03:00, Marco d'Itri  
escreveu:
>On Mar 30, Jonathan Carter  wrote:
>
>> Another big question for me is whether I should really still
>> package/upload/etc from an unstable machine. It seems that it may be prudent
>If we do not use unstable for development then who is going to?
>I think that the real question is whether we should really still use 
>code-signing keys which are not stored in (some kind of) HSM.
>

The backdoor was discovered by someone using the compromised xz-utils *in their 
own machines*. So we are lucky we have people eating our own sid stuff before 
it becomes part of a stable release.

Re: xz backdoor

[Marco d'Itri]

On Mar 30, Christian Kastner  wrote:

> >> Another big question for me is whether I should really still
> >> package/upload/etc from an unstable machine. It seems that it may be 
> >> prudent
> > If we do not use unstable for development then who is going to?
> Are you both talking about unstable hosts, or unstable chroots, or...?
I am talking about our own computers.

Obviously everything is built and rebuilt in unstable chroots.
But I think that we have to eat our dog food.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: xz backdoor

[Cyril Brulebois]

Sirius  (2024-03-30):
> I have seen discussion about shifting away from the whole auto(re)conf
> tooling to CMake or Meson with there being a reasonable drawback to
> CMake.  Is that something being discussed within Debian as well?

Talking about alternatives to autotools:
  
https://git.tukaani.org/?p=xz.git;a=commit;h=328c52da8a2bbb81307644efdb58db2c422d9ba7


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: xz backdoor

[Christian Kastner]

On 2024-03-30 20:20, Russ Allbery wrote:
> I personally specifically want my workstation to be running unstable, so
> I'm watching to see if that's considered unsafe (either, immediately,
> today, or in theory, in the future).
> 
> If I have to use a stable host, I admit I will be sad.  I've been using
> unstable for my personal client and development (not server, never
> exposing services to the Internet) systems for well over a decade (and,
> before, that, testing systems for as long as I've been working on Debian)
> and for me it's a much nicer experience than using stable.

I've heard many people say that, and I believe them. It might also work
for me if I had more time for Debian ($dayjob is unrelated to FOSS) and
thus more time to react to/report issues I find, but when I come home
from work, I really just want a system where I'm practically never
surprised by a significant change, other than security fixes.

Also, I don't possess the skill to quickly work around transient issues
as you mention below.

> It also lets
> me directly and practically dogfood Debian, which has resulted in a fair
> number of bug reports.

I wish I could do that in general. I try to do so during freezes, for
contributing polish to a release, but otherwise I (1) just place my
faith in upstream's own tests and hope autopkgtests have good coverage
so debci catches things, and (2) I rely on the fact that my work in
containers is also a form of dog-fooding.

For more elaborate projects, I do have complex systems set up based on
various environments. They are trivial to set up, tear down, snapshot,
"fork", etc. And they can be locked down pretty tightly.

And it's actually pretty convenient. Reproducing an issue for some other
release, even of a derivative, often only requires trivial changes to a
Dockerfile, like simply replacing the base image name.

(Actually, now that I think about it, apart from my browser, I probably
spend most of my time in terminals within unstable or experimental
containers, and/or connected to containers running services...)

> (This is an analysis specific to me, not general advice, and relies
> heavily on the fact that I'm very good at working around weird problems
> that transiently arise in unstable.)

Best,
Christian

Re: xz backdoor

[Colin Watson]

On Sat, Mar 30, 2024 at 05:12:17PM +0100, Sirius wrote:
> I have seen discussion about shifting away from the whole auto(re)conf
> tooling to CMake or Meson with there being a reasonable drawback to CMake.
> Is that something being discussed within Debian as well?

It's not in general something that Debian can unilaterally change.  And
in a number of cases switching build system would be pretty non-trivial.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: xz backdoor

[Andrey Rakhmatullin]

On Sat, Mar 30, 2024 at 08:52:29PM +0100, Ansgar  wrote:
> Hi,
> 
> On Sun, 2024-03-31 at 00:40 +0500, Andrey Rakhmatullin wrote:
> > On Sat, Mar 30, 2024 at 05:00:26PM +0100, Marco d'Itri wrote:
> > 
> > > I think that the real question is whether we should really still
> > > use 
> > > code-signing keys which are not stored in (some kind of) HSM.
> > What are the options for random DDs for that?
> 
> Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
> Possibly also TPM modules in computers.
> 
> These can usually be used for both OpenPGP and SSH keys.
Sure (though all the discourse around USB keys in the past 10 years or so
has suggested to me that all of them are bad according to one DD or
other).

> If someone cannot afford them, I think Debian paying for them is a good
> investment; Debian buying tokens for all project members would also be
> nice, 
This was even suggested at least once in the past.

> but logistics are probably annoying...
Exactly.


-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Ansgar ]

Hi,

On Sun, 2024-03-31 at 00:40 +0500, Andrey Rakhmatullin wrote:
> On Sat, Mar 30, 2024 at 05:00:26PM +0100, Marco d'Itri wrote:
> 
> > I think that the real question is whether we should really still
> > use 
> > code-signing keys which are not stored in (some kind of) HSM.
> What are the options for random DDs for that?

Yubikeys, Nitrokeys, GNUK, OpenPGP smartcards and similar devices.
Possibly also TPM modules in computers.

These can usually be used for both OpenPGP and SSH keys.

If someone cannot afford them, I think Debian paying for them is a good
investment; Debian buying tokens for all project members would also be
nice, but logistics are probably annoying...

A compromised computer alone is then not enough to get a copy of the
private key: one would also need an exploit for the hardware token.
(A compromised computer can still give temporary access to the key when
it is in use and unlocked; some devices can require pushing a button
for signing, but of course a compromised computer could claim to sign
something different than what gets signed and just show a "wrong PIN"
message to have the user try again.)

If you believe the hardware token to have a backdoor, exploiting it
might still require physical access to the token.

Ansgar

Re: xz backdoor

[Andrey Rakhmatullin]

On Sat, Mar 30, 2024 at 05:00:26PM +0100, Marco d'Itri wrote:
> On Mar 30, Jonathan Carter  wrote:
> 
> > Another big question for me is whether I should really still
> > package/upload/etc from an unstable machine. It seems that it may be prudent
> If we do not use unstable for development then who is going to?
Yup.

> I think that the real question is whether we should really still use 
> code-signing keys which are not stored in (some kind of) HSM.
What are the options for random DDs for that?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Andrey Rakhmatullin]

On Sat, Mar 30, 2024 at 10:49:33AM +0200, Jonathan Carter wrote:
> Another big question for me is whether I should really still
> package/upload/etc from an unstable machine. It seems that it may be prudent
> to consider it best practice to work from stable machines where any private
> keys are involved. For me it's just been so convenient to use unstable
> because it helps track changes that affect my users by the time it hits
> stable and also find bugs early that I care about, but perhaps I just need
> to make that adjustment and find more efficient ways to track unstable
> (perhaps on additional machines / VMs / etc). Not sure how other DDs think
> about this, but I'm also curious how they will deal with this, because
> there's near to no filter between unstable and the outside world, and this
> is probably not the last time someone will try something like this.
For me it's simple: if I'm forced to run my tools not on the host but in
some kind of inconvenient VM/chroot/whatever, I'll just stop contributing.
I'm not even discussing any of that proper Debian setups with keys on
separate airgapped machines, that's just not funny.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: xz backdoor

[Russ Allbery]

Christian Kastner  writes:

> This is both out of convenience (I want my workstation to be based on
> stable) and precisely because of the afforded isolation.

I personally specifically want my workstation to be running unstable, so
I'm watching to see if that's considered unsafe (either, immediately,
today, or in theory, in the future).

If I have to use a stable host, I admit I will be sad.  I've been using
unstable for my personal client and development (not server, never
exposing services to the Internet) systems for well over a decade (and,
before, that, testing systems for as long as I've been working on Debian)
and for me it's a much nicer experience than using stable.  It also lets
me directly and practically dogfood Debian, which has resulted in a fair
number of bug reports.

(This is an analysis specific to me, not general advice, and relies
heavily on the fact that I'm very good at working around weird problems
that transiently arise in unstable.)

But this does come with a security risk because it means a compromised
package could compromise my system much faster than if I were using
testing or, certainly, stable.  That's not a security trade-off that I can
responsibly make entirely for myself, since it affects people who are
using Debian as well.  So I don't get to have the final decision here.

-- 
Russ Allbery (r...@debian.org)  

Re: xz backdoor

[Christian Kastner]

On 2024-03-30 17:00, Marco d'Itri wrote:
> On Mar 30, Jonathan Carter  wrote:
> 
>> Another big question for me is whether I should really still
>> package/upload/etc from an unstable machine. It seems that it may be prudent
> If we do not use unstable for development then who is going to?

Are you both talking about unstable hosts, or unstable chroots, or...?

I do all my development on a stable host, within rootless podman
containers which are trivial to set up. I run final sbuilds and
autopkgtests in QEMU VMs, also trivial to set up.

This is both out of convenience (I want my workstation to be based on
stable) and precisely because of the afforded isolation.

Best,
Christian

Re: xz backdoor

[Sirius]

In days of yore (Fri, 29 Mar 2024), Russ Allbery thus quoth: 
> Russ Allbery  writes:
> > Sirius  writes:
> 
> >> This is quite actively discussed on Fedora lists.
> >> https://www.openwall.com/lists/oss-security/2024/
> >> https://www.openwall.com/lists/oss-security/2024/03/29/4
> 
> >> Worth taking a look if action need to be taken on Debian.
> 
> > The version of xz-utils was reverted to 5.4.5 in unstable yesterday by
> > the security team and migrated to testing today.  Anyone running an
> > unstable or testing system should urgently upgrade.
> 
> I think the big open question we need to ask now is what exactly the
> backdoor (or, rather, backdoors; we know there were at least two versions
> over time) did.  If they only target sshd, that's one thing, and we have a
> bound on systems possibly affected.  But liblzma is linked directly or
> indirectly into all sorts of things such as, to give an obvious example,
> apt-get.  A lot of Debian developers use unstable or testing systems.  If
> the exploit was also exfiltrating key material, backdooring systems that
> didn't use sshd, etc., we have a lot more cleanup to do.
> 
> I think this question can only be answered with reverse-engineering of the
> backdoors, and I personally don't have the skills to do that.

>From what I understand and have read, there is someone that will take a
look at reverse-engineering the .o and figure out what it did. The fact
the exploit looked for /usr/bin/sshd as argv[0] suggests it was targeted
at providing a backdoor into systems via just sshd. To what end, I guess
we will find out. Botnet would be the least worrisome IMHO.

A striking aspect is that this exploit was slow and methodical. This was
no ordinary script-kiddie doing things for giggles. I do wonder what will
shake out of this, but this level of patience and planning does raise
questions.

As you point out, the compression libraries are a weak link. I would think
the authentication and crypto libraries are as well. In this instance,
maybe both Debian and Fedora can breathe a sigh of relief that things got
caught when they did and that the remediation is not man-months or
man-years worth of effort. That said, someone plowing this amount of time
into planting just the one exploit may have other projects sized up where
they can try again.

I have seen discussion about shifting away from the whole auto(re)conf
tooling to CMake or Meson with there being a reasonable drawback to CMake.
Is that something being discussed within Debian as well?


-- 
Kind regards,

/S

Re: xz backdoor

[Marco d'Itri]

On Mar 30, Jonathan Carter  wrote:

> Another big question for me is whether I should really still
> package/upload/etc from an unstable machine. It seems that it may be prudent
If we do not use unstable for development then who is going to?
I think that the real question is whether we should really still use 
code-signing keys which are not stored in (some kind of) HSM.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: xz backdoor

[Pierre-Elliott Bécue]

Jonathan Carter  wrote on 30/03/2024 at 09:49:33+0100:

> Hi Russ
>
> On 2024/03/29 23:38, Russ Allbery wrote:
>> I think the big open question we need to ask now is what exactly the
>> backdoor (or, rather, backdoors; we know there were at least two versions
>> over time) did.
>
> Another big question for me is whether I should really still
> package/upload/etc from an unstable machine. It seems that it may be
> prudent to consider it best practice to work from stable machines
> where any private keys are involved. For me it's just been so
> convenient to use unstable because it helps track changes that affect
> my users by the time it hits stable and also find bugs early that I
> care about, but perhaps I just need to make that adjustment and find
> more efficient ways to track unstable (perhaps on additional machines
> / VMs / etc). Not sure how other DDs think about this, but I'm also
> curious how they will deal with this, because there's near to no
> filter between unstable and the outside world, and this is probably
> not the last time someone will try something like this.

Needing to be able to see how things I package go on when reaching
unstable, I tend to work on testing/unstable laptops.

I took some measures to reduce the risks of a permanent compromission:

 - My main GPG key is not on the machine (it's on a specific device I
   use only on my workstation);
 - My subkeys are rotated periodically (two years-ish I'd say);
 - They are on a YubiKey;
 - My laptop/workstations are hardened (firewall, usbguard,
   non-necessary services are removed, …).

This of course is not enough to mitigate a full-fledged compromission,
but I believe we need to live with some status quo. This time we found
out the compromission "fast". But it could also have reached stable-bpo
or, like other non-voluntary flaws, lived in software for multiple
years.

While I'm fine changing the way I do things, I am not sure that there is
any reasonable extent we could reach in order to prevent such
situations.

-- 
PEB


signature.asc
Description: PGP signature

Re: xz backdoor

[Henrique de Moraes Holschuh]

On Sat, Mar 30, 2024, at 05:49, Jonathan Carter wrote:

> Another big question for me is whether I should really still 
> package/upload/etc from an unstable machine. It seems that it may be 

I have been using stable or old stable + pbuilder for this.  Test runs of the 
results might need a VM though, when stable + container is not enough.

-- 
  Henrique de Moraes Holschuh 

Re: xz backdoor

[Jonathan Carter]

Hi Russ

On 2024/03/29 23:38, Russ Allbery wrote:

I think the big open question we need to ask now is what exactly the
backdoor (or, rather, backdoors; we know there were at least two versions
over time) did.


Another big question for me is whether I should really still 
package/upload/etc from an unstable machine. It seems that it may be 
prudent to consider it best practice to work from stable machines where 
any private keys are involved. For me it's just been so convenient to 
use unstable because it helps track changes that affect my users by the 
time it hits stable and also find bugs early that I care about, but 
perhaps I just need to make that adjustment and find more efficient ways 
to track unstable (perhaps on additional machines / VMs / etc). Not sure 
how other DDs think about this, but I'm also curious how they will deal 
with this, because there's near to no filter between unstable and the 
outside world, and this is probably not the last time someone will try 
something like this.


-Jonathan

Re: xz backdoor

[Russ Allbery]

Moritz Mühlenhoff  writes:
> Russ Allbery  wrote:

>> I think this question can only be answered with reverse-engineering of
>> the backdoors, and I personally don't have the skills to do that.

> In the pre-disclosure discussion permission was asked to share the
> payload with a company specialising in such reverse engineering. If that
> went through, I'd expect results to be publicly available in the next
> days.

Excellent, thank you.

For those who didn't read the analysis on oss-security yet, note that the
initial investigation of the injected exploit indicates that it
deactivates itself if argv[0] is not /usr/sbin/sshd, so there are good
reasons to believe that the problem is bounded to testing or unstable
systems running the OpenSSH server.  If true, this is a huge limiting
factor and in many ways quite relieving compared to what could have
happened.  But the stakes are high enough that hopefully we'll get
detailed confirmation from people with expertise in understanding this
sort of thing.

-- 
Russ Allbery (r...@debian.org)  

Re: xz backdoor

[Moritz Mühlenhoff]

Russ Allbery  wrote:
> I think this question can only be answered with reverse-engineering of the
> backdoors, and I personally don't have the skills to do that.

In the pre-disclosure discussion permission was asked to share the payload
with a company specialising in such reverse engineering. If that went
through, I'd expect results to be publicly available in the next days.

Cheers,
Moritz

Re: xz backdoor

[Russ Allbery]

Russ Allbery  writes:
> Sirius  writes:

>> This is quite actively discussed on Fedora lists.
>> https://www.openwall.com/lists/oss-security/2024/
>> https://www.openwall.com/lists/oss-security/2024/03/29/4

>> Worth taking a look if action need to be taken on Debian.

> The version of xz-utils was reverted to 5.4.5 in unstable yesterday by
> the security team and migrated to testing today.  Anyone running an
> unstable or testing system should urgently upgrade.

I think the big open question we need to ask now is what exactly the
backdoor (or, rather, backdoors; we know there were at least two versions
over time) did.  If they only target sshd, that's one thing, and we have a
bound on systems possibly affected.  But liblzma is linked directly or
indirectly into all sorts of things such as, to give an obvious example,
apt-get.  A lot of Debian developers use unstable or testing systems.  If
the exploit was also exfiltrating key material, backdooring systems that
didn't use sshd, etc., we have a lot more cleanup to do.

I think this question can only be answered with reverse-engineering of the
backdoors, and I personally don't have the skills to do that.

-- 
Russ Allbery (r...@debian.org)  

Re: xz backdoor

[Geert Stappers]

On Fri, Mar 29, 2024 at 09:09:45PM +0100, Sirius wrote:
> Hi there,
> 
> This is quite actively discussed on Fedora lists.
> https://www.openwall.com/lists/oss-security/2024/
> https://www.openwall.com/lists/oss-security/2024/03/29/4
> 
> Worth taking a look if action need to be taken on Debian.
> 

https://tracker.debian.org/news/1515519/accepted-xz-utils-561really545-1-source-into-unstable/

 
Groeten
Geert Stappers
-- 
Silence is hard to parse

Re: xz backdoor

[Russ Allbery]

Sirius  writes:

> This is quite actively discussed on Fedora lists.
> https://www.openwall.com/lists/oss-security/2024/
> https://www.openwall.com/lists/oss-security/2024/03/29/4

> Worth taking a look if action need to be taken on Debian.

The version of xz-utils was reverted to 5.4.5 in unstable yesterday by the
security team and migrated to testing today.  Anyone running an unstable
or testing system should urgently upgrade.

-- 
Russ Allbery (r...@debian.org)  

Re: xz backdoor

[Jérémy Lal]

xz-utils (5.6.1+really5.4.5-1) unstable; urgency=critical



  * Non-maintainer upload by the Security Team.

  * Revert back to the 5.4.5-0.2 version



 -- Salvatore Bonaccorso   Thu, 28 Mar 2024 15:59:38
+0100

Le ven. 29 mars 2024 à 21:17, Sirius  a écrit :

> Hi there,
>
> This is quite actively discussed on Fedora lists.
> https://www.openwall.com/lists/oss-security/2024/
> https://www.openwall.com/lists/oss-security/2024/03/29/4
>
> Worth taking a look if action need to be taken on Debian.
>
> --
> Kind regards,
>
> /S
>
>