Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Henrique de Moraes Holschuh]

On Fri, 11 Nov 2016, Christoph Biedl wrote:
> a proof of concept for all this (I can resist, though). The apt programs
> could obfuscate their request behaviour, the TLS layer could add random
> padding of data and time, but I doubt this would help much.

AFAIK, the TLS layer *does* bit-stuffing and random padding, but it
cannot do that to the point it would help the problem at hand, and still
be usable.

Bitstuffing TLS to the point it could (maybe) deal with the Debian
archive is the wrong solution for the problem anyway, so I won't expand
on this.

-- 
  Henrique Holschuh

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Christoph Biedl]

Henrique de Moraes Holschuh wrote...

> There are some relevant issues, here.
> 
> 1. It does protect against passive snooping *from non-skilled
> attackers*.

Well, yes, no. The tools become better so thinking a few years into
the future sophisticated programs for that purpose might be available to
everyone. Imagine there was a time before wireshark/ethereal, and how
much work pcap analysis was back then.

> 2. It is unknown how much it can protect against passive snooping from
> skilled attackers capable of passive TCP metadata slooping and basic
> traffic analysis *FOR* something like the Debian archive and APT doing
> an update run against the Debian archive

The logical answer is pretty obvious: Not at all. It's a question of
efforts required and my gut feelings tell me it's not very much.

> Do not dismiss (2). TLS is not really designed to be able to fully
> protect object retrieval from a *fully known* *static* object store
> against traffic metadata analysis.   And an apt update run would be even
> worse to protect, as the attacker can [after a small time window from
> the mirror pulse] fully profile the more probable object combinations
> that would be retrieved depending on what version of Debian the user
> has.

Things are worse: There's a small set of clients, and their request
behaviour is quite deterministic. Another snooping aid is usage of
pdiff.

In total, I was not surprised if just given the frame metadata
(direction, high-res timestamp, payload size) it was possible to restore
the actual data transmitted with high accurancy. Even a dget/apt-get
source should have a pretty unique pattern; and I feel tempted to create
a proof of concept for all this (I can resist, though). The apt programs
could obfuscate their request behaviour, the TLS layer could add random
padding of data and time, but I doubt this would help much.

Another "wasn't surprised", applicances might already have that. If not,
the vendors could implement this easily.

> Now, hopefully I got all of that wrong and someone will set me straight.
>  It would make me sleep better at night...

Sorry Dorothy.

Christoph


signature.asc
Description: Digital signature

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[David Kalnischkies]

On Thu, Nov 10, 2016 at 12:39:40PM -0200, Henrique de Moraes Holschuh wrote:
> I'd prefer if we enhanced apt transports to run a lot more protected
> (preferably under seccomp strict) before any such push for enabling
> https transports in apt.  It would reduce the security impact a great
> deal.

I am helplessly optimistic, so I will say it again even if the past
tells me it pointless: Anyone in any way interest in improvements is
more than welcome to join deity@l.d.o / #debian-apt.

Very few things get done by just talking on d-d@ about nice to haves.


> Mind you, at fist look it seems like apt transports will *run as root*
> in Debian jessie.  HOWEVER I didn't do a deep check, just "ps aux" while
> apt was running.  And I didn't check in unstable.  So, I (hopefully)
> could be wrong about this.

For jessie you are right. The few of us took an awful lot of time to
basically reimplement many parts of the acquire subsystem in the last
few years. You can watch Michael talk about it at DC14, me at DC15 and
Julian at DC16 if you like, but the very basic summary is that in
stretch onwards all apt methods run effectively as _apt:nogroups (and
with no-new-privs) & apt itself requires repositories to be signed and
expects more than just SHA1 or MD5 (as usual, that applies to everything
related to apt like aptitude, synaptics, packagekit, apt-file, …).

There is still much we wanna do, but for now we are actually happy that
we seem to have managed to satisfy all the people who responded to those
changes: The army of complainers that it breaks their firewalls, strange
so called sneaker net abominations or other interesting workflows[0] …


> Can you imagine trying to contain an exploit in the wild that will take
> advantage of people trying to update their systems against said exploit
> to spread even further?  Well, this is exactly what would happen.  We

Let the code with no bugs cast the first stone – you could just as well
say that any http bug is if wrapped in https less critical. libcurl
depends on a crapload of stuff we don't actually need because we use it
just for https and not for ftp, samba, …. And then most TLS exploits
tend to be in tricking it to consider a cert valid while it isn't, which
is a big problem for most things, but for apt those kind of bugs are
a lot less critical as we don't trust the transport layer anyhow (if we
treat it more as a MITM annoyance instead of one-and-only security).
As such, completely non-empirical of course, but I think it would be
a net-benefit to have https sources available for use by default even if
its overrated in this context – but you will only die very tired if you
try to explain why https-everywhere is a requirement for your browser
and even most (language specific) package managers to add a tiny layer
of security to them, but our beloved apt doesn't strictly need it for
security (but happily accepts the tiny layer as addition).


As already said, we are open to consider replacing libcurl with
a suitable alternative like e.g. using libgnutls directly – but see
optimistic paragraph above, I still hope that a volunteer will show up…
(as the biggest TLS exploit is usually the implementor who hasn't worked
with the API before and I haven't).

And I would still like to have some for a-t-tor, too. The package is
even way smaller than even the smallest node packages [SCNR] nowadays
and someone with an eye for detail, integration and documentation could
do wonders… but I start to digress.


Best regards

David Kalnischkies

[0] https://xkcd.com/1172/


signature.asc
Description: PGP signature

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Henrique de Moraes Holschuh]

On Mon, Oct 24, 2016, at 00:28, Russ Allbery wrote:
> The value of HTTPS lies in its protection against passive snooping. 

There are some relevant issues, here.

1. It does protect against passive snooping *from non-skilled
attackers*.  And this is not being made anywhere clear enough.

2. It is unknown how much it can protect against passive snooping from
skilled attackers capable of passive TCP metadata slooping and basic
traffic analysis *FOR* something like the Debian archive and APT doing
an update run against the Debian archive (i.e. this comment is NOT valid
for ANY OTHER USE of https).

Do not dismiss (2). TLS is not really designed to be able to fully
protect object retrieval from a *fully known* *static* object store
against traffic metadata analysis.   And an apt update run would be even
worse to protect, as the attacker can [after a small time window from
the mirror pulse] fully profile the more probable object combinations
that would be retrieved depending on what version of Debian the user
has.

apt-transport-https really exists to help people bypass half-hearted
filtering and logging by corporate or ISP web proxies, and it is very
good for that. It is a valid user case, and one we do want to support. 
But it shouldn't be sold as a strong privacy defense, *ever*.

There wouldn't be a reason to not enable apt-transport-https: at this
point of the analysis: it is still not making things really any worse,
so you'd likely have a net gain since it does raise the bar for the
attackers.

However, there are these little real world details called "attack
surface" and "implementation complexity" as sources of exploitable
security vulnerabilities, and the picture changes a lot when you start
looking at that angle:

(up-to-date Debian stable/jessie amd64 system):

ldd /usr/lib/apt/methods/http | cut -d '(' -f 1
linux-vdso.so.1
libapt-pkg.so.4.12 =>
/usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5
/lib64/ld-linux-x86-64.so.2

compared with:

ldd /usr/lib/apt/methods/https | cut -d '(' -f 1
linux-vdso.so.1
libapt-pkg.so.4.12 =>
/usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
libcurl-gnutls.so.4 =>
/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5
libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11
librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1
libssh2.so.1 => /usr/lib/x86_64-linux-gnu/libssh2.so.1
libnettle.so.4 => /usr/lib/x86_64-linux-gnu/libnettle.so.4
libgnutls-deb0.so.28 =>
/usr/lib/x86_64-linux-gnu/libgnutls-deb0.so.28
libgssapi_krb5.so.2 =>
/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2
liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2
libldap_r-2.4.so.2 =>
/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
/lib64/ld-linux-x86-64.so.2
libhogweed.so.2 => /usr/lib/x86_64-linux-gnu/libhogweed.so.2
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10
libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0
libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6
libkrb5support.so.0 =>
/usr/lib/x86_64-linux-gnu/libkrb5support.so.0
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6


I assume I don't have to remind anyone of the security history of the
above library 

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Adrian Bunk]

On Sun, Nov 06, 2016 at 12:03:03AM +0100, Philipp Kern wrote:
> On 2016-11-05 22:23, Adrian Bunk wrote:
> > The solution you are trying to sell is apt-transport-https as default.
> [...]
> > Your solution would be a lot of work with relatively little improvement.
> 
> Well, the client-side exists and works.
>...

Yes and no.

It works, but there is much work left if you want to make that the 
default.

David already mentioned in this discussion where apt-transport-https 
needs improvements.

I did already mention that the current footprint of adding 
apt-transport-https to the installer and small base filesystems
is currently pretty large.
As an example, the installer would require two different TLS libraries
if you just add apt-transport-https.

I would guess there are also other areas that have to be looked at
if that should become the default, like how certificate errors will
be handled in the installer.

> > BTW: The "possible low-effort improvement without tradeoff" is:
> > 
> > Is apt-transport-tor working reliably enough for general usage?
> > Are security updates available immediately through apt-transport-tor?
> > Is there a good reason why apt-transport-tor is not mentioned
> > at the frontpage of http://www.debian.org/security/ ?
> > 
> > My current impression (that might be wrong) is that the technical side
> > would be available, only documentation and perhaps PR (e.g. email to
> > debian-security-announce) are missing.
> 
> If we are limiting ourselves to mirrors run by DSA (which is what happens
> for the backends of the onion balancer), we could have the same with an
> HTTPS-based solution just fine. It'd likely raise the same scalability and
> operational questions as HTTPS. Your proposal here simply has different
> tradeoffs, not none as you claim.

Russ and me were discussing one specific tradeoff.

Let me repeat the relevant problem:
  By discouraging users from using mirrors for security.debian.org,
  Debian is presenting a nearly complete list of all computers in
  the world running Debian stable and their security update status
  and policies on a silver plate to the NSA.

Russ answered:
  It's a tradeoff with freshness of security updates.

With HTTP this tradeoff between "not giving information about Debian 
users on a silver plate to the NSA" and "providing security updates
as soon as possible" exists.

This tradeoff still exists with HTTPS.

Tor offers a solution for this specific problem that does not have
this specific tradeoff.

> Kind regards
> Philipp Kern

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Philipp Kern]

On 2016-11-05 22:23, Adrian Bunk wrote:

The solution you are trying to sell is apt-transport-https as default.

[...]
Your solution would be a lot of work with relatively little 
improvement.


Well, the client-side exists and works. Then it boils down if the mirror 
sponsors would be willing to offer HTTPS in general and from there the 
operational challenges of having the right certs available. Something 
like httpredir would have had it easier because it redirected to 
canonical names owned by the sponsors for which they could offer HTTPS.



(Not limited to security) it is usually worth the effort to start by
properly formulating the problem(s) that should be solved, instead of
limiting yourself to some solutions.


While I generally agree with that statement and see it violated quite 
often myself, you should also give others in the discussion the benefit 
of the doubt here.



BTW: The "possible low-effort improvement without tradeoff" is:

Is apt-transport-tor working reliably enough for general usage?
Are security updates available immediately through apt-transport-tor?
Is there a good reason why apt-transport-tor is not mentioned
at the frontpage of http://www.debian.org/security/ ?

My current impression (that might be wrong) is that the technical side
would be available, only documentation and perhaps PR (e.g. email to
debian-security-announce) are missing.


If we are limiting ourselves to mirrors run by DSA (which is what 
happens for the backends of the onion balancer), we could have the same 
with an HTTPS-based solution just fine. It'd likely raise the same 
scalability and operational questions as HTTPS. Your proposal here 
simply has different tradeoffs, not none as you claim.


Kind regards
Philipp Kern

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Adrian Bunk]

On Tue, Oct 25, 2016 at 11:06:23AM -0700, Russ Allbery wrote:
> Adrian Bunk  writes:
>...
> So, I'm not quite sure how to put this, since I don't know how much work
> you've done professionally in computer security, and I don't want to
> belittle that.  It's entirely possible that we have equivalent levels of
> experience and you just disagree with me, and I want to acknowledge that.

I have some knowledge here and there, but I am definitely not
a Security Expert.

When something is so well-known or obvious that even someone like me 
knows it or is able to figure it out, it has to be pretty well-known
or obvious.

>...
> In the specific case of retrieval of apt packages, use of HTTPS raises the
> bar for the attacker who wants to know what packages you probably have on
> your system from simply parsing the GET commands (information that would
> be captured in any dragnet surveillance database as a matter of course)
> for package names and versions to writing Debian-specific analysis code to
> make (fallible) guesses from traffic analysis.  This is a fairly
> substantial change in the required resources, and makes various things
> (such as retroactive data mining when this particular use case wasn't
> anticipated in advance) considerably harder.

I hope I am not too rude when I state the general problem I have with 
the way you are arguing:
You are not trying to solve a problem,
you are trying to sell a solution.

The solution you are trying to sell is apt-transport-https as default.

The problem would be something like "more security/privacy for users"
or "make it harder for the NSA".

Your solution would be a lot of work with relatively little improvement.

>...
> Yes, you're not going to get absolute security against the NSA with cheap
> wire encryption, but you *do* change the resource battle.

If something makes it more costly for the NSA, the US taxpayer will 
just give them another Billion every year - no party in US congress
would oppose that.

Debian resources are far more limited.

Noone is stopping you from doing the work on the client-side for making 
apt-transport-https the default (that noone seems to be working on) if 
this is important for you - it is your time.

But for any properly formulated problem and a fixed amount of resources,
I doubt the best solution available would include apt-transport-https.

>...
> > By discouraging users from using mirrors for security.debian.org, Debian
> > is presenting a nearly complete list of all computers in the world
> > running Debian stable and their security update status and policies on a
> > silver plate to the NSA.
> 
> It's a tradeoff with freshness of security updates.  Personally, I usually
> use an in-house mirror of security.debian.org for various reasons, and
> it's worth noting that our "discouraging" isn't particularly aggressive.

You are drawing your conclusions from just looking at some solutions.

When I go back one step and think of ways to improve the situation for 
users, I do also see a possible low-effort improvement that does not 
have the tradeoff you are talking about and that does not require a 
local mirror. And when even I am able to come up with something, there 
is likely more.

(Not limited to security) it is usually worth the effort to start by 
properly formulating the problem(s) that should be solved, instead of 
limiting yourself to some solutions.

cu
Adrian


BTW: The "possible low-effort improvement without tradeoff" is:

Is apt-transport-tor working reliably enough for general usage?
Are security updates available immediately through apt-transport-tor?
Is there a good reason why apt-transport-tor is not mentioned
at the frontpage of http://www.debian.org/security/ ?

My current impression (that might be wrong) is that the technical side
would be available, only documentation and perhaps PR (e.g. email to
debian-security-announce) are missing.

apt-transport-tor could also be taken into consideration in the 
unattended-upgrades discussion as a (perhaps non-default) option 
offered for receiving the security updates.

Oh, and when I look at http://www.debian.org/security/ I see
a link "For more information about security issues in Debian, please 
... and a manual called Securing Debian."

The manual says "Version: 3.17, built on Sun, 08 Apr 2012".

I know that writing documentation is less sexy than implementing
technical solutions, but for the problem "more security for users"
proper user documentation is actually more important than many of
the technical solutions.


-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Russ Allbery]

Adrian Bunk  writes:

> If I were looking at the apt traffic, the most interesting for me would
> be the traffic to security.debian.org that a computer running Debian
> stable usually produces.

> Just collecting the data when and how much HTTPS traffic is happening 
> should be sufficient to determine information like the following:
>   What Debian release is running on that computer?
>   Which security relevant packages are installed in that computer?
>   Are security updates downloaded automatically or manually?
>   In the latter case, are they installed in a timely manner?

Again, this requires targeting.  It requires someone go to the effort of
building the software that does this sort of analysis, and then keeping it
up-to-date with changes in the Debian archive structure, transport layer,
package sizes, etc.  (And knowing whether the packages are installed is
quite a bit harder to figure out without doing more active things like
fingerprinting, and even then it's not always possible.)

> When your adversary is powerful enough that he is capable of monitoring
> your traffic with security.debian.org, then apt-transport-https is just 
> snake oil.

So, I'm not quite sure how to put this, since I don't know how much work
you've done professionally in computer security, and I don't want to
belittle that.  It's entirely possible that we have equivalent levels of
experience and you just disagree with me, and I want to acknowledge that.

But, that said, this feeling, which comes across roughly as "this doesn't
completely fix the problem, so why bother with half measures?", is a trap
I see people fall into when they don't have a good feel for the dynamics
of practical security measures.  It's *so* tempting to let the best be the
enemy of the good because you, as an experienced engineer with a complete
mental model of the system being protected, can see flaws in the
protective mitigations and feel like it would be easy to counter them.

This is why, in computer security, it's usually a bad idea to talk about
"fixes" except for specific software-bug vulnerabilities.  Security
measures are often described as "mitigations," and rather than determining
whether or not something is secure, it's usually better to talk about
making things easier or harder for an attacker.

You should generally assume that if a sufficiently funded and motivated
adversary wants to break into your Internet-connected system in
particular, they're going to succeed.  But that doesn't mean that computer
security is useless.  There are multiple other factors in play: a lot of
adversaries care a great deal about not being *detected* (which is a much
easier problem), most attacks are not targeted at all (they're either
spray-and-pray automated attacks or they're dragnet data gathering with no
predetermined goal in mind), and time and resources matter.  To give a
specific example, there are many situations where you don't have to keep a
government out of your stuff permanently.  You just have to keep them out
for long enough that you can get your lawyer involved.

In the specific case of retrieval of apt packages, use of HTTPS raises the
bar for the attacker who wants to know what packages you probably have on
your system from simply parsing the GET commands (information that would
be captured in any dragnet surveillance database as a matter of course)
for package names and versions to writing Debian-specific analysis code to
make (fallible) guesses from traffic analysis.  This is a fairly
substantial change in the required resources, and makes various things
(such as retroactive data mining when this particular use case wasn't
anticipated in advance) considerably harder.

> The NSA might actually be very grateful that there are people who are
> promoting such snake oil as solution, since this lowers the probability
> of people looking for solutions that could make it harder for the NSA.

I truly don't believe this worry makes any sense.

Ubiquitous encryption makes things considerably harder for the NSA because
they have to expend more resources.  There is substantial confirmation of
this from the hissy fits that the NSA has thrown in the past about public
use of encryption, and their repeated attempts to get people to adopt
crypto with back doors.  I know some people think this is all an elaborate
smoke screen, but I think that level of paranoia is unjustified.  The
people working for the NSA aren't superhuman.  Encryption matters, even if
you can still recover metadata with traffic analysis.

Yes, you're not going to get absolute security against the NSA with cheap
wire encryption, but you *do* change the resource battle.  The NSA can
bring almost unlimited resources to bear *on single, high-value targets*,
and those require substantially different precautions.  But I'm just as
worried about the implications of huge databases of information about
people's on-line activities sitting in government databases.  (I'm
actually 

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Alexander Wirt]

On Tue, 25 Oct 2016, Ian Jackson wrote:

> Adrian Bunk writes ("Re: client-side signature checking of Debian archives 
> (Re: When should we https our mirrors?)"):
> > snake oil
> > snake oil
> 
> The phrase "snake oil" is very insulting.  I have asked you several
> times to to stop.
> 
> Publicly CCing listmaster this time.
you are free to do that. But we are no babysitters. Unless things escalate we
won't step in. And sorry, I fail to see where a phrase like "snake oil" would
justify an intervention from listmaster side. 

Alex - Debian Listmaster

P.S. this is my personal oppinion as listmaster and not a coordinated
statement
 

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Ian Jackson]

Adrian Bunk writes ("Re: client-side signature checking of Debian archives (Re: 
When should we https our mirrors?)"):
> snake oil
> snake oil

The phrase "snake oil" is very insulting.  I have asked you several
times to to stop.

Publicly CCing listmaster this time.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Adrian Bunk]

On Mon, Oct 24, 2016 at 04:33:57PM -0700, Russ Allbery wrote:
> Adrian Bunk  writes:
>...
> > I would assume this can be pretty automated, and that by NSA standards
> > this is not a hard problem.
> 
> Since the entire exchange is encrypted, it's not completely trivial to map
> size of data transferred to a specific package (of course, it's even
> harder if we reuse connections).  But the point I'm making is more that
> it's not something that just falls out of an obvious surveillance
> technique that has wide-ranging uses.  It requires someone to write code
> to *specifically* target Debian mirrors, which I think is much less likely
> than just collecting all the data and deciding to analyze it afterwards.
>...

If I were looking at the apt traffic, the most interesting for me would 
be the traffic to security.debian.org that a computer running Debian 
stable usually produces.

Just collecting the data when and how much HTTPS traffic is happening 
should be sufficient to determine information like the following:
  What Debian release is running on that computer?
  Which security relevant packages are installed in that computer?
  Are security updates downloaded automatically or manually?
  In the latter case, are they installed in a timely manner?

When your adversary is powerful enough that he is capable of monitoring
your traffic with security.debian.org, then apt-transport-https is just 
snake oil.

The NSA might actually be very grateful that there are people who are 
promoting such snake oil as solution, since this lowers the probability 
of people looking for solutions that could make it harder for the NSA.

I would assume it is unlikely that the NSA is monitoring the connection 
between me and my nearest Debian mirror. This does of course depend on 
your geographical location.

I would assume it is likely that the NSA is monitoring the connection 
between me and security.debian.org.

By discouraging users from using mirrors for security.debian.org,
Debian is presenting a nearly complete list of all computers in
the world running Debian stable and their security update status
and policies on a silver plate to the NSA.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Russ Allbery]

Paul Wise  writes:

> Debian has Tor onion service frontends to various Debian services,
> including several Debian machines with archive mirrors, this is
> implemented in an automated way using Puppet and onionbalance. So we do
> not rely on Tor exit nodes, just relays and the onion service system.

> https://onion.debian.org/
> https://anonscm.debian.org/cgit/mirror/dsa-puppet.git/tree/modules/onion

Oh, interesting, thank you.  I hadn't realized that.  That definitely
makes Tor more attractive.

-- 
Russ Allbery (r...@debian.org)   

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Paul Wise]

On Tue, Oct 25, 2016 at 7:33 AM, Russ Allbery wrote:

> Tor is easier for us as a project, since we don't really have to do
> anything (assuming we just rely on existing exit nodes).

Debian has Tor onion service frontends to various Debian services,
including several Debian machines with archive mirrors, this is
implemented in an automated way using Puppet and onionbalance. So we
do not rely on Tor exit nodes, just relays and the onion service
system.

https://onion.debian.org/
https://anonscm.debian.org/cgit/mirror/dsa-puppet.git/tree/modules/onion

> SSL is much harder for us as a project

For most debian.org services run by DSA, enabling SSL on a service is
one git commit away, thanks to Lets Encrypt.

Some things like snapshot are harder due to software or other issues.
For snapshot, varnish is the frontend and it doesn't support SSL.

All the debian.org mirrors except ftp.d.o are not actually run by DSA
and DSA occasionally need to change which domain points to which
mirror, so SSL for them is much more complicated.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Russ Allbery]

Adrian Bunk  writes:

> The government operating or having access to the mirror you are using is
> a lot more realistic and easier than the MITM with a fake certificate
> you were talking about.

Both of those were also in the category of things that I think are
unlikely attacks unless the government is specifically targeting you, so
that leaves me confused by what point you're trying to make here.

For the record, MITM with a fake certificate is certainly realistic for a
targeted attack, and is something that one has to think about.  There's a
reason why certificate pinning is a standard part of the defenses in a
higher-security SSL configuration.  I do agree (and in fact this was my
whole point) that it's unlikely a government would burn their certificate
capability this way just to figure out what Debian packages you're
downloading, though.

> I would assume this can be pretty automated, and that by NSA standards
> this is not a hard problem.

Since the entire exchange is encrypted, it's not completely trivial to map
size of data transferred to a specific package (of course, it's even
harder if we reuse connections).  But the point I'm making is more that
it's not something that just falls out of an obvious surveillance
technique that has wide-ranging uses.  It requires someone to write code
to *specifically* target Debian mirrors, which I think is much less likely
than just collecting all the data and deciding to analyze it afterwards.

(That said, it would be possible to reconstruct the necessary data to do
the analysis later, I suppose.  But here too someone has to care at a
level deep enough to go write code to do it, as opposed to just doing ad
hoc queries.)

> Let me try to summarize my point:

> apt-transport-https makes it slightly harder to determine what packages 
> are being transferred, and this is good.

> When someone is seriously worried about a nation-state actor determining 
> what packages he downloads then apt-transport-https is not a solution, 
> and the proper solution is apt-transport-tor.

> I assume you will disagree regarding the word "slightly".
> Are we in agreement regarding the rest of my summary?

I disagree that either of them is a solution.  They're both mitigations,
and if a nation-state attacker is targeting you specifically, I wouldn't
want to rely entirely on either of them.

I agree with you that Tor is a stronger mitigation.

Tor is easier for us as a project, since we don't really have to do
anything (assuming we just rely on existing exit nodes).  SSL is much
harder for us as a project, but is simpler for the end user (and doesn't
require them to take a stance on the merits of Tor as a project; I don't
really want to get into a philosophical argument about the merits of
Internet anonymity, but this is not something everyone thinks is an ideal
to strive for as opposed to a situational fix for very specific risks).
In an ideal world we'd offer both.

> When someone is worried about the confidentiality of the information 
> what packages are installed on a system, only looking at the download 
> side is also missing other areas that might be even more problematic.

This is true.  But the set of problems are largely independent.  While
solving the download side doesn't fix all the other problems, it also
doesn't interfere with fixing all the other problems, and is still needed
to make the overall system more secure.

The reason why addressing the download side is appealing to me is that we
know dragnet surveillance is common and ongoing, so the risk it's
addressing is tangible and we have some reasonably solid information about
how it works.  So while it may not be the most serious problem, it's one
of the problems we understand the best, and therefore are in a good
position to do something about.

> I would be a lot more worried about what reportbug does when a package
> suggests libdvdcss2 - in some jurisdictions this might just be enough
> when the government is looking for a reason to raid your home.

I'm a great believer in ubiquitous encryption, even if it seems silly,
just on the grounds of "why not?".  We should encrypt reportbug traffic
too, if we can.  Yes, a lot of the details get exposed at the other end
anyway (although not necessarily), but it's usually fairly trivial to
encrypt links, and if it is, there's basically no reason not to.

-- 
Russ Allbery (r...@debian.org)   

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Adrian Bunk]

On Mon, Oct 24, 2016 at 09:22:39AM -0700, Russ Allbery wrote:
> Adrian Bunk  writes:
> > On Sun, Oct 23, 2016 at 07:28:23PM -0700, Russ Allbery wrote:
> 
> >>...
> >> The value of HTTPS lies in its protection against passive snooping.  Given
> >> the sad state of the public CA infrastructure, you cannot really protect
> >> against active MITM with HTTPS without certificate pinning.
> 
> > You are implicitely assuming that mirrors can be trusted, 
> > and even that is not true.
> 
> > Who is operating ftp.cn.debian.org, and who has access to the logfiles 
> > on that server?
> 
> So, here, I think you're making the best the enemy of the good, which is
> always tempting in security.
> 
> Everyone (myself included) wants to create a system that's actually
> secure, the same way that we want to create software that works properly.
> But in security, even more so than most software, that's rarely possible.
> There's always another attack.  It's a sliding scale of tradeoffs, and
> just because you're not preventing all attacks doesn't mean that making
> things harder for the attacker isn't useful.
> 
> Looking at this worry in particular, note that you're now assuming that a
> nation-state has compromised a specific mirror (in some way, maybe just by
> running it without regard to the privacy of its users).  As pointed out,
> this only gets partial information, but more importantly, this requires
> the nation-state to actively do something.  They have to now either
> compromise the mirror or be running it in a way that may be discoverable.
>...

The government operating or having access to the mirror you are using is 
a lot more realistic and easier than the MITM with a fake certificate 
you were talking about.

> > When a nation-state actor analyzes all the traffic on a network
> > connection that also happens to carry the traffic between you and the
> > Debian mirror you are using, HTTPS won't make a difference.
> 
> Well, I disagree, largely because I think you're overestimating the
> willingness to do deep data analysis on something that's only a tertiary
> (at best) security objective.
> 
> Dragnet surveillance is the most common nation-state approach at the
> moment primarily because it's *really easy*.  Get a backbone tap
> somewhere, dredge all the unencrypted data, run simple analysis across it
> (pulling out URLs accessed is obviously trivial for unencrypted HTTP),
> throw it into some sort of huge database, and decide what you want to
> query for later.

Pulling out the dates and amounts transferred fot HTTP traffic is
also trivial.

> Doing traffic analysis that requires analyzing encrypted data by object
> size or the like is still certainly *possible* (that was the point that I
> made in my first message to this thread), but it's not *trivial*.  It
> requires someone go write code, go think about the problem, and regularly
> gather information from a mirror to correlate encrypted object sizes with
> packages.  It requires a *human* actually *think* about the problem, and
> then set up and maintain an automated system that requires some level of
> ongoing babysitting.

I would assume this can be pretty automated, and that by NSA standards 
this is not a hard problem.

> Can nation-states do this?  Obviously, yes.  But the goal here isn't to
> prevent them from launching these sorts of attacks.  The goal is to make
> it expensive, to require that they justify a budget to hire people to do
> all this additional work that requires custom automation, and to raise the
> bar across the board to make simplistic dragnet surveillance unrewarding.
> This forces them into making some hard decisions about resource allocation
> and attempting more visible intrusions.  Hard decisions are expensive, not
> just in resources but also in politics.
> 
> It *is* a tradeoff, not perfect security, so we should obviously also take
> into account how much effort it takes on our side and whether we're
> expending less effort than we're forcing from our adversary.  But, in
> general, ubiquitous encryption is totally worth it from that sort of
> effort analysis.  Once you get good systems in place (Let's Encrypt
> seriously changed the calculation here), adding encryption is usually far
> easier than the work required on the snooping side to recover the sort of
> metadata they had access to unencrypted.  I think that's the case here.

Let me try to summarize my point:

apt-transport-https makes it slightly harder to determine what packages 
are being transferred, and this is good.

When someone is seriously worried about a nation-state actor determining 
what packages he downloads then apt-transport-https is not a solution, 
and the proper solution is apt-transport-tor.

I assume you will disagree regarding the word "slightly".
Are we in agreement regarding the rest of my summary?


When someone is worried about the confidentiality of the information 
what packages are installed on a system, only looking at 

signature checking in libcupt (Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?))

[Eugene V. Lyubimkin]

Hi Kristian,

To one of your side questions,

On 24.10.2016 02:33, Kristian Erik Hermansen wrote:
>> 1) Checking chain (e.g. gpgv and its callers) have bugs. True, same as 
>> checking layer for secure transports also have bugs.
> 
> Agreed. Please let me know of a good test case to validate that your
> tools, which are not APT (?), are doing the right things. You said you
> maintained a tool which "downloads and validates Debian archives in a
> similar way APT does", which means not exactly the way APT does. Let
> me know the name of your tool and how to setup some test cases to
> validate your tool is doing things properly. Glad to spend some time
> on it and contribute any potential findings for the community benefit.

The tool I maintain is minor, not widely used package manager, which may or may 
not be worth your time. It's called
Cupt, the sources are at [1a] or [1b]; namely, the checking code at [2] and 
tests for common situations at [3]. One can
play with those test cases, or install the tool and point it [4] to malicious 
servers.

There might be other packages in Debian which access repos not through libapt.


[1a] 
https://alioth.debian.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=cupt/cupt.git;a=tree
[1b] https://github.com/jackyf/cupt
[2] cpp/lib/src/internal/cachefiles.cpp:verifySignature()
[3] test/t/query/repo-signatures/*
[4] same as APT, via /etc/apt/sources.list

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Russ Allbery]

Adrian Bunk  writes:
> On Sun, Oct 23, 2016 at 07:28:23PM -0700, Russ Allbery wrote:

>>...
>> The value of HTTPS lies in its protection against passive snooping.  Given
>> the sad state of the public CA infrastructure, you cannot really protect
>> against active MITM with HTTPS without certificate pinning.

> You are implicitely assuming that mirrors can be trusted, 
> and even that is not true.

> Who is operating ftp.cn.debian.org, and who has access to the logfiles 
> on that server?

So, here, I think you're making the best the enemy of the good, which is
always tempting in security.

Everyone (myself included) wants to create a system that's actually
secure, the same way that we want to create software that works properly.
But in security, even more so than most software, that's rarely possible.
There's always another attack.  It's a sliding scale of tradeoffs, and
just because you're not preventing all attacks doesn't mean that making
things harder for the attacker isn't useful.

Looking at this worry in particular, note that you're now assuming that a
nation-state has compromised a specific mirror (in some way, maybe just by
running it without regard to the privacy of its users).  As pointed out,
this only gets partial information, but more importantly, this requires
the nation-state to actively do something.  They have to now either
compromise the mirror or be running it in a way that may be discoverable.

Security is both technical and politics, and on the political side,
actively doing something is much riskier.  Nation-state surveillance teams
are generally not fond of headlines, and there are political consequences
to being caught with your hand in the cookie jar.  Therefore, technical
measures that force them to take an *active* measure to get data, as
opposed to just passively tapping and siphoning data, are, in practice,
quite effective.  Not because they don't have the technical capabilities
to do that, but because it's much riskier politically to do so, and often
that risk isn't worth the reward from their perspective.

It's true that tapping a single mirror could still be passive, but it's a
much trickier kind of passive than just snooping on a cable.  They have to
be passively collecting the actual server logs in some way, which means
presence on the box or access to some network where those logs are being
sent in the clear.  That does raise the bar substantially.

> Debian would accept debian.nsa.gov as mirror, and the NSA might already 
> operate or have access at some current mirrors.

I'm a little dubious that this is true (I would recommend against it), but
even if so, Debian could also easily revoke that acceptance at any point
if we decided we were uncomfortable with it, which makes it an unreliable
way to gather this sort of data.

> When a nation-state actor analyzes all the traffic on a network
> connection that also happens to carry the traffic between you and the
> Debian mirror you are using, HTTPS won't make a difference.

Well, I disagree, largely because I think you're overestimating the
willingness to do deep data analysis on something that's only a tertiary
(at best) security objective.

Dragnet surveillance is the most common nation-state approach at the
moment primarily because it's *really easy*.  Get a backbone tap
somewhere, dredge all the unencrypted data, run simple analysis across it
(pulling out URLs accessed is obviously trivial for unencrypted HTTP),
throw it into some sort of huge database, and decide what you want to
query for later.

Doing traffic analysis that requires analyzing encrypted data by object
size or the like is still certainly *possible* (that was the point that I
made in my first message to this thread), but it's not *trivial*.  It
requires someone go write code, go think about the problem, and regularly
gather information from a mirror to correlate encrypted object sizes with
packages.  It requires a *human* actually *think* about the problem, and
then set up and maintain an automated system that requires some level of
ongoing babysitting.

Can nation-states do this?  Obviously, yes.  But the goal here isn't to
prevent them from launching these sorts of attacks.  The goal is to make
it expensive, to require that they justify a budget to hire people to do
all this additional work that requires custom automation, and to raise the
bar across the board to make simplistic dragnet surveillance unrewarding.
This forces them into making some hard decisions about resource allocation
and attempting more visible intrusions.  Hard decisions are expensive, not
just in resources but also in politics.

It *is* a tradeoff, not perfect security, so we should obviously also take
into account how much effort it takes on our side and whether we're
expending less effort than we're forcing from our adversary.  But, in
general, ubiquitous encryption is totally worth it from that sort of
effort analysis.  Once you get good systems in place (Let's 

Re: client-side signature checking of Debian archives

[David Kalnischkies]

(Disclaimer: I am a maintainer of apt-transport-tor… but also of
-https and apt itself, so I am biased beyond hope in this matter)

On Sun, Oct 23, 2016 at 07:20:35PM -0700, Russ Allbery wrote:
> Paul Wise  writes:
> > On Mon, Oct 24, 2016 at 7:21 AM, Kristian Erik Hermansen wrote:
> >> The point is to improve privacy.
> 
> > Better privacy than https can be had using Tor:
> 
> > https://onion.debian.org/
> 
> Yeah, but this is *way* harder than just using TLS.  You get much of the
> benefit by using TLS, and Tor comes with a variety of mildly problematic

TLS doesn't give you a lot of privacy in the context of Debian mirrors.
The traffic analyse Russ has hinted at is one thing, but the biggest
privacy issue is actually that you are a Debian user – and that is
communicated in the clear regardless of using HTTPS or not e.g. if you
connect to security.debian.org. Keeping track of then you connect to
figure out how long it takes you to react to DSAs isn't exactly hard
either. Would it be interesting to know which packages you install?
Maybe if I am really interested in you as it takes ages to get to know
all your packages (if you don't happen to do an upgrade to a new major
release), but as the average evil doer I know more than enough already:
Your IP and that you are likely suspect to recent exploits for at least
a few minutes still. That should be enough to add you to my botnet… (or
lets imagine something "less scary": The bar you are in offering
a special two-for-one-beer for Debian users "out of nowhere"…).


> side effects (speed issues,

Maybe its just me being lucky, but speed seems not to be an issue for me
for apt via Tor. Okay, the initial connect takes slightly longer, but
after that is done apts (tor+)http method with its support of pipelining
is actually perfectly capable of maxing out my connection (regardless of
onion or "normal" mirrors I am connecting to) in most cases.


> rather more complicated to set up and keep
> going for the average person,

No. For the average user its a matter of installing apt-transport-tor
and changing sources.list [if you have ideas/patches to enhance this
further feel free to contact us]. You have to do the same for https.
You don't have to go all Tor for everything at once…

(okay, it gets tricky perhaps if your network is blocking connection to
known Tor nodes at which point you need bridges, but the same network
could forbid [non-MITM] HTTPS, so that argument isn't super strong)

Operating an onion service is a different matter of course, but your
average person isn't very likely to setup a good http (or https) mirror
either and you don't absolutely need an onion service. Your usual http
will do. Sure, all-knowing traffic analyse will be capable of perhaps
figuring out what you do in that case, but that chance is a lot lower
the more traffic is routed through the Tor network and the information
that you are a Debian user isn't clearly written on your connection…
(Your are trading it in for "Tor user" which might or might not be
a better label to have at the moment, but given that we are talking
about people out there trying to get you they probably don't need
additional incentive…)


That said, sure, having https would be cool against the casual MITM like
these pesky login-before-you-can-use-our-free-internet portals, but we
already know that. We don't need yet another person coming here and
trying to convince us that HTTPS is the magic bullet we have all been
waiting for because it isn't. Various people have said for various teams
already which technical challenges need to be solved before we can
seriously think about rolling out https on a broad scale and as usual
the problems aren't fixing themselves if only we talk long enough about
them…


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Kristian Erik Hermansen]

On Mon, Oct 24, 2016 at 2:33 AM, Adrian Bunk  wrote:
> You are implicitely assuming that mirrors can be trusted,
> and even that is not true.

No, not actually. Just presuming that NSA doesn't operate ALL mirrors.
Of course they can operate single servers or a number of servers, but
that increases costs and makes it harder to passively collude against
ALL users.

> Who is operating ftp.cn.debian.org, and who has access to the logfiles
> on that server?
>
> Debian would accept debian.nsa.gov as mirror, and the NSA might already
> operate or have access at some current mirrors.

Right, but that's a much smaller subset of ALL.

> When a nation-state actor analyzes all the traffic on a network
> connection that also happens to carry the traffic between you and
> the Debian mirror you are using, HTTPS won't make a difference.

If it doesn't make a difference, send me a PCAP of all your private
traffic captured from an intermediary node :) I mean, if you don't
seem to care, you won't mind me looking through your stuff. And I also
encourage you to configure your browsers and email clients to utilize
only plaintext HTTP / SMTP / IMAP / POP, perhaps on public wifi too,
so we can all read it. You know, I mean, if it "doesn't make a
difference to you" if you use HTTP or HTTPS or other unencrypted
protocols. The reason it matters so much with SecureAPT is because
these are critical protocols running with root privileges on your
system and are leaking a large amount of data about your system
configuration and the security of it. I don't think I need to belabor
that point. HTTPS does make a huge difference and the entire Internet
would not be using it if "didn't make a difference".

We can probably end the thread here because numerous respected @debian
contributors have confirmed the issues with confidentiality and seem
to making efforts in that direction (hopefully for the next release).

-- 
Regards,

Kristian Erik Hermansen
https://www.linkedin.com/in/kristianhermansen

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Adrian Bunk]

On Sun, Oct 23, 2016 at 07:28:23PM -0700, Russ Allbery wrote:
>...
> The value of HTTPS lies in its protection against passive snooping.  Given
> the sad state of the public CA infrastructure, you cannot really protect
> against active MITM with HTTPS without certificate pinning.

You are implicitely assuming that mirrors can be trusted, 
and even that is not true.

Who is operating ftp.cn.debian.org, and who has access to the logfiles 
on that server?

Debian would accept debian.nsa.gov as mirror, and the NSA might already 
operate or have access at some current mirrors.

> But that's
> fine; active attackers are a much, much rarer attack profile.  The most
> likely attack, and the one we're able to protect against here, is passive
> observation of mirror traffic used to build a database of who is using
> what package and at what version.  HTTPS doesn't *prevent* this, but it
> requires the attacker to do much more sophisticated traffic analysis, or
> take the *much* more expensive and *far* riskier step of moving to active
> interference with traffic, neither of which nation-state attackers want to
> do and neither of which they have the resources to do *routinely*.
> 
> It won't help if a nation-state actor is targeting you *in particular*.
> But it helps immensely against dragnet surveillance.

No, it does not help much.

When a nation-state actor analyzes all the traffic on a network 
connection that also happens to carry the traffic between you and
the Debian mirror you are using, HTTPS won't make a difference.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Eugene V. Lyubimkin]

Hi Russ, Kristian,

On 24.10.2016 07:19, Kristian Erik Hermansen wrote:
> On Sun, Oct 23, 2016 at 7:28 PM, Russ Allbery  wrote:
>> The idea is to *add* HTTPS protection on top of the protections we already
>> have.  You're correct that it doesn't give you authentication of the
>> packages without a bunch of work, and we should assume that the general
>> public CA system is compromised.  But that actually doesn't matter much
>> for our purposes, since the point is to greatly increase the cost of
>> gathering data about what packages people have installed.
>>
>> The value of HTTPS lies in its protection against passive snooping.  Given
> 
> Exactly! Much better said than how I originally phrased these issues.
> 
>> what package and at what version.  HTTPS doesn't *prevent* this, but it
>> requires the attacker to do much more sophisticated traffic analysis, or
>> take the *much* more expensive and *far* riskier step of moving to active
>> interference with traffic, neither of which nation-state attackers want to
>> do and neither of which they have the resources to do *routinely*.
>>
>> It won't help if a nation-state actor is targeting you *in particular*.
>> But it helps immensely against dragnet surveillance.
> 
> Again, exactly right and well stated. We can never stop targeted
> attacks, but we can make passive data collection more expensive and
> increase the chances that a targeted attack is detected.

Yes, thank you for explanations, I now get the point of improving more 
confidentiality than integrity here, and warding
off most of passive data gatherers.

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Kristian Erik Hermansen]

On Sun, Oct 23, 2016 at 7:28 PM, Russ Allbery  wrote:
> The idea is to *add* HTTPS protection on top of the protections we already
> have.  You're correct that it doesn't give you authentication of the
> packages without a bunch of work, and we should assume that the general
> public CA system is compromised.  But that actually doesn't matter much
> for our purposes, since the point is to greatly increase the cost of
> gathering data about what packages people have installed.
>
> The value of HTTPS lies in its protection against passive snooping.  Given

Exactly! Much better said than how I originally phrased these issues.

> what package and at what version.  HTTPS doesn't *prevent* this, but it
> requires the attacker to do much more sophisticated traffic analysis, or
> take the *much* more expensive and *far* riskier step of moving to active
> interference with traffic, neither of which nation-state attackers want to
> do and neither of which they have the resources to do *routinely*.
>
> It won't help if a nation-state actor is targeting you *in particular*.
> But it helps immensely against dragnet surveillance.

Again, exactly right and well stated. We can never stop targeted
attacks, but we can make passive data collection more expensive and
increase the chances that a targeted attack is detected.

-- 
Regards,

Kristian Erik Hermansen
https://www.linkedin.com/in/kristianhermansen

Re: client-side signature checking of Debian archives

[Kristian Erik Hermansen]

On Sun, Oct 23, 2016 at 6:46 PM, Paul Wise  wrote:
> Better privacy than https can be had using Tor:
>
> https://onion.debian.org/

If Debian is open to improving SecureAPT's out of the box
configuration by utilizing Tor, then that is fine, but I highly doubt
Debian operators will enjoy the slowness. That is, unless Debian wants
to require that every mirror operator also operate as a Tor exit node
too ;) That will definitely improve the health and speed of the Tor
network! But alas, that is an amount of privacy that I think is much
more complicated to discuss and outside the scope of this thread, as
pointed out by Russ as well.

As also pointed out earlier, Let's Encrypt free CA certificates and
open source configuration utilities make this incredibly easy now for
mirror operators without HTTPS experience. And if you are worried
about the rate limiting, I'm sure that can be removed for trusted
Debian mirrors. The rate limits are squarely targeted at preventing
malicious site operators that may utilize fast-flux methods to conceal
their operations, AFAIK.

-- 
Regards,

Kristian Erik Hermansen
https://www.linkedin.com/in/kristianhermansen

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Russ Allbery]

"Eugene V. Lyubimkin"  writes:

> I'm not sure that benefits outweight the costs. HTTPS requires that I
> trust the third-parties -- mirror provider and CA.  Gpgv doesn't require
> third parties.

It's critical here that we do not drop GPG.  We continue using GPG for the
integrity and authentication part of package retrieval.  If anyone has
proposed replacing the GPG signatures, well, I completely disagree with
that.

The idea is to *add* HTTPS protection on top of the protections we already
have.  You're correct that it doesn't give you authentication of the
packages without a bunch of work, and we should assume that the general
public CA system is compromised.  But that actually doesn't matter much
for our purposes, since the point is to greatly increase the cost of
gathering data about what packages people have installed.

The value of HTTPS lies in its protection against passive snooping.  Given
the sad state of the public CA infrastructure, you cannot really protect
against active MITM with HTTPS without certificate pinning.  But that's
fine; active attackers are a much, much rarer attack profile.  The most
likely attack, and the one we're able to protect against here, is passive
observation of mirror traffic used to build a database of who is using
what package and at what version.  HTTPS doesn't *prevent* this, but it
requires the attacker to do much more sophisticated traffic analysis, or
take the *much* more expensive and *far* riskier step of moving to active
interference with traffic, neither of which nation-state attackers want to
do and neither of which they have the resources to do *routinely*.

It won't help if a nation-state actor is targeting you *in particular*.
But it helps immensely against dragnet surveillance.

-- 
Russ Allbery (r...@debian.org)   

Re: client-side signature checking of Debian archives

[Russ Allbery]

Paul Wise  writes:
> On Mon, Oct 24, 2016 at 7:21 AM, Kristian Erik Hermansen wrote:

>> The point is to improve privacy.

> Better privacy than https can be had using Tor:

> https://onion.debian.org/

Yeah, but this is *way* harder than just using TLS.  You get much of the
benefit by using TLS, and Tor comes with a variety of mildly problematic
side effects (speed issues, rather more complicated to set up and keep
going for the average person, the fact that sadly Tor is also pretty
widely used for malicious activity so you have to be a bit cautious in
*how* you use it, etc.).  By comparison, the work for TLS is all on the
project's part, and then the end user just gets the benefit for nearly
free.

-- 
Russ Allbery (r...@debian.org)   

Re: client-side signature checking of Debian archives

[Paul Wise]

On Mon, Oct 24, 2016 at 7:21 AM, Kristian Erik Hermansen wrote:

> The point is to improve privacy.

Better privacy than https can be had using Tor:

https://onion.debian.org/

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: client-side signature checking of Debian archives

[Kristian Erik Hermansen]

On Sun, Oct 23, 2016 at 4:43 PM, Russ Allbery  wrote:
> susceptible to traffic analysis.  You can make some pretty good guesses
> from the size of the object downloaded, particularly if you can watch over
> time and see what happens when updated packages are released.
>
> Of course, it's much harder than just passively reading the HTTP GET
> commands.  It probably requires someone write code to map object sizes to
> possible packages.

Correct. The point is that it may slightly increase the costs on
Debian and mirrors, but exponentially increase the cost for nation
state attackers. If you want to decrease the traffic analysis issues,
you can also support HTTP/2 (or QUIC udp/443 0-rtt in the future),
which will bundle resource requests intelligently within the same
connection and make traffic analysis a bit more complex / costly.

-- 
Regards,

Kristian Erik Hermansen
https://www.linkedin.com/in/kristianhermansen

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Kristian Erik Hermansen]

On Sun, Oct 23, 2016 at 10:03 AM, Eugene V. Lyubimkin  wrote:
> Thank you for the long list of examples what could go wrong. I'm happy I 
> don't have urgent fixes to apply.

Well, I would say the privacy issues are rather concerning. Security
is generally broken down into at least the following three categories:

* Confidentiality (X)
* Integrity (✓)
* Availability (✓)

Confidentiality is not a current default benefit out of the box with
Secure APT. All choices of user software are exposed. You can even
uniquely identify users by which unpopular packages they get updates
for as they transition their laptop around to various networks.

Availability is covered by the vast numbers of mirrors, so that is covered well.

Integrity is presumed by the GPG / hash verifications. Sure, you have
no publicly known vectors to fix. Presumably there are unknown vectors
the NSA and others have developed, but not made public. There are
really smart people out there that keep their vulnerabilities private
for malintent. This is the concern with leaving clear pathways for
exploitation via unencrypted HTTP channels.

It is also highly likely that at least one member of the Debian
security team has had / will have leaked a private GPG repo signing
key, more than likely by unknown errors in operations or inadvertent
actions. As such, without HTTPS, signing rogue release / package files
for targeting a specific network can take place in secret (targeted
organization would have to detect the tainting at their end-point).
This is because the NSA could specifically target that network with a
rogue "Entity-in-the-Middle" (EitM) attack using the stolen private
key. If HTTPS+HPKP were utilized, it would make pulling off the attack
much more costly, as now they would have to somehow get the mirrors
the target network is using to sync rogue release / package files
without anyone noticing (more easily detectable by mirror system
admins).

And I want to reiterate this attack:

Given a global view of the Internet, the NSA could craft queries like:

"Show us all German systems that originate from network ranges
belonging to  that have never requested a patch
for "

In the above example, they don't need to directly attack APT or
Debian. They can automatically build up a database of CVE attacks that
will work both remotely and locally against the organization, because
they can see that the organization never installed those packages.
This means the unencrypted nature of HTTP thus leaks your entire
organizational security posture to government players for selective
targeting. We know the NSA is doing this, from the Snowden
revelations. See XKeyscore program for more info. There are other
programs that specifically target Linux system administrators. So,
just being on this list might actually put you into a NSA selector
group for targeting as well (be warned, if you didn't already know).

> 1) Checking chain (e.g. gpgv and its callers) have bugs. True, same as 
> checking layer for secure transports also have bugs.

Agreed. Please let me know of a good test case to validate that your
tools, which are not APT (?), are doing the right things. You said you
maintained a tool which "downloads and validates Debian archives in a
similar way APT does", which means not exactly the way APT does. Let
me know the name of your tool and how to setup some test cases to
validate your tool is doing things properly. Glad to spend some time
on it and contribute any potential findings for the community benefit.

> 1a) If an user downloads keys not via packages, downloading may go wrong. 
> True. If I use debian-archive-keyring, I'm fine.

RIght, but there is a bootstrap problem for obtaining trusted keys in
various circumstances though. The SecureAPT wiki even documents those
and instructs the user to use out-of-band methods for obtaining the
key [1], opening up such attacks.

[1] https://wiki.debian.org/SecureApt#How_to_find_and_add_a_key

> 3) Content of packages themselves may be unsecure if they trust some 
> third-party HTTP hosts for downloading their own
> stuff. True, but not relevant to the topic of checking package integrity.

Again, I think the problem is more about CONFIDENTIALITY and not
integrity, even though integrity attacks are likely possible and have
been demonstrated in the past. Surely they will arise again in the
future in a public forum.

> 4) If an user got one malicious key, game is over. True, but so it is if an 
> user got one malicious repo server --
> maintainer scripts from any package have root access to all of your system.

I think the point is that the third-party may not necessarily be
malicious, just that they are not as operationally savvy about
protecting their GPG signing keys as Debian Security Team is, meaning
that the NSA could attack them more easily. This could even be for an
older repo that is no longer operational, but the key is still trusted
in the system from a prior installation. The NSA could 

Re: client-side signature checking of Debian archives

[Russ Allbery]

Ivan Shmakov  writes:

>   My understanding is that the suggestion being discussed is to
>   use TLS /alongside/ the usual Debian/APT signatures – not
>   instead of them; and the primary goal is to improve user’s
>   privacy.  That is: only the mirror operator will remain
>   empowered to know the packages the user’s interested in.

While I have no objections to using TLS for Debian mirrors, it's worth not
overstating the benefits here.  Package retrieval from a public mirror is
susceptible to traffic analysis.  You can make some pretty good guesses
from the size of the object downloaded, particularly if you can watch over
time and see what happens when updated packages are released.

Of course, it's much harder than just passively reading the HTTP GET
commands.  It probably requires someone write code to map object sizes to
possible packages.

-- 
Russ Allbery (r...@debian.org)   

Re: client-side signature checking of Debian archives

[Kristian Erik Hermansen]

On Sun, Oct 23, 2016 at 10:45 AM, Ivan Shmakov  wrote:
> use TLS /alongside/ the usual Debian/APT signatures – not
> instead of them; and the primary goal is to improve user’s
> privacy.  That is: only the mirror operator will remain
>

Exactly right. The point is to improve privacy. Integrity of packages
is not directly vulnerable unless more APT / GPG / parsing
vulnerabilities are identified publicly (NSA surely has some
privately). HTTPS+HPKP doesn't make anything weaker and actually would
also help limit those unknown parsing vectors outlined previously.

-- 
Regards,

Kristian Erik Hermansen
https://www.linkedin.com/in/kristianhermansen

Re: client-side signature checking of Debian archives

[Ivan Shmakov]

> Eugene V Lyubimkin  writes:

[…]

 > I'm not sure that benefits outweigh the costs.  HTTPS requires that
 > I trust the third-parties – mirror provider and CA.  Gpgv doesn't
 > require third parties.

It does; you have to trust whatever source you’ve /initially/
got the public key from.  Also, TLS does /not/ actually preclude
the user from comparing the remote’s key with a copy stored
locally.  For Firefox/HTTPS, the respective functionality could
be found in the Certificate Patrol add-on [1], for instance.

 > To me, that makes HTTPS (even with HPKP) principally weaker than
 > offline medium-agnostic cryptographic content checks.  Or I am wrong
 > here, will the suggested HTTPS+HPKP+… scheme protect me from
 > government players?

My understanding is that the suggestion being discussed is to
use TLS /alongside/ the usual Debian/APT signatures – not
instead of them; and the primary goal is to improve user’s
privacy.  That is: only the mirror operator will remain
empowered to know the packages the user’s interested in.
(As opposed to: the operators of all the networks the APT HTTP
request passes through.)

My concerns would be along the lines of [2] (“Remember that all
mirror sites are donated to Debian: the hardware, […], and the
sysadmin work to keep it running.”)  Specifically, a plain-HTTP
server is easier to configure and maintain.  For one thing, when
your server does /not/ use TLS, you don’t need to be concerned
with the bugs and vulnerabilities of any TLS library whatsoever.

[1] http://patrol.psyced.org/
[2] 
https://lists.debian.org/msgid-search/20161017142819.72lbe3kh346c4h62@exolobe3

-- 
FSF associate member #7257  58F8 0F47 53F5 2EB2 F6A5  8916 3013 B6A0 230E 334A

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Eugene V. Lyubimkin]

Hi,

[ please don't CC me directly ]

On 23.10.2016 17:20, Kristian Erik Hermansen wrote:
> On Sun, Oct 23, 2016 at 7:23 AM, Eugene V. Lyubimkin  
> wrote:
>> I'm a developer of a tool which downloads and validates Debian archives
>> in a similar way APT does.
>>
>> As you use the word "theoretically", that suggests that practically
>> one can bypass the validation. Could you please list all numerous ways
>> to bypass it, so we'd fix our software?
> 
> I will detail more recent issues in the future, but just to start:

Thank you for the long list of examples what could go wrong. I'm happy I don't 
have urgent fixes to apply.

Things you mentioned seem to fall into one of the following categories:

1) Checking chain (e.g. gpgv and its callers) have bugs. True, same as checking 
layer for secure transports also have bugs.

1a) If an user downloads keys not via packages, downloading may go wrong. True. 
If I use debian-archive-keyring, I'm fine.

2) Downloading clients have bugs. True.

3) Content of packages themselves may be unsecure if they trust some 
third-party HTTP hosts for downloading their own
stuff. True, but not relevant to the topic of checking package integrity.

4) If an user got one malicious key, game is over. True, but so it is if an 
user got one malicious repo server --
maintainer scripts from any package have root access to all of your system.


I'm not sure that benefits outweight the costs. HTTPS requires that I trust the 
third-parties -- mirror provider and CA.
Gpgv doesn't require third parties. To me, that makes HTTPS (even with HPKP) 
principally weaker than offline
medium-agnostic cryptographic content checks. Or I am wrong here, will the 
suggested HTTPS+HPKP+... scheme protect me
from government players?

Re: client-side signature checking of Debian archives (Re: When should we https our mirrors?)

[Kristian Erik Hermansen]

Hi :)

On Sun, Oct 23, 2016 at 7:23 AM, Eugene V. Lyubimkin  wrote:
> I'm a developer of a tool which downloads and validates Debian archives
> in a similar way APT does.
>
> As you use the word "theoretically", that suggests that practically
> one can bypass the validation. Could you please list all numerous ways
> to bypass it, so we'd fix our software?

I will detail more recent issues in the future, but just to start:

* Numerous attacks against the downloading client itself (buffer
management flaws to gain code exec, etc)

* Attacks against a GnuPG client parsing public keys or validating signatures
** CVE-2014-4617
** CVE-2013-4402
** CVE-2013-4351
** CVE-2006-6235
** CVE-2006-0455
** CVE-2006-0049
** CVE-2001-0071

* Attacks against weak pre/post-install scripts that retrieve network
data over HTTP

* If your GPG private key is compromised (factored or stolen) without
your knowledge, HTTPS+HPKP helps make arbitrary targeting / infecting
of users much more difficult.

* Potentially trick APT into thinking the install is coming from media
disc and bypass Release file signatures

* If a user adds a GPG key for another third-party product repository
(eg. oracle-java) and that key was generated by NSA to appear as a
legitimate package, HTTPS+HPKP prevents NSA from tampering core
packages too easily (since they have the private key that likely many
users have trusted).

* If a user downloads a GPG public key by searching the public key
servers, NSA can potentially exploit CVE-2012-6085 to overwrite the
trusted local GPG store for trusted keys with a key store of their
choosing. Or also CVE-2008-1530.

* CVE-2012-3587 / CVE-2012-0954 -- APT 0.7/0.8, when using the apt-key
net-update to import keyrings, relies on GnuPG argument order and does
not check GPG subkeys, which might allow remote attackers to install
Trojan horse packages via a man-in-the-middle (MITM) attack.
HTTPS+HPKP would have made this much harder.

* More PitM attacks via CVE-2009-1358 -- "apt-get in apt before 0.7.21
does not check for the correct error code from gpgv, which causes apt
to treat a repository as valid even when it has been signed with a key
that has been revoked or expired, which might allow remote attackers
to trick apt into installing malicious repositories."

* Potentially attack NTP to change the date on a targeted Debian
client system in order to cause potentially exploitable exceptions for
use in a package tampering attack shortly thereafter.

There are other attacks too, but this should suffice for now? I think
the synopsis here is to leverage HTTPS to minimize the exposure from
APT, GPG, parsing / related bugs...unless you truly believe that
programs interpreting untrusted data over HTTP is 100% securely
constructed (quite unlikely).

-- 
Regards,

Kristian Erik Hermansen
https://www.linkedin.com/in/kristianhermansen
https://profiles.google.com/kristianerikhermansen