Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Russ Allbery]

Jonathan Nieder  writes:
> Russ Allbery wrote:

>> I believe at this point the dpkg-buildflags solution has proven
>> reasonably successful and is being widely deployed.  I think we should
>> confirm that the TC agrees with that approach and close out this bug.

> While I understand that the PR effect would be good, I encourage you
> not to do that.  Instead, I'd suggest just closing the bug as
> described at [1]:

> | 5. Sometimes, one side or other is convinced, during the committee's
> |deliberations, by the merit of the other side's arguments. This
> |is a good thing! If it happens, the committee need not make a
> |formal decision

Why?

The solution that was chosen was chosen in part because of the tech-ctte
discussion and because of opinions expressed by the tech-ctte members.  I
believe it's a case where we would have made a formal decision had the
process been working more efficiently, and that it's worthwhile recording
that decision going forward.  I say that in part with my Policy editor hat
on, as there was quite a lot of discussion about this and related topics
in the Policy context and this decision provides some welcome clarity to
how similar situations should be handled going forward.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87d36kne0s@windlord.stanford.edu

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Russ Allbery]

I believe at this point the dpkg-buildflags solution has proven reasonably
successful and is being widely deployed.  I think we should confirm that
the TC agrees with that approach and close out this bug.

I therefore propose the following ballot:

A. The Technical Committee agrees with the dpkg-buildflags approach
   currently in use to enable hardening flags.

B. Further discussion.

I think the remaining open question is whether there is a desire to list
overriding the GCC maintainer and patching GCC to enable the flags for all
compilation on the ballot.  My guess is that there is not; if anyone
disagrees, please speak up and I'll draft a revised ballot including that
option.

If there is no other feedback, I plan to call for a vote on the above
ballot in a few days.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87r4wpp6mu@windlord.stanford.edu

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Raphael Geissert]

[sorry for breaking the thread]

Hi,

Raphael Hertzog wrote:
> Can you do the work of collecting those statistics? Tollef has access
> to a big machine where building all package takes 14h. Roger Leigh used
> it for that kind of research.
> 
> Maybe you can do the rebuild without -Werror=format-security and just grep
> the log to find out those that would fail.

This was already done back at DC10 and the outcome was 8 packages FTBFS[1]. 
However, taking a further look at them now, it looks like the rebuild was not 
done as intended and only packages using h-w were influenced by the new flag.

If the flag is disabled then I think it is pointless to enable -Wformat and -
Wformat-security, as I stated in #587358.

[1] quagga, bist, grap, robodoc, nast, rtpproxy, strongswan, zoem 

Cheers,
-- 
Raphael Geissert - Debian Developer
www.debian.org - get.debian.net


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201108092301.25849.geiss...@debian.org

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Steve Langasek]

On Wed, Jul 27, 2011 at 11:56:39PM +0200, Raphael Hertzog wrote:
> In the course of doing this I discovered that this won't have the
> expected result:
> ---
> export DEB_CFLAGS_MAINT_APPEND = -Wall
> [...]
>   ./configure $(shell dpkg-buildflags --export=configure)
> ---

> Apparently make doesn't export the variables to the sub-shell
> run in this way but only to shells run for commands in the various
> targets. So instead I have to do it this way:
> ./configure $(shell DEB_CFLAGS_MAINT_APPEND="-Wall" dpkg-buildflags 
> --export=configure)

I would be inclined to write this as:

BUILD_FLAGS = $(shell DEB_CFLAGS_MAINT_APPEND="-Wall" dpkg-buildflags 
--export=configure)

build:
./configure $(BUILD_FLAGS)

though a helper implementing this may of course choose to avoid clobbering
the namespace by declaring new make variables.

> Should we go further and provide centralized variables that can be used
> to strip out the precise set of build flags that each hardening "feature"
> adds? For reference /usr/share/hardening-includes/hardening.make does
> provide such variables.

Now that we've suggested complementing DEB_BUILD_OPTIONS with
DEB_BUILD_MAINT_OPTIONS, it stands to reason that we might define some
macros for common cases.

Cheers,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Raphael Hertzog]

On Fri, 29 Jul 2011, Steve Langasek wrote:
> On Tue, Jul 26, 2011 at 11:32:27PM +0200, Raphael Hertzog wrote:
> > TODO: add a "STRIP" operation to the set of operations supported by
> > dpkg-buildflags. DEB_CFLAGS_MAINT_STRIP="--foo --bar" would basically
> > drop all occurrences of --foo and --bar in the returned build flags.
> 
> > QUESTION: Is this ok to assume that all build flags can be "delimited"
> > by a space character?
> 
> Counterexample: -Wl,-z -Wl,defs
> 
> While this *can* also be written as -Wl,-z,defs, I'm not sure there's any
> way to guarantee it will be?

Well, the point of the "strip" operation is to drop the build flags we're
injecting by default so as long as we're able to write all flags that
we're injecting in a way that doesn't require the use of the space, we
should be fine.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110729084448.ge6...@rivendell.home.ouaza.com

Re: Bug#552688: [hert...@debian.org: Bug#552688: Please decide how Debian should enable hardening build flags]

[Kees Cook]

On Fri, Jul 29, 2011 at 12:29:17AM +0200, Raphael Hertzog wrote:
> On Thu, 28 Jul 2011, Kees Cook wrote:
> > That seems fine to me as long as I'm in a position to still be able to fix
> > bugs in the logic. :)
> 
> Well, it's low-maintenance mode I hope so I have no problem merging your
> patches whenever needed.

Cool. Normally it's just been little things -- tweaking features, updating
documentation, or changing arch exclusions.

> > I'm totally fine with h-i going away. The "hardening-check" script will
> > need somewhere to live, though.
> 
> lintian? devscripts? dpkg-dev? I'm not sure what the best place is. But I
> would really like lintian to check if all binaries are built with hardening
> flags. It should probably not report any problem as long as one of
> the hardening feature has been found.
> 
> That way it's still possible to disable some of the hardening features
> without generating a warning that you have to override.

Lintian was where I have been planning to put it, but it's non-trivial. It
is easy to detect certain hardening features (BINDNOW, RELRO), but
stack-protector and _FORTIFY_SOURCE are not possible to detect that they
are strictly _missing_. For example, if a program doesn't use any string
arrays on the stack, it will have no protected stacks. Same for the
protected libc functions with fortify.

> > Do you have an example of what the STRIP stuff would look like for a build?
> > I don't want maintainers to have to know what all the individual flags are,
> > especially since they might change, which is why I did what I did in h-i.
> 
> DEB_CFLAGS_MAINT_STRIP="-fPIE" DEB_LDFLAGS_MAINT_STRIP="-fPIE -pie" 
> dpkg-buildflags
> 
> So yes it requires precise knowledge of the flag in use. To make this
> less obnoxious I can certainly include a copy of the default flags
> in the new /usr/share/dpkg/buildflags.mk file and let maintainer
> use the variables listed there instead of hardcoding the precise set of
> flags.
> 
> (I just did that in my pu/build-flags test branch)

Excellent, I think this is the right thing to do. The flags do slowly
change over time (e.g. adding "--param ssp-buffer-size=4" to stack
protector).

Oh, eek, pointing out that example reminds me about the "can be space
separated?" question ... "--param ssp-buffer-size=4" has a space in the
middle! It should be safe to change this to "--param=ssp-buffer-size=4".
Sorry about missing that.

I'm still not sure what to do about the -fPIC vs -fPIE conflict for
cmake builds, though. I guess the documentation should just specifically
mention it and show how to filter for those cases. And it's not like it's
anything except -fPIC, so I don't think we need a whole system to deal with
it.

> But this is all rather verbose, maybe it's best to keep some separate
> logic to enable/disable hardening features. Other opinions are welcome.
> Maybe with a DEB_BUILD_MAINT_OPTIONS variable.
> 
> DEB_BUILD_MAINT_OPTIONS="hardening=-relro,+pie"

This would be excellent.

> > Hopefully I explained this in the other thread. The situation is that
> > everyone presently using h-w/i expects to build PIE, on architectures
> > that _support_ it, including architectures that should not use it by
> > _default_. So we need an easy way in a specific package to turn on PIE
> > for architectures that support it, but for which we don't want it on by
> > default.
> 
> Maybe something like this:
> DEB_BUILD_MAINT_OPTIONS="hardening=+pie:default"
> DEB_BUILD_MAINT_OPTIONS="hardening=+pie:always"

I think just "+pie" is sufficient, due to how you wrote the supported
tests.

-Kees

-- 
Kees Cook@debian.org


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110728225553.gk4...@outflux.net

Re: Bug#552688: [hert...@debian.org: Bug#552688: Please decide how Debian should enable hardening build flags]

[Raphael Hertzog]

On Thu, 28 Jul 2011, Kees Cook wrote:
> > It would not be reasonable for dpkg-dev to depend on hardening-includes so
> > my plan was basically to move this logic into dpkg-dev. But instead of
> > duplicating it we can find a way for hardening-includes to reuse the logic
> > that would be integrated in dpkg-dev.
> 
> That seems fine to me as long as I'm in a position to still be able to fix
> bugs in the logic. :)

Well, it's low-maintenance mode I hope so I have no problem merging your
patches whenever needed.

> I'm totally fine with h-i going away. The "hardening-check" script will
> need somewhere to live, though.

lintian? devscripts? dpkg-dev? I'm not sure what the best place is. But I
would really like lintian to check if all binaries are built with hardening
flags. It should probably not report any problem as long as one of
the hardening feature has been found.

That way it's still possible to disable some of the hardening features
without generating a warning that you have to override.

> Do you have an example of what the STRIP stuff would look like for a build?
> I don't want maintainers to have to know what all the individual flags are,
> especially since they might change, which is why I did what I did in h-i.

DEB_CFLAGS_MAINT_STRIP="-fPIE" DEB_LDFLAGS_MAINT_STRIP="-fPIE -pie" 
dpkg-buildflags

So yes it requires precise knowledge of the flag in use. To make this
less obnoxious I can certainly include a copy of the default flags
in the new /usr/share/dpkg/buildflags.mk file and let maintainer
use the variables listed there instead of hardcoding the precise set of
flags.

(I just did that in my pu/build-flags test branch)

But this is all rather verbose, maybe it's best to keep some separate
logic to enable/disable hardening features. Other opinions are welcome.
Maybe with a DEB_BUILD_MAINT_OPTIONS variable.

DEB_BUILD_MAINT_OPTIONS="hardening=-relro,+pie"

> Hopefully I explained this in the other thread. The situation is that
> everyone presently using h-w/i expects to build PIE, on architectures
> that _support_ it, including architectures that should not use it by
> _default_. So we need an easy way in a specific package to turn on PIE
> for architectures that support it, but for which we don't want it on by
> default.

Maybe something like this:
DEB_BUILD_MAINT_OPTIONS="hardening=+pie:default"
DEB_BUILD_MAINT_OPTIONS="hardening=+pie:always"

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110728222917.gg3...@rivendell.home.ouaza.com

Re: Bug#552688: [hert...@debian.org: Bug#552688: Please decide how Debian should enable hardening build flags]

[Kees Cook]

On Wed, Jul 27, 2011 at 05:13:30PM +0200, Raphael Hertzog wrote:
> On Wed, 27 Jul 2011, Kees Cook wrote:
> > > Assuming that all those improvements are done, the consensus was that
> > > it's fine for dpkg-buildflags to start emitting the hardening build
> > > flags by default. According to Ubuntu's experience only a few dozen of
> > > packages are broken by the presence of these flags and those packages
> > > should just be updated to use the new STRIP operation to drop the
> > > problematic flags. This could be dealt as part of a wheezy release goal.
> > 
> > And a large portion of them are already fixed since Ubuntu reported the
> > bugs to Debian and most were fixed.
> 
> How are they fixed? By adding DEB_BUILD_HARDENING_* := 0 in their
> environment?

The Ubuntu gcc patch doesn't handle the DEB_BUILD_HARDENING_* flags, so
the bulk of the fixes were for things that _FORTIFY_SOURCE and
stack-protector caught, so the code itself was fixed either upstream or
with a Debian-carried patch.

Things that were built in Ubuntu with hardening-wrapper/includes were
to explicitly gain PIE (since gcc already turned everything else on),
so when they went weird they were fixed the same way (code fixes,
upstream or Debian-carried patch). Most of the packages that had an
Ubuntu delta of just h-w/i were taken by the Debian maintainers, so all
of those packages should be sensible.

> > - by what mechanism will dpkg-buildflags use hardening-includes? It
> >   wouldn't make sense to duplicate the existing arch-specific logic
> >   that lives in hardening-includes.
> 
> It would not be reasonable for dpkg-dev to depend on hardening-includes so
> my plan was basically to move this logic into dpkg-dev. But instead of
> duplicating it we can find a way for hardening-includes to reuse the logic
> that would be integrated in dpkg-dev.

That seems fine to me as long as I'm in a position to still be able to fix
bugs in the logic. :)

> That said, why should hardening-includes last any longer if
> dpkg-buildflags offers everything it does?

I'm totally fine with h-i going away. The "hardening-check" script will
need somewhere to live, though. And, as mentioned in the other thread, we
need to figure out a sensible transition for the PIE bits, since it will
likely start life in dpkg-buildflags globally disabled.

> > - should the hardening flags presence still be controlled by the env
> >   variables that are exposed as the existing interfaces defined by
> >   hardening-wrapper/hardening-includes?
> 
> If that's how current debian packages have been fixed, possibly yes at the
> start but we would emit a warning explaining that package have to be
> updated to use the new STRIP dpkg-buildflags operation.

Maintainers transitioning from hardening-wrapper should just skip straight
to STRIP since it's the same work moving from -wrapper to -includes.

Maintainers transitioning from hardening-includes should be able to easily
switch to STRIP, so I'm not sure it's worth supporting the h-i flags.

Do you have an example of what the STRIP stuff would look like for a build?
I don't want maintainers to have to know what all the individual flags are,
especially since they might change, which is why I did what I did in h-i.

> And at some point, the support for those env variables should be dropped.
> 
> > - there needs to be a way to identify those architectures that are
> >   "register starved", since those should _not_ get the PIE flags by
> >   default (e.g. i386 should not get PIE, but amd64 should get PIE by
> >   default). Right now if one uses hardening-wrapper, it's expected
> >   that everything that can be enabled is enabled, so you gain PIE
> >   even on i386 at the moment.
> 
> Not sure I understand your problem. What's difficult in excluding
> i386 from the set of architectures where PIE is used?

Hopefully I explained this in the other thread. The situation is that
everyone presently using h-w/i expects to build PIE, on architectures
that _support_ it, including architectures that should not use it by
_default_. So we need an easy way in a specific package to turn on PIE
for architectures that support it, but for which we don't want it on by
default.

-Kees

-- 
Kees Cook@debian.org


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110728205742.gx4...@outflux.net

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Kees Cook]

Hi Matthias,

On Thu, Jul 28, 2011 at 06:55:11PM +0200, Matthias Klose wrote:
> On 07/27/2011 11:56 PM, Raphael Hertzog wrote:
> >On Tue, 26 Jul 2011, Raphael Hertzog wrote:
> >>Assuming that all those improvements are done, the consensus was that
> >>it's fine for dpkg-buildflags to start emitting the hardening build
> >>flags by default. According to Ubuntu's experience only a few dozen of
> >>packages are broken by the presence of these flags and those packages
> >>should just be updated to use the new STRIP operation to drop the
> 
> for strip you need to know the exact options to be passed; wouldn't
> it be better to have them stripped by something like `nohardening'?

Do you mean via DEB_BUILD_OPTIONS? That can't really be set _in_ a
package, so we'd need something else, IIUC.

> >One thing that is really not clear to me is how we should handle PIE.
> >It's not enabled by default by the gcc patch. This means that it's not
> >safe to enable it by default in dpkg-buildflags because we have no idea of
> >its impact. While all the other options have been well tested thanks to
> >Ubuntu, this one was not.
> 
> >Yet it seems that we should still aim to use it
> >by default at some point. How should we handle that transition?
> 
> I did see performance penalties of more than 20% on i386 and armel
> when enabling PIE by default. This shouldn't be the scope of this
> discussion, and I still don't see value to rebuild the whole archive
> using PIE.

I recall it being no more than 15% in the worst case, but yes, that was
for the register-starved architectures. I totally agree: it should not be
the default for i386. I would like to see more benchmarks for armel and
amd64, though.

The only part of PIE, I think, should be that it should be able to be
enabled trivially, so that the hardening-wrapper/includes transition is
easy for maintainers.

> >The current implementation in my branch is that PIE is disabled by defaut
> >but if you set DEB_BUILD_HARDENING_PIE=1 then it will be used. This was
> >easily done on top of the compatibility layer with
> >hardening-includes/hardening-wrapper but I'm not convinced it's an
> >interface we want to use for this transition.
> >
> >In that case, it means that we should rebuild the archive with PIE
> >enabled, see what breaks, report bugs and ask people to add
> >DEB_CFLAGS_MAINT_STRIP="-fPIE" DEB_LDFLAGS_MAINT_STRIP="-fPIE -pie"
> >where required. Once most packages have been fixed, we can add
> >PIE to the default flags. Does this sound reasonable?
> 
> No, there's no value having cc1 built with PIE, same for number
> crunching libraries, doubtful for interpreters.

Libraries are already PIC. PIE will only make a difference for the main
binary. But, as you say, worst-case situations tend to be things that use
very few library calls, like interpreters, compilers, etc. I would agree that
cc1 doesn't need to be PIE. I would, however, argue that interpreters
should be PIE, since they are frequently dealing with external input
(just think of all the utf-8 vulnerabilities that have happened over
the last few years).

But, as you mention, this is out of scope really. Let's get the rest in
place and work to make the transition not accidentally trigger feature
loss for maintainers.

-Kees

-- 
Kees Cook@debian.org


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110728204219.gw4...@outflux.net

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Kees Cook]

On Wed, Jul 27, 2011 at 11:56:39PM +0200, Raphael Hertzog wrote:
> One thing that is really not clear to me is how we should handle PIE.
> It's not enabled by default by the gcc patch. This means that it's not
> safe to enable it by default in dpkg-buildflags because we have no idea of
> its impact. While all the other options have been well tested thanks to
> Ubuntu, this one was not. Yet it seems that we should still aim to use it
> by default at some point. How should we handle that transition?

While I did not make it a default in Ubuntu yet, I have done archive
rebuilds with PIE enabled. It was about 2 years ago, but they were still
interesting. The majority of things compiled fine, but the logs were huge,
so I, unfortunately, deleted them a while ago.

I really want have it be the default for at least amd64. There are a
few issues that come to mind:

- speed impact in places
  Some things are genuinely slower with PIE (but I don't have amd64
  benchmarks -- I only measured i386 at the time). My feeling is that
  speed sensitive packages (e.g. cc1 as doko points out) would then have
  the option of disabling it for their specific build. The default case,
  I think, is worth it, though I haven't actually ever been able to
  keep all the rebuilds I did with testing to see if there were any
  problems with a running system built that way.

- assembly FTBFSes
  There are a few packages (usually codecs etc) that have non-relocatable
  assembly in them, so PIE builds fail to handle it, or to find an available
  register to use for thunking. This is, I feel, a case-by-case issue.
  Ironically, these are usually the things I'd like to see built PIE more
  than other things! :)

- non-PIC .a file relocation
  Using PIE by default means that packages shipping non-PIC .a files
  suddenly produce relocatable .a files. If a package that links against
  them isn't building as PIE too, it will FTBFS.

> The current implementation in my branch is that PIE is disabled by defaut
> but if you set DEB_BUILD_HARDENING_PIE=1 then it will be used. This was
> easily done on top of the compatibility layer with
> hardening-includes/hardening-wrapper but I'm not convinced it's an
> interface we want to use for this transition.

If someone chose to build-dep on hardening-wrapper/hardening-includes, they
expect to have built PIE, so I think that the dpkg-buildflags default
should likely depend on that in some way.

The problem here is that h-w/i defaults to PIE-when-supported rather than
PIE-when-supported-and-desired, so having a maintainer explicitly set
DEB_BUILD_HARDENING_PIE=1 will trigger FTBFS on the architectures that
don't support it. I think we'll need some other flag instead that means
"PIE if possible" when moving from dpkg-buildflags from h-w/i.

It might be possible to reorganize hardening.make to have
_HARDENED_PIE_CFLAGS and _HARDENED_PIE_LDFLAGS only be populated on archs
that support PIE. Right now, the selection logic is part of
HARDENING_CFLAGS and HARDENING_LDFLAGS. Or they could be exposed as a
separate set?

There's a lot of ways to do this. I'm not sure what is best. What's
important to me is that maintainers that were using h-w/i don't suddenly
end up with builds that aren't PIE, since they explicitly chose to build
with PIE (unless they also explicitly chose to disable it).

> In that case, it means that we should rebuild the archive with PIE
> enabled, see what breaks, report bugs and ask people to add
> DEB_CFLAGS_MAINT_STRIP="-fPIE" DEB_LDFLAGS_MAINT_STRIP="-fPIE -pie"
> where required. Once most packages have been fixed, we can add
> PIE to the default flags. Does this sound reasonable?

Well, this can only be done once the archive is using dpkg-buildflags,
which will be a long transition, IIUC.

> Should we go further and provide centralized variables that can be used
> to strip out the precise set of build flags that each hardening "feature"
> adds? For reference /usr/share/hardening-includes/hardening.make does
> provide such variables.

It seemed like a good idea to me (which is why hardening.make has it), so
I'd support that again. Having a common way to control the flags seems like
a very good idea.

There's also the complex case of some build systems passing -fPIC to
everything, which supersedes -fPIE unfortunately. I think this is a GCC
bug, personally, but I haven't had time to hunt it down.

-Kees

-- 
Kees Cook@debian.org


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110728203402.gv4...@outflux.net

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Matthias Klose]

On 07/27/2011 11:56 PM, Raphael Hertzog wrote:

Hi,

On Tue, 26 Jul 2011, Raphael Hertzog wrote:

Assuming that all those improvements are done, the consensus was that
it's fine for dpkg-buildflags to start emitting the hardening build
flags by default. According to Ubuntu's experience only a few dozen of
packages are broken by the presence of these flags and those packages
should just be updated to use the new STRIP operation to drop the


for strip you need to know the exact options to be passed; wouldn't it be better 
to have them stripped by something like `nohardening'?



In the course of doing this I discovered that this won't have the
expected result:
---
export DEB_CFLAGS_MAINT_APPEND = -Wall
[...]
./configure $(shell dpkg-buildflags --export=configure)
---

Apparently make doesn't export the variables to the sub-shell
run in this way but only to shells run for commands in the various
targets. So instead I have to do it this way:
./configure $(shell DEB_CFLAGS_MAINT_APPEND="-Wall" dpkg-buildflags 
--export=configure)

One thing that is really not clear to me is how we should handle PIE.
It's not enabled by default by the gcc patch. This means that it's not
safe to enable it by default in dpkg-buildflags because we have no idea of
its impact. While all the other options have been well tested thanks to
Ubuntu, this one was not.



Yet it seems that we should still aim to use it
by default at some point. How should we handle that transition?


I did see performance penalties of more than 20% on i386 and armel when enabling 
PIE by default. This shouldn't be the scope of this discussion, and I still 
don't see value to rebuild the whole archive using PIE.



The current implementation in my branch is that PIE is disabled by defaut
but if you set DEB_BUILD_HARDENING_PIE=1 then it will be used. This was
easily done on top of the compatibility layer with
hardening-includes/hardening-wrapper but I'm not convinced it's an
interface we want to use for this transition.

In that case, it means that we should rebuild the archive with PIE
enabled, see what breaks, report bugs and ask people to add
DEB_CFLAGS_MAINT_STRIP="-fPIE" DEB_LDFLAGS_MAINT_STRIP="-fPIE -pie"
where required. Once most packages have been fixed, we can add
PIE to the default flags. Does this sound reasonable?


No, there's no value having cc1 built with PIE, same for number crunching 
libraries, doubtful for interpreters.


  Matthias


--
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e31946f.9000...@debian.org

Re: Bug#552688: Please decide how Debian should enable hardening build flags

[Raphael Hertzog]

Hi,

On Tue, 26 Jul 2011, Raphael Hertzog wrote:
> Assuming that all those improvements are done, the consensus was that
> it's fine for dpkg-buildflags to start emitting the hardening build
> flags by default. According to Ubuntu's experience only a few dozen of
> packages are broken by the presence of these flags and those packages
> should just be updated to use the new STRIP operation to drop the
> problematic flags. This could be dealt as part of a wheezy release goal.

I have prepared a branch that implements all this, that also includes
hardening build flags and that modify dpkg's debian/rules to actually
use dpkg-buildflags in the way we expect maintainers to use it.

See 
http://anonscm.debian.org/gitweb/?p=users/hertzog/dpkg.git;a=shortlog;h=refs/heads/pu/build-flags
You can grab it with "git clone git://git.debian.org/~hertzog/dpkg.git -b
pu/build-flags".

In the course of doing this I discovered that this won't have the
expected result:
---
export DEB_CFLAGS_MAINT_APPEND = -Wall
[...]
./configure $(shell dpkg-buildflags --export=configure)
---

Apparently make doesn't export the variables to the sub-shell
run in this way but only to shells run for commands in the various
targets. So instead I have to do it this way:
./configure $(shell DEB_CFLAGS_MAINT_APPEND="-Wall" dpkg-buildflags 
--export=configure)

One thing that is really not clear to me is how we should handle PIE.
It's not enabled by default by the gcc patch. This means that it's not
safe to enable it by default in dpkg-buildflags because we have no idea of
its impact. While all the other options have been well tested thanks to
Ubuntu, this one was not. Yet it seems that we should still aim to use it
by default at some point. How should we handle that transition?

The current implementation in my branch is that PIE is disabled by defaut
but if you set DEB_BUILD_HARDENING_PIE=1 then it will be used. This was
easily done on top of the compatibility layer with
hardening-includes/hardening-wrapper but I'm not convinced it's an
interface we want to use for this transition.

In that case, it means that we should rebuild the archive with PIE
enabled, see what breaks, report bugs and ask people to add
DEB_CFLAGS_MAINT_STRIP="-fPIE" DEB_LDFLAGS_MAINT_STRIP="-fPIE -pie"
where required. Once most packages have been fixed, we can add
PIE to the default flags. Does this sound reasonable?

Should we go further and provide centralized variables that can be used
to strip out the precise set of build flags that each hardening "feature"
adds? For reference /usr/share/hardening-includes/hardening.make does
provide such variables.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110727215639.gb9...@rivendell.home.ouaza.com

Re: Bug#552688: [hert...@debian.org: Bug#552688: Please decide how Debian should enable hardening build flags]

[Raphael Hertzog]

Hi,

On Wed, 27 Jul 2011, Kees Cook wrote:
> > TODO: revert debian/buildflags support, and implement
> > support for the environment variable DEB__MAINT_ which
> > work exactly like the corresponding DEB__ except it's
> > meant to be used by the package maintainer within debian/rules.
> 
> I'm not sure how this will interact with hardening options, but okay.

It's not really relevant for hardening options except that if we want to
make dpkg-buildflags a mandatory interface to retrieve the complete
set of build flags, it's important that the interface it offers can be used
in all cases.

> > QUESTION: Is this ok to assume that all build flags can be "delimited"
> > by a space character?
> 
> For the hardening flags, yes.

The question was more general because it's a generic interface for
dpkg-buildflags and it should handle any build flag that might
realistically be used.

> > Assuming that all those improvements are done, the consensus was that
> > it's fine for dpkg-buildflags to start emitting the hardening build
> > flags by default. According to Ubuntu's experience only a few dozen of
> > packages are broken by the presence of these flags and those packages
> > should just be updated to use the new STRIP operation to drop the
> > problematic flags. This could be dealt as part of a wheezy release goal.
> 
> And a large portion of them are already fixed since Ubuntu reported the
> bugs to Debian and most were fixed.

How are they fixed? By adding DEB_BUILD_HARDENING_* := 0 in their
environment?

> I see three remaining issues:

I think all those issues are to be sorted between you and me, and
do not need the involvment of the technical committee (but obviously
I always welcome review by anyone even of TC members :)).

> - by what mechanism will dpkg-buildflags use hardening-includes? It
>   wouldn't make sense to duplicate the existing arch-specific logic
>   that lives in hardening-includes.

It would not be reasonable for dpkg-dev to depend on hardening-includes so
my plan was basically to move this logic into dpkg-dev. But instead of
duplicating it we can find a way for hardening-includes to reuse the logic
that would be integrated in dpkg-dev.

All the code is in libdpkg-perl and we can decide to have a specific
function that retrieves only the hardening build flags instead of all the
build flags.

That said, why should hardening-includes last any longer if
dpkg-buildflags offers everything it does?

> - should the hardening flags presence still be controlled by the env
>   variables that are exposed as the existing interfaces defined by
>   hardening-wrapper/hardening-includes?

If that's how current debian packages have been fixed, possibly yes at the
start but we would emit a warning explaining that package have to be
updated to use the new STRIP dpkg-buildflags operation.

And at some point, the support for those env variables should be dropped.

> - there needs to be a way to identify those architectures that are
>   "register starved", since those should _not_ get the PIE flags by
>   default (e.g. i386 should not get PIE, but amd64 should get PIE by
>   default). Right now if one uses hardening-wrapper, it's expected
>   that everything that can be enabled is enabled, so you gain PIE
>   even on i386 at the moment.

Not sure I understand your problem. What's difficult in excluding
i386 from the set of architectures where PIE is used?

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110727151330.gb17...@rivendell.home.ouaza.com