Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec

[Guillem Jover]

Hi!

On Tue, 2023-10-31 at 10:52:40 +0100, Guillem Jover wrote:
> I guess I could do it the other way, and given this is apparently
> causing issues as reported by Adrian, and as seen recently from
> the referenced bug report which might require patching a specific
> package to disable PIE there, I'm inclined to completely mask PIE
> in dpkg-buildflags for alpha and ia64 in around a couple of weeks,
> if I don't hear anything.

I've now queued locally a commit masking this for alpha and ia64 and
will be including it in dpkg 1.22.2, probably by the end of the weekend.
To the porters, please let me know if you'd prefer these not to be
masked.

> If PIE (via specs files) appears to work on x32, and changing the
> defaults in gcc is too much to bother, I think leaving it as is looks
> fine by me. But if this is causing problems as well and the x32 porters
> (if there's any remaining :), want to mask it alongside the other ports,
> let me know and I can also flip the switch for that one.

If the porters would also like to see x32 masked, let me know and I
can also include it, otherwise I'll leave it as it is now.

Thanks,
Guillem

Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec

[Guillem Jover]

Hi!

On Tue, 2023-07-04 at 13:12:48 +0300, Adrian Bunk wrote:
> On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote:
> > On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
> > > There are some problems with this:
> > >
> > > 1. PIE should either be default or not be used
> > >
> > > I suspect x32 might be able to default to PIE without problems
> > > (there might just not be enough interest left to change the default).
> > >
> > > On alpha the toolchain has already become quite brittle
> > > with frequent issues like (reproducible) linker segfaults,
> > > any variations that affects the toolchain are bad.
> > >
> > > It is for the port maintainers to decide whether or not PIE
> > > is considered stable on a port, and accordingly either make
> > > it default (which also avoids the other issues below) or not.
> > >
> > > It is clear that a non-PIE architecture would no longer be
> > > considered suitable as release architecture.
> >
> > In general the way this is handled in dpkg, is that if the flags
> > supposedly work on that arch they are allowed, but if they are not
> > supported or are broken then they are masked.

I recalled this report before the 1.22.1 release, but didn't feel it
appropriate to push a last minute change for it, before giving some
advance notice, and because I was also expecting some word from the
relevant arch porters.

I guess I could do it the other way, and given this is apparently
causing issues as reported by Adrian, and as seen recently from
the referenced bug report which might require patching a specific
package to disable PIE there, I'm inclined to completely mask PIE
in dpkg-buildflags for alpha and ia64 in around a couple of weeks,
if I don't hear anything.

> > > Please drop pie-{compile,link}.spec, on the architectures
> > > where it has any effect it is doing more harm than good.
> >
> > For example hppa has pie masked for build flags. If the porters for
> > alpha and/or ia64 consider that they should also get pie masked for
> > those arches, I'm fine doing the changes. Although that means on those
> > ports it will not be possible to enable pie at all, even if asked for
> > explicitly, as in «hardening=+pie».
>
> Semi-PIE non-release architectures shouldn't exist, PIE on an·
> architecture should be a binary option decided by the porters
> for this architecture.
>
> If there are any porters left on an architecture and they consider PIE·
> stable, they can always ask the gcc maintainer to change the default.[1]
>
> >...
> > > The lowest effort fix would be to patch debian/rules of affected
> > > packages to disable hardening=+pie on affected architectures,
> > > but that would still be spending time on working around a problem
> > > that shouldn't exist.
> >
> > Yeah, that does not look like the right thing to be spending time on.
> >...
>
> As long as we have at least one semi-PIE architecture, this is the only·
> realistic option for porters. If the porters for alpha and/or ia64 want·
> to have pie masked, such changes will still be required for x32.

If PIE (via specs files) appears to work on x32, and changing the
defaults in gcc is too much to bother, I think leaving it as is looks
fine by me. But if this is causing problems as well and the x32 porters
(if there's any remaining :), want to mask it alongside the other ports,
let me know and I can also flip the switch for that one.

Thanks,
Guillem

Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec

[Guillem Jover]

Hi!

On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
> Package: dpkg-dev
> Version: 1.21.22
> Severity: normal
> X-Debbugs-Cc: debian-alpha@lists.debian.org, debian-i...@lists.debian.org

> Since stretch all release architectures are using PIE by default,
> and all future release architectures (including riscv64) will also
> use PIE by default.
> 
> Many packages in Debian are building with hardening=+all, and the
> effect regarding PIE is "enable PIE for this package on some obscure
> ports architectures that don't have it enabled by default" which is
> unlikely to be what the maintainer intended.
> 
> There are also some pre-stretch "hardening=+pie" left
> in some packages.

Yeah, I've never been very satisfied with our pie handling. :/

> There are some problems with this:
> 
> 1. PIE should either be default or not be used
> 
> I suspect x32 might be able to default to PIE without problems
> (there might just not be enough interest left to change the default).
> 
> On alpha the toolchain has already become quite brittle
> with frequent issues like (reproducible) linker segfaults,
> any variations that affects the toolchain are bad.
> 
> It is for the port maintainers to decide whether or not PIE
> is considered stable on a port, and accordingly either make
> it default (which also avoids the other issues below) or not.
> 
> It is clear that a non-PIE architecture would no longer be
> considered suitable as release architecture.

In general the way this is handled in dpkg, is that if the flags
supposedly work on that arch they are allowed, but if they are not
supported or are broken then they are masked.

> 2. It causes weird issues on undersupported architectures
> 
> gluegen2 passes LDFLAGS to ld instead of gcc.
> 
> Several packages have relocation errors only on affected
> architectures.
> 
> ...
> 
> Such issues could be debugged and fixed, but in practice
> trying to handle such issues that happen only with
> pie-{compile,link}.spec creates additional work that frustrates
> the few people keeping these non-release architectures alive.

Regardless of this report, I think this would still be worthwhile,
as otherwise you cannot for example disable them globally on arches
where it is a builtin in the compiler (as those will also need the
spec files.

> The lowest effort fix would be to patch debian/rules of affected
> packages to disable hardening=+pie on affected architectures,
> but that would still be spending time on working around a problem
> that shouldn't exist.

Yeah, that does not look like the right thing to be spending time on.

> 3. It breaks some cases of static linking
> 
> Linking a package with hardening=+all against a static library
> from a package not using hardening=+all cannot work on the
> affected architectures.
> 
> Static linking is relatively rare, but I remember requesting binNMUs
> for static linking cases to fix FTBFS on release architectures when
> the default changed before stretch.

Hmm, ah or even packages not respecting DEB_BUILD_OPTIONS, right.

> Please drop pie-{compile,link}.spec, on the architectures
> where it has any effect it is doing more harm than good.

For example hppa has pie masked for build flags. If the porters for
alpha and/or ia64 consider that they should also get pie masked for
those arches, I'm fine doing the changes. Although that means on those
ports it will not be possible to enable pie at all, even if asked for
explicitly, as in «hardening=+pie».

Thanks,
Guillem

[RFC] The PIE unholy mess

[Guillem Jover]

Hi!

I'd like to get some feedback from porters and package maintainers,
given that this affects at least both groups. Some background first.

One of the reasons PIE has in the past not been enabled by default in
dpkg-buildflags, is because it introduced some slowness on some arches,
although this has somewhat changed recently. But also because
unconditionally setting it, breaks at least PIC builds. So PIE got
enabled recently by default in gcc, as it could easily control when it
is relevant. Although this has been done only for release architectures.

At about the same time this was being considered, I realized that dpkg
could enable this "safely" by using gcc specs files. But this is in
any case also required to be able to disable PIE when it is implicitly
enabled by default in gcc. So we'll need specs files no matter what,
at least for now.

While adapting dpkg-buildflags to cover for the new gcc defaults, I
unintentionally enabled PIE by default on all architectures, and when
I noticed, it seemed to make sense to leave it like that, because:

  * All previous build flags from dpkg-buildflags have always been
enabled globally and only blacklisted completely when they have
been non-functional.
  * It's a more consistent interface for packages, as they get built
with the same flags everywhere. Otherwise we'd get PIE enabled by
default in release arches, disabled by default elsewhere, and
enabled or disabled depending on the package requesting so.
  * It will mean that PIE coverage reporting will be shadowed in
lintian, because the tags only cover i386 and amd64, so maintainers
will probably stop enabling them globally.

Matthias Klose recently filed an unclear report (#848129) on dpkg-dev
requesting to not enable PIE globally from dpkg-buildflags, and pretty
much immediately added a patch into gcc [P] to ignore dpkg-buildflags
PIE -specs flags if DEB_BUILD_OPTIONS or DEB_BUILD_MAINT_OPTIONS did
not enable PIE explicitly (I only fully understood the request after
seeing the gcc patch).

  [P] 


Besides this being completely broken, as DEB_BUILD_MAINT_OPTIONS
does not even need to be exported from debian/rules, nor from the
dpkg architecture.mk fragment, or when dpkg-buildflags is used with its
--export=configure or --export=cmdline. It's also a layer violation.
It also breaks unrelated stuff as now gcc emits notes when it thinks
the -specs option should not be passed. And while I could certainly
have started an arms-race by adding counter-measures by randomizing
the filenames or something similarly ugly, that'd be pretty much silly
and childish.

For better or worse, this does not affect the release architectures,
so by extension it should not affect the release, but it still sucks.

So, I'd like to know how people feel about the requested interface
(i.e. not enabling PIE globally from dpkg-buildflags). If there's
consensus that porters and maintainers want that, I'll just change
dpkg-buildflags to do this, even though I think it's a very
suboptiomal behavior.

Alternatively, porters could as well request PIE be enabled by default
in gcc on their port, which could make this also globally enabled.

Thanks,
Guillem

Re: Bug#845193: dpkg: recent -specs PIE changes break openssl

[Guillem Jover]

Hi!

On Thu, 2016-11-24 at 14:52:33 +, Thorsten Glaser wrote:
> clone 845193 -1
> reassign -1 dpkg
> retitle -1 dpkg: please do not add -specs= flags only on some architectures
> thanks

I'm afraid I'll have to wontfix this because it is not really
implementable. See below… :/

> Guillem Jover dixit:
> >Those specs files should make it possible to build stuff with PIE
> 
> Yes, but they *do* break anything that
> - acts on the CFLAGS (and LDFLAGS) variables
> - uses klcc or other compiler wrappers that don't understand -specs
> - uses clang or pcc or whatever other compilers

The default dpkg build flags have always been tied to the specific
language compiler version currently marked as the default (for C that
would currently be gcc-6).

> Worse, they break *differently* on whether…
> 
> >Precisely to make the behavior consistent on all architectures, dpkg
> >enables PIE (conditionally if no other flags marks it as to be
> >disabled) on all architectures were gcc has not enabled this by
> >default.
> 
> … that. And that is just plain wrong. Either dpkg should inject
> -specs= stuff on all architectures or on none. Differing like this
> just invites hidden and hard to track down bugs.

As long as gcc enables PIE on a subset, there will be need to inject
some form of specs on either subset of those arches, either on
hardening=+pie or on hardening=-pie, pick yout poison. :(

Injecting just the raw -fno-PIE and -no-pie does not work, so when you
need to disable those we need to pass via specs files.

> Please get an agreement with the GCC maintainer on how to proceed
> from here.
>
> Personally, I’d still prefer for GCC to behave as on other systems,
> i.e. not to enable PIE by default, and to have it done completely
> within dpkg, but I can *also* live with it being done exclusively
> in GCC.
> 
> Either are *much* better than the current way.

Well you and me both, I'm just adapting the dpkg-buildflags to the
current gcc situation. :/

> >Also BTW the gcc maintainer asked that porters
> >interested could request PIE to be enabled by default in gcc.
> 
> What difference does it make on whether GCC or dpkg enables PIE?

It means it should at least get the same behavior from gcc as all
official ports, it is more transparent and should not suffer from
unbalanced passing of CFLAGS w/o LDFLAGS or the other way around,
for example. Of course that does not mean that package might still
not fail, in case they try to link PIE code into a shared library
or similar.

> The two last quote sections make it clear that any architecture
> that currently has PIE enabled in dpkg should have it enabled in
> GCC in the first place.

Enabling new build flags in dpkg has always been done globally,
except for specific blacklists on things that were completely broken
in the toolchain, which have then been enabled eventually when they
got fixed.

Having a subset of architectures is a pain for maintainers as they
get different resulting objects depending on the architectures, it
also changes the semantics from before the gcc default change, as
previously explicitly enabling PIE was global, and now would be for
a subset.

Or worse, the new semantics would need to be that by default you get
PIE on a subset but if you pass hardening=+pie on each package you get
it enabled globally? Pretty unintuitive IMO.

> (Did dpkg enable that on porters’ requests?
> It does not look like that to me. This smells like overstepping
> boundaries.)

If porters are unhappy about this, I'll revert PIE for those ports in
dpkg, but this will not make the situation less of a mess, hardening=-pie
will still need the specs files on ports were gcc enables it by default,
and hardening=+pie might need them on the ones that do not…

> tl;dr: I don’t care as much _which_ of GCC xor dpkg does it,
> as long as only one does it. FFS, just enable it on all of them
> unless known to absolutely not work; that’d still be better than
> the current mess.

Well I think we should be enabling all hardening flags directly in
gcc, but now that we have the specs files I guess it would not be
too bad to enable them just in dpkg, but I agree either would be
preferable. :/

Thanks,
Guillem

Re: Porter roll call for Debian Stretch

[Guillem Jover]

Hi!

On Sun, 2016-08-21 at 08:22:09 +0200, Niels Thykier wrote:
> Kurt Roeckx:
> > On Wed, Aug 17, 2016 at 10:05:06PM +0200, ni...@thykier.net wrote:
> >>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
> >>also apply to this port? [0]
> > 
> > If -fPIE is the default will -fPIC override it?
> > 
> > It will also default to tell the linker to use -pie, but then
> > don't do it when you want to create a shared library?
> 
> I believe so.  At least, Ubuntu made PIE default in their compiler
> without having to change all SO packages to still build.

As I mentioned on IRC, the Gentoo people seems to have done so as well
in their Gentoo Hardened Toolchain project
.

> Admittedly, it could also be that GCC uses some built "compiler spec"
> files for this case (a la an implicit "-specs=$FILE"), which are similar
> to those Redhat uses for this purposes (see [1] for an example of such
> files).

I think there are some bugs tracked in Fedora about this
, And this
is what I based the dpkg patch on as a potential alternative instead
of modifying our toolchain, which I think would be preferable. But see
below.

> >>From what I understand, depending on what the .o file is going to
> > be used for you want different things:
> > - shared library: -fPIC
> > - executable: -fPIC or -fPIE both work, but prefer -fPIE
> > - static library: Same as executables
> > 
> > For static libraries we now have a policy to not use -fPIC,
> > should that then get replaced by using -fPIE?

> Honestly, I had not thought about that.  From the looks of it, de facto
> we will end up with -fPIE being the default for static libraries as well.
>   I would be supporting that change on the assumption that it requires
> less work from individual maintainers (and presumably has no notable
> downsides in the final result).

I think that's what we are getting right now for any package enabling
the "pie" build flags feature anyway.

> [1] Example spec files for this case seems to be something like:
> 
> pie-compile.specs
> """
> *cc1_options:
> + %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}
> """
> 
> pie-link.specs:
> """
> *self_spec:
> + %{!shared:%{!r:-pie}}
> """
> 
> NB: I manually carved them out of a diff without testing them.

That would be
,
which neither I tested. Testing should first be done for gcc, g++, gcj,
gobjc, gobjc++, gccgo (although I've not managed to use shared libraries
with latest releases, but used to be able to) and gfortran. If someone
wants to test these first to see if it works and then later on do an
archive rebuild, that would also be appreciated.

At that point I could then at least switch the current implementation
so that we can just enable the "pie" feature per package regardless of
it building prorgrams or shared libraries. Enabling it by default
globally would require the usual process described in the dpkg FAQ.

Thanks,
Guillem

Re: Bug#255270: xfree86: libglide3 has now ia64 and amd64 support

[Guillem Jover]

Hi,

On Wed, Jul 14, 2004 at 11:19:13AM -0500, Branden Robinson wrote:
> On Sat, Jun 19, 2004 at 11:58:45PM +0200, Guillem Jover wrote:
> > I've ported libglide3 to amd64 and ia64. So now xfree86 can Build-Depend
> > on libglide3-dev on those arches.

> I've integrated this patch:
> 
> $ svn log -r 1640 svn://necrotic.deadbeast.net/xfree86

Thanks Branden!

> If you and your fellow Glide enthusiasts could test the XFree86 SVN trunk
> on i386, amd64, ia64, and alpha, I sure would appreciate it.
> 
> Instructions[1] for building the trunk are available.
> 
> [1] http://necrotic.deadbeast.net/xsf/XFree86/HACKING.txt

Could someone with an amd64, ia64 or alpha (I'll take care of i386)
and a 3Dfx card with any of the following chipsets:

Voodoo Banshee, Voodoo 3, Voodoo 4 or Voodoo 5

build an xfree86 package from XSF trunk (you should have already a
libglide3{-dev,} build for your arch) and report back if it does
hardware acceleration (glxinfo) or any build problem to this bug
report?

thanks,
guillem

Re: glide: added alpha support, need testers

[Guillem Jover]

Hi Dannis and Gianluca,

On Thu, Aug 28, 2003 at 07:59:16PM +0200, Dannis 't Hart wrote:
> On Wed, 2003-08-27 at 07:59, Guillem Jover wrote:
> > I'm trying to fix #174035 (glide3-alpha FTBFS), so I've unified both
> > packages and now glide supports alpha. But escher.d.o is down and I
> > cannot build it. Can someone try building it ?
> > And also I would like someone with an Alpha and a 3Dfx graphic card
> > to install and test it with a 3D program and give me the output of
> > glxinfo.

> > The apt repo is:
> > 
> > deb http://www.hadrons.org/~guillem/debian sid/binary/
> > deb-src http://www.hadrons.org/~guillem/debian sid/source/
> > 
> 
> Perhaps, if you send me some quick&dirty instructions, I can do it
> sooner.

I forgot, you'll need a unstable (sid) system if you have stable
or testing, please use debootstrap to generate one and chroot and
build there.

Ok a step-by-step guide. Once you have added the deb lines and done a
apt-get update and installed build-essential and fakeroot, as a normal
user do:

apt-get source glide
cd glide-2002.04.10
fakeroot debian/rules binary 2>&1 | tee ../glide-build.log

And in the upper dir should be two .deb, libglide3 and libglide3-dev.
Install them. (Re)start the X System and run:

glxinfo > glide-glxinfo.log

Also run some 3D application and see if it renders correctly, tuxracer
for example.

Then send me all logs.

Gianluca, I think you have a broken source dir or an old dbs (at least
0.24). Try removing the source dir and unpacking with:

dpkg-source -x glide_2002.04.10-3.dsc

and then build with the previous fakeroot line.

Thanks a lot to both,
guillem

glide: added alpha support, need testers

[Guillem Jover]

[ Please respect the MFT as I'm not suscribed ]

Hello,

I'm trying to fix #174035 (glide3-alpha FTBFS), so I've unified both
packages and now glide supports alpha. But escher.d.o is down and I
cannot build it. Can someone try building it ?
And also I would like someone with an Alpha and a 3Dfx graphic card
to install and test it with a 3D program and give me the output of
glxinfo.

If noone has the hardware, but it builds I'll upload in a few days and
ask for glide3-alpha removal, as upstream we are unifiyng with 64bit
support anyway.

The apt repo is:

deb http://www.hadrons.org/~guillem/debian sid/binary/
deb-src http://www.hadrons.org/~guillem/debian sid/source/

thanks,
guillem