Re: [Stretch] Status for architecture qualification

[Marc Rosen]

If it would be helpful, I have access to two Sun T2000 machines, in a 
server room, that are in good working condition (one has a bad RAM chip, 
but everything else seems to work fine). If it would be helpful for the 
project, I'd be happy to have them be used. That being said, the 
machines are currently off, and I probably won't be able to access the 
LOC until September (or very late august).


~ Marc Rosen

Sysadmin of the JHU ACM

On 6/5/16 5:48 AM, Niels Thykier wrote:

John Paul Adrian Glaubitz:

Hi Niels!

On 06/05/2016 12:01 PM, Niels Thykier wrote:

Beyond mips64el, we are not aware of any new architectures for Stretch.

I kindly ask you to:

  * Porters, please assert if your architecture is targeting Stretch.

To give some insight what's happening in Debian Ports. We have two candidates 
which
I think would qualify for inclusion in a stable release. There is also a third
potential candidate for future releases of Debian once the hardware has become
available.


Thanks. :)


ppc64:

[...]

AFAICT, it is not in unstable and I have not heard of any attempts to
bootstrap it there yet.  Who is working on it and what is the ETA?


sparc64:

[...]


Thanks for the update.  For every one working on this, please keep in
mind that it can take quite a while to be set up and included in testing
(even without missing hardware).

If you are unable to acquire (promise of) the necessary hardware within
a month or two, making it into a Stretch will probably be very hard.


sh4:

[...]

Thanks,
Adrian

[...]

While it sounds exciting, I doubt it will be ready for Stretch (I
presume this was the "potential candidate for a future release").

Thanks,
~Niels

Re: [Stretch] Status for architecture qualification

[Niels Thykier]

Steven Chamberlain:
> Hi,
> 

Hi,

> John Paul Adrian Glaubitz wrote:
>> I have invested lots of time and effort to get sparc64 into a usable state 
>> in Debian.
>> We are close to 11.000 installed packages. Missing packages include Firefox,
>> Thunderbird/Icedove, golang and LibreOffice to name the most important ones.
> 
> Is there some way to define 'core'[0] packages as blockers for testing
> migration, and arch release qualification;  but other packages not?
> 

There is no current definition and I doubt it will be trivial to agree
on a definition.  Also, the moment you want to keep the set
self-contained (by including build-depends) it very easily explodes
unless you patch packages to disable "optional" features.

> Many of these ports would be useful if just a base system was released,
> and preferably having stable/security updates for that part (otherwise
> it is difficult for users to try it, developers to work on it, or DSA to
> support buildds for it;  all of which are limitations on ports' further
> growth).
> 

Assuming we use your definition as basis, you probably also want the
packages necessary to support a DSA maintained buildd.  Otherwise it
seems it fail one of your own proposed requirements.

> Trying to have *every* package build and stay built on every port, and
> supported for the lifetime of stable, is a lot of work without much
> purpose sometimes.  And it's unreasonable for any one port to block
> testing migration of a package on all arches, unless it is something
> really essential.
> 
> This might be done either:
>   * in the official archive, with relaxed rules for testing migration
> and more frequently de-crufting of out-of-date packages;

I suspect this will be a lot of work and an uphill battle as the our
current tooling is not really written for this case.  At the very least,
I fear a lot of extra special cases in Britney that I cannot easily deal
with.

>   * creating a mini testing/stable suite based on debian-ports.org?
> where maybe only the core packages are candidates to migrate.
> 

At least short term, this probably a lot more tractable. I am happy to
provide help with setting up a Britney instance for ports.  Though we
would need some way to provide a architecture specific version of the
"core" packages.

> [0]: I'd define core packages as everything needed to install, boot, and
> then build packages on that arch.  The rebootstrap project gives us some
> idea of what those are;  but add to that the kernel and any bootloaders.
> Being able to rebootstrap, should be part of the arch release
> qualification anyway IMHO.
> 
> Regards,
> 

Hmm, the rebootstrap idea is interesting as a new requirement.  I will
look into it.

Thanks,
~Niels





signature.asc
Description: OpenPGP digital signature

Re: [Stretch] Status for architecture qualification

[Oleg Endo]

Hi,

On Sun, 2016-06-05 at 13:26 +0200, John Paul Adrian Glaubitz wrote:

> sh4:
> 

> 
> The two biggest issues with sh4 are currently with binutils and the 
> kernel. binutils has problems when building Qt5:
> 

There is in fact another big elephant in the room, which I have
mentioned several times before...

The atomic model that is currently being used on sh4-linux works only
on SH3* and SH4* single-core systems.  Packages built for the current
sh4-linux will not work on multi-core systems like SH7786...

There should be two distinct builds .. sh4-linux with the current
atomic model (-matomic-model=soft-gusa) and sh4a-linux with the newer 
-matomic-model=hard-llcs.  I think it would be the easiest thing for
Debian to switch the whole thing to SH4A only and drop the older SH4.

I will add a GCC configure option to allow changing the default atomic
model.  Currently it's hardcoded to -matomic-model=soft-gusa for sh3
-linux or sh4*-linux.

Cheers,
Oleg

(GCC SH Maintainer)

Re: [Stretch] Status for architecture qualification

[Holger Levsen]

thanks to everyone explaining arch:any to me :)

-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: [Stretch] Status for architecture qualification

[Steven Chamberlain]

Hi,

John Paul Adrian Glaubitz wrote:
> I have invested lots of time and effort to get sparc64 into a usable state in 
> Debian.
> We are close to 11.000 installed packages. Missing packages include Firefox,
> Thunderbird/Icedove, golang and LibreOffice to name the most important ones.

Is there some way to define 'core'[0] packages as blockers for testing
migration, and arch release qualification;  but other packages not?

Many of these ports would be useful if just a base system was released,
and preferably having stable/security updates for that part (otherwise
it is difficult for users to try it, developers to work on it, or DSA to
support buildds for it;  all of which are limitations on ports' further
growth).

Trying to have *every* package build and stay built on every port, and
supported for the lifetime of stable, is a lot of work without much
purpose sometimes.  And it's unreasonable for any one port to block
testing migration of a package on all arches, unless it is something
really essential.

This might be done either:
  * in the official archive, with relaxed rules for testing migration
and more frequently de-crufting of out-of-date packages;
  * creating a mini testing/stable suite based on debian-ports.org?
where maybe only the core packages are candidates to migrate.

[0]: I'd define core packages as everything needed to install, boot, and
then build packages on that arch.  The rebootstrap project gives us some
idea of what those are;  but add to that the kernel and any bootloaders.
Being able to rebootstrap, should be part of the arch release
qualification anyway IMHO.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: [Stretch] Status for architecture qualification

[Niels Thykier]

John Paul Adrian Glaubitz:
> Hi Niels!
> 
> On 06/05/2016 12:01 PM, Niels Thykier wrote:
>> Beyond mips64el, we are not aware of any new architectures for Stretch.
>>
>> I kindly ask you to:
>>
>>  * Porters, please assert if your architecture is targeting Stretch.
> 
> To give some insight what's happening in Debian Ports. We have two candidates 
> which
> I think would qualify for inclusion in a stable release. There is also a third
> potential candidate for future releases of Debian once the hardware has become
> available.
> 

Thanks. :)

> ppc64:
> 
> [...]

AFAICT, it is not in unstable and I have not heard of any attempts to
bootstrap it there yet.  Who is working on it and what is the ETA?

> 
> sparc64:
> 
> [...]
> 

Thanks for the update.  For every one working on this, please keep in
mind that it can take quite a while to be set up and included in testing
(even without missing hardware).

If you are unable to acquire (promise of) the necessary hardware within
a month or two, making it into a Stretch will probably be very hard.

> sh4:
> 
> [...]
> 
> Thanks,
> Adrian
> 
> [...]

While it sounds exciting, I doubt it will be ready for Stretch (I
presume this was the "potential candidate for a future release").

Thanks,
~Niels




signature.asc
Description: OpenPGP digital signature

Re: [Stretch] Status for architecture qualification

[peter green]

On 05/06/16 13:00, Holger Levsen wrote:

On Sun, Jun 05, 2016 at 01:26:39PM +0200, John Paul Adrian Glaubitz wrote:
   

ppc64:

This architecture is basically on par with the release architectures. We have 
over
11.000 packages installed
 

[...]
   

sparc64:
We are close to 11.000 installed packages.
 

I'm not sure whether you are talking about source or binary packages but
sid/amd64 has over 24000 source packages and over 5 binary packages,
so I would call the above "on par". Or what am I missing?

   
I presume he is talking about source packages in the "installed" state 
in wana-build. For comparison this is currently 12153 for amd64 and 
10937 for mips.


(this leads to the interesting conclusion that about half the source 
packages in sid are arch-all only)

Re: How-to: get FFB support in X11 on Debian/sparc64 (was: Re: Framebuffer support in Debian)

[Romain Dolbeau]

2016-03-06 10:00 GMT+01:00 Romain Dolbeau :
> 3) apply the attached patch to the /dist directory to fix some minor
compilation issues

Updated patch with an API change to EnableDisableFBAccess.

Crdially,

-- 
Romain Dolbeau


ffb.1.patch
Description: Binary data

Re: [Stretch] Status for architecture qualification

[Christian Seiler]

On 06/05/2016 02:00 PM, Holger Levsen wrote:
> On Sun, Jun 05, 2016 at 01:26:39PM +0200, John Paul Adrian Glaubitz wrote:
>> ppc64:
>>
>> This architecture is basically on par with the release architectures. We 
>> have over
>> 11.000 packages installed
> [...]
>> sparc64:
>> We are close to 11.000 installed packages. 
> 
> I'm not sure whether you are talking about source or binary packages but
> sid/amd64 has over 24000 source packages and over 5 binary packages,
> so I would call the above "on par". Or what am I missing?

But around 12000 of those source packages only build Arch: all packages.
If you look at amd64's buildd stats in sid, there are ~12000 source
packages in the Installed state:

https://buildd.debian.org/status/architecture.php?a=amd64=sid

i386 also has ~12000; arm64, armhf, armel, powerpc and ppc64 have ~11500
each; mipsel has ~11300 and mips has ~11000.

Arch: all has ~15000 source packages in the Installed state (but that
also counts packages that build both Arch: !all and Arch: all.

From these statistics, sparc64 appears to be a tiny bit behind mips in
terms of archive coverage, but not by much.

Regards,
Christian



signature.asc
Description: OpenPGP digital signature

Re: [Stretch] Status for architecture qualification

[John Paul Adrian Glaubitz]

On 06/05/2016 02:00 PM, Holger Levsen wrote:
> I'm not sure whether you are talking about source or binary packages but
> sid/amd64 has over 24000 source packages and over 5 binary packages,
> so I would call the above "on par". Or what am I missing?

There are just around 12,000 source packages with arch:amd64 in sid:

glaubitz@wuiet:~$ wanna-build -A sparc64 -d unstable --list=installed | wc -l
10828
glaubitz@wuiet:~$ wanna-build -A ppc64 -d unstable --list=installed | wc -l
10990
glaubitz@wuiet:~$ wanna-build -A amd64 -d unstable --list=installed | wc -l
12154
glaubitz@wuiet:~$

The rest is arch:all:

glaubitz@wuiet:~$ wanna-build -A all -d unstable --list=installed | wc -l
15672
glaubitz@wuiet:~$

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



signature.asc
Description: OpenPGP digital signature

Re: [Stretch] Status for architecture qualification

[Holger Levsen]

On Sun, Jun 05, 2016 at 01:26:39PM +0200, John Paul Adrian Glaubitz wrote:
> ppc64:
> 
> This architecture is basically on par with the release architectures. We have 
> over
> 11.000 packages installed
[...]
> sparc64:
> We are close to 11.000 installed packages. 

I'm not sure whether you are talking about source or binary packages but
sid/amd64 has over 24000 source packages and over 5 binary packages,
so I would call the above "on par". Or what am I missing?


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: [Stretch] Status for architecture qualification

[John Paul Adrian Glaubitz]

Hi Niels!

On 06/05/2016 12:01 PM, Niels Thykier wrote:
> Beyond mips64el, we are not aware of any new architectures for Stretch.
> 
> I kindly ask you to:
> 
>  * Porters, please assert if your architecture is targeting Stretch.

To give some insight what's happening in Debian Ports. We have two candidates 
which
I think would qualify for inclusion in a stable release. There is also a third
potential candidate for future releases of Debian once the hardware has become
available.

ppc64:

This architecture is basically on par with the release architectures. We have 
over
11.000 packages installed with some fluctuation due to the frequent necessary 
binNMUs
for the Haskell packages. Moreover, yaboot, the bootloader used on many powerpc
machines is now supporting ppc64, too. So building usable debian-installer CD 
images
should be possible:

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=322540

Although yaboot still currently FTBFS on ppc64, so someone has to look into 
that.

Both openSUSE and Fedora provide official installation images for ppc64, 
upstream
development in the toolchain and kernel is active, too.

> http://dl.fedoraproject.org/pub/fedora-secondary/releases/23/Server/ppc64/iso/
> http://download.opensuse.org/ports/ppc/distribution/13.2/iso/

To set up buildds and porterboxes, virtual machines could be set up on the same
POWER servers that are used to provide powerpc and ppc64el machines, provided
that there are still enough resources available.

sparc64:

Since sparc (32-bit userland, 64-bit kernel) was dropped after Wheezy, 
development
focus shifted to sparc64 which is fully 64-bit. Upstream development was 
recently
picked up again and both kernel and toolchain are again in active development 
with
many developers being employed by Oracle. They have released a reference 
platform
last fall:

> https://oss.oracle.com/projects/linux-sparc/

I have invested lots of time and effort to get sparc64 into a usable state in 
Debian.
We are close to 11.000 installed packages. Missing packages include Firefox,
Thunderbird/Icedove, golang and LibreOffice to name the most important ones. I
haven't looked into golang yet, but Firefox, Thunderbird and LibreOffice have 
good
chances to be working soon. LibreOffice has just some tests failing in the 
testsuite
thanks to James Clarke who has ported the architecture-dependent code from 
sparc to
sparc64. Firefox and Thunderbird require some fixing in the JavaScript engine:

> https://bugzilla.mozilla.org/show_bug.cgi?id=1275204

There are also some pending patches which fix binutils/gold and gcc-5/6 after 
which
even more packages should build without problems:

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=809509
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818934
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792921

Otherwise, many packages mostly fail to build from source because of some bad
programming practices which provokes unaligned access:

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=806208

To set up buildds and porterboxes, we would need new hardware. We have some 
older
SPARC Fire T2000 machines running as buildds plus one beefy SPARC-T5 (192 GiB 
RAM,
192 CPU threads), but those are not really suitable as official DSA machines. I
have reached out to Oracle and asked for hardware donations for that matter.

sh4:

This architecture is also known as SuperH and is actually no longer actively
developed and marketed by its developing company, Renesas.

However, there have been interesting developments for this architecture, it is
becoming open source and therefore potentially interesting for many Debian users
and developers who are concerned with privacy and free computing:

> https://lwn.net/Articles/647636/

This has become possible since - as explained in the article above - all patents
related to SuperH are expiring one after another meaning that the hardware can 
be
reimplemented without infringing any patents.

The current result of this effort is the J-Core project:

> http://j-core.org/

The big advantage of Super-H/J-Core over existing open source architectures like
OpenRISC is that both kernel and toolchain are already fully supported so that
Debian can be installed on these machines without any limitations.

We are currently at around 9100 installed packages, a large number of packages 
will
soon be able to build once I have finished bootstrapping GHC (the Haskell 
compiler)
which takes some time on my older SH-7785LCR hardware (still building for 14 
days).


The two biggest issues with sh4 are currently with binutils and the kernel. 
binutils
has problems when building Qt5:

> https://sourceware.org/bugzilla/show_bug.cgi?id=17739

While the kernel needs some additional syscalls to be wired up in the interface 
which
prevents gtk+3.0_3.20.x being built on sh4:

> https://bugzilla.kernel.org/show_bug.cgi?id=119121

As the SH port of the Linux kernel was recently turned into maintained state 
again
after two 

[Stretch] Status for architecture qualification

[Niels Thykier]

Hi members of DSA, Security, RT and all porters.

While the freeze still seem far away, I think it is time to start with
the architecture qualifications.

For starters, here are the architectures we are aware of:

 * amd64, i386, armel, armhf, arm64, mips, mipsel, powerpc, ppc64el,
   s390x
   - *No* blockers at this time from RT, DSA nor security.
   - s390, ppc64el and all arm ports have DSA concerns.
   - armel has a RT concern about lack of buildds (only 2)

 * mips64el (NEW)
   - No DSA buildd (RT blocker)
   - Rebuild after import not complete (RT Blocker)
   - Not yet in testing (due to the above).

 * kfreebsd-i386, kfreebsd-amd64
   - Not included in Jessie due to lack of sustainable man-power (RT
 blocker)
   - No news of the situation having changed
   - If there is no news on the situation after DebConf16, I will
 assume kfreebsd-* will not target Stretch.

Beyond mips64el, we are not aware of any new architectures for Stretch.

I kindly ask you to:

 * Porters, please assert if your architecture is targeting Stretch.
 * Review the list architectures and the reported blockers/concerns
 * Reply to this mail with updates to these blockers/concerns (if any).
   - DSA/Security/RT: Please use the word "blocker" or "RC" for issues
 that *must* be solved for you to be willing to support the
 architecture.

Thanks,
~Niels




signature.asc
Description: OpenPGP digital signature