Re: armel after Stretch

[Stefan Monnier]

> jenkins.d.n would be the place to put full-system cross-grading tests.
> piuparts would be the place to put per-package cross-grading tests.

FWIW, I just went through a cross-grade of a Lil'Debi chroot from armel
to armhf.  I wouldn't recommend it to someone who's not used to command
line use of dpkg and apt.  But it worked nicely in the end (the
get-selection/set-selection step seemed to create trouble more than
anything, OTOH).


Stefan

Re: armel after Stretch

[Russ Allbery]

Ian Jackson  writes:

> I think I will probably do the i386->amd64 crossgrade on chiark some
> time during the supported life of stretch.  I certainly don't expect
> it to be straightforward and I wouldn't dare trust it to a script.

> Just a data point.

Adding a similar data point for two of my personal servers.

-- 
Russ Allbery (r...@debian.org)   

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Ian Jackson]

Christoph Biedl writes ("Re: armel after Stretch (was: Summary of the ARM ports 
BoF at DC16)"):
> Doing this by hand is of course neither fast nor simple. The migration
> script you requested could change that, however it's a delicate job,
> full of pitfalls, desaster if anything goes wrong, so nobody would want
> to take the burden of maintaining it. I bet there still are a lot of
> hand-crafted i386 systems out there that never made it to amd64 because
> the admins don't dare to. I bet as well these systems are so filled with
> quirks any automated migration will to unable to handle them.

I think I will probably do the i386->amd64 crossgrade on chiark some
time during the supported life of stretch.  I certainly don't expect
it to be straightforward and I wouldn't dare trust it to a script.

Just a data point.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

[ limiting to devel- ]

Wouter Verhelst wrote...

> I think a proper procedure should involve a script that:

[ sane criteria ]

> We currently don't have anything remotely like the above, and I think we
> should.

Yes, but I doubt it would be used a lot. There's a wide-spread culture
of re-installing instead of upgrading, also centralized management for
system configuration and "cloud storage" for users' data make that
even easier.

Even I used architecture migration for a clean-up: New installation in
parallel, then sync the interesting stuff (this requires some additional
hacks to make it more or less carefree). And I always found a lot of
cruft that supports the notion this was the right choice.

Doing this by hand is of course neither fast nor simple. The migration
script you requested could change that, however it's a delicate job,
full of pitfalls, desaster if anything goes wrong, so nobody would want
to take the burden of maintaining it. I bet there still are a lot of
hand-crafted i386 systems out there that never made it to amd64 because
the admins don't dare to. I bet as well these systems are so filled with
quirks any automated migration will to unable to handle them.

So, however is in charge of such a script, they can expect to have a lot
of hard work while constantly receiving flames from people where the
procedure failed. Doesn't quite look tempting.

Christoph


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

Lennart Sorensen wrote...

> I actually highly doubt there are that many armv7 boxes running armel.
> armhf was a nice performance improvement and worth the hassle to reinstall
> if you had such a box in the first place.  I think most armel systems
> are probably armv5, often the marvell chips.  Not sure if anyone is
> running it on Raspberry pi (Original, not 2 or 3) systems (...)

That would be me. If somebody has instructions how to build (or: where
to get) a current u-boot that boots a vanilla kernel, resulting in a
system that does *not* see 8kIRQ/sec, I'll happily take a hint.

At the moment, I run the 4.1 series based on the huge Raspbian patch,
which is quite painful. Forwarding to even 4.4 failed.

Lesson learned: Never buy hardware that's not supported mainline, or
will be in a forseeable time.

Christoph

PS: I think it's about time to restrict this to debian-arm, Reply-To:
set.


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wouter Verhelst]

On Sat, Dec 17, 2016 at 08:15:15PM +0800, Paul Wise wrote:
> On Sat, 2016-12-17 at 09:45 +0100, Wouter Verhelst wrote:
> 
> > Yes, but that still says:
> 
> Ack.
> 
> > I think a proper procedure should involve a script that:
> > 
> > - is packaged in Debian;
> 
> Ack.
> 
> > - checks whether the hardware it's running on has all the hardware
> >   requirements for the new architecture
> 
> apt show arch-test
> 
> > - is properly tested to work in (almost) all situations;
> 
> jenkins.d.n would be the place to put full-system cross-grading tests.
> piuparts would be the place to put per-package cross-grading tests.
> 
> > - is a properly supported way to move from one ABI to another.
> 
> What do you mean by "properly supported"?

Where we don't go "it might break, and if it does, you get to keep the
pieces", but instead "if it breaks, that's a bug and we will fix it, and
try to help you recover from that".

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
   people in the world who think they really understand all of its rules,
   and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Sun, Dec 18, 2016 at 11:22 PM, Roger Shimizu wrote:

> Is there any way to simplify?

Remove the obsolete armel binaries where they occur and then mark the
packages as NFU on armel:

https://wiki.debian.org/ftpmaster_Removals
https://wiki.debian.org/PackagesArchSpecific

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

On Sat, Dec 10, 2016 at 12:22 PM, Wookey  wrote:
>
> We can do poor-mans partial arch by just being fairly agressive about
> disabling armel for packages that are broken or not suitable. Not very
> clever or efficient, but it is easy to do and requires no infra or
> tooling changes at all. So long as someone is volunteering for that
> (easy but unexciting) work that could work.

If I understand correctly, you mean for those packages broke on armel
(or to be broken in the future), we can modify debian/control as following:

From:
  Architecture: any
To:
  Architecture: any [!armel]

I'm not sure whether statement above works or not, because debian
policy didn't mention this [0][1].
It only mentioned statement like "any" or "linux-any".

If we cannot use "any [!armel]", we have to list all the ARCHes
without armel, so it's horrible like:
  Architecture: alpha, amd64, arm64, armhf, hppa, i386, m68k, mips,
mips64, mips64el, mipsel, mipsn32, mipsn32el, or1k, powerpc,
powerpcspe, ppc64, ppc64el, s390, s390x, sh4, sparc, sparc64, tilegx,
x32

And we need to write a few ARCHes that may be supported later [2]:
mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el

Is there any way to simplify?

[0] 
https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Architecture
[1] 
https://www.debian.org/doc/debian-policy/ch-customized-programs.html#s-arch-spec
[2] https://anonscm.debian.org/cgit/kernel/linux.git/tree/debian/config/defines

Cheers,
--
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Sat, 2016-12-17 at 09:45 +0100, Wouter Verhelst wrote:

> Yes, but that still says:

Ack.

> I think a proper procedure should involve a script that:
> 
> - is packaged in Debian;

Ack.

> - checks whether the hardware it's running on has all the hardware
>   requirements for the new architecture

apt show arch-test

> - is properly tested to work in (almost) all situations;

jenkins.d.n would be the place to put full-system cross-grading tests.
piuparts would be the place to put per-package cross-grading tests.

> - is a properly supported way to move from one ABI to another.

What do you mean by "properly supported"?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Adam Borowski]

On Sat, Dec 17, 2016 at 09:45:01AM +0100, Wouter Verhelst wrote:
> On Thu, Dec 15, 2016 at 11:19:34AM +0800, Paul Wise wrote:
> > On Thu, Dec 15, 2016 at 1:40 AM, Wouter Verhelst wrote:
> > 
> > > One way in which the need to keep armel around would be reduced is if we
> > > could somehow upgrade from armel machines to armhf ones, without
> > > requiring a reinstall.
> > 
> > There is a script for that here:
> > 
> > https://wiki.debian.org/CrossGrading

Considering my inept attempts to make such a script, this one is way too
fragile to actually use.  In my experience, you need to semi-manually (dpkg
-i) convert the transitively-essential set, as otherwise apt will either
throw its hands up or propose "remove world" as a solution.

> Yes, but that still says:
> 
> A full backup is also strongly recommended

Duh.  That'd be true even if it was a supported, well-tested operation.

> I think a proper procedure should involve a script that:
> 
> - is packaged in Debian;

I doubt we can make one and have it in unstable before Xmas (the NEW
freeze).

> - checks whether the hardware it's running on has all the hardware
>   requirements for the new architecture (i.e., in case of armel to
>   armhf, can support the ARM ABI required by armhf; or in case of i386
>   to amd64, is a processor that supports the x86_64 ABI), and produces a
>   proper error message in case the required support isn't there;

Done!  This is exactly what "arch-test" is for.

> - is properly tested to work in (almost) all situations;

Ha ha ha!

> - is a properly supported way to move from one ABI to another.
> 
> We currently don't have anything remotely like the above, and I think we
> should.

Here's my very early WIP: https://github.com/kilobyte/crossgrade
It does only a bunch of checks, compares versions, builds and installs the
essential set but doesn't go anywhere further.

It also tries to reinvent the wheel in order to be resilient wrt partially
aborted upgrades, I now realize I could have considered apt-perl libraries
"essential" so they'd always work.

And it's not like crossgrading is easily possible anyway:
Unpacking libpam-modules:amd64 (1.1.8-3.3) ...
dpkg: error processing archive 
/tmp/apt-dpkg-install-afa8lY/0-libpam-modules_1.1.8-3.3_amd64.deb (--unpack):
 trying to overwrite shared '/usr/share/man/man8/pam_exec.8.gz', which is 
different from other instances of package libpam-modules:amd64

At least my recovery code works properly, and after manual
  dpkg --force-overwrite 
/var/cache/apt/archives/libpam-modules_1.1.8-3.3_amd64.deb
it resumes and succeeds until the end of implemented part.

Then there is the multiarch interpreter problem.


> [1] save that, I think, at the time multiarch didn't exist yet. Yes,
> this was an "interesting" experience :-)

Even for release architectures, binNMU versions are often not in sync.  When
you want something second-class (most of my crossgrading at home was to and
from x32), multiarch works only in theory not practice.


Meow!
-- 
Autotools hint: to do a zx-spectrum build on a pdp11 host, type:
  ./configure --host=zx-spectrum --build=pdp11

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wouter Verhelst]

On Thu, Dec 15, 2016 at 11:19:34AM +0800, Paul Wise wrote:
> On Thu, Dec 15, 2016 at 1:40 AM, Wouter Verhelst wrote:
> 
> > One way in which the need to keep armel around would be reduced is if we
> > could somehow upgrade from armel machines to armhf ones, without
> > requiring a reinstall.
> 
> There is a script for that here:
> 
> https://wiki.debian.org/CrossGrading

Yes, but that still says:

A full backup is also strongly recommended as this procedure is still very
much work in progress. Reinstalling is still the safer option. You have
been warned! 

I think a proper procedure should involve a script that:

- is packaged in Debian;
- checks whether the hardware it's running on has all the hardware
  requirements for the new architecture (i.e., in case of armel to
  armhf, can support the ARM ABI required by armhf; or in case of i386
  to amd64, is a processor that supports the x86_64 ABI), and produces a
  proper error message in case the required support isn't there;
- is properly tested to work in (almost) all situations;
- is a properly supported way to move from one ABI to another.

We currently don't have anything remotely like the above, and I think we
should.

[1] save that, I think, at the time multiarch didn't exist yet. Yes,
this was an "interesting" experience :-)

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
   people in the world who think they really understand all of its rules,
   and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Re: Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Juan Carlos Romero]

Hello.


recent u-boot for kirkwood armel devices:

http://forum.doozan.com/read.php?3,12381


Regards

-- 
Nos vemos en los bares.

J. Carlos

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Vagrant Cascadian]

On 2016-12-16, Roger Shimizu  wrote:
> On Sat, Dec 10, 2016 at 11:31 PM, Christoph Biedl
>  wrote:
>>> Is it possible to put a bootloader like u-boot in the flash partitions
>>> and have it load the Linux kernel and initrd from elsewhere?

There's no technical reason this wouldn't be possible, just a matter of
getting patches in upstream u-boot for that platform adding support for
loading from other media (SD, USB, sata, etc.), filesystem support and
so on, and staying withint the size constraints for that platform.

You might be able to use an SPL loader as a first-stage loader early in
the boot process to load a more capable u-boot from other flash
partitions and/or other media and/or filesystems. With the space
reserved for a linux kernel on flash media, that would presumably give
quite a bit of space for a full-featured u-boot.


>> That how I've been running my Dockstars through all the years. As as
>> far as I know this worked with the Debian kernels as well (I use my
>> own kernels for reasons).
>
> If I understand correctly, it means boot like:
>   u-boot shipped by original vendor => self built u-boot => kernel
> (Debian's or your own customized one)

While technically possible, u-boot upstream discourages chainloaded
u-boot. So while you could maybe make patches to get it working for a
particular board and set of vendor and upstream u-boot versions, I'm not
sure if patches would be accepted for upstream u-boot.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

[CC Vagrant, u-boot pkg maintainer]

On Sat, Dec 10, 2016 at 11:31 PM, Christoph Biedl
 wrote:
> Paul Wise wrote...
>>
>> Is it possible to put a bootloader like u-boot in the flash partitions
>> and have it load the Linux kernel and initrd from elsewhere?
>
> That how I've been running my Dockstars through all the years. As as
> far as I know this worked with the Debian kernels as well (I use my
> own kernels for reasons).

If I understand correctly, it means boot like:
  u-boot shipped by original vendor => self built u-boot => kernel
(Debian's or your own customized one)

I'm very interesting to this idea.
Is there any manual or blog post I can take reference?

Thank you!
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Thu, Dec 15, 2016 at 1:40 AM, Wouter Verhelst wrote:

> One way in which the need to keep armel around would be reduced is if we
> could somehow upgrade from armel machines to armhf ones, without
> requiring a reinstall.

There is a script for that here:

https://wiki.debian.org/CrossGrading

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Lennart Sorensen]

On Wed, Dec 14, 2016 at 06:40:22PM +0100, Wouter Verhelst wrote:
> On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:
> [...asking for armel to be retained...]
> 
> One way in which the need to keep armel around would be reduced is if we
> could somehow upgrade from armel machines to armhf ones, without
> requiring a reinstall.
> 
> After all, armel has been around longer than armhf has, which means that
> there may be some machines out in the wild that were installed (and
> upgraded) when armel existed but armhf did not yet (or at least, was not
> stable yet). Some of those machines might be armv7 machines that would
> be perfectly capable of running the armhf port, except that it wasn't
> around yet when they were first installed, and switching to armhf
> without reinstalling isn't possible.
> 
> I once did try to do a similar migration on my Thecus (from arm to
> armel, rather than armel to armhf), but that failed miserably; and since
> I hadn't installed the firmware update to be able to access the console
> so as to figure out what went wrong, that essentially bricked the
> machine.
> 
> If there was a supported and tested way to upgrade older armel
> installations on hardware that actually works with armhf, then those
> machines wouldn't need to be able to run armel anymore, and part of this
> problem would go away...

I actually highly doubt there are that many armv7 boxes running armel.
armhf was a nice performance improvement and worth the hassle to reinstall
if you had such a box in the first place.  I think most armel systems
are probably armv5, often the marvell chips.  Not sure if anyone is
running it on Raspberry pi (Original, not 2 or 3) systems or if those
all run Rasbian armhf instead.

-- 
Len Sorensen

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wouter Verhelst]

On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:
[...asking for armel to be retained...]

One way in which the need to keep armel around would be reduced is if we
could somehow upgrade from armel machines to armhf ones, without
requiring a reinstall.

After all, armel has been around longer than armhf has, which means that
there may be some machines out in the wild that were installed (and
upgraded) when armel existed but armhf did not yet (or at least, was not
stable yet). Some of those machines might be armv7 machines that would
be perfectly capable of running the armhf port, except that it wasn't
around yet when they were first installed, and switching to armhf
without reinstalling isn't possible.

I once did try to do a similar migration on my Thecus (from arm to
armel, rather than armel to armhf), but that failed miserably; and since
I hadn't installed the firmware update to be able to access the console
so as to figure out what went wrong, that essentially bricked the
machine.

If there was a supported and tested way to upgrade older armel
installations on hardware that actually works with armhf, then those
machines wouldn't need to be able to run armel anymore, and part of this
problem would go away...

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
   people in the world who think they really understand all of its rules,
   and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wookey]

On 2016-12-13 23:42 +0100, Adam Borowski wrote:
> On Tue, Dec 13, 2016 at 09:21:48PM +0100, Christoph Biedl wrote:
> > W. Martin Borgert wrote...
> > > The forementioned hardware needs < 0.5 W, the manufacturer even
> > > claims 0.18 W. AFAIK, most newer ARM boards that are capable to
> > > run Debian need more energy or am I wrong?
> > 
> > So let me play the devil's advocate another time: My Dockstar runs
> > 24/7 and allegedly consumes 5 watts. Replacing it with a board that
> > takes a tenth, the electricity bill will be ten euros less. Depending
> > on the price for the replacement, that might be worth a thought.
> > 
> > It certainly is if you're still running a WRT54G at some 15 watts

15 really? This page says 3-8W. Fairly sure mine was more like 9 (and
the replacement ~4) https://forum.openwrt.org/viewtopic.php?id=2162
 
> Don't forget that your time spent ordering a replacement, configuring it,
> replacing the OS, etc, is not free either.

I think Martin is counting Energy/emissions, not cost. So extra time
spent making lower-power stuff work is investment in efficiency
(within reason). 

> > > (Furthermore, any replacement of hardware has many environmental
> > > effects apart of energy consumption: Use of rare materials,
> > > production side effects, transport, waste problems, etc.)
> > 
> > Controlling does not care beyond the bills. And there might be
> > transition costs as well (testing new hardware, deployment etc).

It seems that you misunderstand: the object of the exercise is not
(money) cost-minimisation, but environmental cost
minimisation. Ultimately we can make as much money as we like (it's
just numbers), other resources are much more limited, which is why I
prefer Mr Borgert's methodology.
 
> If the old box still matches your needs, it's almost never cost-effective to
> replace it, unless you manage a fleet of those and can do the replacement in
> batches.

This is not true. For example, an old PC using 70W is always worth
replacing ASAP with a 5W arm box if you can, assuming that you still
need the service for long enough to cover the embodied emissions of
the new box (which won't be very long at 65W difference, how long
exactly depending on supply carbon intensity (~6 months in the UK).

(Perhaps getting a little OT now.)

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Kurt Roeckx]

On Wed, Dec 07, 2016 at 03:53:31PM +, Steve McIntyre wrote:
> AFAIK there are potentially still similar problems with ARMv5 - lack
> of architcture-defined barrier primitives for C++11 atomics to
> work. (I'd love to be corrected on this if people know better!) This
> is one of the key points here. More and more software is expecting to
> use these primitives, and a lack of them is a major problem. To
> demonstrate using gcc, you can see that lock-free atomics only started
> appearing in ARMv6 and were improved in ARMv7:
> 
> $ for arch in 4 5 6 7-a; do echo ARMv${arch}; echo | g++ -march=armv${arch} 
> -dM -E - | grep -i lock_free; done
> ARMv4
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 1
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 1
> #define __GCC_ATOMIC_INT_LOCK_FREE 1
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LONG_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 1
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 1
> ARMv5
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 1
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 1
> #define __GCC_ATOMIC_INT_LOCK_FREE 1
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LONG_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 1
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 1
> ARMv6
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 1
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 2
> #define __GCC_ATOMIC_INT_LOCK_FREE 2
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
> #define __GCC_ATOMIC_LONG_LOCK_FREE 2
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 1
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 1
> ARMv7-a
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 2
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 2
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 2
> #define __GCC_ATOMIC_INT_LOCK_FREE 2
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
> #define __GCC_ATOMIC_LONG_LOCK_FREE 2
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 2
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 2
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 2

What you're actually showing is that even for ARMv4 they are
sometimes lock free by using the kernel support.

> There are kernel helpers available to provide some atomic support, but
> they'll be very slow compared to real hardware support at this level.

I was under the impression that that's not the case:
https://lwn.net/Articles/314561/


Kurt

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Adam Borowski]

On Tue, Dec 13, 2016 at 09:21:48PM +0100, Christoph Biedl wrote:
> W. Martin Borgert wrote...
> > The forementioned hardware needs < 0.5 W, the manufacturer even
> > claims 0.18 W. AFAIK, most newer ARM boards that are capable to
> > run Debian need more energy or am I wrong?
> 
> So let me play the devil's advocate another time: My Dockstar runs
> 24/7 and allegedly consumes 5 watts. Replacing it with a board that
> takes a tenth, the electricity bill will be ten euros less. Depending
> on the price for the replacement, that might be worth a thought.
> 
> It certainly is if you're still running a WRT54G at some 15 watts
> where a TP-Link 741 costs less than 20 euros and takes some two or
> three watts.

A rough rule of thumb is that 1 watt-year costs $1, this is approximately
true in most of the world.  US is $1, China/India are at $0.7, Poland is at
$1.2; unfortunately for you Germany is an outlier at $3.  (All of these
prices are for residential customers, taken from three random pages on the
Interwebs.)

Don't forget that your time spent ordering a replacement, configuring it,
replacing the OS, etc, is not free either.

> > (Furthermore, any replacement of hardware has many environmental
> > effects apart of energy consumption: Use of rare materials,
> > production side effects, transport, waste problems, etc.)
> 
> Controlling does not care beyond the bills. And there might be
> transition costs as well (testing new hardware, deployment etc).

If the old box still matches your needs, it's almost never cost-effective to
replace it, unless you manage a fleet of those and can do the replacement in
batches.

> Nevertheless, there is a point where supporting old hardware makes
> little to no sense. Defining that point is hard and includes personals
> preferences as well. Given the arguments in this thread I'm less sure
> armel already is at that point but it surely will come.

Yeah, the time of volunteers (porters) is not free either.  As the ARM team
decided they don't want to do the work, and no one else stepped up, you have 
five
more years of support for that armel box.  Ie, there's no rush to replace it,
it's pretty likely it will stop being adequate for your needs before then,
and if not, well, assume the box will sort-of break by then.  Losing
security support is a nice sort of failure as you're warned years in
advance.


Meow!
-- 
u-boot problems can be solved with the help of your old SCSI manuals, the
parts that deal with goat termination.  You need a black-handled knife, and
an appropriate set of candles (number and color matters).  Or was it a
silver-handled knife?  Crap, need to look that up.

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

W. Martin Borgert wrote...

> The forementioned hardware needs < 0.5 W, the manufacturer even
> claims 0.18 W. AFAIK, most newer ARM boards that are capable to
> run Debian need more energy or am I wrong?

So let me play the devil's advocate another time: My Dockstar runs
24/7 and allegedly consumes 5 watts. Replacing it with a board that
takes a tenth, the electricity bill will be ten euros less. Depending
on the price for the replacement, that might be worth a thought.

It certainly is if you're still running a WRT54G at some 15 watts
where a TP-Link 741 costs less than 20 euros and takes some two or
three watts.

> (Furthermore, any replacement of hardware has many environmental
> effects apart of energy consumption: Use of rare materials,
> production side effects, transport, waste problems, etc.)

Controlling does not care beyond the bills. And there might be
transition costs as well (testing new hardware, deployment etc).

Nevertheless, there is a point where supporting old hardware makes
little to no sense. Defining that point is hard and includes personals
preferences as well. Given the arguments in this thread I'm less sure
armel already is at that point but it surely will come.

On the other hand it hurts to see Debian will no longer support
hardware that is still being sold. For me, it's a deja-vu: ARMv4
boards (v4 as in: No thumb) were sold until at least 2009 in some NAS
boxes. When I started playing with it, arm(OABI) left the building.

Christoph


signature.asc
Description: Digital signature

Re: armel after Stretch

[Steve McIntyre]

Roger Shimizu wrote:
>On Thu, Dec 8, 2016 at 12:53 AM, Steve McIntyre  wrote:
>>
>> There are kernel helpers available to provide some atomic support, but
>> they'll be very slow compared to real hardware support at this level.
>
>Are those kernel helper already reached Debian?
>Or there's still some work here?

As Ben said - they're in the kernel already. But various packages
*may* still need port work to use them.

>> It's a similar thing, but further up the curve - that's all. We added
>> armhf (as the equivalent of i686, maybe) a while back, targeting
>> ARMv7. The much older CPU designs based on ARMv4 and ARMv5 are getting
>> harder to support than the i586 here. The world has moved on,
>> basically.
>
>Just like the world already moves on to amd64, but Debian still
>support i386 and i686,

Not so much - we don't support *actual* i386 machines any more, nor
i486 nor most i586.

>I think the community need armel and armhf, for quite a long time if
>it's feasible.
>
>> You're the first armel developer to offer to dive in here, so that's a
>> good start!
>
>Thanks, but my knowledge don't cover the lower part such as building 
>toolchains,
>still need someone with more experience.

OK.

>>  * *If* we wanted to try the partial architecture thing, that will
>>need some effort to make it work. That's not well-defined right
>>now, as it's only a vague concept at best (sorry!)
>
>Maybe we can avoid to do partial arch?

As it's still very much a vague concept that'd be easier, yes. :-)

>>  * There are going to be some packages that just won't work,
>>particularly JIT compilers and other code generators that assume
>>ARM == ARMv7. Fixing those up might raneg between feasible and
>>~impossible depending on the size of the codebase...
>
>If something breaks, I guess it breaks now already.
>We need to fix before Stretch.
>If the issue is fixed, I think there's no need to remove armel
>immediately after releasing Stretch.

That will be up to the release team, basically. I'm not going to be
actively pushing for the removal of armel personally, but as we've
seen recently (with the removal of powerpc for stretch) what matters
is having multiple active porters working on fixing things steadily
throughout the release cycle. So long as we (you!) can demonstrate
that, armel can survive.

It's not wonderful for users that things go EOL over time, but it
happens. We're also giving people plenty of warning that this might be
coming, so interested people can step up and provide that support to
keep things going.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
< sladen> I actually stayed in a hotel and arrived to find a post-it
  note stuck to the mini-bar saying "Paul: This fridge and
  fittings are the correct way around and do not need altering"

Re: armel after Stretch

[Steve McIntyre]

On Fri, Dec 09, 2016 at 06:42:19PM -1000, Julien Cristau wrote:
>On 12/09/2016 05:22 PM, Wookey wrote:
>> We can do poor-mans partial arch by just being fairly agressive about
>> disabling armel for packages that are broken or not suitable. Not very
>> clever or efficient, but it is easy to do and requires no infra or
>> tooling changes at all. So long as someone is volunteering for that
>> (easy but unexciting) work that could work.
>> 
>We wouldn't necessarily want to call the result a Debian release, though.

Nod. Also (AIUI) fairly likely to break release work for testing
migration etc. unless people are very careful...

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
< Aardvark> I dislike C++ to start with. C++11 just seems to be
handing rope-creating factories for users to hang multiple
instances of themselves.

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Timo Jyrinki]

2016-12-09 0:12 GMT+02:00 Christoph Biedl :
> Same here. My Dockstars (orion5x/kirkwood) still work like a charm and
> it gives a bad feeling having to trash them some day just because
> there's no support any more.
>
> On the other hand, they face another problem I guess is typical for
> that generation, just by the age: Memory.

There are new kirkwood devices still being produced and my <2y old one
(and still on sale in some places) is 2.0GHz one with 1GB RAM.

Just pointing out that ARMv5 is less a certain generation of CPUs as
such but a selection of instruction set by a chip manufacturer. It
will of course become rarer over time.

Thanks to all who have maintained armel so far, I'll be a happy user
for the duration of stretch support.

-Timo

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Stephen Kitt]

On Sun, 11 Dec 2016 09:14:09 +0100, Marc Haber 
wrote:
> And, while we're at it, how is "locale" pronounced? I have a native
> speaker in my filter bubble who claims that it's short for "local
> environment" and therefore pronounced "local e", but that sounds wrong
> for me. Is the example pronouncement given on
> http://dict.leo.org/dictQuery/m-vocab/ende/de.html?searchLoc=0=ende=de=0=off=locale=basic=on
> correct even in AmE?

Yes, locale isn't a neologism; it refers to a place:
https://www.merriam-webster.com/dictionary/locale

The pronunciation given on LEO is correct.

Regards,

Stephen


pgpazyIkvLdan.pgp
Description: OpenPGP digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Marc Haber]

On Thu, 8 Dec 2016 23:12:20 +0100, Christoph Biedl
 wrote:
>Locale generation needs a lot of RAM. You can work around it by
>installing locales-all which however takes long time to install on
>slow flash drives. Or disable locales entirely. Err.

Or build a locale-local package that only contains the locales you
want. Does the locales package by any chance offer infrastructure to
do that (like a build-locale-deb command, maybe)?

And, while we're at it, how is "locale" pronounced? I have a native
speaker in my filter bubble who claims that it's short for "local
environment" and therefore pronounced "local e", but that sounds wrong
for me. Is the example pronouncement given on
http://dict.leo.org/dictQuery/m-vocab/ende/de.html?searchLoc=0=ende=de=0=off=locale=basic=on
correct even in AmE?

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[W. Martin Borgert]

On 2016-12-10 21:40, Adam Borowski wrote:
> Once you go down from 3GB memory that P4 in my cellar has to 128MB your
> armel box is limited to, the number of uses gets sharply limited.  Even
> simple "apt update" takes ages.

The (multiple!) machines with 128 MB we run, have a dedicated
repository with only few hundreds of packages. This way apt is
"fast enough" and does not run oom.
Thanks, germinate and reprepro!

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Adam Borowski]

On Sat, Dec 10, 2016 at 05:54:29PM +0100, Christoph Biedl wrote:
> Certainly, with some limits though. At some point new hardware is that
> much more energy efficient the inital cost pays off over the intended
> time of usage. Want my old P4 server?

That P4 server can drive a bunch of disks and serve as an adequate
backup/etc endpoint.  You don't need a big CPU or gobs of memory for that
task, its CPU will stay mostly idle too.  And more importantly, it can run
regular current amd64 Debian.  Heck, I run 15 vservers on mine and I see no
performance needs it doesn't meet.  Obviously, I'm not insane enough to try
to run compiles, etc -- I got actual modern machines for that, there's just
no need to upgrade this one.

Once you go down from 3GB memory that P4 in my cellar has to 128MB your
armel box is limited to, the number of uses gets sharply limited.  Even
simple "apt update" takes ages.

As it takes $20-30 to buy an armhf or arm64 SoC with 2GB memory and a CPU
that runs circles around both our machines, it's no longer cost effective to
try to keep that old armel thing running -- I'm speaking about human effort
not the negligible monetary cost.


Meow!
-- 
u-boot problems can be solved with the help of your old SCSI manuals, the
parts that deal with goat termination.  You need a black-handled knife, and
an appropriate set of candles (number and color matters).  Or was it a
silver-handled knife?  Crap, need to look that up.

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[W. Martin Borgert]

On 2016-12-10 17:54, Christoph Biedl wrote:
> Maximum RAM is 128 Mbytes. Wouldn't buy this to run Debian on it.

Depends on the use case. My employer is using it successfully
since years. Of course: No X, no database, no big data, but some
hungry Pythons and a web interface :~)

> At some point new hardware is that
> much more energy efficient the inital cost pays off over the intended
> time of usage. Want my old P4 server?

The forementioned hardware needs < 0.5 W, the manufacturer even
claims 0.18 W. AFAIK, most newer ARM boards that are capable to
run Debian need more energy or am I wrong?

(Furthermore, any replacement of hardware has many environmental
effects apart of energy consumption: Use of rare materials,
production side effects, transport, waste problems, etc.)

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

W. Martin Borgert wrote...

> Quoting Ben Hutchings :
> >Also, dedicated tiny flash partitions for the kernel and initrd.  I
> >wouldn't be surprised to be find that by the time we want to release
> >buster we can't build a useful kernel that fits into the 2 MB partition
> >that most of these devices seem to have.
> 
> Non-HF devices can be very different, look at e.g. this ARM926EJ-S one:
> http://www.taskit.de/stamp9g20.html

Maximum RAM is 128 Mbytes. Wouldn't buy this to run Debian on it.

> >As it is, stretch will be supported until 2020, maybe 2022 on armel.
> >Is it really worthwhile to add another 2 years to that?
> 
> This depends on the effort, of course. But for environmental reasons
> I'ld say the longer the better.

Certainly, with some limits though. At some point new hardware is that
much more energy efficient the inital cost pays off over the intended
time of usage. Want my old P4 server?

Christoph


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

Paul Wise wrote...

> On Fri, Dec 9, 2016 at 8:53 AM, Ben Hutchings wrote:
> 
> > Also, dedicated tiny flash partitions for the kernel and initrd.  I
> > wouldn't be surprised to be find that by the time we want to release
> > buster we can't build a useful kernel that fits into the 2 MB partition
> > that most of these devices seem to have.
> 
> Is it possible to put a bootloader like u-boot in the flash partitions
> and have it load the Linux kernel and initrd from elsewhere?

That how I've been running my Dockstars through all the years. As as
far as I know this worked with the Debian kernels as well (I use my
own kernels for reasons).

Christoph


signature.asc
Description: Digital signature

Re: armel after Stretch

[Julien Cristau]

On 12/09/2016 05:22 PM, Wookey wrote:
> We can do poor-mans partial arch by just being fairly agressive about
> disabling armel for packages that are broken or not suitable. Not very
> clever or efficient, but it is easy to do and requires no infra or
> tooling changes at all. So long as someone is volunteering for that
> (easy but unexciting) work that could work.
> 
We wouldn't necessarily want to call the result a Debian release, though.

Cheers,
Julien

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Ben Hutchings]

On Sat, 2016-12-10 at 09:54 +0800, Paul Wise wrote:
> On Fri, Dec 9, 2016 at 8:53 AM, Ben Hutchings wrote:
> 
> > Also, dedicated tiny flash partitions for the kernel and initrd.  I
> > wouldn't be surprised to be find that by the time we want to release
> > buster we can't build a useful kernel that fits into the 2 MB partition
> > that most of these devices seem to have.
> 
> Is it possible to put a bootloader like u-boot in the flash partitions
> and have it load the Linux kernel and initrd from elsewhere?

It may well be, and that has been proposed before, but someone will
need to do the work to make flash-kernel upgrade (or at least detect
and support upgrades) to this configuration.

Ben.

-- 
Ben Hutchings
Logic doesn't apply to the real world. - Marvin Minsky



signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wookey]

On 2016-12-07 15:53 +, Steve McIntyre wrote:
> On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:

> >I'm ARM porter on armel/marvell (orion5x/kirkwood).
> >Stretch will be frozen and released soon, which makes me bit depressed, 
> >because it means armel will be dropped out of unstable/testing as the 
> >conclusion of Cape Town BoF.
> >
> >> Possible future options for armel:
> >> 
> >>  * Partial armel architecture?
> >> 
> >>We've talked about this in the past for various architectures, but
> >>nobody has stepped up to work on it. The vast majority of the
> >>outstanding armel use cases are going to be in headless servers so
> >>we could maybe drop the desktop tasks and dependencies and keep
> >>things like web server / mail server tasks that are much simpler.

> >>Downside: There's no clear plan for how to create/maintain a
> >>partial architecture, let alone how to release one. Potentially
> >>huge amount of work needed.

We can do poor-mans partial arch by just being fairly agressive about
disabling armel for packages that are broken or not suitable. Not very
clever or efficient, but it is easy to do and requires no infra or
tooling changes at all. So long as someone is volunteering for that
(easy but unexciting) work that could work.

Obviously something fancier and more centralised/general would be
'better' but it requires a different skill-set and realistically will
probably take a long time.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Fri, Dec 9, 2016 at 8:53 AM, Ben Hutchings wrote:

> Also, dedicated tiny flash partitions for the kernel and initrd.  I
> wouldn't be surprised to be find that by the time we want to release
> buster we can't build a useful kernel that fits into the 2 MB partition
> that most of these devices seem to have.

Is it possible to put a bootloader like u-boot in the flash partitions
and have it load the Linux kernel and initrd from elsewhere?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: armel after Stretch

[Christian Seiler]

On 12/09/2016 01:53 AM, Ben Hutchings wrote:
> On Thu, 2016-12-08 at 23:12 +0100, Christoph Biedl wrote:
>> Roger Shimizu wrote...
>>
>>> I'm ARM porter on armel/marvell (orion5x/kirkwood).
>>> Stretch will be frozen and released soon, which makes me bit depressed, 
>>> because it means armel will be dropped out of unstable/testing as the 
>>> conclusion of Cape Town BoF.
>>
>> Same here. My Dockstars (orion5x/kirkwood) still work like a charm and
>> it gives a bad feeling having to trash them some day just because
>> there's no support any more.
>>
>> On the other hand, they face another problem I guess is typical for
>> that generation, just by the age: Memory.
> [...]
> 
> Also, dedicated tiny flash partitions for the kernel and initrd.  I
> wouldn't be surprised to be find that by the time we want to release
> buster we can't build a useful kernel that fits into the 2 MB partition
> that most of these devices seem to have.

I'd like to mention my package tiny-initramfs here, which can help
keep the size required for the initramfs down quite a bit (without
modules it's ~12-14 kiB (!) with gzip on armel, modules increase it
by their own size, but nothing else; separate /usr is explicitly
supported), and may be useful for at least some of that hardware:

https://tracker.debian.org/pkg/tiny-initramfs
https://github.com/chris-se/tiny-initramfs/

That doesn't help if there's a limit on the kernel image alone and
that is exceeded, of course.

Regards,
Christian

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Ben Hutchings]

On Fri, 2016-12-09 at 22:14 +0900, Roger Shimizu wrote:
> On Fri, 09 Dec 2016 00:53:17 +
> > Ben Hutchings  wrote:
> 
> > Also, dedicated tiny flash partitions for the kernel and initrd.  I
> > wouldn't be surprised to be find that by the time we want to release
> > buster we can't build a useful kernel that fits into the 2 MB partition
> > that most of these devices seem to have.
> 
> Actually those limitations are only for a few old orion5x models.
> For Linkstation kirkwood series I maintain for, kernel can be 2.5MB
> and initrd can be 8MB.

OK, that's much less of a problem.  Please add this information to the
comment on size limitations (debian/config/armel/defines).

> I'm not sure about the exact limitation size because the vendor doesn't
> disclose this specification clearly.

I think we should assume that whatever fits in the flash partition will
work, unless we discover another limitation in the boot loader.

Ben.

> But size listed above should be much easier to achieve for kernel 
> maintainers.
> 
> > As it is, stretch will be supported until 2020, maybe 2022 on armel. 
> > Is it really worthwhile to add another 2 years to that?
> 
> If the effort to maintain the armel is limited under certain level,
> and the those hardware are still in good condition, IMHO the longer
> the better.
> 
> Cheers,
-- 
Ben Hutchings
Logic doesn't apply to the real world. - Marvin Minsky



signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[W. Martin Borgert]

Quoting Ben Hutchings :

Also, dedicated tiny flash partitions for the kernel and initrd.  I
wouldn't be surprised to be find that by the time we want to release
buster we can't build a useful kernel that fits into the 2 MB partition
that most of these devices seem to have.


Non-HF devices can be very different, look at e.g. this ARM926EJ-S one:
http://www.taskit.de/stamp9g20.html


As it is, stretch will be supported until 2020, maybe 2022 on armel.
Is it really worthwhile to add another 2 years to that?


This depends on the effort, of course. But for environmental reasons
I'ld say the longer the better.

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

On Fri, 09 Dec 2016 00:53:17 +
Ben Hutchings  wrote:

> Also, dedicated tiny flash partitions for the kernel and initrd.  I
> wouldn't be surprised to be find that by the time we want to release
> buster we can't build a useful kernel that fits into the 2 MB partition
> that most of these devices seem to have.

Actually those limitations are only for a few old orion5x models.
For Linkstation kirkwood series I maintain for, kernel can be 2.5MB
and initrd can be 8MB.

I'm not sure about the exact limitation size because the vendor doesn't
disclose this specification clearly.
But size listed above should be much easier to achieve for kernel 
maintainers.

> As it is, stretch will be supported until 2020, maybe 2022 on armel. 
> Is it really worthwhile to add another 2 years to that?

If the effort to maintain the armel is limited under certain level,
and the those hardware are still in good condition, IMHO the longer
the better.

Cheers,
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1


pgpZLHHHcQrXl.pgp
Description: PGP signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Ben Hutchings]

On Thu, 2016-12-08 at 23:12 +0100, Christoph Biedl wrote:
> Roger Shimizu wrote...
> 
> > I'm ARM porter on armel/marvell (orion5x/kirkwood).
> > Stretch will be frozen and released soon, which makes me bit depressed, 
> > because it means armel will be dropped out of unstable/testing as the 
> > conclusion of Cape Town BoF.
> 
> Same here. My Dockstars (orion5x/kirkwood) still work like a charm and
> it gives a bad feeling having to trash them some day just because
> there's no support any more.
> 
> On the other hand, they face another problem I guess is typical for
> that generation, just by the age: Memory.
[...]

Also, dedicated tiny flash partitions for the kernel and initrd.  I
wouldn't be surprised to be find that by the time we want to release
buster we can't build a useful kernel that fits into the 2 MB partition
that most of these devices seem to have.

As it is, stretch will be supported until 2020, maybe 2022 on armel. 
Is it really worthwhile to add another 2 years to that?

Ben.

-- 
Ben Hutchings
Logic doesn't apply to the real world. - Marvin Minsky



signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

Roger Shimizu wrote...

> I'm ARM porter on armel/marvell (orion5x/kirkwood).
> Stretch will be frozen and released soon, which makes me bit depressed, 
> because it means armel will be dropped out of unstable/testing as the 
> conclusion of Cape Town BoF.

Same here. My Dockstars (orion5x/kirkwood) still work like a charm and
it gives a bad feeling having to trash them some day just because
there's no support any more.

On the other hand, they face another problem I guess is typical for
that generation, just by the age: Memory. So for quite some time I
wanted to start a thread here but there are too many other things I
have to take care of. Now however that the discussion has started ...
I would have written something like the following:


As far as I know, there is no such thing as "minimal hardware
requirement" for Debian besides some lines in the installer pages. For
RAM, the minimum value is 128 megabytes, and I think it's about time
to raise that. Yes, you can run Debian on that but it's not fun:

Locale generation needs a lot of RAM. You can work around it by
installing locales-all which however takes long time to install on
slow flash drives. Or disable locales entirely. Err.

Side-effects of over-eagerly used xz compression. This is a relict of
time when xz was pretty new and maintainers added the -9 compression
option, probably completely unaware this makes no sense for small
packages, unlike gzip or bzip2. Also, even unpacking unconditionally
allocates some 65 Mbyte memory then, easiliy driving a 128 Mbyte-box
to the limits. One of the worst is the traceroute package that
actually carries some 110 kbyte; however I was unable to install it
due to the above problem.

The amount of data apt deals with reflects more or less the size of
the Packages indexes. As they get bigger each release, this will
become a problem. Already now apt gets OOM-killed every now and then.


About xz, I considered doing a MBF (severity normal the most) asking
to end this, it's about 45 packages.

About apt I have an idea to provide "reduced" Packages indexes. They
would be smaller so the memory usage is no longer a problem, also apt
should become somewhat faster. But since it's virtually impossible to
create a complete subset, I'd declare incompleteness a feature: If
somebody manages to hit the limits by requesting installation of
packages that are referenced only - tough luck, use the full index,
please. But somebody would have to promote and maintain that feature.


Another way to deal with this however was to raise the minimum memory
requirements. To 256 MB, or perhaps even 512 MB. It feels wrong since
it works around a problem instead of solving it. But since all
components become more greedy over time, this will have to be done
sooner or later anyway. And will render armel de-facto obsolete.

Still I wouldn't mind armel in buster, perhaps restricted to armv5.
But I understand the odds aren't quite in favour.

Christoph


signature.asc
Description: Digital signature

Re: armel after Stretch

[Ben Hutchings]

On Thu, 2016-12-08 at 20:19 +0900, Roger Shimizu wrote:
> Dear Steve,
> 
> Thanks for your comments!
> Very informative!
> 
> > On Thu, Dec 8, 2016 at 12:53 AM, Steve McIntyre  wrote:
> > 
> > There are kernel helpers available to provide some atomic support, but
> > they'll be very slow compared to real hardware support at this level.
> 
> Are those kernel helper already reached Debian?
> Or there's still some work here?
[...]

The cmpxchg32 helper has been there since 2.6.12 while the cmpxchg64
helper was added in 3.1.  So all supported Debian releases have them.

Ben.

-- 
Ben Hutchings
When in doubt, use brute force. - Ken Thompson



signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch

[Roger Shimizu]

Dear Steve,

Thanks for your comments!
Very informative!

On Thu, Dec 8, 2016 at 12:53 AM, Steve McIntyre  wrote:
>
> There are kernel helpers available to provide some atomic support, but
> they'll be very slow compared to real hardware support at this level.

Are those kernel helper already reached Debian?
Or there's still some work here?

>>>Downside: as above if we try to do the partial architecture route;
>>>if we don't we'll still have to support a full range of software on
>>>older hardware.
>>
>>I don't see any detailed downside reason here. I think armel dropping
>>v4t is just like i386 dropping 586-class CPU [0]. If we can dropping
>>586-class CPU support for i386, by changing a few configure files in
>>gcc/dpkg/kernel packages, why we cannot do the same for armel?
>
> It's a similar thing, but further up the curve - that's all. We added
> armhf (as the equivalent of i686, maybe) a while back, targeting
> ARMv7. The much older CPU designs based on ARMv4 and ARMv5 are getting
> harder to support than the i586 here. The world has moved on,
> basically.

Just like the world already moves on to amd64, but Debian still
support i386 and i686,
I think the community need armel and armhf, for quite a long time if
it's feasible.

> You're the first armel developer to offer to dive in here, so that's a
> good start!

Thanks, but my knowledge don't cover the lower part such as building toolchains,
still need someone with more experience.

>  * It will need somebody happy to dive into the lower levels of the
>various toolchains to verify support for atomics and make things
>work where it's not already, by adding support for the kernel
>helpers.

> On Wed, Dec 07, 2016 at 05:05:58PM +0100, Emilio Pozuelo Monfort wrote:
>>https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820535
>>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64735

I guess above 2 links, provided by Emilio (thanks so much!), can
resolve the concern on building toolchains.

>  * *If* we wanted to try the partial architecture thing, that will
>need some effort to make it work. That's not well-defined right
>now, as it's only a vague concept at best (sorry!)

Maybe we can avoid to do partial arch?

>  * There are going to be some packages that just won't work,
>particularly JIT compilers and other code generators that assume
>ARM == ARMv7. Fixing those up might raneg between feasible and
>~impossible depending on the size of the codebase...

If something breaks, I guess it breaks now already.
We need to fix before Stretch.
If the issue is fixed, I think there's no need to remove armel
immediately after releasing Stretch.

Please correct me if I missed something. Thank you!

Cheers,
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: armel after Stretch

[Steve McIntyre]

On Wed, Dec 07, 2016 at 05:05:58PM +0100, Emilio Pozuelo Monfort wrote:
>On 07/12/16 16:53, Steve McIntyre wrote:
>>  * It will need somebody happy to dive into the lower levels of the
>>various toolchains to verify support for atomics and make things
>>work where it's not already, by adding support for the kernel
>>helpers.
>
>There has been some recent work on this, but it seems stalled and it's not 
>clear
>whether it works on all the armel targeted hardware (e.g. v4t), see:
>
>https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820535
>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64735

ACK. I'd missed the updates there - thanks for the links!

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Managing a volunteer open source project is a lot like herding
 kittens, except the kittens randomly appear and disappear because they
 have day jobs." -- Matt Mackall

Re: armel after Stretch

[Emilio Pozuelo Monfort]

On 07/12/16 16:53, Steve McIntyre wrote:
>  * It will need somebody happy to dive into the lower levels of the
>various toolchains to verify support for atomics and make things
>work where it's not already, by adding support for the kernel
>helpers.

There has been some recent work on this, but it seems stalled and it's not clear
whether it works on all the armel targeted hardware (e.g. v4t), see:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820535
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64735

Emilio

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Steve McIntyre]

On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:
>[ intentionally keep d-d CCed ]
>
>On Fri, 22 Jul 2016 02:36:05 +0100
>Steve McIntyre  wrote:
>
>> [ Please note the cross-post and Reply-To ]
>> 
>> Hi folks,
>> 
>> As promised, here's a quick summary of what was discussed at the ARM
>> ports BoF session in Cape Town.
>
>Thanks for the summary!
>
>I'm ARM porter on armel/marvell (orion5x/kirkwood).
>Stretch will be frozen and released soon, which makes me bit depressed, 
>because it means armel will be dropped out of unstable/testing as the 
>conclusion of Cape Town BoF.
>
>So I'm writing to ask if there's any chance ...
>
>> We spoke about the past/present/future state of ARM ports in Debian.
>> 
>> armel
>> =
>> 
>> armel is the longest-running of our current ARM ports in Debian,
>> starting in 2007. It's starting to become difficult to support. Debian
>> is the only common distro still supporting ARMv4t. While there is
>> still good upstream support in most major packages (e.g. Linux and
>> gcc), we're seeing a growing number of issues. Some packages
>> (e.g. NodeJS) don't support pre-ARMv7 at all any more. Another problem
>> is the lack of direct support for C++11 atomics - it's possible to use
>> a kernel helper to cover for this lack, but performance will be
>> terrible.
>> 
>> Possible future options for armel:
>> 
>>  * Partial armel architecture?
>> 
>>We've talked about this in the past for various architectures, but
>>nobody has stepped up to work on it. The vast majority of the
>>outstanding armel use cases are going to be in headless servers so
>>we could maybe drop the desktop tasks and dependencies and keep
>>things like web server / mail server tasks that are much simpler.
>> 
>>Downside: There's no clear plan for how to create/maintain a
>>partial architecture, let alone how to release one. Potentially
>>huge amount of work needed.
>> 
>>  * Update to ARMv5t (and maybe go headless)
>> 
>>We might be able to extend the life of armel by upping the minimum
>>spec, and would be able to continue supporting Kirkwood and Orion
>>based machines (e.g. NAS boxes) for a while longer. This would kill
>>support for v4t devices like Openmoko, but there are precious few
>>of these older devices still around.
>
>Dropping v4t devices seems won't cause big problem, as you said
>there's few devices around currently. So I personally prefer this
>option.

AFAIK there are potentially still similar problems with ARMv5 - lack
of architcture-defined barrier primitives for C++11 atomics to
work. (I'd love to be corrected on this if people know better!) This
is one of the key points here. More and more software is expecting to
use these primitives, and a lack of them is a major problem. To
demonstrate using gcc, you can see that lock-free atomics only started
appearing in ARMv6 and were improved in ARMv7:

$ for arch in 4 5 6 7-a; do echo ARMv${arch}; echo | g++ -march=armv${arch} -dM 
-E - | grep -i lock_free; done
ARMv4
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __GCC_ATOMIC_POINTER_LOCK_FREE 1
#define __GCC_ATOMIC_INT_LOCK_FREE 1
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
#define __GCC_ATOMIC_LONG_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
ARMv5
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __GCC_ATOMIC_POINTER_LOCK_FREE 1
#define __GCC_ATOMIC_INT_LOCK_FREE 1
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
#define __GCC_ATOMIC_LONG_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
ARMv6
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __GCC_ATOMIC_POINTER_LOCK_FREE 2
#define __GCC_ATOMIC_INT_LOCK_FREE 2
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
#define __GCC_ATOMIC_LONG_LOCK_FREE 2
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
ARMv7-a
#define __GCC_ATOMIC_CHAR_LOCK_FREE 2
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
#define __GCC_ATOMIC_BOOL_LOCK_FREE 2
#define __GCC_ATOMIC_POINTER_LOCK_FREE 2
#define __GCC_ATOMIC_INT_LOCK_FREE 2
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
#define __GCC_ATOMIC_LONG_LOCK_FREE 2
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 2
#define __GCC_ATOMIC_LLONG_LOCK_FREE 2
#define __GCC_ATOMIC_SHORT_LOCK_FREE 2

There are kernel helpers available to provide some atomic support, but
they'll be very slow compared to real hardware support at this level.

>>Downside: as above if we try to do the partial architecture route;
>>if we don't we'll still have to support a full range of software on
>>older hardware.
>
>I don't see any detailed 

armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

[ intentionally keep d-d CCed ]

On Fri, 22 Jul 2016 02:36:05 +0100
Steve McIntyre  wrote:

> [ Please note the cross-post and Reply-To ]
> 
> Hi folks,
> 
> As promised, here's a quick summary of what was discussed at the ARM
> ports BoF session in Cape Town.

Thanks for the summary!

I'm ARM porter on armel/marvell (orion5x/kirkwood).
Stretch will be frozen and released soon, which makes me bit depressed, 
because it means armel will be dropped out of unstable/testing as the 
conclusion of Cape Town BoF.

So I'm writing to ask if there's any chance ...

> We spoke about the past/present/future state of ARM ports in Debian.
> 
> armel
> =
> 
> armel is the longest-running of our current ARM ports in Debian,
> starting in 2007. It's starting to become difficult to support. Debian
> is the only common distro still supporting ARMv4t. While there is
> still good upstream support in most major packages (e.g. Linux and
> gcc), we're seeing a growing number of issues. Some packages
> (e.g. NodeJS) don't support pre-ARMv7 at all any more. Another problem
> is the lack of direct support for C++11 atomics - it's possible to use
> a kernel helper to cover for this lack, but performance will be
> terrible.
> 
> Possible future options for armel:
> 
>  * Partial armel architecture?
> 
>We've talked about this in the past for various architectures, but
>nobody has stepped up to work on it. The vast majority of the
>outstanding armel use cases are going to be in headless servers so
>we could maybe drop the desktop tasks and dependencies and keep
>things like web server / mail server tasks that are much simpler.
> 
>Downside: There's no clear plan for how to create/maintain a
>partial architecture, let alone how to release one. Potentially
>huge amount of work needed.
> 
>  * Update to ARMv5t (and maybe go headless)
> 
>We might be able to extend the life of armel by upping the minimum
>spec, and would be able to continue supporting Kirkwood and Orion
>based machines (e.g. NAS boxes) for a while longer. This would kill
>support for v4t devices like Openmoko, but there are precious few
>of these older devices still around.

Dropping v4t devices seems won't cause big problem, as you said there's few 
devices around currently.
So I personally prefer this option.

>Downside: as above if we try to do the partial architecture route;
>if we don't we'll still have to support a full range of software on
>older hardware.

I don't see any detailed downside reason here.
I think armel dropping v4t is just like i386 dropping 586-class CPU [0].
If we can dropping 586-class CPU support for i386, by changing a few 
configure files in gcc/dpkg/kernel packages, why we cannot do the same for 
armel?

[0] https://lists.debian.org/debian-devel-announce/2016/05/msg1.html

I'm a "high-level" ARM porter, merely knowing the basic device-tree stuff 
to support new device.
So I'm lack of knowledge in lower chipset level and maybe missed something 
here.
Could you kindly help to list the detailed work other than listed above?

>Will need volunteers to make this work in either form, and while
>some people are prepared to *help* (e.g. bdale and zumbi), nobody
>stepped up in the BoF to lead such an effort. It needs both skill
>and commitment of time.

If specific working items are clearly listed, I think there would be someone 
stepping out, since the armel user/devel base is still big.

Thanks and look forward to your feeback!

Cheers!
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1


pgpF1UF7rpzU6.pgp
Description: PGP signature