Re: armel/armhf arch qualification for buster: call for DSA, Security, toolchain concernsj
Hi Roger! On 07/23/2018 10:42 AM, Roger Shimizu wrote: > I talked to a few people about keeping armel in buster, during 1st and > 2nd day in debcamp. > Seems the blocker is just the buildd server hardware, and memory size it has. According to my colleague Alex Graf at SUSE, you can definitely build ARMv7 on arm64 using chroots. The only problem with chroots is that "uname -a" shows "armv8" which some userspace applications are stumbling over. openSUSE/SLE builds armv7 packages in OBS/IBS using a fully emulated system using KVM. As for the hardware, you should watch out for hardware with ARM Cortex Cores. Alternatively, X-Gene 1. ThunderX, ThunderX2 and Centriq are definitely not supported. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: armel/armhf arch qualification for buster: call for DSA, Security, toolchain concernsj
Dear armel/armhf shakeholders, I talked to a few people about keeping armel in buster, during 1st and 2nd day in debcamp. Seems the blocker is just the buildd server hardware, and memory size it has. On Fri, Jun 29, 2018 at 7:04 PM, W. Martin Borgert wrote: > > Quoting Uwe Kleine-König : >> >> If the concerns are mostly about the hardware not being rackable, there >> is a rackable NAS by Netgear: >> >> >> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs > > This seems to be out of stock and discontinued, unfortunately. This is still available in amazon: - https://www.amazon.com/dp/B00MQK14KC > Anyway, I'm relatively sure, that I can convince my boss to sponsor/donate > both armel and armhf hardware for Debian, if that is of any help. Or arm64 > used in "32 bits mode". I think DSA team prefers armel or armhf real hardware (not just developing boards). So it'll be super great if you (or your boss) can kindly sponsor some armel/armhf hardwares that support to install 4GB memory. Thanks! Cheers, -- Roger Shimizu, GMT +9 Tokyo PGP/GPG: 4096R/6C6ACD6417B3ACB1
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
> On Jun 29, 2018, at 9:50 AM, Jonathan Wiltshire wrote: > >> On Fri, Jun 29, 2018 at 11:04:26PM +0900, Roger Shimizu wrote: >> I see armel is already not a candidate for buster [0]. >> So it seems we can discuss armhf, but no armel at all. >> I don't agree with this idea. >> And I think we should treat armel and armhf equally. > > Please review the mail which originated this thread [1] where you will see > that both armel and armhf are affected by DSA's concern. If I understand > correctly, virtualisation of architectures in general is a possible > solution but there are problems in the case of these two. I have just talked to a colleague at SUSE about this and apparently Alex Graf from SUSE’s QEMU/KVM team has fixed many bugs regarding ARM32 on ARM64 virtualization. If I understand correctly, SUSE builds ARMv7 on ARM64 without problems. I have reached out to Alex Graf and asked him for clarification on what possibilities we currently have. Adrian
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 11:04:26PM +0900, Roger Shimizu wrote: > I see armel is already not a candidate for buster [0]. > So it seems we can discuss armhf, but no armel at all. > I don't agree with this idea. > And I think we should treat armel and armhf equally. Please review the mail which originated this thread [1] where you will see that both armel and armhf are affected by DSA's concern. If I understand correctly, virtualisation of architectures in general is a possible solution but there are problems in the case of these two. At the end of the day, if Debian can't reliably run builders for an architecture we do not do users a service by pretending to be able to support it in a formal release. A freeze may be for Christmas, but stable is for at least five+ years. -- Jonathan Wiltshire j...@debian.org Debian Developer http://people.debian.org/~jmw 4096R: 0xD3524C51 / 0A55 B7C5 1223 3942 86EC 74C3 5394 479D D352 4C51
Re: armel/armhf arch qualification for buster: call for DSA, Security, toolchain concernsj
Open Network Linux developed for the Open Compute Project has a vested interest in maintaining support for armel at least. I would be interested in sponsoring or donating rack mountable switches using armel processors. On Fri, Jun 29, 2018 at 3:04 AM, W. Martin Borgert wrote: > Quoting Uwe Kleine-König : > >> If the concerns are mostly about the hardware not being rackable, there >> is a rackable NAS by Netgear: >> >> https://www.netgear.com/business/products/storage/readynas/ >> RN2120.aspx#tab-techspecs >> > > This seems to be out of stock and discontinued, unfortunately. > > Anyway, I'm relatively sure, that I can convince my boss to sponsor/donate > both armel and armhf hardware for Debian, if that is of any help. Or arm64 > used in "32 bits mode". > >
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 10:04 PM, Uwe Kleine-König wrote: > Hello Julien, > > On 06/29/2018 11:23 AM, Julien Cristau wrote: >>> If the concerns are mostly about the hardware not being rackable, there >>> is a rackable NAS by Netgear: >>> >>> >>> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs >>> >>> with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 >>> GiB) are good enough. The machine can run mainline Linux[1]. I think >>> U-Boot doesn't support this machine in mainline though. >>> >> Rackable, while good, is only part of it. The main part is remote >> management. I'm not seeing any mention of ipmi or anything like that in >> the datasheet? > > you can access the serial console, but I don't think there is built-in > support for something IPMI-like. > >> 2G is also way too little memory these days for a new buildd. > > Then the machine is out, the amount of RAM isn't upgradable. I don't think 2GB is not enough for 32-bit machine. I see armel is already not a candidate for buster [0]. So it seems we can discuss armhf, but no armel at all. I don't agree with this idea. And I think we should treat armel and armhf equally. Both armel and armhf are working fine are millions of boards and embedded devices, and have stable quality [1]. They deserve the support from a community driven distro. [0] https://release.debian.org/buster/arch_qualify.html [1] https://lists.debian.org/debian-arm/2017/11/msg00061.html Cheers, -- Roger Shimizu, GMT +9 Tokyo PGP/GPG: 4096R/6C6ACD6417B3ACB1
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
Hello Julien, On 06/29/2018 11:23 AM, Julien Cristau wrote: >> If the concerns are mostly about the hardware not being rackable, there >> is a rackable NAS by Netgear: >> >> >> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs >> >> with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 >> GiB) are good enough. The machine can run mainline Linux[1]. I think >> U-Boot doesn't support this machine in mainline though. >> > Rackable, while good, is only part of it. The main part is remote > management. I'm not seeing any mention of ipmi or anything like that in > the datasheet? you can access the serial console, but I don't think there is built-in support for something IPMI-like. > 2G is also way too little memory these days for a new buildd. Then the machine is out, the amount of RAM isn't upgradable. Best regards Uwe signature.asc Description: OpenPGP digital signature
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 12:06 PM, John Paul Adrian Glaubitz wrote: > On 06/29/2018 10:41 AM, Luke Kenneth Casson Leighton wrote: >> On Fri, Jun 29, 2018 at 8:16 AM, Uwe Kleine-König >> wrote: >> In short, the hardware (development boards) we're currently using to build armel and armhf packages aren't up to our standards, and we really, really want them to go away when stretch goes EOL (expected in 2020). We urge arm porters to find a way to build armhf packages in VMs or chroots on server-class arm64 hardware. >> >> from what i gather the rule is that the packages have to be built >> native. is that a correct understanding or has the policy changed? > > Native in the sense that the CPU itself is not emulated which is the case > when building arm32 packages on arm64. ok. that's clear. thanks john. > I think that building on arm64 after fixing the bug in question is the > way to move forward. I'm surprised the bug itself hasn't been fixed yet, > doesn't speak for ARM. if you mean ARM hardware (OoO), it's too late. systems are out there with OoO speculative execution bugs in the hardware (and certainly more to be found), and they're here to stay unfortunately. if you mean that buildd on 32-bit systems could be modified to pass "-Wl,--no-keep-memory" to all linker phases to see if that results in the anticipated dramatic reduction in memory usage, that's straightforward to try, nothing to do with ARM themselves. l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On 06/29/2018 10:41 AM, Luke Kenneth Casson Leighton wrote: > On Fri, Jun 29, 2018 at 8:16 AM, Uwe Kleine-König > wrote: > >>> In short, the hardware (development boards) we're currently using to >>> build armel and armhf packages aren't up to our standards, and we >>> really, really want them to go away when stretch goes EOL (expected in >>> 2020). We urge arm porters to find a way to build armhf packages in >>> VMs or chroots on server-class arm64 hardware. > > from what i gather the rule is that the packages have to be built > native. is that a correct understanding or has the policy changed? Native in the sense that the CPU itself is not emulated which is the case when building arm32 packages on arm64. We're also building i386 packages on amd64 and we used to build powerpc packages on ppc64 (and we will continue to do that once the move of powerpc to ports has been completed). I think that building on arm64 after fixing the bug in question is the way to move forward. I'm surprised the bug itself hasn't been fixed yet, doesn't speak for ARM. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
armel/armhf arch qualification for buster: call for DSA, Security, toolchain concernsj
Quoting Uwe Kleine-König : If the concerns are mostly about the hardware not being rackable, there is a rackable NAS by Netgear: https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs This seems to be out of stock and discontinued, unfortunately. Anyway, I'm relatively sure, that I can convince my boss to sponsor/donate both armel and armhf hardware for Debian, if that is of any help. Or arm64 used in "32 bits mode".
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
[s/debian-ports/debian-arm/] On 06/29/2018 09:16 AM, Uwe Kleine-König wrote: > Hello, > > On Wed, Jun 27, 2018 at 08:03:00PM +, Niels Thykier wrote: >> armel/armhf: >> >> >> * Undesirable to keep the hardware running beyond 2020. armhf VM >>support uncertain. (DSA) >>- Source: [DSA Sprint report] >> >> [DSA Sprint report]: >> https://lists.debian.org/debian-project/2018/02/msg4.html > > In this report Julien Cristau wrote: > >> In short, the hardware (development boards) we're currently using to >> build armel and armhf packages aren't up to our standards, and we >> really, really want them to go away when stretch goes EOL (expected in >> 2020). We urge arm porters to find a way to build armhf packages in >> VMs or chroots on server-class arm64 hardware. > > If the concerns are mostly about the hardware not being rackable, there > is a rackable NAS by Netgear: > > > https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs > > with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 > GiB) are good enough. The machine can run mainline Linux[1]. I think > U-Boot doesn't support this machine in mainline though. > Rackable, while good, is only part of it. The main part is remote management. I'm not seeing any mention of ipmi or anything like that in the datasheet? 2G is also way too little memory these days for a new buildd. Cheers, Julien
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 8:16 AM, Uwe Kleine-König wrote: > Hello, > > On Wed, Jun 27, 2018 at 08:03:00PM +, Niels Thykier wrote: >> armel/armhf: >> >> >> * Undesirable to keep the hardware running beyond 2020. armhf VM >>support uncertain. (DSA) >>- Source: [DSA Sprint report] >> >> [DSA Sprint report]: >> https://lists.debian.org/debian-project/2018/02/msg4.html > > In this report Julien Cristau wrote: > >> In short, the hardware (development boards) we're currently using to >> build armel and armhf packages aren't up to our standards, and we >> really, really want them to go away when stretch goes EOL (expected in >> 2020). We urge arm porters to find a way to build armhf packages in >> VMs or chroots on server-class arm64 hardware. from what i gather the rule is that the packages have to be built native. is that a correct understanding or has the policy changed? > > If the concerns are mostly about the hardware not being rackable, there > is a rackable NAS by Netgear: > > > https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs > > with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 > GiB) are good enough. no matter how much RAM there is it's never going to be "enough", and letting systems go into swap is also not a viable option [2] i've been endeavouring to communicate the issue for many many years wrt building (linking) of very large packages, for a long, *long* time. as it's a strategic cross-distro problem that's been very very slowly creeping up on *all* distros as packages inexorably creep up in size, reaching people about the problem and possible solutions is extremely difficult. eventually i raised a bug on binutils and it took several months to communicate the extent and scope of the problem even to the developer of binutils: https://sourceware.org/bugzilla/show_bug.cgi?id=22831 the problem is that ld from binutils by default, unlike gcc which looks dynamically at how much RAM is available, loads absolutely all object files into memory and ASSUMES that swap space is going to take care of any RAM deficiencies. unfortunately due to the amount of cross-referencing that takes place in the linker phase this "strategy" causes MASSIVE thrashing, even if one single object file is sufficient to cause swapping. this is particularly pertinent for systems which compile with debug info switched on as it is far more likely that a debug compile will go into swap, due to the amount of RAM being consumed. firefox now requires 7GB of resident RAM, making it impossible to compile on 32-bit systems webkit-based packages require well over 2GB RAM (and have done for many years). i saw one scientific package a couple years back that could not be compiled for 32-bit systems either. all of this is NOT the fault of the PACKAGES [1], it's down to the fact that *binutils* - ld's default memory-allocation strategy - is far too aggressive. the main developer of ld has this to say: Please try if "-Wl,--no-keep-memory" works. now, that's *not* a good long-term "solution" - it's a drastic, drastic hack that cuts the optimisation of keeping object files in memory stone dead. it'll work... it will almost certainly result in 32-bit systems being able to successfully link applications that previously failed... but it *is* a hack. someone really *really* needs to work with the binutils developer to *properly* solve this. if any package maintainer manages to use the above hack to successfully compile 32-bit packages that previously completely ran out of RAM or otherwise took days to complete, please do put a comment to that effect in the binutiols bugreport, it will help everyone in the entire GNU/Linux community to do so. l. [1] really, it is... developers could easily split packages into dynamic-loadable modules, where each module easily compiles well below even 2GB or 1GB of RAM. they choose not to, choosing instead to link hundreds of object files into a single executable (or library). asking so many developers to change their strategy however... yyeah :) big task, i ain't taking responsibility for that one. [2] the amount of memory being required for the linker phase of large packages over time goes up, and up, and up, and up... when is it going to stop? never. so just adding more RAM is never going to "solve" the problem, is it? it just *avoids* the problem. letting even 64-bit systems go into swap is a huge waste of resources as builds that go into swap will consume far more resources and time. so *even on 64-bit systems* this needs solving.
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
Hello, On Wed, Jun 27, 2018 at 08:03:00PM +, Niels Thykier wrote: > armel/armhf: > > > * Undesirable to keep the hardware running beyond 2020. armhf VM >support uncertain. (DSA) >- Source: [DSA Sprint report] > > [DSA Sprint report]: > https://lists.debian.org/debian-project/2018/02/msg4.html In this report Julien Cristau wrote: > In short, the hardware (development boards) we're currently using to > build armel and armhf packages aren't up to our standards, and we > really, really want them to go away when stretch goes EOL (expected in > 2020). We urge arm porters to find a way to build armhf packages in > VMs or chroots on server-class arm64 hardware. If the concerns are mostly about the hardware not being rackable, there is a rackable NAS by Netgear: https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 GiB) are good enough. The machine can run mainline Linux[1]. I think U-Boot doesn't support this machine in mainline though. Apart from that the people in #debian-arm (e.g. Sledge) seem to be positive that at least armhf should be fine to be built on arm64 hardware. Best regards Uwe [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/armada-xp-netgear-rn2120.dts signature.asc Description: PGP signature