RE: Stretch bootstrap for sparc64 and sparc, optimized for ultrasparc3
Hello Adrian, On Thu, Sep 7, 2017, at 07:22 PM, John Paul Adrian Glaubitz wrote: > We are actually planning on implementing Britney for Debian Ports which > means that Debian Ports would also get a testing release. That would be great! > You can just easily cross-compile a native gcc compiler with sbuild. > That's actually > very easy using the cross-build toolchain available in Debian. Thanks for the pointer, this will help me a lot! > There isn't really a point in bootstrapping for sparc these days unless > you set the default CPU to sparv8 to be able to run Debian on a sun4m > machine. sun4u or newer machines should use sparc64. > Your efforts sound like you're trying to reinvent the wheel in my opinion. We have different perspectives on those ;-) > I think it makes more sense to help to make Britney and hence testing > available in Debian Ports. One does not exclude the other. Do you know who is working on that? Thanks, Tom
Re: Stretch bootstrap for sparc64 and sparc, optimized for ultrasparc3
Hi! On 09/08/2017 10:44 AM, Tom Turelinckx wrote: You can just easily cross-compile a native gcc compiler with sbuild. That's actually very easy using the cross-build toolchain available in Debian. Thanks for the pointer, this will help me a lot! We are currently missing a cross-build-essential-sparc64 meta-package though, I haven't found the time yet to file a bug report with a patch yet. Currently, I'm building this package manually and pre-install it into the build chroot. There isn't really a point in bootstrapping for sparc these days unless you set the default CPU to sparv8 to be able to run Debian on a sun4m machine. sun4u or newer machines should use sparc64. Your efforts sound like you're trying to reinvent the wheel in my opinion. We have different perspectives on those ;-) Well, my point is: There isn't much development going on for sparc anymore, so you're investing time into a more or less dead target. There is no objective reason not to use sparc64 on an UltraSPARC machine. If you have some applications that need more performance, you can compile them as 32-bit binaries individually. It's not unlikely that you will run into trouble with several packages because they haven't been built for sparc for a very long time. I mean, if you're fixing those packages and send in patches, I won't be unhappy, but I'm just trying to tell you, it won't be without quite some pain. I think it makes more sense to help to make Britney and hence testing available in Debian Ports. One does not exclude the other. Do you know who is working on that? You should come join #debian-ports and also talk to Niels Thykier and James Clarke. And, no, one does not exclude the other. But joining forces makes more sense than trying to be a one-man army. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Stretch bootstrap for sparc64 and sparc, optimized for ultrasparc3
On 09/07/2017 07:05 PM, Tom Turelinckx wrote: Because I want to move forward with upgrading some of those machines to a newer release, and replacing some of the Sun Fire-series hardware with SPARC Enterprise-series hardware, I'm working on bootstrapping Stretch for both sparc64 and sparc, preferably --with-cpu(-32/-64)=ultrasparc3 instead of ultrasparc. We are actually planning on implementing Britney for Debian Ports which means that Debian Ports would also get a testing release. I'm not sure what the current status is though. Bootstrapping Stretch for sparc is less straightforward, as the latest/last packages available from snapshot are from two years ago. Still, there is a large amount of packages available that is not terribly old. Thanks to excellent work by Helmut Grohne and others [2], it's also possible to cross-compile a recent version of the most important packages from source. But why 32-bit sparc? All the current development happens in sparc64. There isn't really a point in bootstrapping for sparc these days unless you set the default CPU to sparv8 to be able to run Debian on a sun4m machine. sun4u or newer machines should use sparc64. It took only minimal patches to get rebootstrap to work against Stretch 9.0. It finishes successfully, and I have repositories available for both sparc and sparc64. Unfortunately, build-essential is not (yet) fully complete: rebootstrap does not (yet) produce a native gcc. At jenkins.debian.net, builds considered successful finish in the same state, so I guess producing all the build dependencies to produce a native gcc is (currently) out of the scope of rebootstrap. You can just easily cross-compile a native gcc compiler with sbuild. That's actually very easy using the cross-build toolchain available in Debian. Having Stretch repositories with a significant number of packages available for both sparc64 and sparc would open up a lot of opportunities for testing and actually using the port, and having them optimized for ultrasparc3 would allow useful performance testing on a broad range of currently relevant hardware. If I succeed in building the repositories, I hope to make them public. I think it makes more sense to help to make Britney and hence testing available in Debian Ports. Your efforts sound like you're trying to reinvent the wheel in my opinion. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Stretch bootstrap for sparc64 and sparc, optimized for ultrasparc3
Hi, On Thu, Sep 7, 2017, at 11:19 AM, John Paul Adrian Glaubitz wrote: > On 09/07/2017 10:30 AM, Tom Turelinckx wrote: >> Not all of those may be necessary anymore, but I've been doing it like >> that since squeeze and up to the current sid on dozens of machines, and >> it works reliably: when the first disk fails, I am able to boot from the >> second disk. > > On a sidenote: Would you mind enabling popcon for your sparc64 > installations, > so we get more counts of people running Debian on sparc64 hardware? Enabling it for my sparc64 installations wouldn't be very useful, as those are just two or three machines (V210, V240, T5140) with one or two LXC containers each, and they're only used for experiments, so they're not representative. Enabling it for my sparc installations would cause quite a spike in the Wheezy deployments, as those are 15-20 machines (V120, V210, V240, V440) with ~4 LXC containers each. Those currently can't be upgraded because Sid is too volatile. Because I want to move forward with upgrading some of those machines to a newer release, and replacing some of the Sun Fire-series hardware with SPARC Enterprise-series hardware, I'm working on bootstrapping Stretch for both sparc64 and sparc, preferably --with-cpu(-32/-64)=ultrasparc3 instead of ultrasparc. Just bootstrapping Stretch 9.0 for sparc64 is relatively straightforward thanks to excellent work by Adrian Glaubitz and others: for most of the binary packages the correct version is readily available from snapshot.debian.org. I've built a repository of binary packages to create Stretch 9.0 for sparc64 that has either the correct version of each package or slightly older, up to build-essential and some additional packages such as the kernel. >From this repository I can create a pbuilder basetgz that's very close to >Stretch 9.0, and allows to build additional packages to bring the >"very-close-to-9.0" repo to "really-9.0", as well as up to 9.1. I've also >installed a physical machine using a fairly old sid cd where all package >versions were older than Stretch 9.0, then upgraded using this repository, so >I have a physical machine that's "very-close-to-9.0" but hasn't seen any >packages "from the future" to run pbuilder on. The only problem here is how to automatically select the correct binNMU for sparc64 for a given source package version, for Stretch 9.0. I think it can't be automated (correctly), so I verified it manually for the packages that I've done, but it's unrealistic to do so for the entire archive. Once a certain amount of packages has been made available through this method, it seems easier to start (automatically) recompiling additional packages from source, rather than (manually) pulling in the correct binNMU from snapshot... And if I do start recompiling a large amount of packages, I intend to optimize them for ultrasparc3 rather than ultrasparc, based on this [1] remark by David Miller. Bootstrapping Stretch for sparc is less straightforward, as the latest/last packages available from snapshot are from two years ago. Still, there is a large amount of packages available that is not terribly old. Thanks to excellent work by Helmut Grohne and others [2], it's also possible to cross-compile a recent version of the most important packages from source. It took only minimal patches to get rebootstrap to work against Stretch 9.0. It finishes successfully, and I have repositories available for both sparc and sparc64. Unfortunately, build-essential is not (yet) fully complete: rebootstrap does not (yet) produce a native gcc. At jenkins.debian.net, builds considered successful finish in the same state, so I guess producing all the build dependencies to produce a native gcc is (currently) out of the scope of rebootstrap. I'm working on creating a useful native pbuilder chroot for sparc (similar to what I have for sparc64), either by pulling packages from rebootstrap into a chroot created from snapshot, or by pulling packages from snapshot into the cross-compiling chroot from rebootstrap. I'm also investigating botch and dose to determine the optimal order for mass-building all the packages. Because so many packages are available, build dependencies will probably not be a problem, but by doing them in the optimal order, it might be possible to get correctly ultrasparc3-optimized versions of all packages in one or maybe two runs, and I think a random order might require more such runs. Having Stretch repositories with a significant number of packages available for both sparc64 and sparc would open up a lot of opportunities for testing and actually using the port, and having them optimized for ultrasparc3 would allow useful performance testing on a broad range of currently relevant hardware. If I succeed in building the repositories, I hope to make them public. Tom > [1] https://patchwork.ozlabs.org/comment/927979/ > [2] https://wiki.debian.org/HelmutGrohne/rebootstrap
