Re: Altivec in baseline for ppc64?
Hi On 13/07/2021 21:35, Karoly Balogh wrote: > Just to note it here, this CPU family is also used by some relatively > recent desktop-class PowerPC machines, like A-Eon's AmigaOne X5000 (and > its "CyrusPlus" motherboard). So it's not just about the support of some > obscure embedded dev board. e5500 is also used by NXP in their T1040 QorIQ network appliances I recompiled and ported most of Debian 8 packages to these boxes about three to four years ago. It went pretty smoth even if in some cases i had to patch some upstream source code because of inline ASM code or hardcoded compiler options in Makefiles. Hope fully it was very few software we were not using in network use case. Initial build hase been done using 5 dual cpu Powermac G5. Once it hase been proven working, and build chain boot strapped i recompiled a few thousands of packages using a Power 8 installed with the packages built on Macs So in some cases like mine it can be not about embeded boards or Amiga clones :) and i am still an Amiga lover and owner 30 years after :P Any ways i'll be happy to share and talk about this any days. Cheers W. -- kind regards, William https://forum.armwizard.org ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ wbonnet@(armwizard|firmwaretoolkit|neuralnet-studio).org ⢿⡄⠘⠷⠚⠋⠀ GPG fingerprint: 7189 DC8E 15B9 B3E4 EA3E 902B 8EAC F0B9 25A5 9D48 ⠈⠳⣄ OpenPGP_signature Description: OpenPGP digital signature
Re: Altivec in baseline for ppc64?
> On 13. Jul 2021, at 21:44, Karoly Balogh wrote: > > Hi, > > On Tue, 13 Jul 2021, John Paul Adrian Glaubitz wrote: > >>> However my understanding is that this port supports any powerpc64 CPU, >>> including some that don’t have Altivec (e.g. POWER4 or POWER5). This >>> is also what the main wiki page for PPC64 says: >>> https://wiki.debian.org/PPC64 >>> >>> Can someone please clarify the situation? >> >> The ppc64 originally used the ppc64 baseline including AltiVec e.g >> PowerPC970, (64-Bit PowerMac). However, the previous port maintainer >> decided he wanted to support embedded systems such as the PowerPC E5500 >> which does not support AltiVec. > > Just to note it here, this CPU family is also used by some relatively > recent desktop-class PowerPC machines, like A-Eon's AmigaOne X5000 (and > its "CyrusPlus" motherboard). So it's not just about the support of some > obscure embedded dev board. > > Charlie Charlie, I didn’t see that you have already posted a note because of the X5000. Thanks for the hint. Cheers, Christian
Re: Altivec in baseline for ppc64?
> On 13. Jul 2021, at 21:05, John Paul Adrian Glaubitz > wrote: > > Hi Sébastien! > >> On 7/13/21 1:55 PM, Sébastien Villemot wrote: >> The wiki page that synthesizes architecture specificities indicates >> that Altivec is included in the baseline for the ppc64 port: >> https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 >> >> However my understanding is that this port supports any powerpc64 CPU, >> including some that don’t have Altivec (e.g. POWER4 or POWER5). This is >> also what the main wiki page for PPC64 says: >> https://wiki.debian.org/PPC64 >> >> Can someone please clarify the situation? > > The ppc64 originally used the ppc64 baseline including AltiVec e.g PowerPC970, > (64-Bit PowerMac). However, the previous port maintainer decided he wanted to > support embedded systems such as the PowerPC E5500 which does not support > AltiVec. This was the right decision. Some users use Debian PPC64 on their AmigaOne X5000 machines and these machines don’t have AltiVec.
Re: Altivec in baseline for ppc64?
On Tue, Jul 13, 2021 at 2:20 PM Sébastien Villemot wrote:
>
> Le mardi 13 juillet 2021 à 20:06 +0200, Mathieu Malaterre a écrit :
> > On Tue, Jul 13, 2021 at 7:21 PM Sébastien Villemot
> > wrote:
> > > Le mardi 13 juillet 2021 à 18:56 +0200, Mathieu Malaterre a écrit :
> > > >
> > > > On Tue, Jul 13, 2021 at 2:04 PM Sébastien Villemot
> > > > wrote:
> > > > >
> > > > > The wiki page that synthesizes architecture specificities indicates
> > > > > that Altivec is included in the baseline for the ppc64 port:
> > > > > https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64
> > > > >
> > > > > However my understanding is that this port supports any powerpc64 CPU,
> > > > > including some that don’t have Altivec (e.g. POWER4 or POWER5). This
> > > > > is
> > > > > also what the main wiki page for PPC64 says:
> > > > > https://wiki.debian.org/PPC64
> > > > >
> > > > > Can someone please clarify the situation?
> > > > >
> > > > > (I’m asking because I’m the maintainer of the openblas package, and
> > > > > knowing whether Altivec is available or not, and more generally what
> > > > > is
> > > > > in the baseline, is essential for proper packaging).
> > > >
> > > > I do not believe that you can do much as a packager. You cannot assume
> > > > anything on the target arch. You need to do the same thing as ffmpeg
> > > > is doing for avx2/sse4 on amd64, you need to do runtime detection. So
> > > > unless upstream is doing something very clever you cannot compile blas
> > > > using any of the fancy altivec instructions :(
> > > >
> > > > The man page for ld.so mentions something about optimized libraries
> > > > (search for "/usr/lib/sse2/"), but this is currently not in use in
> > > > Debian (AFAIK).
> > >
> > > Actually OpenBLAS has its own runtime detection mechanism, which is
> > > used to select the best linear algebra kernel for the current CPU
> > > (those kernels are mainly written in assembly, and take advantage of
> > > available ISA extensions). This mechanism is used on several archs,
> > > including ppc64el (so at runtime, OpenBLAS chooses between a POWER8 and
> > > a POWER9 kernel; there is even a POWER10 kernel already available).
> > >
> > > However, I cannot enable this mechanism on ppc64 and powerpc, because
> > > the runtime detection only works for POWER6 and above, and my
> > > understanding is that for these two ports the baseline is lower. Hence
> > > on these two archs, only one kernel is included in the package binaries
> > > (currently POWER4 for ppc64 and PPCG4 for powerpc). For optimal
> > > performance, users should recompile OpenBLAS locally (as indicated in
> > > the package description and in README.Debian).
> >
> > There are plenty of people on this mailing list that could test/verify
> > that. Is there a quick way to check that your openblas package is
> > compiled correctly for ppc32 and ppc64 (like a verbose mode) ? Did you
> > do any experiment on perotto.debian.net ?
>
> perotto.debian.net is POWER8, so it’s clearly well above the baseline.
> The package runs fine there, but that does not tell anything about
> baseline violation.
>
> Verifying that the package compiled fine and passed its testsuite on
> build daemons does not give any information about baseline violation
> either, because buildds are probably above the baseline as well. FYI,
> the most recent build logs are there:
> https://buildd.debian.org/status/package.php?p=openblas=experimental
> (there is a problem with powerpc in experimental; but the version in
> sid compiled).
>
> If nobody has the relevant knowledge, then the only option is to test
> the package on the oldest possible hardware. The easiest way to test it
> is to recompile it locally (since this will exercise the testsuite).
I can provide SSH access to a PowerMac G5 with Altivec. That should
test the delineation between Altivec and PWR{5-10}.
If OpenBLAS needs to do 64-bit math, then I have the routines cribbed
away that performs 64-bit addition and subtraction using 32x4 vectors.
The routines have to handle carry/borrow themselves. My experience
with Crypto++ and algos like ChaCha20 demonstrate it is profitable.
Send over your SSH public key/authorized_keys, if interested.
Jeff
Re: Altivec in baseline for ppc64?
Hi, On Tue, 13 Jul 2021, John Paul Adrian Glaubitz wrote: > > However my understanding is that this port supports any powerpc64 CPU, > > including some that don’t have Altivec (e.g. POWER4 or POWER5). This > > is also what the main wiki page for PPC64 says: > > https://wiki.debian.org/PPC64 > > > > Can someone please clarify the situation? > > The ppc64 originally used the ppc64 baseline including AltiVec e.g > PowerPC970, (64-Bit PowerMac). However, the previous port maintainer > decided he wanted to support embedded systems such as the PowerPC E5500 > which does not support AltiVec. Just to note it here, this CPU family is also used by some relatively recent desktop-class PowerPC machines, like A-Eon's AmigaOne X5000 (and its "CyrusPlus" motherboard). So it's not just about the support of some obscure embedded dev board. Charlie
Re: Altivec in baseline for ppc64?
Hi, >> I am however not sure that my current choices for the ppc64 and powerpc >> baselines are optimal, hence this thread. If i can help you i'll be happy to. I have several Powermac G5 running current Debian i can use one to test your packages as long as you tell me please a little about the software to test (i don't know OpenBLAS). On the other hand i am used to build Debian packages on PPC64 and cpu support issues. an keep on tal talking about this "live on IRC". If needed we can send technical "conclusions on the list. How about this ? :) cheers W. -- kind regards, William https://forum.armwizard.org ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ wbonnet@(armwizard|firmwaretoolkit|neuralnet-studio).org ⢿⡄⠘⠷⠚⠋⠀ GPG fingerprint: 7189 DC8E 15B9 B3E4 EA3E 902B 8EAC F0B9 25A5 9D48 ⠈⠳⣄ OpenPGP_signature Description: OpenPGP digital signature
Re: Altivec in baseline for ppc64?
On 7/13/21 6:56 PM, Mathieu Malaterre wrote: > I do not believe that you can do much as a packager. You cannot assume > anything on the target arch. You need to do the same thing as ffmpeg > is doing for avx2/sse4 on amd64, you need to do runtime detection. So > unless upstream is doing something very clever you cannot compile blas > using any of the fancy altivec instructions :( For performance-related libraries I'm okay with AltiVec, we're enabling it for NSS2 as well and so far no user has reported any issues. If that changes in the future, i.e. someone is actually complaining, we can still revisit this issue. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Altivec in baseline for ppc64?
Hi Sébastien! On 7/13/21 1:55 PM, Sébastien Villemot wrote: > The wiki page that synthesizes architecture specificities indicates > that Altivec is included in the baseline for the ppc64 port: > https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 > > However my understanding is that this port supports any powerpc64 CPU, > including some that don’t have Altivec (e.g. POWER4 or POWER5). This is > also what the main wiki page for PPC64 says: > https://wiki.debian.org/PPC64 > > Can someone please clarify the situation? The ppc64 originally used the ppc64 baseline including AltiVec e.g PowerPC970, (64-Bit PowerMac). However, the previous port maintainer decided he wanted to support embedded systems such as the PowerPC E5500 which does not support AltiVec. I wasn't really a fan of that change but my stance is that we should use AltiVec in packages where it makes sense as the majority of the ppc64 port users will have a machine that suppport AltiVec. If they run into an issue with these packages on non-AltiVec systems, they can still file a bug. > (I’m asking because I’m the maintainer of the openblas package, and > knowing whether Altivec is available or not, and more generally what is > in the baseline, is essential for proper packaging). Please go ahead and enabled AltiVec as I don't think it makes much sense to use BLAS on machines without any SIMD support. If any user complains about compatibility issues, please feel free to bring up the issue here again. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Altivec in baseline for ppc64?
Le mardi 13 juillet 2021 à 20:06 +0200, Mathieu Malaterre a écrit : > On Tue, Jul 13, 2021 at 7:21 PM Sébastien Villemot > wrote: > > Le mardi 13 juillet 2021 à 18:56 +0200, Mathieu Malaterre a écrit : > > > > > > On Tue, Jul 13, 2021 at 2:04 PM Sébastien Villemot > > > wrote: > > > > > > > > The wiki page that synthesizes architecture specificities indicates > > > > that Altivec is included in the baseline for the ppc64 port: > > > > https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 > > > > > > > > However my understanding is that this port supports any powerpc64 CPU, > > > > including some that don’t have Altivec (e.g. POWER4 or POWER5). This is > > > > also what the main wiki page for PPC64 says: > > > > https://wiki.debian.org/PPC64 > > > > > > > > Can someone please clarify the situation? > > > > > > > > (I’m asking because I’m the maintainer of the openblas package, and > > > > knowing whether Altivec is available or not, and more generally what is > > > > in the baseline, is essential for proper packaging). > > > > > > I do not believe that you can do much as a packager. You cannot assume > > > anything on the target arch. You need to do the same thing as ffmpeg > > > is doing for avx2/sse4 on amd64, you need to do runtime detection. So > > > unless upstream is doing something very clever you cannot compile blas > > > using any of the fancy altivec instructions :( > > > > > > The man page for ld.so mentions something about optimized libraries > > > (search for "/usr/lib/sse2/"), but this is currently not in use in > > > Debian (AFAIK). > > > > Actually OpenBLAS has its own runtime detection mechanism, which is > > used to select the best linear algebra kernel for the current CPU > > (those kernels are mainly written in assembly, and take advantage of > > available ISA extensions). This mechanism is used on several archs, > > including ppc64el (so at runtime, OpenBLAS chooses between a POWER8 and > > a POWER9 kernel; there is even a POWER10 kernel already available). > > > > However, I cannot enable this mechanism on ppc64 and powerpc, because > > the runtime detection only works for POWER6 and above, and my > > understanding is that for these two ports the baseline is lower. Hence > > on these two archs, only one kernel is included in the package binaries > > (currently POWER4 for ppc64 and PPCG4 for powerpc). For optimal > > performance, users should recompile OpenBLAS locally (as indicated in > > the package description and in README.Debian). > > There are plenty of people on this mailing list that could test/verify > that. Is there a quick way to check that your openblas package is > compiled correctly for ppc32 and ppc64 (like a verbose mode) ? Did you > do any experiment on perotto.debian.net ? perotto.debian.net is POWER8, so it’s clearly well above the baseline. The package runs fine there, but that does not tell anything about baseline violation. Verifying that the package compiled fine and passed its testsuite on build daemons does not give any information about baseline violation either, because buildds are probably above the baseline as well. FYI, the most recent build logs are there: https://buildd.debian.org/status/package.php?p=openblas=experimental (there is a problem with powerpc in experimental; but the version in sid compiled). If nobody has the relevant knowledge, then the only option is to test the package on the oldest possible hardware. The easiest way to test it is to recompile it locally (since this will exercise the testsuite). Note that nobody complained for years about the situation of openblas on powerpc and ppc64. So maybe that’s a sign that the current setting is fine (either the baseline is respected, or nobody uses the package). -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
Re: Altivec in baseline for ppc64?
On Tue, Jul 13, 2021 at 7:21 PM Sébastien Villemot wrote: > > Hi Mathieu, > > Le mardi 13 juillet 2021 à 18:56 +0200, Mathieu Malaterre a écrit : > > > > On Tue, Jul 13, 2021 at 2:04 PM Sébastien Villemot > > wrote: > > > > > > The wiki page that synthesizes architecture specificities indicates > > > that Altivec is included in the baseline for the ppc64 port: > > > https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 > > > > > > However my understanding is that this port supports any powerpc64 CPU, > > > including some that don’t have Altivec (e.g. POWER4 or POWER5). This is > > > also what the main wiki page for PPC64 says: > > > https://wiki.debian.org/PPC64 > > > > > > Can someone please clarify the situation? > > > > > > (I’m asking because I’m the maintainer of the openblas package, and > > > knowing whether Altivec is available or not, and more generally what is > > > in the baseline, is essential for proper packaging). > > > > I do not believe that you can do much as a packager. You cannot assume > > anything on the target arch. You need to do the same thing as ffmpeg > > is doing for avx2/sse4 on amd64, you need to do runtime detection. So > > unless upstream is doing something very clever you cannot compile blas > > using any of the fancy altivec instructions :( > > > > The man page for ld.so mentions something about optimized libraries > > (search for "/usr/lib/sse2/"), but this is currently not in use in > > Debian (AFAIK). > > Actually OpenBLAS has its own runtime detection mechanism, which is > used to select the best linear algebra kernel for the current CPU > (those kernels are mainly written in assembly, and take advantage of > available ISA extensions). This mechanism is used on several archs, > including ppc64el (so at runtime, OpenBLAS chooses between a POWER8 and > a POWER9 kernel; there is even a POWER10 kernel already available). > > However, I cannot enable this mechanism on ppc64 and powerpc, because > the runtime detection only works for POWER6 and above, and my > understanding is that for these two ports the baseline is lower. Hence > on these two archs, only one kernel is included in the package binaries > (currently POWER4 for ppc64 and PPCG4 for powerpc). For optimal > performance, users should recompile OpenBLAS locally (as indicated in > the package description and in README.Debian). There are plenty of people on this mailing list that could test/verify that. Is there a quick way to check that your openblas package is compiled correctly for ppc32 and ppc64 (like a verbose mode) ? Did you do any experiment on perotto.debian.net ? > I am however not sure that my current choices for the ppc64 and powerpc > baselines are optimal, hence this thread. > > -- > ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot > ⣾⠁⢠⠒⠀⣿⡁ Debian Developer > ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name > ⠈⠳⣄ https://www.debian.org >
Re: Altivec in baseline for ppc64?
Hi Mathieu, Le mardi 13 juillet 2021 à 18:56 +0200, Mathieu Malaterre a écrit : > > On Tue, Jul 13, 2021 at 2:04 PM Sébastien Villemot > wrote: > > > > The wiki page that synthesizes architecture specificities indicates > > that Altivec is included in the baseline for the ppc64 port: > > https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 > > > > However my understanding is that this port supports any powerpc64 CPU, > > including some that don’t have Altivec (e.g. POWER4 or POWER5). This is > > also what the main wiki page for PPC64 says: > > https://wiki.debian.org/PPC64 > > > > Can someone please clarify the situation? > > > > (I’m asking because I’m the maintainer of the openblas package, and > > knowing whether Altivec is available or not, and more generally what is > > in the baseline, is essential for proper packaging). > > I do not believe that you can do much as a packager. You cannot assume > anything on the target arch. You need to do the same thing as ffmpeg > is doing for avx2/sse4 on amd64, you need to do runtime detection. So > unless upstream is doing something very clever you cannot compile blas > using any of the fancy altivec instructions :( > > The man page for ld.so mentions something about optimized libraries > (search for "/usr/lib/sse2/"), but this is currently not in use in > Debian (AFAIK). Actually OpenBLAS has its own runtime detection mechanism, which is used to select the best linear algebra kernel for the current CPU (those kernels are mainly written in assembly, and take advantage of available ISA extensions). This mechanism is used on several archs, including ppc64el (so at runtime, OpenBLAS chooses between a POWER8 and a POWER9 kernel; there is even a POWER10 kernel already available). However, I cannot enable this mechanism on ppc64 and powerpc, because the runtime detection only works for POWER6 and above, and my understanding is that for these two ports the baseline is lower. Hence on these two archs, only one kernel is included in the package binaries (currently POWER4 for ppc64 and PPCG4 for powerpc). For optimal performance, users should recompile OpenBLAS locally (as indicated in the package description and in README.Debian). I am however not sure that my current choices for the ppc64 and powerpc baselines are optimal, hence this thread. -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
Re: Altivec in baseline for ppc64?
Hi Sébastien, On Tue, Jul 13, 2021 at 2:04 PM Sébastien Villemot wrote: > > Hi, > > The wiki page that synthesizes architecture specificities indicates > that Altivec is included in the baseline for the ppc64 port: > https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 > > However my understanding is that this port supports any powerpc64 CPU, > including some that don’t have Altivec (e.g. POWER4 or POWER5). This is > also what the main wiki page for PPC64 says: > https://wiki.debian.org/PPC64 > > Can someone please clarify the situation? > > (I’m asking because I’m the maintainer of the openblas package, and > knowing whether Altivec is available or not, and more generally what is > in the baseline, is essential for proper packaging). I do not believe that you can do much as a packager. You cannot assume anything on the target arch. You need to do the same thing as ffmpeg is doing for avx2/sse4 on amd64, you need to do runtime detection. So unless upstream is doing something very clever you cannot compile blas using any of the fancy altivec instructions :( The man page for ld.so mentions something about optimized libraries (search for "/usr/lib/sse2/"), but this is currently not in use in Debian (AFAIK). 2cts
Altivec in baseline for ppc64?
Hi, The wiki page that synthesizes architecture specificities indicates that Altivec is included in the baseline for the ppc64 port: https://wiki.debian.org/ArchitectureSpecificsMemo#ppc64 However my understanding is that this port supports any powerpc64 CPU, including some that don’t have Altivec (e.g. POWER4 or POWER5). This is also what the main wiki page for PPC64 says: https://wiki.debian.org/PPC64 Can someone please clarify the situation? (I’m asking because I’m the maintainer of the openblas package, and knowing whether Altivec is available or not, and more generally what is in the baseline, is essential for proper packaging). -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
