Re: [OE-core] ARM tunings was Re: [PATCH 3/7] conf/machine/include: Cleanup MIPS tunings to match README
On Mon, 2012-04-09 at 16:44 -0500, Mark Hatle wrote: Depends on the distribution and reasons for these feeds. What is typical is that a base distribution will be generated for a common compatible (reasonable) architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, etc) for the target arch, i.e. armv7a. Then you have a couple of packages hand-tuned for size, speed, or other that define or not thumb and add even a higher level of optimization. It's possible, folks do it today.. but it's not always obvious. (I have existing customers today that run a mix like I described through their own package feed like system. They really don't care at all that the core system is tuned for a given processor -- they only care that their specific applications and certain areas are specifically tuned to their use-cases.) Note, this is not what I would consider a typical use-case! Sorry, I'm not quite sure I understand what point you're trying to make here. Are you describing what your customers are currently doing with OE, or are you saying that this is something that they would like to do with OE but don't feel they are able to at present, or something else? I'm still not entirely clear on what you feel is broken about the current state of the ARM tunings. What exactly is the scenario that works with the traditional workstaton/server Linux OS and can't be replicated with OE? But, all that aside, it seems ultimately that the exact way the PACKAGE_ARCHs are structured ought to be a DISTRO policy decision and not something that's mandated by the underlying infrastructure. That would perhaps remove some of the need for tinkering with these things in oe-core itself. p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] ARM tunings was Re: [PATCH 3/7] conf/machine/include: Cleanup MIPS tunings to match README
On 4/10/12 4:23 AM, Phil Blundell wrote: On Mon, 2012-04-09 at 16:44 -0500, Mark Hatle wrote: Depends on the distribution and reasons for these feeds. What is typical is that a base distribution will be generated for a common compatible (reasonable) architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, etc) for the target arch, i.e. armv7a. Then you have a couple of packages hand-tuned for size, speed, or other that define or not thumb and add even a higher level of optimization. It's possible, folks do it today.. but it's not always obvious. (I have existing customers today that run a mix like I described through their own package feed like system. They really don't care at all that the core system is tuned for a given processor -- they only care that their specific applications and certain areas are specifically tuned to their use-cases.) Note, this is not what I would consider a typical use-case! Sorry, I'm not quite sure I understand what point you're trying to make here. Are you describing what your customers are currently doing with OE, or are you saying that this is something that they would like to do with OE but don't feel they are able to at present, or something else? The company I work for has an existing product that does not use OE. The customers using this have requested from us packages tagged with different package architectures to indicate the tuning and optimization information so that they can create multiple boards with the same software running on them. (This is closer to the traditional workstation/server model in my experience.) The multiple boards have a common set of OS applications that run on them. ARMv5 for the most part. The customers then have optimized applications with or without thumb, and optimized for a variety of different ARM parts. They then use the binary packages to assemble the filesystems, and perform field upgrades, on these boards as they are put into use. The installation system uses a best to least best match when doing assembly actions. So if the part is an ARMv7a, it will first look for ARMv7a w/ thumb, vfp and neon, not finding that, ARMv7a w/ thumb and vfp, then ARMv7a w/ thumb, then ARMv5 w/ thumb and vfp, then ARMv5 w/ thumb, and finally fall back to the ARMv5 binaries. I'm still not entirely clear on what you feel is broken about the current state of the ARM tunings. What exactly is the scenario that works with the traditional workstaton/server Linux OS and can't be replicated with OE? With the current implementation, there is no package differentiation between thumb and non-thumb binaries. I accept why this is in OE-core and I can live with that. However, there are other binaries that are theoretically optimized from specific Cortex models to the more generic ARMv7a tunings. Currently they all use the same package arch, which means I can't tell which CPU they're really for -- and this model (above) of best to least best match doesn't work. On a pure embedded model, I doubt anyone would do this. Thus it is a fairly unique embedded use-case, but a common Workstation/Server use case that is being replicated. But, all that aside, it seems ultimately that the exact way the PACKAGE_ARCHs are structured ought to be a DISTRO policy decision and not something that's mandated by the underlying infrastructure. That would perhaps remove some of the need for tinkering with these things in oe-core itself. I intend, after this release, to propose changes to differentiate the models in oe-core. If the oe-core folks do not feel this is necessary, they I will maintain them on my own as I feel necessary to cover the above use-case. I can very much understand that in OE, for ARM specifically the package arch is simply indicating basic compatibility and not ABI ISA Optimization like it is on other architectures. --Mark p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] ARM tunings was Re: [PATCH 3/7] conf/machine/include: Cleanup MIPS tunings to match README
On Tue, 2012-04-10 at 12:39 -0500, Mark Hatle wrote: The installation system uses a best to least best match when doing assembly actions. So if the part is an ARMv7a, it will first look for ARMv7a w/ thumb, vfp and neon, not finding that, ARMv7a w/ thumb and vfp, then ARMv7a w/ thumb, then ARMv5 w/ thumb and vfp, then ARMv5 w/ thumb, and finally fall back to the ARMv5 binaries. This sounds like exactly the behaviour you would get with the current ARM tunings (except the Thumb bit which, as previously discussed, I think is somewhat misguided in the first place). The existing ARM tunings do seem to correctly encode VFP and Neon-ness. Which part isn't working for you? Maybe you could give a concrete example of where exactly it falls down. I can very much understand that in OE, for ARM specifically the package arch is simply indicating basic compatibility and not ABI ISA Optimization like it is on other architectures. Well, I would consider ABI ISA to be a fairly big part of basic compatibility. It is true that we don't currently encode optimisations, but as I previously mentioned I don't think many (perhaps any) other distributions do that either, and it's perhaps debatable whether it would be a very useful thing to do in the general case. For individual packages you can obviously force the issue in your DISTRO configuration anyway. p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
[OE-core] ARM tunings was Re: [PATCH 3/7] conf/machine/include: Cleanup MIPS tunings to match README
On 4/9/12 4:03 PM, Phil Blundell wrote: On Mon, 2012-04-09 at 15:25 -0500, Mark Hatle wrote: And just to be extra clear, I consider it a defect if we can produce a package with the same name for two different tune settings.. (the exception being the hell that is ARM and thumb namings.) While you might consider that a defect (and it probably is a defensible position to do so), it hasn't historically been considered such in OE. The PACKAGE_ARCH value has, traditionally, been concerned purely with ISA and ABI (i.e. answering the question can I execute this code?) rather than optimisations. For example, the tune-arm926ejs.inc and tune-xscale.inc files in current oe-core both end up setting PACKAGE_ARCH to armv5tte (sic). But those are quite different processors and have different tuning requirements, so the binaries you get are unlikely to be the same. If you were to take the view that the PACKAGE_ARCH must uniquely identify one set of binaries then obviously each of these tunings (and probably all the ARM cpu-specific tunings) would need to set PACKAGE_ARCH to some unique value. I do, and thus the hell that is ARM. I could not currently generate a single package feed that work would on a variety of devices (like a traditional workstaton/server Linux OS would.) While this isn't a big issue in the embedded space where we should hopefully be aware of the tunings and configuration were are using, it is still a problem. As the systems get larger, the requirement for common pages feeds increases, leading to the problem being, well a problem. (ARM is starting to be considered for Carrier Grade systems, many of which have very common requirements to traditional server Linux. A set of established binaries and the vendors just want to drop in optimized applications.) On ARM what we currently have defined is: (tune) - (package arch) arm1136jfs - armv6-vfp arm920t - armv4t arm926ejs - armv5te arm9tdmi - armv4t armv4b - armv4b armv4tb - armv4tb armv4 - armv4 armv4t - armv4t armv5b - armv5b armv5b-vfp - armv5b-vfp armv5eb - armv5eb armv5eb-vfp - armv5eb-vfp armv5ehfb-vfp - armv5ehfb-vfp armv5ehf-vfp - armv5ehf-vfp armv5e-vfp - armv5e-vfp armv5hfb-vfp - armv5hfb-vfp armv5hf-vfp - armv5hf-vfp armv5tb - armv5tb armv5tb-vfp - armv5b-vfp armv5teb - armv5teb armv5teb-vfp - armv5teb-vfp armv5tehfb-vfp - armv5tehfb-vfp armv5tehf-vfp - armv5tehf-vfp armv5te - armv5te armv5te-vfp - armv5te-vfp armv5thfb-vfp - armv5hfb-vfp armv5thf-vfp - armv5hf-vfp armv5 - armv5 armv5t - armv5t armv5t-vfp - armv5-vfp armv5-vfp - armv5-vfp armv6b - armv6b-vfp armv6hfb - armv6hfb-vfp armv6hf - armv6hf-vfp armv6tb - armv6tb-vfp armv6thfb - armv6thfb-vfp armv6thf - armv6thf-vfp armv6 - armv6-vfp armv6t - armv6t-vfp armv7ab-neon - armv7ab-vfp-neon armv7ab - armv7ab-vfp armv7ahfb-neon - armv7ahfb-vfp-neon armv7ahfb - armv7ahfb-vfp armv7ahf-neon - armv7ahf-vfp-neon armv7ahf - armv7ahf-vfp armv7a-neon - armv7a-vfp-neon armv7atb-neon - armv7ab-vfp-neon armv7atb - armv7ab-vfp armv7athfb - armv7ahfb-vfp armv7athf-neon - armv7ahf-vfp-neon armv7athf - armv7ahf-vfp armv7a - armv7a-vfp armv7at-neon - armv7a-vfp-neon armv7at - armv7a-vfp cortexa8hf-neon - armv7ahf-vfp-neon cortexa8hf - armv7ahf-vfp cortexa8-neon - armv7a-vfp-neon cortexa8thf - armv7ahf-vfp cortexa8 - armv7a-vfp cortexa8t - armv7a-vfp cortexa9hf-neon - armv7ahf-vfp-neon cortexa9hf - armv7ahf-vfp cortexa9-neon - armv7a-vfp-neon cortexa9thf - armv7ahf-vfp cortexa9 - armv7a-vfp cortexa9t - armv7a-vfp cortexm1 - armv7a-vfp cortexm3 - armv7m-vfp cortexr4 - armv7r-vfp ep9312 - ep9312 iwmmxt - iwmmxt strongarm - armv4 Add in to that one of the tunings -- not indicated by the package arch of thumb enabled or not, and its difficult to know exactly what is in a package without extracting and examining it. But I consider this to be a quirk of the ARM architecture as implemented in OE. --Mark p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] ARM tunings was Re: [PATCH 3/7] conf/machine/include: Cleanup MIPS tunings to match README
On Mon, 2012-04-09 at 16:21 -0500, Mark Hatle wrote: I do, and thus the hell that is ARM. I could not currently generate a single package feed that work would on a variety of devices (like a traditional workstaton/server Linux OS would.) Well, actually, you could in fact do exactly that. What you couldn't necessarily do with the tunings as they exist right now is generate a package feed which is optimised for (as opposed to works on) all those devices. But it isn't clear to me that you could do that with a traditional workstaton/server kind of distribution either. In the x86 world, for example, the majority of the big distros do not bother to ship individually-tuned binaries for different processor types, certainly not for the entire distribution. Add in to that one of the tunings -- not indicated by the package arch of thumb enabled or not There are multiple reasons why this isn't indicated by the PACKAGE_ARCH. Firstly, it's irrelevant: on v5T or newer, the question of whether a given package is using Thumb-state or not has no ABI impact and there is no reason for anyone to care at a compatibility level. Second, it may be unpredictable: the compiler is at liberty (although current versions of gcc don't exploit this latitude) to switch arbitrarily between ARM-state and Thumb-state as it sees fit to get the best performance. And thirdly, it's just another piece of distro policy in the same way as compiling for -O2 vs -Os (which we also don't encode into PACKAGE_ARCH) is. p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] ARM tunings was Re: [PATCH 3/7] conf/machine/include: Cleanup MIPS tunings to match README
On 4/9/12 4:30 PM, Phil Blundell wrote: On Mon, 2012-04-09 at 16:21 -0500, Mark Hatle wrote: I do, and thus the hell that is ARM. I could not currently generate a single package feed that work would on a variety of devices (like a traditional workstaton/server Linux OS would.) Well, actually, you could in fact do exactly that. What you couldn't necessarily do with the tunings as they exist right now is generate a package feed which is optimised for (as opposed to works on) all those devices. But it isn't clear to me that you could do that with a traditional workstaton/server kind of distribution either. In the x86 world, for example, the majority of the big distros do not bother to ship individually-tuned binaries for different processor types, certainly not for the entire distribution. Depends on the distribution and reasons for these feeds. What is typical is that a base distribution will be generated for a common compatible (reasonable) architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, etc) for the target arch, i.e. armv7a. Then you have a couple of packages hand-tuned for size, speed, or other that define or not thumb and add even a higher level of optimization. It's possible, folks do it today.. but it's not always obvious. (I have existing customers today that run a mix like I described through their own package feed like system. They really don't care at all that the core system is tuned for a given processor -- they only care that their specific applications and certain areas are specifically tuned to their use-cases.) Note, this is not what I would consider a typical use-case! Add in to that one of the tunings -- not indicated by the package arch of thumb enabled or not There are multiple reasons why this isn't indicated by the PACKAGE_ARCH. Firstly, it's irrelevant: on v5T or newer, the question of whether a given package is using Thumb-state or not has no ABI impact and there is no reason for anyone to care at a compatibility level. Second, it may be unpredictable: the compiler is at liberty (although current versions of gcc don't exploit this latitude) to switch arbitrarily between ARM-state and Thumb-state as it sees fit to get the best performance. And thirdly, it's just another piece of distro policy in the same way as compiling for -O2 vs -Os (which we also don't encode into PACKAGE_ARCH) is. I agree, on ARM the tunings and optimizations between regular and thumb do not impact the ABI what-so-ever. And so far compilers have to be explicitly set to do thumb or tranditional ARM mode.. so in the end developers are looking into the performance and size impacts of each of these configuration and making changes as they see fit to best meet their needs. These are unique cases though, the majority of the software built for the core OS uses a single policy -- it's when something needs to be further optimized that this comes into play. At this point, I'd like to better differentiate the ARM package arches.. I don't care so much about the thumb enabled or not.. but the other tune settings are things I do care about. I started to change that for the last update and decided it was a rat-hole I was not willing to go down at this point. At some point in the future, I will look at, and document the differences in the tunings according to GCC configurations -- to get a good idea of what is and isn't producing the same binaries based on various arch and tune settings. --Mark p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
