Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 16.04.2012 17:30, Mark Hatle wrote: On 4/16/12 10:15 AM, Andreas Oberritter wrote: On 16.04.2012 16:42, Richard Purdie wrote: On Mon, 2012-04-16 at 15:55 +0200, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote: now, after having repacked all binary tarballs that had mipsel or mipsel-nf in their name and contents, and after having changed all occurrences of mipsel and mipsel-nf in my local recipes (where appropriate), and after having rebuilt everything from scratch again, it came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg, because no mipsel packages are being generated. That's what I told before, right? How is this breaking opkg? We often have architectures listed in there for which there are no packages generated (all, noarch and any spring to mind)? Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 18.04.2012 13:40, Richard Purdie wrote: On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote: now, after having repacked all binary tarballs that had mipsel or mipsel-nf in their name and contents, and after having changed all occurrences of mipsel and mipsel-nf in my local recipes (where appropriate), and after having rebuilt everything from scratch again, it came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg, because no mipsel packages are being generated. That's what I told before, right? How is this breaking opkg? We often have architectures listed in there for which there are no packages generated (all, noarch and any spring to mind)? Downloading http://10.0.0.1/mipsel/Packages.gz. wget: server returned error: HTTP/1.1 404 Not Found Collected errors: * opkg_download: Failed to download http://10.0.0.1/mipsel/Packages.gz, wget returned 1. Regards, Andreas ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 18.04.2012 13:45, Andreas Oberritter wrote:
On 18.04.2012 13:40, Richard Purdie wrote:
On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote:
now, after having repacked all binary tarballs that had mipsel or
mipsel-nf in their name and contents, and after having changed all
occurrences of mipsel and mipsel-nf in my local recipes (where
appropriate), and after having rebuilt everything from scratch again, it
came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg,
because no mipsel packages are being generated. That's what I told
before, right?
How is this breaking opkg? We often have architectures listed in there
for which there are no packages generated (all, noarch and any spring to
mind)?
Downloading http://10.0.0.1/mipsel/Packages.gz.
wget: server returned error: HTTP/1.1 404 Not Found
Collected errors:
* opkg_download: Failed to download http://10.0.0.1/mipsel/Packages.gz, wget
returned 1.
That is, because although noarch and any are part of ${PACKAGE_ARCHS},
they are not part of ${PACKAGE_EXTRA_ARCHS}.
distro-feed-configs.bb uses all ${PACKAGE_EXTRA_ARCHS} ${MACHINE_ARCH}.
Regards,
Andreas
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Wed, Apr 18, 2012 at 01:45:54PM +0200, Andreas Oberritter wrote: On 18.04.2012 13:40, Richard Purdie wrote: On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote: now, after having repacked all binary tarballs that had mipsel or mipsel-nf in their name and contents, and after having changed all occurrences of mipsel and mipsel-nf in my local recipes (where appropriate), and after having rebuilt everything from scratch again, it came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg, because no mipsel packages are being generated. That's what I told before, right? How is this breaking opkg? We often have architectures listed in there for which there are no packages generated (all, noarch and any spring to mind)? Downloading http://10.0.0.1/mipsel/Packages.gz. wget: server returned error: HTTP/1.1 404 Not Found Collected errors: * opkg_download: Failed to download http://10.0.0.1/mipsel/Packages.gz, wget returned 1. I had a lot of those (e.g. because armv7a-vfp-neon was including 20 arm*feed.conf variants in /etc/opkg most of them empty - without Packages.gz). So I've added filter to distro-feed-configs http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa to add only feeds I'm generating (and I also don't want armv5* packages installed on armv7a-vfp-neon target unless user explicitly adds armv5* feed). Cheers, -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote: On Wed, Apr 18, 2012 at 01:45:54PM +0200, Andreas Oberritter wrote: On 18.04.2012 13:40, Richard Purdie wrote: On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote: now, after having repacked all binary tarballs that had mipsel or mipsel-nf in their name and contents, and after having changed all occurrences of mipsel and mipsel-nf in my local recipes (where appropriate), and after having rebuilt everything from scratch again, it came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg, because no mipsel packages are being generated. That's what I told before, right? How is this breaking opkg? We often have architectures listed in there for which there are no packages generated (all, noarch and any spring to mind)? Downloading http://10.0.0.1/mipsel/Packages.gz. wget: server returned error: HTTP/1.1 404 Not Found Collected errors: * opkg_download: Failed to download http://10.0.0.1/mipsel/Packages.gz, wget returned 1. I had a lot of those (e.g. because armv7a-vfp-neon was including 20 arm*feed.conf variants in /etc/opkg most of them empty - without Packages.gz). So I've added filter to distro-feed-configs http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa to add only feeds I'm generating (and I also don't want armv5* packages installed on armv7a-vfp-neon target unless user explicitly adds armv5* feed). This is the better solution. I think we need to get a better default feed-config generation mechanism into the core. Distros may still need to tweak it but it would be good to share some of the best practises... Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 18.04.2012 14:00, Richard Purdie wrote: On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote: On Wed, Apr 18, 2012 at 01:45:54PM +0200, Andreas Oberritter wrote: On 18.04.2012 13:40, Richard Purdie wrote: On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote: now, after having repacked all binary tarballs that had mipsel or mipsel-nf in their name and contents, and after having changed all occurrences of mipsel and mipsel-nf in my local recipes (where appropriate), and after having rebuilt everything from scratch again, it came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg, because no mipsel packages are being generated. That's what I told before, right? How is this breaking opkg? We often have architectures listed in there for which there are no packages generated (all, noarch and any spring to mind)? Downloading http://10.0.0.1/mipsel/Packages.gz. wget: server returned error: HTTP/1.1 404 Not Found Collected errors: * opkg_download: Failed to download http://10.0.0.1/mipsel/Packages.gz, wget returned 1. I had a lot of those (e.g. because armv7a-vfp-neon was including 20 arm*feed.conf variants in /etc/opkg most of them empty - without Packages.gz). So I've added filter to distro-feed-configs http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa to add only feeds I'm generating (and I also don't want armv5* packages installed on armv7a-vfp-neon target unless user explicitly adds armv5* feed). This is the better solution. I think we need to get a better default feed-config generation mechanism into the core. Distros may still need to tweak it but it would be good to share some of the best practises... Did you look at the patch? Which default setting of SUPPORTED_EXTRA_ARCHS do you suggest? Do you think it's feasible to add every single downloadable arch to this variable? If a user of my distro decides to build it for some arm or x86 cpu, should he need to know which archs to add at this place? I don't think that's user-friendly and I don't know what's so bad about removing something that probably hasn't helped anybody. Regards, Andreas ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
Op 18 apr. 2012, om 14:00 heeft Richard Purdie het volgende geschreven: On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote: On Wed, Apr 18, 2012 at 01:45:54PM +0200, Andreas Oberritter wrote: On 18.04.2012 13:40, Richard Purdie wrote: On Wed, 2012-04-18 at 13:30 +0200, Andreas Oberritter wrote: now, after having repacked all binary tarballs that had mipsel or mipsel-nf in their name and contents, and after having changed all occurrences of mipsel and mipsel-nf in my local recipes (where appropriate), and after having rebuilt everything from scratch again, it came to my attention that mipsel in PACKAGE_EXTRA_ARCHS breaks opkg, because no mipsel packages are being generated. That's what I told before, right? How is this breaking opkg? We often have architectures listed in there for which there are no packages generated (all, noarch and any spring to mind)? Downloading http://10.0.0.1/mipsel/Packages.gz. wget: server returned error: HTTP/1.1 404 Not Found Collected errors: * opkg_download: Failed to download http://10.0.0.1/mipsel/Packages.gz, wget returned 1. I had a lot of those (e.g. because armv7a-vfp-neon was including 20 arm*feed.conf variants in /etc/opkg most of them empty - without Packages.gz). So I've added filter to distro-feed-configs http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa to add only feeds I'm generating (and I also don't want armv5* packages installed on armv7a-vfp-neon target unless user explicitly adds armv5* feed). This is the better solution. I think we need to get a better default feed-config generation mechanism into the core. Distros may still need to tweak it but it would be good to share some of the best practises... That's exactly why I invented FEED_ARCH a few years ago, which is now changed to BASE_PACKAGE_ARCH. The angstrom-feed-configs recipe which distro-feed-configs was based on doesn't support from the the above drawbacks. This discussion about using all PACKAGE_*_ARCHS to generate URLs pops up every other year with people concluding that distro-feed-configs sucks. I can only recommend to have an integrated solution to manage your binary feeds instead of trying to fit it into the (broken) distro-feed-config model. regards, Koen ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote: On 18.04.2012 14:00, Richard Purdie wrote: On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote: I had a lot of those (e.g. because armv7a-vfp-neon was including 20 arm*feed.conf variants in /etc/opkg most of them empty - without Packages.gz). So I've added filter to distro-feed-configs http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa to add only feeds I'm generating (and I also don't want armv5* packages installed on armv7a-vfp-neon target unless user explicitly adds armv5* feed). This is the better solution. I think we need to get a better default feed-config generation mechanism into the core. Distros may still need to tweak it but it would be good to share some of the best practises... Did you look at the patch? Which default setting of SUPPORTED_EXTRA_ARCHS do you suggest? I did. I didn't say the above patch was a perfect solution. Do you think it's feasible to add every single downloadable arch to this variable? If a user of my distro decides to build it for some arm or x86 cpu, should he need to know which archs to add at this place? This is a place where the build system meets and interfaces with the distro. No one policy in the build system is going to fit every distro's needs, not should we ever aim to so. I think my comment above shows what I think is the correct intention, we should aim to find common ground and share solutions where possible. You could simply decide your distro only supports PACKAGE_ARCH for a given target, you could also decide to filter through the list PACKAGE_ARCHs and remove certain values based on a whitelist or blacklist. Its a policy decision and one the distro needs to decide on ultimately. I don't think that's user-friendly and I don't know what's so bad about removing something that probably hasn't helped anybody. I disagree. I am sorry you're feeling some pain on this particular topic, that wasn't intended and I'm hoping we can get that resolved and move forward quickly. Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 18.04.2012 14:45, Richard Purdie wrote: On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote: On 18.04.2012 14:00, Richard Purdie wrote: On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote: I had a lot of those (e.g. because armv7a-vfp-neon was including 20 arm*feed.conf variants in /etc/opkg most of them empty - without Packages.gz). So I've added filter to distro-feed-configs http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa to add only feeds I'm generating (and I also don't want armv5* packages installed on armv7a-vfp-neon target unless user explicitly adds armv5* feed). This is the better solution. I think we need to get a better default feed-config generation mechanism into the core. Distros may still need to tweak it but it would be good to share some of the best practises... Did you look at the patch? Which default setting of SUPPORTED_EXTRA_ARCHS do you suggest? I did. I didn't say the above patch was a perfect solution. Do you think it's feasible to add every single downloadable arch to this variable? If a user of my distro decides to build it for some arm or x86 cpu, should he need to know which archs to add at this place? This is a place where the build system meets and interfaces with the distro. No one policy in the build system is going to fit every distro's needs, not should we ever aim to so. At least we should have defaults that actually work for someone. Now we don't and considering that distro-feed-configs.bb is the only place where PACKAGE_EXTRA_ARCHS is actually used, this would be very easy to accomplish. Especially because it worked well by default before Mark broke it. I guess it's indeed better to just override the necessary bits in my distro instead of trying to get working defaults upstream. Regards, Andreas ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Wed, Apr 18, 2012 at 10:21:38AM -0500, Mark Hatle wrote:
On 4/18/12 9:37 AM, Andreas Oberritter wrote:
On 18.04.2012 14:45, Richard Purdie wrote:
On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote:
On 18.04.2012 14:00, Richard Purdie wrote:
On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote:
I had a lot of those (e.g. because armv7a-vfp-neon was including 20
arm*feed.conf variants in /etc/opkg most of them empty - without
Packages.gz).
So I've added filter to distro-feed-configs
http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa
to add only feeds I'm generating (and I also don't want armv5* packages
installed on armv7a-vfp-neon target unless user explicitly adds armv5*
feed).
This is the better solution. I think we need to get a better default
feed-config generation mechanism into the core. Distros may still need
to tweak it but it would be good to share some of the best practises...
Did you look at the patch? Which default setting of
SUPPORTED_EXTRA_ARCHS do you suggest?
I did. I didn't say the above patch was a perfect solution.
Do you think it's feasible to add
every single downloadable arch to this variable? If a user of my distro
decides to build it for some arm or x86 cpu, should he need to know
which archs to add at this place?
This is a place where the build system meets and interfaces with the
distro. No one policy in the build system is going to fit every distro's
needs, not should we ever aim to so.
At least we should have defaults that actually work for someone. Now we
don't and considering that distro-feed-configs.bb is the only place
where PACKAGE_EXTRA_ARCHS is actually used, this would be very easy to
accomplish. Especially because it worked well by default before Mark
broke it.
PACKAGE_EXTRA_ARCHS is also used by Zypper, RPM configuration and other
places.
In those cases it is a full list of all available (and compatible) package
architecture types.
Coming from the RPM world, it seems very odd to me that a set of
extra_archs
can not list well, extra compatible archs without causing an error. I have
no
idea how to reconcile this behavior, without making a package manager
distro-feed specific solution. (For RPM we absolutely want the existing
behavior.)
The problem Andreas is seeing is not fatal AFAIK.. just couple (or a
lot) of 404 (Packages files not available) while doing opkg update is
not nice for end user.
Downloading many existing Packages files without any Package in it
is also suboptimal, but maybe good start.. so we can teach
meta/classes/package_ipk.bbclass:package_update_index_ipk() to create
Packages files not only for existing
ipkgarchs=${ALL_MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
but for all (replace if [ -e $pkgdir/ ]; then with something like
if [ ! -e $pkgdir/ ]; then mkdir -p $pkgdir; fi)
Cheers,
I guess it's indeed better to just override the necessary bits in my
distro instead of trying to get working defaults upstream.
Regards,
Andreas
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
--
Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
signature.asc
Description: Digital signature
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
Op 18 apr. 2012, om 17:46 heeft Martin Jansa het volgende geschreven:
On Wed, Apr 18, 2012 at 10:21:38AM -0500, Mark Hatle wrote:
On 4/18/12 9:37 AM, Andreas Oberritter wrote:
On 18.04.2012 14:45, Richard Purdie wrote:
On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote:
On 18.04.2012 14:00, Richard Purdie wrote:
On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote:
I had a lot of those (e.g. because armv7a-vfp-neon was including 20
arm*feed.conf variants in /etc/opkg most of them empty - without
Packages.gz).
So I've added filter to distro-feed-configs
http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa
to add only feeds I'm generating (and I also don't want armv5* packages
installed on armv7a-vfp-neon target unless user explicitly adds armv5*
feed).
This is the better solution. I think we need to get a better default
feed-config generation mechanism into the core. Distros may still need
to tweak it but it would be good to share some of the best practises...
Did you look at the patch? Which default setting of
SUPPORTED_EXTRA_ARCHS do you suggest?
I did. I didn't say the above patch was a perfect solution.
Do you think it's feasible to add
every single downloadable arch to this variable? If a user of my distro
decides to build it for some arm or x86 cpu, should he need to know
which archs to add at this place?
This is a place where the build system meets and interfaces with the
distro. No one policy in the build system is going to fit every distro's
needs, not should we ever aim to so.
At least we should have defaults that actually work for someone. Now we
don't and considering that distro-feed-configs.bb is the only place
where PACKAGE_EXTRA_ARCHS is actually used, this would be very easy to
accomplish. Especially because it worked well by default before Mark
broke it.
PACKAGE_EXTRA_ARCHS is also used by Zypper, RPM configuration and other
places.
In those cases it is a full list of all available (and compatible) package
architecture types.
Coming from the RPM world, it seems very odd to me that a set of
extra_archs
can not list well, extra compatible archs without causing an error. I have
no
idea how to reconcile this behavior, without making a package manager
distro-feed specific solution. (For RPM we absolutely want the existing
behavior.)
The problem Andreas is seeing is not fatal AFAIK.. just couple (or a
lot) of 404 (Packages files not available) while doing opkg update is
not nice for end user.
Downloading many existing Packages files without any Package in it
is also suboptimal, but maybe good start.. so we can teach
meta/classes/package_ipk.bbclass:package_update_index_ipk() to create
Packages files not only for existing
ipkgarchs=${ALL_MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
but for all (replace if [ -e $pkgdir/ ]; then with something like
if [ ! -e $pkgdir/ ]; then mkdir -p $pkgdir; fi)
That implies you're exposing feeds straight from OE, which is a bad, bad idea.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 18.04.2012 19:01, Koen Kooi wrote:
Op 18 apr. 2012, om 17:46 heeft Martin Jansa het volgende geschreven:
On Wed, Apr 18, 2012 at 10:21:38AM -0500, Mark Hatle wrote:
On 4/18/12 9:37 AM, Andreas Oberritter wrote:
On 18.04.2012 14:45, Richard Purdie wrote:
On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote:
On 18.04.2012 14:00, Richard Purdie wrote:
On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote:
I had a lot of those (e.g. because armv7a-vfp-neon was including 20
arm*feed.conf variants in /etc/opkg most of them empty - without
Packages.gz).
So I've added filter to distro-feed-configs
http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa
to add only feeds I'm generating (and I also don't want armv5* packages
installed on armv7a-vfp-neon target unless user explicitly adds armv5*
feed).
This is the better solution. I think we need to get a better default
feed-config generation mechanism into the core. Distros may still need
to tweak it but it would be good to share some of the best practises...
Did you look at the patch? Which default setting of
SUPPORTED_EXTRA_ARCHS do you suggest?
I did. I didn't say the above patch was a perfect solution.
Do you think it's feasible to add
every single downloadable arch to this variable? If a user of my distro
decides to build it for some arm or x86 cpu, should he need to know
which archs to add at this place?
This is a place where the build system meets and interfaces with the
distro. No one policy in the build system is going to fit every distro's
needs, not should we ever aim to so.
At least we should have defaults that actually work for someone. Now we
don't and considering that distro-feed-configs.bb is the only place
where PACKAGE_EXTRA_ARCHS is actually used, this would be very easy to
accomplish. Especially because it worked well by default before Mark
broke it.
PACKAGE_EXTRA_ARCHS is also used by Zypper, RPM configuration and other
places.
In those cases it is a full list of all available (and compatible) package
architecture types.
Coming from the RPM world, it seems very odd to me that a set of
extra_archs
can not list well, extra compatible archs without causing an error. I have
no
idea how to reconcile this behavior, without making a package manager
distro-feed specific solution. (For RPM we absolutely want the existing
behavior.)
The problem Andreas is seeing is not fatal AFAIK.. just couple (or a
lot) of 404 (Packages files not available) while doing opkg update is
not nice for end user.
Downloading many existing Packages files without any Package in it
is also suboptimal, but maybe good start.. so we can teach
meta/classes/package_ipk.bbclass:package_update_index_ipk() to create
Packages files not only for existing
ipkgarchs=${ALL_MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
but for all (replace if [ -e $pkgdir/ ]; then with something like
if [ ! -e $pkgdir/ ]; then mkdir -p $pkgdir; fi)
That implies you're exposing feeds straight from OE, which is a bad, bad idea.
Can you please elaborate on why this is a bad idea?
Regards,
Andreas
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
Op 18 apr. 2012, om 20:10 heeft Andreas Oberritter het volgende geschreven:
On 18.04.2012 19:01, Koen Kooi wrote:
Op 18 apr. 2012, om 17:46 heeft Martin Jansa het volgende geschreven:
On Wed, Apr 18, 2012 at 10:21:38AM -0500, Mark Hatle wrote:
On 4/18/12 9:37 AM, Andreas Oberritter wrote:
On 18.04.2012 14:45, Richard Purdie wrote:
On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote:
On 18.04.2012 14:00, Richard Purdie wrote:
On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote:
I had a lot of those (e.g. because armv7a-vfp-neon was including 20
arm*feed.conf variants in /etc/opkg most of them empty - without
Packages.gz).
So I've added filter to distro-feed-configs
http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa
to add only feeds I'm generating (and I also don't want armv5*
packages
installed on armv7a-vfp-neon target unless user explicitly adds armv5*
feed).
This is the better solution. I think we need to get a better default
feed-config generation mechanism into the core. Distros may still need
to tweak it but it would be good to share some of the best practises...
Did you look at the patch? Which default setting of
SUPPORTED_EXTRA_ARCHS do you suggest?
I did. I didn't say the above patch was a perfect solution.
Do you think it's feasible to add
every single downloadable arch to this variable? If a user of my distro
decides to build it for some arm or x86 cpu, should he need to know
which archs to add at this place?
This is a place where the build system meets and interfaces with the
distro. No one policy in the build system is going to fit every distro's
needs, not should we ever aim to so.
At least we should have defaults that actually work for someone. Now we
don't and considering that distro-feed-configs.bb is the only place
where PACKAGE_EXTRA_ARCHS is actually used, this would be very easy to
accomplish. Especially because it worked well by default before Mark
broke it.
PACKAGE_EXTRA_ARCHS is also used by Zypper, RPM configuration and other
places.
In those cases it is a full list of all available (and compatible) package
architecture types.
Coming from the RPM world, it seems very odd to me that a set of
extra_archs
can not list well, extra compatible archs without causing an error. I
have no
idea how to reconcile this behavior, without making a package manager
distro-feed specific solution. (For RPM we absolutely want the existing
behavior.)
The problem Andreas is seeing is not fatal AFAIK.. just couple (or a
lot) of 404 (Packages files not available) while doing opkg update is
not nice for end user.
Downloading many existing Packages files without any Package in it
is also suboptimal, but maybe good start.. so we can teach
meta/classes/package_ipk.bbclass:package_update_index_ipk() to create
Packages files not only for existing
ipkgarchs=${ALL_MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
but for all (replace if [ -e $pkgdir/ ]; then with something like
if [ ! -e $pkgdir/ ]; then mkdir -p $pkgdir; fi)
That implies you're exposing feeds straight from OE, which is a bad, bad
idea.
Can you please elaborate on why this is a bad idea?
The main reason is that packages will get recreated with different MD5 sums,
upsetting opkg. And worse, they will have different contents because PR bumps
get left out a lot (but less than they used to). It's analogous to using
WORKDIR/rootfs for nfsroot. It might work for you, but in general it's a bad
idea.
What we do in angstrom is disallow duplicate files to get uploaded, so
foo_1.0-r0.ipk won't get overwritten by a 'newer' foo_1.0-r0.ipk, it will only
go away if foo_1.0-r1.ipk gets uploaded.
regards,
Koen
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Wed, Apr 18, 2012 at 07:01:12PM +0200, Koen Kooi wrote:
Op 18 apr. 2012, om 17:46 heeft Martin Jansa het volgende geschreven:
On Wed, Apr 18, 2012 at 10:21:38AM -0500, Mark Hatle wrote:
On 4/18/12 9:37 AM, Andreas Oberritter wrote:
On 18.04.2012 14:45, Richard Purdie wrote:
On Wed, 2012-04-18 at 14:08 +0200, Andreas Oberritter wrote:
On 18.04.2012 14:00, Richard Purdie wrote:
On Wed, 2012-04-18 at 13:54 +0200, Martin Jansa wrote:
I had a lot of those (e.g. because armv7a-vfp-neon was including 20
arm*feed.conf variants in /etc/opkg most of them empty - without
Packages.gz).
So I've added filter to distro-feed-configs
http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=236aa553bb0f82f741c6edb793e96f421f24f4fa
to add only feeds I'm generating (and I also don't want armv5*
packages
installed on armv7a-vfp-neon target unless user explicitly adds armv5*
feed).
This is the better solution. I think we need to get a better default
feed-config generation mechanism into the core. Distros may still need
to tweak it but it would be good to share some of the best practises...
Did you look at the patch? Which default setting of
SUPPORTED_EXTRA_ARCHS do you suggest?
I did. I didn't say the above patch was a perfect solution.
Do you think it's feasible to add
every single downloadable arch to this variable? If a user of my distro
decides to build it for some arm or x86 cpu, should he need to know
which archs to add at this place?
This is a place where the build system meets and interfaces with the
distro. No one policy in the build system is going to fit every distro's
needs, not should we ever aim to so.
At least we should have defaults that actually work for someone. Now we
don't and considering that distro-feed-configs.bb is the only place
where PACKAGE_EXTRA_ARCHS is actually used, this would be very easy to
accomplish. Especially because it worked well by default before Mark
broke it.
PACKAGE_EXTRA_ARCHS is also used by Zypper, RPM configuration and other
places.
In those cases it is a full list of all available (and compatible)
package
architecture types.
Coming from the RPM world, it seems very odd to me that a set of
extra_archs
can not list well, extra compatible archs without causing an error. I
have no
idea how to reconcile this behavior, without making a package manager
distro-feed specific solution. (For RPM we absolutely want the existing
behavior.)
The problem Andreas is seeing is not fatal AFAIK.. just couple (or a
lot) of 404 (Packages files not available) while doing opkg update is
not nice for end user.
Downloading many existing Packages files without any Package in it
is also suboptimal, but maybe good start.. so we can teach
meta/classes/package_ipk.bbclass:package_update_index_ipk() to create
Packages files not only for existing
ipkgarchs=${ALL_MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
but for all (replace if [ -e $pkgdir/ ]; then with something like
if [ ! -e $pkgdir/ ]; then mkdir -p $pkgdir; fi)
That implies you're exposing feeds straight from OE, which is a bad, bad idea.
I'm rsyncing feed (whole deploy dir) to exposed location only after whole build
is finished and package-indexes regenerated, so I think I'm quite safe.
--
Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
signature.asc
Description: Digital signature
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries -- effectively eliminating mips32 as a tune * Removes the tune entries (cleans up the tunes), no compatibility -- but it's unlikely anyone was directly specifying
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 04/16/2012 09:55 PM, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries -- effectively eliminating mips32 as a tune * Removes the tune entries (cleans up the tunes), no compatibility -- but it's unlikely anyone was directly specifying mips32 as their
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Mon, 2012-04-16 at 15:55 +0200, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries --
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 16.04.2012 16:42, Richard Purdie wrote: On Mon, 2012-04-16 at 15:55 +0200, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries -- effectively eliminating mips32 as a tune * Removes the tune
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 16.04.2012 16:37, Robert Yang wrote: On 04/16/2012 09:55 PM, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries -- effectively eliminating mips32 as a tune * Removes the tune
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 4/16/12 10:15 AM, Andreas Oberritter wrote: On 16.04.2012 16:42, Richard Purdie wrote: On Mon, 2012-04-16 at 15:55 +0200, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries -- effectively eliminating mips32 as a tune * Removes the tune entries (cleans up
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On Mon, 2012-04-16 at 22:37 +0800, Robert Yang wrote: On 04/16/2012 09:55 PM, Andreas Oberritter wrote: On 10.04.2012 21:28, Andreas Oberritter wrote: I'm fine with either way that restores mips/mipsel for mips32 targets *before the release*, because the online update feeds broke and all packages had to be built again from scratch. Richard, can you please state your opinion about this issue? The mips package feed (e.g. qemumips) is now broken since weeks. Can I I'm a little confused with this, I've just done a build with MACHINE = qemumips, both build and runqemu worked well. Builds will work fine however the package architecture for mips32 changes from mipsel to mips32el which broken things for people with existing package feeds. Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] MIPS vs MIPS32 tunings -- summary and questions
On 10.04.2012 19:53, Mark Hatle wrote: We still do not have a clean answer for how to resolve the concerns in the recent thread conf/machine/include: Cleanup MIPS tunings to match README. The following is in response to a request I received to summarize the discussion so far, and include the options to resolve the issue for the current OE-Core release. If you are interested in this, please be sure to read until the end before commenting. Background: About 2 weeks ago, in response to a number of patches sent for PowerPC issues, I set to the task of documenting and cleaning up the various tune files. It was discovered that since they were originally implemented, a number of minor conflicts and defects had crept into the system. The recent patch set added a number of README files and attempted to resolve any duplication, or confusion between items. During this work it was discovered that there were two tunings that produced the same package architecture: mips (tune), optimized for mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), optimized for mips32 - o32 ABI, produced packages also with a mips arch. While mips1 should work on a mips32 system, the reverse is not true. There was no way to distinguish, in a package feed, the difference between the two sets of binaries. I updated the MIPS tune files to resolve this issue. The result was: mips (tune), mips1 - o32 ABI, produced packages with a mips arch mips32 (tune), mips32 - o32 ABI, produced packages with a mips32 arch This lead to the thread mentioned above. At first there were concerns that the GNU target arch had changed (from mips to mips32), this was not the case. The only change is in the produced package arch names. So the package feeds and image generation are the only components affected by this change. After various discussion with folks, such as Khem Raj, it is unlikely that anyone would be using oe-core with a mips1 target. There may be some mips3 or mips4 targets, but we find it highly unlikely based on our current experience. Khem suggested resolving this my simply making the mips include mips32 as the default optimization. Image generation was verified to produce the same images before and after this change for the qemumips target. I am unable to verify the package feeds, as I do not have a suitable setup for this. So possible solutions to this particular issue, which we do need on prior to the upcoming release: 1) Revert the behavior and match that last release. We have two tunes that produce different binaries w/ the same mips package arch. * This preserves previous behavior, but IMHO continues to implement the defect mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips package arch 2) Keep it as it is currently checked in. Provide the ability to build a basic mips and a more optimized mips32 tuned target and package set. * This fixes the defect and provides the opportunity for mips to be a basic common package arch, while mips32 (or additional mips3? mips4? mips32r2?) tunes could be used to augment this for specific systems. mips (tune) - mips1 processor, o32 ABI - mips package arch mips32 (tune) - mips32 processor, o32 ABI - mips32 package arch If the tune should reflect the optimization, then mips should be renamed to mips1 and specified explicitly using -march=mips1, in order to protect against changing defaults when using a newer compiler. However, as Phil pointed out, there are many more optimization settings, e.g. -O2 vs. -Os, that aren't encoded into the package arch, so the goal to have distinct package archs for different binaries won't be reached. I don't see what a common mips package arch would give us. Within OE, you'd usually compile all your applications for the package arch of your target system. Adding compatible package archs to the feed just increases the complexity of online updates. 3) Define only one mips tune, with a target package arch of mips. Changing the basic mips tune, and corresponding mips package arch to include mips32 optimizations and instructions. * This preserves the mips tune, but changes the behavior of the tune from default compiler, to mips32 optimization * Anyone requiring mips3 or mips4 will need to add a tune, and that tune will not be compatible with mips Also, mips1 could be added back anytime if anybody starts using it. mips (tune) - mips32 processor, o32 ABI - mips package arch 3a) Preserve the mips32 tune entries, but define it as being equal to mips * Preserves the tune entries for compatibility, but is anyone directly using them? 3b) Remove the mips32 tune entries -- effectively eliminating mips32 as a tune * Removes the tune entries (cleans up the tunes), no compatibility -- but it's unlikely anyone was directly specifying mips32 as their machine's DEFAULTTUNE.
