[fedora-arm] Patching aarch64 support into Fedora (Round 4)
Based on recent IRC discussions, some changes were made to the patchify script [0] and the Fedora package archive scanned again. The overall goal is to try to find all of those Fedora packages that use autoconf or ./configure to build but that do not yet have the updates to recognize 'aarch64' as a valid target (i.e., they need newer versions of config.sub or config.guess). And, if they do not, generate a patch that will add the latest config.* files. What changed? The parsing of the spec file needed to look at the %build and %prep sections is a bit smarter. It used to see some uses of autoconf/configure in the initial prologue for a spec file (e.g., BuildRequires: autoconf) as a proper use of autoconf when it clearly is not. Additionally, if autoreconf is being used outside of an if-block, and it is using -i/--install and -f/--force, these are packages that will correct themselves on build so do not need patching [1]. These changes should eliminate some false positives and catch some packages that may have inadvertently been skipped over. The list of packages needing patching is here: http://fedorapeople.org/~ahs3/patchify/run4/patched.pkgs This lists 2027 packages that need patching (most of which already have bugs filed against them, but there were some new ones found (341) and others removed (290). I'll be comparing this against the aarch64 tracking bug [2] to be sure we're in sync and all the right notifications have been made. Note that all of the package lists and a README describing what each list represents are also at: http://fedorapeople.org/~ahs3/patchify/run4/ Some data I found interesting from this scan [3] are: -- there are 572 dead packages in the archive -- 196 packages already use autoreconf -i -f which is higher than I thought it might be -- there are 105 cases where it's difficult to determine the source directory name when all you have is the package name -- a lot of packages use ./configure, but most of them do not need or use config.sub/config.guess And the best one: 579 packages have already been modified to use the latest config.sub/config.guess and are already able to handle aarch64. So, action items for me are: (1) make sure the AArch64 tracking bug is listing bugs for the right list of packages, and (2) help close as many of those bugs as I can. If you can help, please do. Notes: [0] git://fedorapeople.org/~ahs3/patchify.git [1] The specific package list is: http://fedorapeople.org/~ahs3/patchify/run4/autoreconf.pkgs [2] https://bugzilla.redhat.com/show_bug.cgi?id=922257 [3] This is the output summary from the scan: --- Results --- Total packages examined: 12088 Total dead packages: 572 Total packages using autoreconf properly: 196 Total patches available: 2027 Total packages using ./configure but not config.*: 8353 Total packages using config.* with aarch64 in them: 579 --- Configuration --- 365 use autoconf in some recognized form 196 use autoreconf with correct options 3512 use configure in some recognized form 7715 do not mention autoconf or configure --- Error Conditions --- 0 cases of unclassified errors 251 cases where we could not unpack the SRPM 1 cases where we there was no spec file 105 cases where we could not find the source directory 0 cases where we could not cp -r the source directory 0 cases where find config.* reported errors 4 cases where we could not overwrite a config.* file -- ciao, al --- Al Stone Software Engineer Red Hat, Inc. a...@redhat.com --- ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On 03/06/2013 10:39 PM, Brendan Conoboy wrote:
On 03/06/2013 07:24 PM, Dennis Gilmore wrote:
Talking with the X guys they run autoreconf in %build so all of their
packages are false positives.
Good catch. Al is updating patchify to also scan %build for such
occurrences. We'll have an updated list tomorrow.
Sorry for the delayupdated results can be found here:
http://fedorapeople.org/~ahs3/patchify/run3/patched.pkgs
That list is all the packages that use config.{guess,sub}
and have not already been patched upstream for aarch64 -- a
total of 1976 packages that I've been able to find. Some
434 have already been patched.
Note that there may still be false positives; I did not
eliminate the packages that use some form of autoconf because
it's not clear one can assume that autoconf will actually be
invoked during the %build -- or at least I can't unless I'm
willing to do some extensive parsing of the spec file that I
have not done so far (it would probably easier to just build
the package).
The packages that use ./configure in some form in the spec
file (2623 of them) are listed here:
http://fedorapeople.org/~ahs3/patchify/run3/configure.pkgs
This lists 2623 packages. In actuality, by searching for
the existence of config.{guess,sub} there are only 2459.
The remainder are the results of the regex not being able
to separate out, for example, './configure' in a comment
vs an actual use.
So, 2459 total packages using ./configure; patches were
created for 1976 of them, 434 were already patched, and
5 ran into errors while creating a patch. I'm missing
44 somewhere still that I'll see if I can find.
I've also stashed copies of the log from running the script
(run3.log) and several other lists of the various packages
found (e.g., dead packages, packages not using config*, and
so on) in that same directory on fp.o. Holler if there's
questions...
--
ciao,
al
---
Al Stone
Software Engineer
Red Hat, Inc.
a...@redhat.com
---
___
arm mailing list
arm@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On Mon, 04 Mar 2013 19:44:16 -0800 Brendan Conoboy b...@redhat.com wrote: Hi everybody, Fedora 19 has many of the enablers for native aarch64 in core packages such as glibc and gcc. Many more packages would compile if config.guess and config.sub recognized aarch64 as a valid architecture, but only the latest version of autoconf knows about aarch64. At last week's fedora-arm meeting we talked about the viability of automatically patching such packages. The outcome of that discussion was that we should first identify how many packages need such a patch, then decide what to do based on that number. Red Hat's Al Stone has written a script which automatically generates patches for each package that needs it (And updates its spec file). After a complete run we can say that ~1850 packages need such a patch. The number may be larger, but it is definitely not smaller, as his only considers autoconf-using packages. If another auto configuration system is in use by a number of packages it too many need updating (cmake?). Now that we have this number, what do we want to do? I see a few options: 1. Do nothing, trip over this issue at least 1850 times during bootstrap. 2. Mail all package owners asking for action. 3. Proven packager commits the patches, package owners take them out once unnecessary. 4. Run autoconf during build, incurring wrath of any packager whose package isn't compatible with the latest autoconf. 5. Your much more sensible idea goes here. What say you? I say we need the list of packages effected. then we can evaluate how critical it is that we take action. Dennis ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On 03/06/2013 06:37 AM, Dennis Gilmore wrote: I say we need the list of packages effected. then we can evaluate how critical it is that we take action. Here is the initial list: http://people.fedoraproject.org/~blc/fedora-arm/patchify/configure.pkgs I think it may actually be much larger. Al and I are going back and forth on fixing up patchify to identify more cases. -- Brendan Conoboy / Red Hat, Inc. / b...@redhat.com ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On Wed, Mar 6, 2013 at 6:11 PM, Brendan Conoboy b...@redhat.com wrote: On 03/06/2013 06:37 AM, Dennis Gilmore wrote: I say we need the list of packages effected. then we can evaluate how critical it is that we take action. Here is the initial list: http://people.fedoraproject.org/~blc/fedora-arm/patchify/configure.pkgs I think it may actually be much larger. Al and I are going back and forth on fixing up patchify to identify more cases. Ultimately there's a number of things like alsa* globus* as well as a lot of X related stuff where upstream should be contacted because they do have regular releases so there's no reason why they shouldn't be fixed by default from upstream. Peter ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On Wed, 2013-03-06 at 10:11 -0800, Brendan Conoboy wrote: On 03/06/2013 06:37 AM, Dennis Gilmore wrote: I say we need the list of packages effected. then we can evaluate how critical it is that we take action. Here is the initial list: http://people.fedoraproject.org/~blc/fedora-arm/patchify/configure.pkgs I think it may actually be much larger. Al and I are going back and forth on fixing up patchify to identify more cases. I notice that gcc is missing from the list. I had to patch the config files in the isl subdir for the aarch64 build. https://bugzilla.redhat.com/show_bug.cgi?id=917749 --Mark ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On 03/05/2013 11:44 AM, Brendan Conoboy wrote: [snip..] Red Hat's Al Stone has written a script which automatically generates patches for each package that needs it (And updates its spec file). After a complete run we can say that ~1850 packages need such a patch. The number may be larger, but it is definitely not smaller, as his only considers autoconf-using packages. If another auto configuration system is in use by a number of packages it too many need updating (cmake?). For the curious, the script used is in git://fedorapeople.org/~ahs3/patchify.git -- ciao, al --- Al Stone Software Engineer Red Hat, Inc. a...@redhat.com --- ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On Tue, Mar 5, 2013 at 3:44 AM, Brendan Conoboy b...@redhat.com wrote: Hi everybody, Fedora 19 has many of the enablers for native aarch64 in core packages such as glibc and gcc. Many more packages would compile if config.guess and config.sub recognized aarch64 as a valid architecture, but only the latest version of autoconf knows about aarch64. At last week's fedora-arm meeting we talked about the viability of automatically patching such packages. The outcome of that discussion was that we should first identify how many packages need such a patch, then decide what to do based on that number. Red Hat's Al Stone has written a script which automatically generates patches for each package that needs it (And updates its spec file). After a complete run we can say that ~1850 packages need such a patch. The number may be larger, but it is definitely not smaller, as his only considers autoconf-using packages. If another auto configuration system is in use by a number of packages it too many need updating (cmake?). Now that we have this number, what do we want to do? I see a few options: 1. Do nothing, trip over this issue at least 1850 times during bootstrap. 2. Mail all package owners asking for action. 3. Proven packager commits the patches, package owners take them out once unnecessary. 4. Run autoconf during build, incurring wrath of any packager whose package isn't compatible with the latest autoconf. 5. Your much more sensible idea goes here. What say you? Firstly where is the list of 1850 packages that are affected? Also we do need to know whether the likes of cmake do need work or what else might need work. There's 13K packages in Fedora, of that how many are autoconf driven (so what sort of percentage is the 1850, it might give us an idea of what has been updated). I don't see 1 and 4 as an option. Option 2 should read File a bug against every affected package with it linked to a tracking bug asking them for action I think a 5) file a rel-eng bug with the patches and script and ask them to do it which was I believe the plan prior to mass rebuild but the script/list wasn't ready in time/ Ultimately we need to get this done and any analysis of other non autoconf build platforms that might need the work too prior to the branching which is scheduled to happen on March 12. That gives us a week to get this sorted. Peter ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Re: [fedora-arm] Patching aarch64 support into Fedora
On Mon, Mar 04, 2013 at 07:44:16PM -0800, Brendan Conoboy wrote: 1. Do nothing, trip over this issue at least 1850 times during bootstrap. 2. Mail all package owners asking for action. How about posting the list and the suggested action to correct it to fedora-devel, as an initial pass? -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
[fedora-arm] Patching aarch64 support into Fedora
Hi everybody, Fedora 19 has many of the enablers for native aarch64 in core packages such as glibc and gcc. Many more packages would compile if config.guess and config.sub recognized aarch64 as a valid architecture, but only the latest version of autoconf knows about aarch64. At last week's fedora-arm meeting we talked about the viability of automatically patching such packages. The outcome of that discussion was that we should first identify how many packages need such a patch, then decide what to do based on that number. Red Hat's Al Stone has written a script which automatically generates patches for each package that needs it (And updates its spec file). After a complete run we can say that ~1850 packages need such a patch. The number may be larger, but it is definitely not smaller, as his only considers autoconf-using packages. If another auto configuration system is in use by a number of packages it too many need updating (cmake?). Now that we have this number, what do we want to do? I see a few options: 1. Do nothing, trip over this issue at least 1850 times during bootstrap. 2. Mail all package owners asking for action. 3. Proven packager commits the patches, package owners take them out once unnecessary. 4. Run autoconf during build, incurring wrath of any packager whose package isn't compatible with the latest autoconf. 5. Your much more sensible idea goes here. What say you? -- Brendan Conoboy / Red Hat, Inc. / b...@redhat.com ___ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
