On 04/03/2024 16:42, Christophe Lyon wrote:
> On Mon, 4 Mar 2024 at 16:41, Richard Earnshaw <richard.earns...@arm.com>
> wrote:
>>
>>
>>
>> On 04/03/2024 15:36, Richard Earnshaw (lists) wrote:
>> > On 04/03/2024 14:46, Christophe Lyon via Gcc wrote:
>> >> On Mon, 4 Mar 2024 at 12:25, Jonathan Wakely <jwakely....@gmail.com>
>> >> wrote:
>> >>>
>> >>> On Mon, 4 Mar 2024 at 10:44, Christophe Lyon via Gcc <gcc@gcc.gnu.org>
>> >>> wrote:
>> >>>>
>> >>>> Hi!
>> >>>>
>> >>>> On Mon, 4 Mar 2024 at 10:36, Thomas Schwinge <tschwi...@baylibre.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Hi!
>> >>>>>
>> >>>>> On 2024-03-04T00:30:05+0000, Sam James <s...@gentoo.org> wrote:
>> >>>>>> Mark Wielaard <m...@klomp.org> writes:
>> >>>>>>> On Fri, Mar 01, 2024 at 05:32:15PM +0100, Christophe Lyon wrote:
>> >>>>>>>> [...], I read
>> >>>>>>>> https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration
>> >>>>>>>> <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration>
>> >>>>>>>> <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration
>> >>>>>>>> <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration>>
>> >>>>>>>> which basically says "run autoreconf in every dir where there is a
>> >>>>>>>> configure script"
>> >>>>>>>> but this is not exactly what autoregen.py is doing. IIRC it is based
>> >>>>>>>> on a script from Martin Liska, do you know/remember why it follows a
>> >>>>>>>> different process?
>> >>>>>>>
>> >>>>>>> CCing Sam and Arsen who helped refine the autoregen.py script, who
>> >>>>>>> might remember more details. We wanted a script that worked for both
>> >>>>>>> gcc and binutils-gdb. And as far as I know autoreconf simply didn't
>> >>>>>>> work in all directories. We also needed to skip some directories that
>> >>>>>>> did contain a configure script, but that were imported (gotools,
>> >>>>>>> readline, minizip).
>> >>>>>>
>> >>>>>> What we really need to do is, for a start, land tschwinge/aoliva's
>> >>>>>> patches [0]
>> >>>>>> for AC_CONFIG_SUBDIRS.
>> >>>>>
>> >>>>> Let me allocate some time this week to get that one completed.
>> >>>>>
>> >>>>>> Right now, the current situation is janky and it's nowhere near
>> >>>>>> idiomatic autotools usage. It is not a comfortable experience
>> >>>>>> interacting with it even as someone who is familiar and happy with
>> >>>>>> using
>> >>>>>> autotools otherwise.
>> >>>>>>
>> >>>>>> I didn't yet play with maintainer-mode myself but I also didn't see
>> >>>>>> much
>> >>>>>> point given I knew of more fundamental problems like this.
>> >>>>>>
>> >>>>>> [0]
>> >>>>>> https://inbox.sourceware.org/gcc-patches/oril72c4yh....@lxoliva.fsfla.org/
>> >>>>>>
>> >>>>>> <https://inbox.sourceware.org/gcc-patches/oril72c4yh....@lxoliva.fsfla.org/>
>> >>>>>>
>> >>>>>> <https://inbox.sourceware.org/gcc-patches/oril72c4yh....@lxoliva.fsfla.org/
>> >>>>>>
>> >>>>>> <https://inbox.sourceware.org/gcc-patches/oril72c4yh....@lxoliva.fsfla.org/>>
>> >>>>>
>> >>>>
>> >>>> Thanks for the background. I didn't follow that discussion at that time
>> >>>> :-)
>> >>>>
>> >>>> So... I was confused because I noticed many warnings when doing a simple
>> >>>> find . -name configure |while read f; do echo $f;d=$(dirname $f) &&
>> >>>> autoreconf -f $d && echo $d; done
>> >>>> as suggested by https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration
>> >>>> <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration>
>> >>>> <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration
>> >>>> <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration>>
>> >>>>
>> >>>> Then I tried with autoregen.py, and saw the same.... and now just
>> >>>> checked Sourceware's bot logs and saw the same numerous warnings at
>> >>>> least in GCC (didn't check binutils yet). Looks like this is
>> >>>> "expected" ....
>> >>>>
>> >>>> I started looking at auto-regenerating these files in our CI a couple
>> >>>> of weeks ago, after we received several "complaints" from contributors
>> >>>> saying that our precommit CI was useless / bothering since it didn't
>> >>>> regenerate files, leading to false alarms.
>> >>>> But now I'm wondering how such contributors regenerate the files
>> >>>> impacted by their patches before committing, they probably just
>> >>>> regenerate things in their subdir of interest, not noticing the whole
>> >>>> picture :-(
>> >>>>
>> >>>> As a first step, we can probably use autoregen.py too, and declare
>> >>>> maintainer-mode broken. However, I do notice that besides the rules
>> >>>> about regenerating configure/Makefile.in/..., maintainer-mode is also
>> >>>> used to update some files.
>> >>>> In gcc:
>> >>>> fixincludes: fixincl.x
>> >>>> libffi: doc/version.texi
>> >>>> libgfortran: some stuff :-)
>> >>>> libiberty: functions.texi
>> >>>
>> >>> My recently proposed patch adds the first of those to gcc_update, the
>> >>> other should be done too.
>> >>> https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html
>> >>> <https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html>
>> >>> <https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html
>> >>> <https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html>>
>> >>>
>> >>
>> >> This script touches files such that they appear more recent than their
>> >> dependencies,
>> >> so IIUC even if one uses --enable-maintainer-mode, it will have no effect.
>> >> For auto* files, this is "fine" as we can run autoreconf or
>> >> autoregen.py before starting configure+build, but what about other
>> >> files?
>> >> For instance, if we have to test a patch which implies changes to
>> >> fixincludes/fixincl.x, how should we proceed?
>> >> 1- git checkout (with possibly "wrong" timestamps)
>> >> 2- apply patch-to-test
>> >> 3- contrib/gcc_update -t
>> >> 4- configure --enable-maintainer-mode
>> >
>> > If you ran
>> >
>> > git reset --hard master // restore state to 'master'
>> > contrib/gcc_update // pull latest code
>> >
>> > then anything coming from upstream will be touched automatically. You
>> > really don't want to re-touch the files after patching unless you're sure
>> > they've all been patched correctly, it will break if there's anything
>> > regenerated that's missing.
>> >
>> > R.
>>
>> Alternatively, if you did
>>
>> git reset --hard master // restore state to 'master'
>> contrib/gcc_update // pull latest code
>> patch
>> contrib/gcc_update -t
>>
>> and the second command touched files that weren't listed in the patches,
>> then you'd know that there was something missing from the patch set. You'd
>> probably still need to go back to the pristine trunk version though, to get
>> the state into something that could be maintainer-mode regenerated.
>>
>
> We have been using contrib/gcc_update -t for years (even before the
> public CI) because we noticed random timestamp problems after 'git
> clone/checkout'. Maybe that was caused by our mirror, I don't
> remember. Anyway after a fresh clone/checkout, we had some random
> build failures because the build process was failing to update some
> files; probably trying to run the host's too recent autoconf...
> Actually I'm not sure about the original purpose of gcc_summary, it
> dates back to 1999 and the commit message is a bit... short :-)
Tools like git (and svn before it) don't try to maintain time-stamps on
patches, the tool just modifies the file and the timestamp comes from the time
of the modification. That's fine if there is nothing regenerated within the
repository (it's pure original source), but will cause problems if there are
generated files as their time stamps aren't necessarily correct. `gcc_update
--touch` addresses that by ensuring all the generated files are retouched when
needed.
R.
>
> The "pull latest code" step you mention is part of what I call "git
> clone/checkout".
>
> Anyway, this doesn't fix the problem that --enable-maintainer-mode is broken
> :-)
>
> Thanks,
>
> Christophe
>
>> R.
>>
>> >
>> >>
>> >> I guess --enable-maintainer-mode would be largely (if not completely)
>> >> disabled by step 3 above?
>> >> And we should probably swap steps 2 and 3, so that the effects of
>> >> patch-to-test are handled by make?
>> >>
>> >> Thanks,
>> >>
>> >> Christophe
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> in binutils/bfd:
>> >>>> gdb/sim
>> >>>> bfd/po/SRC-POTFILES.in
>> >>>> bfd/po/BLD-POTFILES.in
>> >>>> bfd/bfd-in2.h
>> >>>> bfd/libbfd.h
>> >>>> bfd/libcoff.h
>> >>>> binutils/po/POTFILES.in
>> >>>> gas/po/POTFILES.in
>> >>>> opcodes/i386*.h
>> >>>> gdb/copying.c
>> >>>> gdb/data-directory/*.xml
>> >>>> gold/po/POTFILES.in
>> >>>> gprof/po/POTFILES.in
>> >>>> gfprofng/doc/version.texi
>> >>>> ld/po/SRC-POTFILES.in
>> >>>> ld/po/BLD-POTFILES.in
>> >>>> ld: ldgram/ldlex... and all emulation sources
>> >>>> libiberty/functions.texi
>> >>>> opcodes/po/POTFILES.in
>> >>>> opcodes/aarch64-{asm,dis,opc}-2.c
>> >>>> opcodes/ia64 msp430 rl78 rx z8k decoders
>> >>>>
>> >>>> How are these files "normally" expected to be updated? Do people just
>> >>>> manually uncomment the corresponding maintainer lines in the Makefiles
>> >>>> and update manually? In particular we hit issues several times with
>> >>>> files under opcodes, that we don't regenerate currently. Maybe we
>> >>>> could build binutils in maintainer-mode at -j1 but, well....
>> >>>>
>> >>>> README-maintainer-mode in binutils/gdb only mentions a problem with
>> >>>> 'make distclean' and maintainer mode
>> >>>> binutils/README-how-to-make-a-release indicates to use
>> >>>> --enable-maintainer-mode, and the sample 'make' invocations do not
>> >>>> include any -j flag, is that an indication that only -j1 is supposed
>> >>>> to work?
>> >>>> Similarly, the src-release.sh script does not use -j.
>> >>>>
>> >>>> Thanks,
>> >>>>
>> >>>> Christophe
>> >>>>
>> >>>>>
>> >>>>> Grüße
>> >>>>> Thomas
>> >
