Re: gcc git locked out for hours second day in a row
Hello, Le 12/06/2024 à 16:57, Jakub Jelinek a écrit : On Wed, Jun 12, 2024 at 04:53:38PM +0200, Mikael Morin wrote: Perhaps you could create a mirror version of the repo and do some experiments locally on that to identify where the bottle-neck is coming from? Not sure where to start for that.Are the hooks published somewhere? Yes: https://github.com/AdaCore/git-hooks/tree/master Note, we use some tweaks on top of that, but that is mostly for the release branches and trunk, so it would be interesting to just try to reproduce that with the stock AdaCore git hooks. I have finally taken some time to investigate this hook slowness, and here are my findings. My tests were run with configs commit-extra-checker and commit-email-formatter disabled, and hooks.update-hook set to a minimal script (either "true" or "sleep 1"). With that config, I could not reproduce the slowness pushing to refs/users/mikael/*. The push finishes in less than a minute. However, trying to push to a normal tag, there is some email count check coming into play, and I can reproduce some slowness (details below). This email count check shouldn't happen on the gcc repository in my use case (as email checks don't apply to user references), but the slowness could well happen in other cases than email count check depending on the configuration, as the problem relates to the size of the list of new commits and is not restricted to email count. Anyway, even with email count check triggering, each tag takes less than 2 minutes to be rejected in my test. With 330 tags to process, that would make an upper bound of 11 hours before rejecting the push in my test (I killed it after a few minutes). On the other hand, with the information you gave upthread, the hook on the gcc repository seemed to be still processing the first tag after a few hours (assuming they are processed in alphabetical order, which seems to be the case). So this still doesn't explain what was happening on the gcc repository. Regarding the email count check slowness I mentioned above, I traced it back to the updates.AbstractUpdate class, whose (procedural) new_commits_for_ref attribute is a list of "new" commits, containing both really new commits and commits newly on the branch to be updated, but already known to the repository. For a tag or branch creation, a list of "new on the branch" commits would be huge as everything is new, so parent commits of the oldest "repository-new" commit are not picked up. But in my test the list still amounts to a little less than 80,000 commits, basically what happened on trunk in the last 8 years. Anything that walks such a big list is bound to be slow. To sum up: - The hooks support checking "new on the branch" commits additionally to "new on the repository" commits, and that is a feature, not a bug. - In my use case, that means that the hooks process 80,000 commits, even if only 330 of them are new on the repository. - As the hook is called on a per-reference basis, the same commits would be processed over and over again for every reference in my use case, so the best would be to push them one by one, in order. - I still don't know why it took hours (without even finishing) to process just one tag the other day on the gcc repository. Nikael
Re: gcc git locked out for hours second day in a row
Le 12/06/2024 à 16:34, Richard Earnshaw (lists) a écrit : On 12/06/2024 14:23, Mikael Morin via Gcc wrote: Le 12/06/2024 à 14:58, Jonathan Wakely a écrit : On Wed, 12 Jun 2024 at 13:57, Mikael Morin via Gcc wrote: Le 12/06/2024 à 13:48, Jakub Jelinek a écrit : Hi! Yesterday the gcc git repository was locked for 3 hours locked by user mikael at 2024-06-11 13:27:44.301067 (pid = 974167) 78:06 python hooks/update.py refs/users/mikael/tags/fortran-dev_merges/r10-1545 c2f9fe1d8111b9671bf0aa8362446516fd942f1d process until overseers killed it but today we have the same situation for 3 ours and counting again: locked by user mikael at 2024-06-12 08:35:48.137564 (pid = 2219652) 78:06 python hooks/update.py refs/users/mikael/tags/toto cca005166dba2cefeb51afac3ea629b3972acea3 It is possible we have some bug in the git hook scripts, but it would be helpful trying to understand what exactly you're trying to commit and why nobody else (at least to my knowledge) has similarly stuck commits. The effect is that nobody can push anything else to gcc git repo for hours. Jakub Yes, sorry for the inconvenience. I tried pushing a series of tags labeling merge points between the fortran-dev branch and recent years master. Just pushing tags should not cause a problem, assuming all the commits being tagged already exist. What exactly are you pushing? Well, the individual commits to be merged do exist, but the merge points don't and they are what I'm trying to push. To be clear, the branch hasn't seen any update for years, and I'm trying to reapply what happened on trunk since, in a step-wise manner. With 300 merges I'm summing up 6 commits of history. The number of merge points is a bit high (329) but I expected it to be a manageable number. I tried again today with just the most recent merge point, but it got stuck again. I should try with the oldest one, but I'm afraid locking the repository again. I waited for the push to finish for say one hour before killing it yesterday, and no more than 15 minutes today. Unfortunately, killing the process doesn't seem to unlock things on the server side. It may be a misconfiguration on my side, but I have never had this problem before. Sorry again. Perhaps you could create a mirror version of the repo and do some experiments locally on that to identify where the bottle-neck is coming from? Not sure where to start for that.Are the hooks published somewhere?
Re: gcc git locked out for hours second day in a row
Le 12/06/2024 à 14:58, Jonathan Wakely a écrit : On Wed, 12 Jun 2024 at 13:57, Mikael Morin via Gcc wrote: Le 12/06/2024 à 13:48, Jakub Jelinek a écrit : Hi! Yesterday the gcc git repository was locked for 3 hours locked by user mikael at 2024-06-11 13:27:44.301067 (pid = 974167) 78:06 python hooks/update.py refs/users/mikael/tags/fortran-dev_merges/r10-1545 c2f9fe1d8111b9671bf0aa8362446516fd942f1d process until overseers killed it but today we have the same situation for 3 ours and counting again: locked by user mikael at 2024-06-12 08:35:48.137564 (pid = 2219652) 78:06 python hooks/update.py refs/users/mikael/tags/toto cca005166dba2cefeb51afac3ea629b3972acea3 It is possible we have some bug in the git hook scripts, but it would be helpful trying to understand what exactly you're trying to commit and why nobody else (at least to my knowledge) has similarly stuck commits. The effect is that nobody can push anything else to gcc git repo for hours. Jakub Yes, sorry for the inconvenience. I tried pushing a series of tags labeling merge points between the fortran-dev branch and recent years master. Just pushing tags should not cause a problem, assuming all the commits being tagged already exist. What exactly are you pushing? Well, the individual commits to be merged do exist, but the merge points don't and they are what I'm trying to push. To be clear, the branch hasn't seen any update for years, and I'm trying to reapply what happened on trunk since, in a step-wise manner. With 300 merges I'm summing up 6 commits of history. The number of merge points is a bit high (329) but I expected it to be a manageable number. I tried again today with just the most recent merge point, but it got stuck again. I should try with the oldest one, but I'm afraid locking the repository again. I waited for the push to finish for say one hour before killing it yesterday, and no more than 15 minutes today. Unfortunately, killing the process doesn't seem to unlock things on the server side. It may be a misconfiguration on my side, but I have never had this problem before. Sorry again.
Re: gcc git locked out for hours second day in a row
Le 12/06/2024 à 14:14, Mark Wielaard a écrit : Hi, On Wed, 2024-06-12 at 13:48 +0200, Jakub Jelinek via Gcc wrote: Yesterday the gcc git repository was locked for 3 hours locked by user mikael at 2024-06-11 13:27:44.301067 (pid = 974167) 78:06 python hooks/update.py refs/users/mikael/tags/fortran-dev_merges/r10-1545 c2f9fe1d8111b9671bf0aa8362446516fd942f1d process until overseers killed it but today we have the same situation for 3 ours and counting again: locked by user mikael at 2024-06-12 08:35:48.137564 (pid = 2219652) 78:06 python hooks/update.py refs/users/mikael/tags/toto cca005166dba2cefeb51afac3ea629b3972acea3 Just for the record, I kill the process and removed the git- hooks::update.token.lock file. Sorry that killed mikael's push, but hopefully that makes it possible for others to push again. No problem, I had interrupted the push long before on my side.
Re: gcc git locked out for hours second day in a row
Le 12/06/2024 à 13:48, Jakub Jelinek a écrit : Hi! Yesterday the gcc git repository was locked for 3 hours locked by user mikael at 2024-06-11 13:27:44.301067 (pid = 974167) 78:06 python hooks/update.py refs/users/mikael/tags/fortran-dev_merges/r10-1545 c2f9fe1d8111b9671bf0aa8362446516fd942f1d process until overseers killed it but today we have the same situation for 3 ours and counting again: locked by user mikael at 2024-06-12 08:35:48.137564 (pid = 2219652) 78:06 python hooks/update.py refs/users/mikael/tags/toto cca005166dba2cefeb51afac3ea629b3972acea3 It is possible we have some bug in the git hook scripts, but it would be helpful trying to understand what exactly you're trying to commit and why nobody else (at least to my knowledge) has similarly stuck commits. The effect is that nobody can push anything else to gcc git repo for hours. Jakub Yes, sorry for the inconvenience. I tried pushing a series of tags labeling merge points between the fortran-dev branch and recent years master. The number of merge points is a bit high (329) but I expected it to be a manageable number. I tried again today with just the most recent merge point, but it got stuck again. I should try with the oldest one, but I'm afraid locking the repository again. I waited for the push to finish for say one hour before killing it yesterday, and no more than 15 minutes today. Unfortunately, killing the process doesn't seem to unlock things on the server side. It may be a misconfiguration on my side, but I have never had this problem before. Sorry again.
Re: gfortran-dg-runtest, torture options
Le 13/08/2013 20:23, Janis Johnson a écrit : > On 08/13/2013 04:06 AM, Thomas Schwinge wrote: >> Hi! >> >> I noticed something strange in the libgomp testresults (but not >> necessarily specific to libgomp): an "arbitrary" set of the Fortran >> execution tests are run just for -O, and others for each of the full set >> of torture options: -O0, -O1, -O2, and so on. After some time I realized >> it's the set of tests that contain an explicit »dg-do run« directive that >> are run for all torture levels, and the tests that inherit the default >> »set dg-do-what-default run« from libgomp/testsuite/lib/libgomp.exp are >> only run for -O. This is coming from the special handling in >> gcc/testsuite/lib/gfortran-dg.exp:gfortran-dg-test (which seems to be >> present approximately "forever"). gfortran-dg.exp is (obviously) supposed to handle the gcc/testsuite/gfortran.dg/* tests, which exercise the fortran front-end. The tests come in mostly two flavors: - compile ones checking the error reporting and lack of failed assert, segmentation fault etc. - run ones checking the code generated. Only the run time ones get checked against multiple optimization options because they are the ones concerned with code generation. I suppose the gomp implementers just picked gfortran-dg.exp to have the parsing of error messages coming from fortran tests, and they inherited from the above. >> Should this consider the >> dg-do-what-default case, too? For gfortran.dg, the default is compile IIRC, so it's OK as is. For libgomp, I guess more consistency wouldn't harm. To be honest I would expect the default case to be first resolved to either compile or run and then handled as if it was explicitly specified. That's not how it works, it seems. :-/ Mikael
Re: Building C++ with --enable-languages=c,fortran
On Saturday 06 August 2011 00:52:02 Thomas Koenig wrote: > Hello world, > > I just noticed that C++ now appears to be built by default, even when > only the C and fortran are specified. Yes, but it doesn't make much difference in practice. The only difference I saw is the debugging symbols including the full prototype (not only the function name) in the debugger. WRT to bootstrap time, as usual: it's too long. Mikael
Re: RFH: Impose code-movement restrictions and value assumption (for ASYNCHRONOUS/Coarrays)
Hello,
On Wednesday 13 July 2011 15:46:37 Tobias Burnus wrote:
>
> void some_function(void);
>
> void
> sub (int *restrict non_aliasing_var)
> {
>*non_aliasing_var = 5;
>some_function ();
>if (*non_aliasing_var != 5)
> foobar_();
> }
>
Couldn't we simulate the desired behaviour with more than one decl, one of
them const qualified? Like so:
void
sub (int *restrict non_aliasing_var)
{
*non_aliasing_var = 5;
{
const int *non_aliasing_var_tmp = non_aliasing_var;
some_function ();
if (*non_aliasing_var_tmp != 5)
foobar_();
}
}
It would probably be a hell to implement however.
Mikael
Re: Original commit history for gfortran
On Sunday 19 June 2011 20:30:17 Christopher Bergström wrote: > Nothing cloudy > > 1) Vet the codebase (stated this clearly) > 2) Listen to what people say - (What needs to be worked on, are people > open to things like dual licensing, what's the future of Fortran, > etc..) > > For whatever reason someone at Apple has decided to work on "flang". > I'm not sure if the code is public or even a serious effort at all. > Apple certainly has the resources to toss at it, but outside of > someone's personal hobby project I can only think of one reason to > spend time on it. > > 1) Our goal (PathScale) is to implement F2K3/8 Why not extend the existing frontend? Is it in such a bad shape that rewriting a new IR generator is preferable? Is it g95 only that you plan to import or some of gfortran as well? I personally see no problem gfortran being reused in pathscale's compiler as long as pathscale's contribution is libre (free). It can even improve code quality to make gfortran backend independant (probably not much as the IR generation is quite separated already, but who knows?), and that would give us an open64 (this one is a "really free" compiler I think) backend at the same time. Not bad. Mikael
Re: GCC 4.5.3 Status Report (2011-04-21), branch now frozen
On Thursday 21 April 2011 11:30:49 Richard Guenther wrote: > Status > == > > A first release candidate for GCC 4.5.3 is beeing made. The branch > is now frozen until after the final 4.5.3 release. All changes > require explicit release manager approval. > > > Quality Data > > > Priority # Change from Last Report > --- --- > P10 > P2 112 - 7 > P30 - 9 > --- --- > Total 112 - 5 Total 112 - 16 Do you think nobody reads your reports? :-p
Re: [RFC] gfortran's coarray (library version): configure/build and the testsuite
On Tuesday 05 April 2011 22:01:03 Tobias Burnus wrote:
> Hello,
>
> Fortran 2008 has a build in parallelization (Coarray [Fortran], CAF)
> [1]. gfortran did the first steps to a communication-library version
> [2]. The library will be based MPI.
>
> There are two issues I like to discuss in this email:
>
> a) configuring and building
> b) Test-suite support
>
>
> Let's start with (b) which is more important for me. The current scheme
> is that the user somehow compiles the communication library (libcaf) [2]
> and then builds and links doing something like:
>mpif90 -fcoarray=lib fortran.f90 -lcaf_mpi
> or alternatively
>gfortran -fcoarray=lib fortran.f90 -lcaf_mpi
> -I/usr/lib64/mpi/gcc/openmpi/include -L/usr/lib64/mpi/gcc/openmpi/lib64
> -lmpi
> with some -I, -L -l are added. (Cf. "mpif90 -show" of
> some MPI implementations.) The resulting program is then run using, e.g.,
>mpiexec -n 3 ./a.out
> Alternatively, it could be just "-lcaf_single" which is run like normal
> ("./a.out").
>
> Thus, one needs some means to add link and compile options - and a means
> to add an (optional) run command. Those one would probably pass via
> environment variables.
As I said in the other mail, I think we should try to test everything that is
testable without the user requiring it or configuring anything.
We could have a gfortran.dg/caf (like gomp,vect,...) dir in which every test
is tested against -fcoarray=single and with -fcoarray=lib for every libcaf_*.
>
> One would then either only run the tests if the environment variable is
> set - or if "libcaf_single.a" is installed by default (cf. below), one
> could default to linking that version if no environment variable is set.
> Then "make check-gfortran" could then always run the CAF library checks
> - otherwise, only conditionally.
>
> What do you think? Do you have comments how this should be done? I also
> wouldn't mind if someone could help as I am not really comfortable with
> the test-suite setup nor do I know Lisp well.
All is well, it is TCL, not Lisp. ;-)
> Thus, the first question is: Should one build and install single.c
> (libcaf_single.a) by default? (Might also relate to (a), namely how the
> test suite is handled.)
It is a small library to be put in a gcc-owned (thus it shouldn't be too
disturbing) directory. So, yes, in my opinion, we can install it by default.
The same goes for libcaf_mpi. Not as small, yet still reasonable.
We should install it if mpi is available.
>
> And the second question is: Should one be able to configure and build
> mpi.c (libcaf_mpi.a) by some means? I think users interested in could
> also do the procedure of [2] - or let their admin do it. (For Linux
> distributions one would run into the problem that they typically offer
> several MPI implementations, e.g. Open MPI and MPICH2, which couldn't be
> handled that way.)
We should try to have [2] done automatically if possible.
After all, all that is needed is a mpicc in the PATH (and the corresponding
mpif90 for the testsuite) so the corresponding autoconf code shouldn't be too
convoluted.
For the multiple MPI implementations, either we accept to choose only one at
configure time, or we could ship a stripped down mpi.h containing only the
interfaces we care about. As the interfaces are standard (aren't they?) we
could link with any standard-compliant implementation afterwards.
With that, no configury needed, we install unconditianally the libcaf_mpi.a
file, but some manual tweaking is needed at make check time to choose one mpi
implementation. Another downside is a burden on us to keep the file up-to-
date.
Mikael
[testsuite] How to conditionally skip a single dejagnu directive
Hello,
I have a runnable (fortran) test on which I want to check that a
conditional part is optimized away.
Here are the corresponding tree-dump directives :
! { dg-final { scan-tree-dump-times "_gfortran_abort" 32 "original" } }
! { dg-final { scan-tree-dump-times "_gfortran_abort" 30 "optimized" } }
This works, but as the fortran run test is a torture one (it tries
several optimization options), there is a single failure for the -O0
case (because there is no optimization).
I would like to skip the dg directive if the options match "-O0" but on
the tree dump only.
There is dg-skip-if, but it skips the whole test, and I don't want to
skip unrelated dg directives. Especially I don't want to skip the
testcase execution.
So, is there something existing to skip a single dump scan based on the
compilation options ?
Thanks in advance
Mikael
Re: [Patch, Committed] Re: New GNU Fortran reviewers.
Daniel Kraft wrote: > > I've taken the freedom to move myself as well as Mikael to the reviewer > section (Janus was already moved) and committed the attached patch to > MAINTAINERS. Thanks, I was too lazy :s. > > Cheers, > Daniel >
Re: Errors on your web page
Steve Kargl wrote: > Dear Absoft, > > Please fix the errors on your web page: > > http://www.absoft.com/Absoft_Windows_Compiler.htm > > The comparison chart is not only misleading, it is a down > right lie! The last column is labeled "GNU g77 gfortran". > g77 and gfortran are completely different compilers! gfortran > is a Fortran 95, and in fact, gfortran probably supports more > Fortran 2003 and draft Fortran 2008 features than Absoft's product. > Your chart clearly states that gfortran is not a 'Native f95 > compiler". What does 'native' mean, here? > > The chart indicates that gfortran doesn't use SSE. Well, > gfortran can use SSE if your processor has these features. > You need to use the options -msse -msse2 -msse3 -mssse3 > -msse4.1 -msse4.2 and -msse4 to get sse or one the -march= > options. > > What to you mean by "top 10% performance" in your chart? > gfortran performs as well as or better than Absoft's product > on many of the Polyhedron Benchmarks. Your PDF file under the > Linux Benchmark is quite misleading. You are comparing gfortran > 4.1.2 (released on 2/17/07) against your product. Try > comparing to gfortran 4.3.2 (the current release) or > the upcoming gfortran 4.4.0 release. You do not include > gfortran in the Windows benchmarch, and the MacOS X > comparison uses 4.3.0. Wow, that's consistent! > > Your chart states that gfortran doesn't support Mil Std 1753. > In fact, gfortran has supported Mil Std 1753 for a very long > time. Please fix your chart. > > What do you mean by "Mixed Platform Licenses"? gfortran can > be run on more architectures and operating systems than Absoft's > product and you only need to understanding the GPL (an Open > Source License, ie., it's free). > > Note there is a win64 version of gfortran provided by www.equation.com. > Again, your chart is misleading. > Do you really think they will change their webpage to one that explicitly states that their compiler has a competitor with comparable features, which is at least as fast, supports more platforms and is available for free? I would be very surprised if they do. Mikael
