Re: [PATCH v5 4/5] c++modules: report imported CMI files as dependencies
On Wed, Jul 19, 2023 at 17:11:08 -0400, Nathan Sidwell wrote: > GCC is neither of these descriptions. a CMI does not contain the transitive > closure of its imports. It contains an import table. That table lists the > transitive closure of its imports (it needs that closure to do remapping), > and > that table contains the CMI pathnames of the direct imports. Those pathnames > are absolute, if the mapper provded an absolute pathm or relative to the CMI > repo. > > The rationale here is that if you're building a CMI, Foo, which imports a > bunch > of modules, those imported CMIs will have the same (relative) location in > this > compilation and in compilations importing Foo (why would you move them?) Note > this is NOT inhibiting relocatable builds, because of the CMI repo. But it is inhibiting distributed builds because the distributing tool would need to know: - what CMIs are actually imported (here, "read the module mapper file" (in CMake's case, this is only the modules that are needed; a single massive mapper file for an entire project would have extra entries) or "act as a proxy for the socket/program specified" for other approaches); - read the CMIs as it sends to the remote side to gather any other CMIs that may be needed (recursively); Contrast this with the MSVC and Clang (17+) mechanism where the command line contains everything that is needed and a single bolus can be sent. And relocatable is probably fine. How does it interact with reproducible builds? Or are GCC CMIs not really something anyone should consider for installation (even as a "here, maybe this can help consumers" mechanism)? > On 7/18/23 20:01, Ben Boeckel wrote: > > Maybe I'm missing how this *actually* works in GCC as I've really only > > interacted with it through the command line, but I've not needed to > > mention `a.cmi` when compiling `use.cppm`. Is `a.cmi` referenced and > > read through some embedded information in `b.cmi` or does `b.cmi` > > include enough information to not need to read it at all? If the former, > > distributed builds are going to have a problem knowing what files to > > send just from the command line (I'll call this "implicit thin"). If the > > latter, that is the "fat" CMI that I'm thinking of. > > please don't use perjorative terms like 'fat' and 'thin'. Sorry, I was internally analogizing to "thinLTO". --Ben
Re: [PATCH v5 4/5] c++modules: report imported CMI files as dependencies
On 7/18/23 20:01, Ben Boeckel wrote: On Tue, Jul 18, 2023 at 16:52:44 -0400, Jason Merrill wrote: On 6/25/23 12:36, Ben Boeckel wrote: On Fri, Jun 23, 2023 at 08:12:41 -0400, Nathan Sidwell wrote: On 6/22/23 22:45, Ben Boeckel wrote: On Thu, Jun 22, 2023 at 17:21:42 -0400, Jason Merrill wrote: On 1/25/23 16:06, Ben Boeckel wrote: They affect the build, so report them via `-MF` mechanisms. Why isn't this covered by the existing code in preprocessed_module? It appears as though it is neutered in patch 3 where `write_make_modules_deps` is used in `make_write` (or will use that name Why do you want to record the transitive modules? I would expect just noting the ones with imports directly in the TU would suffice (i.e check the 'outermost' arg) FWIW, only GCC has "fat" modules. MSVC and Clang both require the transitive closure to be passed. The idea there is to minimize the size of individual module files. If GCC only reads the "fat" modules, then only those should be recorded. If it reads other modules, they should be recorded as well. For clarification, given: * a.cppm ``` export module a; ``` * b.cppm ``` export module b; import a; ``` * use.cppm ``` import b; ``` in a "fat" module setup, `use.cppm` only needs to be told about `b.cmi` because it contains everything that an importer needs to know about the `a` module (reachable types, re-exported bits, whateve > With the "thin" modules, `a.cmi` must be specified when compiling `use.cppm` to satisfy anything that may be required transitively (e.g., a return GCC is neither of these descriptions. a CMI does not contain the transitive closure of its imports. It contains an import table. That table lists the transitive closure of its imports (it needs that closure to do remapping), and that table contains the CMI pathnames of the direct imports. Those pathnames are absolute, if the mapper provded an absolute pathm or relative to the CMI repo. The rationale here is that if you're building a CMI, Foo, which imports a bunch of modules, those imported CMIs will have the same (relative) location in this compilation and in compilations importing Foo (why would you move them?) Note this is NOT inhibiting relocatable builds, because of the CMI repo. Maybe I'm missing how this *actually* works in GCC as I've really only interacted with it through the command line, but I've not needed to mention `a.cmi` when compiling `use.cppm`. Is `a.cmi` referenced and read through some embedded information in `b.cmi` or does `b.cmi` include enough information to not need to read it at all? If the former, distributed builds are going to have a problem knowing what files to send just from the command line (I'll call this "implicit thin"). If the latter, that is the "fat" CMI that I'm thinking of. please don't use perjorative terms like 'fat' and 'thin'. But wouldn't the transitive modules be dependencies of the direct imports, so (re)building the direct imports would first require building the transitive modules anyway? Expressing the transitive closure of dependencies for each importer seems redundant when it can be easily derived from the direct dependencies of each module. I'm not concerned whether it is transitive or not, really. If a file is read, it should be reported here regardless of the reason. Note that caching mechanisms may skip actually *doing* the reading, but the dependencies should still be reported from the cached results as-if the real work had been performed. --Ben -- Nathan Sidwell
Re: [committed] - Re: [patch] OpenMP/Fortran: Non-rectangular loops with constant steps other than 1 or -1 [PR107424]
Hi Tobias! On 2023-07-19T10:26:12+0200, Tobias Burnus wrote: > Now committed as Rev. r14-2634-g85da0b40538fb0 On devel/omp/gcc-13 branch, the corresponding commit b003e6511754dce475f7f5b0c5cd887a177e41b3 "OpenMP/Fortran: Non-rectangular loops with constant steps other than 1 or -1 [PR107424]" introduces a regression: PASS: libgomp.fortran/loop-transforms/unroll-2.f90 -O0 (test for excess errors) [-PASS:-]{+FAIL:+} libgomp.fortran/loop-transforms/unroll-2.f90 -O0 execution test Etc. spawn [open ...] 4 8 10 11 Program aborted. Backtrace: #0 0x400f9c in test at [...]/libgomp.fortran/loop-transforms/unroll-2.f90:85 #1 0x400fd3 in main at [...]/libgomp.fortran/loop-transforms/unroll-2.f90:59 Grüße Thomas > Changes: > > * I missed to updated another 'sorry' (msg wording change) - now fixed; > I also added it to the sorry-testcase file non-rectangular-loop-5.f90. > > * I decided to retire the PR as several issues have been fixed and the > original title did not fit any more. The remaining issue is now tracked > in PR110735 (i.e. handling step != const, both the generic and possibly > a simpler special case). > > * I added a link to the PR to libgomp.texi such that one can find out > what is only partially supported for Fortran. > > Thanks, > > Tobias > > PS: Otherwise, the following still applies: > > On 18.07.23 14:11, Tobias Burnus wrote: >> Comments regarding the validity of the Fortran assumptions are welcome! >> >> This patch now uses a 'simple' loop for OpenMP loops with >> a constant loop-step size. Before, it only did so for step = ±1. >> (Otherwise, a count variable is used from which the original >> loop index variable is calculated from.) >> >> For details, see the attached patch or >> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107424#c12 >> (comment 12 + 14 plus the email linked in comment 12). >> >> Comments? Remarks? If there are none, I will relatively soonish >> commit the attached patch to mainline, only. > commit 85da0b40538fb0d17d89de1e7905984668e3dfef > Author: Tobias Burnus > Date: Wed Jul 19 10:18:49 2023 +0200 > > OpenMP/Fortran: Non-rectangular loops with constant steps other than 1 or > -1 [PR107424] > > Before this commit, gfortran produced with OpenMP for 'do i = 1,10,2' > the code > for (count.0 = 0; count.0 < 5; count.0 = count.0 + 1) > i = count.0 * 2 + 1; > > While such an inner loop can be collapsed, a non-rectangular could not. > With this commit and for all constant loop steps, a simple loop such > as 'for (i = 1; i <= 10; i = i + 2)' is created. (Before only for the > constant steps of 1 and -1.) > > The constant step permits to know the direction (increasing/decreasing) > that is required for the loop condition. > > The new code is only valid if one assumes no overflow of the loop > variable. > However, the Fortran standard can be read that this must be ensured by > the user. Namely, the Fortran standard requires (F2023, 10.1.5.2.4): > "The execution of any numeric operation whose result is not defined by > the arithmetic used by the processor is prohibited." > > And, for DO loops, F2023's "11.1.7.4.3 The execution cycle" has the > following: The number of loop iterations handled by an iteration count, > which would permit code like 'do i = huge(i)-5, huge(i),4'. However, > in step (3), this count is not only decremented by one but also: > "... The DO variable, if any, is incremented by the value of the > incrementation parameter m3." > And for the example above, 'i' would be 'huge(i)+3' in the last > execution cycle, which exceeds the largest model number and should > render the example as invalid. > > PR fortran/107424 > > gcc/fortran/ChangeLog: > > * trans-openmp.cc (gfc_nonrect_loop_expr): Accept all > constant loop steps. > (gfc_trans_omp_do): Likewise; use sign to determine > loop direction. > > libgomp/ChangeLog: > > * libgomp.texi (Impl. Status 5.0): Add link to new PR110735. > * testsuite/libgomp.fortran/non-rectangular-loop-1.f90: Enable > commented tests. > * testsuite/libgomp.fortran/non-rectangular-loop-1a.f90: Remove > test file; tests are in non-rectangular-loop-1.f90. > * testsuite/libgomp.fortran/non-rectangular-loop-5.f90: Change > testcase to use a non-constant step to retain the 'sorry' test. > * testsuite/libgomp.fortran/non-rectangular-loop-6.f90: New test. > > gcc/testsuite/ChangeLog: > > * gfortran.dg/gomp/linear-2.f90: Update dump to remove > the additional count variable. > --- > gcc/fortran/trans-openmp.cc| 18 +- > gcc/testsuite/gfortran.dg/gomp/linear-2.f90| 4 +- >
[committed] - Re: [patch] OpenMP/Fortran: Non-rectangular loops with constant steps other than 1 or -1 [PR107424]
Now committed as Rev. r14-2634-g85da0b40538fb0 Changes: * I missed to updated another 'sorry' (msg wording change) - now fixed; I also added it to the sorry-testcase file non-rectangular-loop-5.f90. * I decided to retire the PR as several issues have been fixed and the original title did not fit any more. The remaining issue is now tracked in PR110735 (i.e. handling step != const, both the generic and possibly a simpler special case). * I added a link to the PR to libgomp.texi such that one can find out what is only partially supported for Fortran. Thanks, Tobias PS: Otherwise, the following still applies: On 18.07.23 14:11, Tobias Burnus wrote: Comments regarding the validity of the Fortran assumptions are welcome! This patch now uses a 'simple' loop for OpenMP loops with a constant loop-step size. Before, it only did so for step = ±1. (Otherwise, a count variable is used from which the original loop index variable is calculated from.) For details, see the attached patch or https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107424#c12 (comment 12 + 14 plus the email linked in comment 12). Comments? Remarks? If there are none, I will relatively soonish commit the attached patch to mainline, only. - Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955 commit 85da0b40538fb0d17d89de1e7905984668e3dfef Author: Tobias Burnus Date: Wed Jul 19 10:18:49 2023 +0200 OpenMP/Fortran: Non-rectangular loops with constant steps other than 1 or -1 [PR107424] Before this commit, gfortran produced with OpenMP for 'do i = 1,10,2' the code for (count.0 = 0; count.0 < 5; count.0 = count.0 + 1) i = count.0 * 2 + 1; While such an inner loop can be collapsed, a non-rectangular could not. With this commit and for all constant loop steps, a simple loop such as 'for (i = 1; i <= 10; i = i + 2)' is created. (Before only for the constant steps of 1 and -1.) The constant step permits to know the direction (increasing/decreasing) that is required for the loop condition. The new code is only valid if one assumes no overflow of the loop variable. However, the Fortran standard can be read that this must be ensured by the user. Namely, the Fortran standard requires (F2023, 10.1.5.2.4): "The execution of any numeric operation whose result is not defined by the arithmetic used by the processor is prohibited." And, for DO loops, F2023's "11.1.7.4.3 The execution cycle" has the following: The number of loop iterations handled by an iteration count, which would permit code like 'do i = huge(i)-5, huge(i),4'. However, in step (3), this count is not only decremented by one but also: "... The DO variable, if any, is incremented by the value of the incrementation parameter m3." And for the example above, 'i' would be 'huge(i)+3' in the last execution cycle, which exceeds the largest model number and should render the example as invalid. PR fortran/107424 gcc/fortran/ChangeLog: * trans-openmp.cc (gfc_nonrect_loop_expr): Accept all constant loop steps. (gfc_trans_omp_do): Likewise; use sign to determine loop direction. libgomp/ChangeLog: * libgomp.texi (Impl. Status 5.0): Add link to new PR110735. * testsuite/libgomp.fortran/non-rectangular-loop-1.f90: Enable commented tests. * testsuite/libgomp.fortran/non-rectangular-loop-1a.f90: Remove test file; tests are in non-rectangular-loop-1.f90. * testsuite/libgomp.fortran/non-rectangular-loop-5.f90: Change testcase to use a non-constant step to retain the 'sorry' test. * testsuite/libgomp.fortran/non-rectangular-loop-6.f90: New test. gcc/testsuite/ChangeLog: * gfortran.dg/gomp/linear-2.f90: Update dump to remove the additional count variable. --- gcc/fortran/trans-openmp.cc| 18 +- gcc/testsuite/gfortran.dg/gomp/linear-2.f90| 4 +- libgomp/libgomp.texi | 4 +- .../libgomp.fortran/non-rectangular-loop-1.f90 | 537 ++--- .../libgomp.fortran/non-rectangular-loop-1a.f90| 374 -- .../libgomp.fortran/non-rectangular-loop-5.f90 | 22 +- .../libgomp.fortran/non-rectangular-loop-6.f90 | 196 7 files changed, 494 insertions(+), 661 deletions(-) diff --git a/gcc/fortran/trans-openmp.cc b/gcc/fortran/trans-openmp.cc index c88ee3c7656..cf741cebf91 100644 --- a/gcc/fortran/trans-openmp.cc +++ b/gcc/fortran/trans-openmp.cc @@ -5374,10 +5374,10 @@ gfc_nonrect_loop_expr (stmtblock_t *pblock, gfc_se *sep, int loop_n, if