Hi folks, > On 1 Oct 2025, at 16:55, Andre Vehreschild <[email protected]> wrote: > > Hi Iain, > > there is: > > git checkout devel/gfortran-test > > which gives an older state. Then doing > > git am pr88076_v4_9.patch > > (yes, only that last file) should bring you the state this mail published.
I wil try what has been suggested - but I guess that also applying the patch series to current trunk ought to work? > When you then go to > > libgfortran/caf/shmem/shared_memory.c > > and search for "mmap" you'll see what I do. Yes, I know and tried MMAP_FIXED. > It works for some cases and in other corrupts the address space on MacOS. > (Works fine on Linux all the time). The base address is "send" from the > supervisor process to its workers (called images in Fortran) using an > environment variable. At the moment I am using the "proposal" method. I.e., I > give MacOS the base address where I want that shared memory segment and just > specify MMAP_SHARED. You might need to “propose” an address that is in a range that is not subject to other uses (i.e. especially that other uses could have ALSR applied which would mean that you’d see a random unavailablity). The sanitizers and the PCH code both need to take this into account. > On Linux, I am done. I get the shared memory always on the desired address. Well - I’d assume that on Linux editions with PIE executables and ASLR enabled you’d have to make similar considerations? (I am not so familiar with these facilities on Linux). > > On MacOS the address is sometimes accepted on the first try, and sometimes > needs several thousand tries to be accepted. But finally MacOS accepts it. > This > is so indeterministic to me. I expect that is totally intentional :) .. if you have an address in a range that conflicts with ALSR. > > I am really curious what I am doing wrong for MacOS being so exceptional. it’s plausibly not only the OS - but also the default conditions that apply on the OS - for macOS: PIE and ASLR are defaults (and on Arm64 macOS mandated by security policy). That might not be the situation on the Linux distribution you are using (IDK, just speculating). > The basic reason for all this fuss is, that an approach using a shared > memory segment for implementing Fortran's coarrays was proposed. Coarrays are > a > language intrinsic parallel programming paradigm. You may compare it with MPI > where data is send to and retrieved from other processes. The caf_shmem > approach > just uses a big memory segment to store the coarrays in and allows for fast > access to memory that is owned by a different process participating in the > same > computation. I hope this broad description helps a little bit. thanks Iain > > Regards, > Andre > > On Wed, 1 Oct 2025 15:13:00 +0100 > Iain Sandoe <[email protected]> wrote: > >> Hi Andre >> >>> On 1 Oct 2025, at 14:52, Andre Vehreschild <[email protected]> wrote: >> >>> attached series of patches adds more stable support for running shared >>> memory multi process Fortran programs on MacOS. The only new part is the >>> last (the pr88076_v4_9.patch). Changes in the former files are only results >>> of rebasing. >>> >>> On MacOS shared memory coarrays face the issue of having to have the shared >>> memory segment at the same (virtual) base address in all processes/images, >>> but the OS not always complying to our wish. The supervisor of the coarray >>> program therefore now is able to restart an image, when the OS is not >>> complying to put the shared memory segment where we need to have it. The >>> shared memory segment needs to be at the same address in all images, >>> because else shared mutexes tend malfunction. I haven't found any >>> documentation from Apple or the MachOS kernel why this is that way, but it >>> unfortunately is. The number of restarts of images is limited. I.e. this is >>> not going on forever, but one can either set a (documented) environment >>> variable, or got with the default of 4000 at the moment. The count on >>> restarts is total on all images and not per image. >> >> I would expect that if one wanted a shared memory segment then it would be >> allocated in one place and the address made available to other consumers; >> there are provisions for asking for a particular address from mmap too …. >> >> Maybe I can help with this - but I would need to understand first what you >> are >> currently trying to do - and what results it gets. I think last time I tried >> the coarrays stuff there were build problems so I did not even get this far. >> >> Is there a development branch somewhere I can build? >> >>> It took me quite some time to figure what was going on and I had to fix >>> libsanitizer on macOS for newer OS releases, >> >> How, exactly, did you do this? >> >> The sanitizers are currently disabled on newer macOS because the API to get >> the address of the dynamic linker uses Apple Blocks which GCC does not yet >> support. I have in mind a work-around but not had a chance to try it yet, >> >>> which being unfamiliar took me >>> even longer. There might be other issues fixed unintentionally by these >>> patches. So feel free to test. >> >> I will try to fit this in - if you can point me at a suitable branch - to >> make sure I’m looking at the same thing as you. >> >> thanks >> Iain >> >>> Regtested ok on x86_64-pc-linux-gnu / F41, x86_64-w64-win32 / MSYS2 / >>> UCRT64 and x86_64-apple-darwin24.something. (Testing on MSYS2/UCRT64 gives >>> tons of fails of 'excess output' tests, which stem from gfortran emitting >>> some escape code, which doesn't even print, and expect picking it up.) >>> >>> Regards, >>> Andre >>> >>> On Wed, 10 Sep 2025 13:31:01 -0700 >>> Jerry D <[email protected]> wrote: >>> >>>> On 9/2/25 2:51 AM, Andre Vehreschild wrote: >>>>> Hi all, >>>>> >>>>> attached series of patches updates the coarray shmem gfortran series of >>>>> patches. It now has support for building on Window's Cygwin and Window's >>>>> MSYS2/UCRT64. Building on the latter is quite tricky. When you want to do >>>>> that, I recommend getting the source package for gcc-15 from the msys >>>>> website and adapting it. Nevertheless it is very fiddly to get it to >>>>> build. Most of the regression test on MSYS2/UCRT64 report "Excess errors" >>>>> on the compile step, which is due to gfortran emitting some escape codes, >>>>> that do not print, but "expect" regards them as output. >>>>> >>>>> I have deliberately (rebased and) updated the gfortran-test branch also, >>>>> so that you can get the newest version from there, too. >>>>> >>>>> Regtests ok on x86_64-pc-linux / F41 and x86_64-pc-cygwin / Win10. >>>>> >>>>> After hopefully having Windows wrestled down now, I will tackle MacOS not >>>>> having a sane process parallel mutex. >>>>> >>>>> Have fun testing. >>>>> >>>>> Regards, >>>>> Andre >>>> >>>> Hi all, >>>> >>>> Pinging here as a reminder. I have managed to apply the new patches but I >>>> have not had time to test. I also found a Cygwin installation on one of my >>>> computers here but have not tried anything there yet. >>>> >>>> Andre,, this is going to take some time and I hope to get to this. >>>> >>>> Regards, >>>> >>>> Jerry >>> >>> >>> -- >>> Andre Vehreschild * Email: vehre ad gmx dot de >>> <pr88076_v4_1.patch><pr88076_v4_2.patch><pr88076_v4_3.patch><pr88076_v4_4.patch><pr88076_v4_5.patch><pr88076_v4_6.patch><pr88076_v4_7.patch><pr88076_v4_8.patch><pr88076_v4_9.patch> >>> >> > > > -- > Andre Vehreschild * Email: vehre ad gmx dot de
