On 9/21/18 1:16 PM, Damian Rouson wrote:> On Fri, Sep 21, 2018 at 9:25
AM Jerry DeLisle <jvdeli...@charter.net> wrote:
>
>> The actual library source code is contained mostly in one source file.
>
> There are as many files as there are options for the underlying
> parallel programming
> model. The default is MPI, but I've co-authored conference papers
last year
> and this year in which the OpenCoarrays OpenSHEM option outperformed MPI.
> One paper even described a platform on which OpenSHMEM was the only
option
> beyond a few thousand cores because the required MPI features were
immature on
> that platform. Early versions of OpenCoarrays also provided GASNet
> and ARMCI options.
> I recommend against tying gfortran to MPI only.
I agree with you on this point. Perhaps the Opencoarrays implementation
should somehow do some runtime introspection to allow the library to
sync to whatever is desired on a given system. The gfortran interface
was designed to be generic. Implementation should be more dynamic in run
time linking and abstracted in such a way that OpenCoarrays could be
compiled stand alone and use something like "plugins" to allow post
build the determination of what which interface to use.
I am by no means a software expert in these techniques, but they are
becoming common practice in other areas, for example linux/Gnu kernel
modules
>
>> After all the attempts to integrate into the GNU build systems without
>> much success my thinking has shifted.
>
> Thanks for all your efforts!
>
>> Keep in mind that the OpenCoarrays
>> implementation is quite dependent on gfortran and in fact has to do
>> special things in the build dependent on the version of gcc/gfortran a
>> user happens to use. I dont think this is a good situation.
>
> I agree. Possibly OpenCoarrays could drop support for older gfortran
versions
> at some point to avoid maintaining code that exists solely to support
compiler
> versions that are several years old.
See my comments above about pluggable modules. Maybe libgfortran should
have this pluggable interface and Opencoarrays provide the plugins.
Think how useful it would be to be able to choose the backend at time of
execution based on a simple envoronment variable set by the user.
>
>>
>> 1) Focus on distribution packages such as Fedora, Debian, Ubuntu,
>> Windows, etc. Building of these packages needs to be automated into the
>> distributions.
>
> This is the option that the OpenCoarrays documentation recommends as
easiest for
> most users.
Agree.
>
>> 2) Take the one source file, edit out all the macros that define
>> prefixes to function calls, hard code the gfortran prefixes etc and fork
>> it directly into the libgfortran library under GPL with attributions to
>> the original developers as appropriate.
>
> See above. Also, this means that changes in the gfortran repository
would not
> propagate back upstream unless each gfortran developer agrees to
> distribute his or her
> work under both GPL and BSD. Even that is only feasible if the copied
> files stay cohesive
The flip of this would be to have the OpenCorrays developers to agree to
the GPL and release under both. The libgfortran license says:
"Under Section 7 of GPL version 3, you are granted additional
permissions described in the GCC Runtime Library Exception, version
3.1, as published by the Free Software Foundation."
Probably worth a fresh look.
> and don't reference code outside the copied file. I think it's more
> likely that copying the code
> into gfortran would be a branch point, after which the relevant files
> would diverge and
> work on the GPL side would be harder to fund than the BSD side.
>
> Most commercial entities are more likely to contribute to a
> BSD-licensed project than a
> GPL-licensed one. Over the past several months, one commercial
compiler vendor
> authorized one of their developers to contribute to OpenCoarrays. and
> another commercial
> compiler vendor invited community input on whether to use OpenCoarrays
> during a public
> teleconference. The prospect of commercial support is the motivation
> for using BSD.
I really have no commercial interest. So I will not comment on GPL vs
BSD other than referring to the multitude of FSF recommendations about
why one should choose one of the FSF flavors rather than BSD.
>
>> Strategy 2 does have some advantages. For example, eliminating the need
>> for separate CAF and CAFRUN scripts which are a wrapper on gfortran.
>
> Even in the case of just one underlying parallel programming model,
> this is tricky. To wit, Cray uses
> a compiler wrapper and a program launcher. Intel was able to
> eliminate the compiler wrapper,
> but still required a program launcher for distributed-memory execution
> until recently. I don't
> know the details, but I've heard it was not trivial for Intel to
> accomplish this and I imagine it would be
> even more complicated if they weren't hardwiring Intel MPI into their
back-end.
Well here is one commercial entity that did not shy away from
'hardwiring' MPI, Regardless, using plugins would resolve concerns about
which MPI to use or whether to use shared memory or some other model.
>
>> People would just have to stop thinking about it and just use it.
>
> The same would be true if someone could coax the GCC build system to
> build OpenCoarrays
> just as it builds other prerequisites. The big difference is that
> OpenCoarrays is a prerequisite
> for using gfortran rather than for building gfortran so it needs to be
> built after gfortran rather
> than before like other prerequisites. The real problem is finding
> anyone who can work the
> proper magic in the GCC build system.
I dont see this as the real problem. The forking idea would resolve this
fairly easily. Above, you mentioned concern about locking into MPI. Do
the packaged versions of OpenCoarrays not lock into MPI, either OpenMPI
or MPICH? I have not tried one yet since I am waiting for the Fedora one
to hit the release.
If the tight coupling is needed maybe there ought to be a set of
libraries or modules, one for each "backend". (Back to my pluggable
modules concept) The more I think about it the more I think this is the
fundamental design issue.
>
> Thanks for your input. I hope my response is helpful.
>
> Damian
>
As always, best regards.
Jerry
