This is very well put, Michael. I still suggest that moving to C++17 is a better option.
best regards Janek Michael Hofmann said: (by the date of Wed, 6 May 2020 09:17:40 +0200) > There are simple and established solutions to the problem of having an old > system compiler or standard libraries on a cluster-type computing system. > The Environment Modules system (http://modules.sourceforge.net/) is one of > them. Local compiler installs or even Docker are others. Any sysadmin worth > their salt can solve these issues for users quite easily, if they just > wanted to. (They could additionally upgrade the system, if they just wanted > to.) > > > However, conservatism certainly has value. Particularly when you > advertise yourself as a production library (not a research or proof of > concept toy). > > Eigen development is slow. The code has reached a very stable state. > The last major release was in 2018. These facts are probably good for most > users of Eigen. > > I heavily disagree. In my opinion, there is absolutely no value in this > kind of conservatism. Instead, it *only* harms. > > Conservatism in library development has little effect on "production > quality". In fact, I'd argue that libraries that choose to move faster are > often better, including for production use cases, since they have > judiciously cleaned up their cruft, thus attract a larger developer and > user base and have a healthier ecosystem. It is proven that in software > development, small lead and cycle times and frequent "deploys", be it > upstream or downstream, in libraries or services, are strongly beneficial. > At least part of the reason why Eigen development has slowed to a halt is > said refusal to move with the times, at the right time (i.e. not six years > too late). > > We have a classic case of "code rot" here, due to changing requirements > from the user base over time. Both the internals of Eigen have become > harder to maintain (things are not expressed as simply and elegantly as > they could be) and the externals harder to use (crufty interfaces, slow > compilation speeds, myriads of spurious warnings on newer compilers, etc.). > > Because nothing happened for years due to the conservative governance > model, modernizing the code has now accumulated to a huge task, and it's on > a developer base that has already shrunken quite a bit, judging from the > average level of activity on this mailing list. This greatly exacerbates > the problem. Continuous and incremental modernization in a timely(!) manner > is the best way to prevent this scenario from happening. This includes (but > is not limited to) moving to new C++ standards around the time they are > introduced. > > It doesn't really matter whether one calls a newer version 3.x or 4.x -- > these are just semantics in the end. No matter how a project is governed > (SemVer or not, level of promised API or ABI stability, etc.), the > fundamental insight has to be that *old versions do not go away* and can > still be used by anyone who wishes to do so. Conservative users may choose > to upgrade more rarely, at the cost of much larger pain when doing so. (A > model that I do not believe in.) Most users should be encouraged to migrate > to the latest released version quickly, and to be part of the continuous, > incremental maintenance cycle. > > I have been an Eigen user and defender for years, but I cannot recommend > this library anymore to others without giving huge caveats ("outdated", > "seems effectively abandoned"). A strong commitment to modernity and moving > with the times by the maintainers is the best and maybe only way to bring > back users and thus developers. As part of this, I do hope that Eigen > governance commits to moving to C++17 in the very near future. > > Cheers, > Michael > > On Wed, May 6, 2020 at 5:53 AM Rob McDonald <[email protected]> > wrote: > > > I don't have access to the machine, so I can't answer the detailed > > questions. > > > > My application isn't super intensive -- but it can be used to generate > > input files for much more expensive computations and optimization studies. > > Consequently, it sometimes gets run on big iron. > > > > My user who recently had problems because my app developed a dependency on > > std::regexp -- which isn't really supported by gcc 4.8. The problem is > > less the compiler and more the standard libraries installed on the > > machine. He is not building on the big machine -- he builds on a local vm > > on a laptop and then transfers the binary up. They also prefer to do their > > local setup on images that are as similar as possible to the computation > > environment. Perhaps they could set up local compilers and libraries, but > > that is a much bigger hassle than using system installed standard libraries. > > > > The machine is not at TACC. It has nearly 250k cores and 1PB memory. It > > was installed when RHEL 7 was brand new. I have no idea how many cores the > > users are using -- a small fraction no doubt. > > > > Rob > > > > > > On Tue, May 5, 2020 at 8:01 PM Jeff Hammond <[email protected]> > > wrote: > > > >> Is the primary compiler on said supercomputer really /usr/bin/g++? How > >> many nodes are users running on when they build apps with this compiler? > >> > >> My guess is most of the users of that system use mpicxx or something > >> similar from modules that provides a trivial solution to this problem. If > >> you’re referring to TACC, I’m sure they’ll recommend a bunch of compilers > >> besides GCC-4.8. > >> > >> RHEL has devtoolset for a reason. All major HPC systems have modules for > >> a dozen or more compilers that don’t live in /usr. Eigen shouldn’t hold > >> itself back for people who refuse to help themselves. > >> > >> Jeff > >> > >> On Tue, May 5, 2020 at 3:21 PM Rob McDonald <[email protected]> > >> wrote: > >> > >>> I certainly understand the need to move forward. However, conservatism > >>> certainly has value. Particularly when you advertise yourself as a > >>> production library (not a research or proof of concept toy). > >>> > >>> Just as an example, the main national supercomputer for a major US > >>> government research agency is running RHEL 7. That means gcc 4.8.x and > >>> the > >>> standard libraries that go with it. That means C++11 and most of the > >>> stuff > >>> that goes with it -- but not everything. > >>> > >>> RHEL 8 was only released 364 days ago (depending on your time zone). It > >>> is certainly not unreasonable for installed high value production > >>> computing > >>> assets like the above machine to still be running RHEL 7. While it would > >>> be great if RHEL moved compilers forward more quickly -- that isn't how it > >>> works. > >>> > >>> The fact is -- Eigen development is slow. The code has reached a very > >>> stable state. The last major release was in 2018. These facts are > >>> probably good for most users of Eigen. > >>> > >>> If permission to jump to some advanced version means someone is going to > >>> go through and wholesale clean up the code and move it forward -- who is > >>> that someone? Where is this development energy coming from? Users making > >>> a request doesn't mean there is anyone to get the job done. > >>> > >>> Eigen 3.x should remain pretty much the way it is. I hope/assume that > >>> any big jump like this would start by creating an Eigen 4.x branch/fork on > >>> GitHub and moving forward from there -- hopefully the energetic people > >>> calling for this change are the ones who will step forward to do the work. > >>> > >>> Rob > >>> > >>> > >>> > >>> On Tue, May 5, 2020 at 3:01 PM Janek Kozicki (yade) < > >>> [email protected]> wrote: > >>> > >>>> I suggest to move forward, and use C++17. There is no reason to > >>>> stick with old libraries. > >>>> > >>>> If you prefer to keep for a year or two the ability to compile with > >>>> C++14, then place such obsolete code inside some #ifdef #else #endif, > >>>> so that more efficient code is generated with C++17 and less efficient > >>>> code when someone insists on compiling with C++14. > >>>> > >>>> In yade[1] we have moved to C++17 when building packages for ubuntu > >>>> focal 20.04 or debian buster or bullseye. The C++14 is used only when > >>>> building packages for older systems such as ubuntu xenial 16.04. > >>>> > >>>> Yade can compile with eigen using C++17 without any problems. Which > >>>> is good. For example the coinor library had some problems with C++17: > >>>> and older version worked only with C++17. > >>>> > >>>> best regards > >>>> Janek > >>>> > >>>> [1] https://yade-dem.org/doc/ and https://gitlab.com/yade-dev/trunk > >>>> > >>>> > >>>> > >>>> Rasmus Munk Larsen said: (by the date of Tue, 5 May 2020 14:14:19 > >>>> -0700) > >>>> > >>>> > I agree with you, but it seems the maintainers are very busy with > >>>> other > >>>> > projects, and I am guessing they do not have time for major > >>>> initiatives to > >>>> > clean up Eigen. We have dropped support for c++03 in the Tensor > >>>> library, > >>>> > but a lot of things could be cleaned up if we more fully embraced > >>>> c++11 or > >>>> > c++14. At this point, I only see this happening slowly, or if > >>>> somebody from > >>>> > the open source community is willing to help. > >>>> > > >>>> > On Tue, May 5, 2020 at 2:05 PM Patrik Huber <[email protected]> > >>>> wrote: > >>>> > > >>>> > > Hi all, > >>>> > > > >>>> > > I'm a quite sad to see this post got no replies. I can only add > >>>> that I > >>>> > > fully agree and think along the very same lines. I would love to > >>>> see Eigen > >>>> > > move forward, move to C++14 in the 3.5 release, and drop the old > >>>> cruft. > >>>> > > It's 2020 now, 6 years after C++14. > >>>> > > > >>>> > > Best wishes, > >>>> > > Patrik > >>>> > > > >>>> > > On Wed, 8 Apr 2020 at 17:20, Martin Beeger <[email protected] > >>>> > > >>>> > > wrote: > >>>> > > > >>>> > >> Hallo! > >>>> > >> > >>>> > >> This is continuation of the discussion from last year about > >>>> > >> compatibility, but I wanted to bring it up again, because I have > >>>> some > >>>> > >> new data from my experience. > >>>> > >> > >>>> > >> Our company uses Eigen extensively and has a larger ecosystem > >>>> built on > >>>> > >> it, as do many of the voices on this list. We are in the embedded > >>>> world, > >>>> > >> so we are often hindered to adapt quickly to new tools, very much > >>>> like > >>>> > >> the HPC community has been. > >>>> > >> But we managed to move to C++14 with our codebase in 2019. When > >>>> doing > >>>> > >> that, we somewhat monitored performance and compile time during the > >>>> > >> adaption. > >>>> > >> > >>>> > >> When we starting compiling our C++98 codebase as is with C+14, > >>>> > >> performance already went up slightly. C++14 does silently move or > >>>> elide > >>>> > >> copies, and modern compiler have better optimizers, and the > >>>> quality of > >>>> > >> implementation of STL types got better. So if you are at all > >>>> bound by > >>>> > >> performance of STL types, I would really recommend using a new > >>>> compiler > >>>> > >> in C++14 mode even with your old code. > >>>> > >> > >>>> > >> This was the obvious part. Another part was less obvious. We > >>>> profiled > >>>> > >> compile time, which didn't change much after the switch and > >>>> started to > >>>> > >> look into stuff which was expensive to compile. And then > >>>> selectively in > >>>> > >> places, where the compile time was bad we simplified the code using > >>>> > >> C++14 features to improve compile time. This allowed us with very > >>>> > >> localized changes to cut our compile time by almost in half, > >>>> while, at > >>>> > >> the same time, making the code in question often a lot simpler and > >>>> new > >>>> > >> features (e.g. for performance improvements) easier to implement. > >>>> > >> > >>>> > >> This is also the experience which was observed with other template > >>>> heavy > >>>> > >> codebases like boost::mpl in comparison to hana and other stuff. > >>>> The > >>>> > >> gains in simplicity and compile time, especially from constexpr and > >>>> > >> lambda features are not minor. They can often cut your code in > >>>> half and > >>>> > >> more than double the compile time. > >>>> > >> > >>>> > >> The problem is, I am at a point where its hard to do much more to > >>>> > >> improve the compile time of my codebase significantly, because > >>>> most of > >>>> > >> the compile time is brought in by 2 libraries: boost.test and > >>>> Eigen. We > >>>> > >> will most likely at some point abandon boost.test due to this, > >>>> like many > >>>> > >> others have already (which slowed the development and improvement > >>>> of > >>>> > >> boost.test further, while the alternatives got better and getting > >>>> into > >>>> > >> this downwards spiral). I would be vary happy if I am not forced to > >>>> > >> abandon Eigen after the great 10 years we had with this library. In > >>>> > >> order to avoid this I pulled a lot of tricks like explicit template > >>>> > >> instatiations, tricks to reduce includes, even pimpl-like > >>>> encapsulation > >>>> > >> at performance cost to isolate from the problem, but that gets you > >>>> only > >>>> > >> so far. > >>>> > >> > >>>> > >> I may well be that my use case is special, but I strongly assume > >>>> that > >>>> > >> new users which today want to adopt Eigen and have to look into its > >>>> > >> internals (as you inevitable need to do at some points), will see > >>>> how > >>>> > >> its written and quickly run for alternatives. This amount of > >>>> macros, > >>>> > >> boilerplate and similar stuff will be an argument against this > >>>> great > >>>> > >> library some day and this day may already have come. > >>>> > >> > >>>> > >> The important part is here: Eigen compile time and internal > >>>> expression > >>>> > >> template engine code readability it was great by 2010s standards, > >>>> it was > >>>> > >> ok by 2015 standards, it is borderline by 2020 standards, and > >>>> unless > >>>> > >> something changes, it will be unacceptable by 2025 standards, > >>>> unless the > >>>> > >> Eigen library moves along. > >>>> > >> > >>>> > >> As C++ users, we do care about backwards compatibility greatly and > >>>> that > >>>> > >> is even good for me, but we should not go the C way and care about > >>>> too > >>>> > >> ancient compilers. The C++ comittee doesn't (that why int is > >>>> required to > >>>> > >> be 2s complement in C++20), so Eigen library maintainers IMHO > >>>> should > >>>> > >> follow. > >>>> > >> > >>>> > >> What are the chances to get Eigen 3.4 out of the door with C++98 > >>>> support > >>>> > >> and drop it on the devel branch afterwards and jump to C++14? > >>>> > >> What is the Eigen promises about how old yours compilers may be? > >>>> > >> Can we explicitly agree on a statement like: We vow to support up > >>>> to 3 > >>>> > >> year old compilers (or 5 years)? > >>>> > >> > >>>> > >> If we could agree on clear and conservative rules like we will go 3 > >>>> > >> years back or 5 years back and state these on the Eigen front page, > >>>> > >> Eigen user may look at our codebase and be much more willing to > >>>> accept > >>>> > >> older standards code, knowing that it will improve over time and > >>>> that > >>>> > >> the user gets some useful guarantees about the future in return. > >>>> > >> There is agrument to be made that if you use a 5+ years old > >>>> compiler, > >>>> > >> you really do not care about performance, something Eigen uses > >>>> generally > >>>> > >> care about. So you are not in a targeted user group of Eigen. > >>>> Little HPC > >>>> > >> clusters do not at all offer any way to install a more recent > >>>> compilers > >>>> > >> that 5 years (this has changed a lot from a decade ago) and even > >>>> in the > >>>> > >> embedded world, vendors tend to drop support or upgrade for > >>>> platforms > >>>> > >> with more than 5 years old toolchains too (this has also very much > >>>> > >> changed from 10 years ago). > >>>> > >> > >>>> > >> Eigen users should be able to get a clear answer on the question > >>>> when we > >>>> > >> drop C++98 (of if). That belongs on the front page IMHO. > >>>> > >> > >>>> > >> Kind regards, > >>>> > >> Martin > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >>>> > >>>> -- > >>>> -- > >>>> Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. > >>>> Gdańsk University of Technology > >>>> Faculty of Applied Physics and Mathematics > >>>> Department of Theoretical Physics and Quantum Information > >>>> -- > >>>> http://yade-dem.org/ > >>>> http://pg.edu.pl/jkozicki (click English flag on top right) > >>>> > >>>> > >>>> -- > >> Jeff Hammond > >> [email protected] > >> http://jeffhammond.github.io/ > >> > > -- -- Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. Gdańsk University of Technology Faculty of Applied Physics and Mathematics Department of Theoretical Physics and Quantum Information -- http://yade-dem.org/ http://pg.edu.pl/jkozicki (click English flag on top right)
