Hi Alan,
I will have a look at this - it is a pretty lengthy email and I have never used cdash before ;). It is probably not that complicated, but I do want to get a clear idea about what it means. Regards, Arjen > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@beluga.phys.uvic.ca] > Sent: Tuesday, November 29, 2016 3:39 AM > To: PLplot development list > Subject: [Plplot-devel] Convenient submission of dashboards has now been > enabled > > To Arjen and Hazen: > > I have now enabled the convenient submission of a dashboard (defined as the > collection of essentially all results from a run of ctest) to our cdash > dartboard server, > my.cdash.org, with results displayed at > <http://my.cdash.org/index.php?project=PLplot_git>. This step was completely > straightforward, i.e, it only required introduction of the 15-line file, > CTestConfig.cmake that is actually provided by my.cmake.org when I created the > PLplot_git project there. See the commit message for d7192bb for details. > > See also the commit messages for 9adf35e and e187a6c which implement opt-in > dashboard submission for each ctest that is run by our comprehensive test > script > and which eliminates an annoying truncation problem that was occurring for my > previously submitted dashboards. > > Will you both please try submitting dashboards to prove this process works > well for > you on the various platforms accessible to you? > > The whole submission process is straightforward and should work for any Unix > platform (Cygwin, MinGW-w64/MSYS2, Mac OS X, and Linux should all qualify). > Either you run scripts/comprehensive_test.sh with the "do_submit_dashboard > yes" > option or else you can configure PLplot with cmake as normal (using the - > DBUILD_TEST=ON option), build the "all" target (as you likely do normally as > well), > and the only extra step required _after_ the "all" > target is built is to execute > > ctest -D Experimental > > That command (also used in our comprehensive test script when "-- > do_submit_dashboard yes" runs all the ctest tests that are enabled by our > build > system as normal PLUS with that -D option, ctest collects the dashboard of > those > results and sends it off to my.cash.org as configured by CTestConfig.cmake. > And > you can then review those results (and my previous ones) at > <http://my.cdash.org/index.php?project=PLplot_git>. > > At first these results seem extremely terse (organized as one dashboard per > line), > but you soon realize that virtually every field for a given dashboard is a > clickable link > starting a hierarchy of further clickable links to make complete information > on a > given ctest run accessible in a very nicely organized way. > > The only slight concern I have with regard to these dashboard results is the > false > positive count of 14 warnings for the cmake run. > Apparently, my use of "WARNING:" in many of the messages emitted by our build > system is triggering this. I think that is likely a ctest bug, and instead > the dashboard > should report a warning only when there is an official cmake warning message. > > Anyhow, the number of configuration warnings these dashboard summaries show > should be taken with a large grain of salt. > > In sum, simply remember to configure, build, and then execute > > ctest -D Experimental > > whenever there is an interesting git commit you would like to test or > whenever you > would like to test your own work before a commit. Then visit > <http://my.cdash.org/index.php?project=PLplot_git> to look at those results > in detail > and to compare them with ctest results submitted by others. > > Now I have implemented this nice new facility for submitting dashboards, I > have put > together a table that compares this facility with the travis-ci capability > recently > implemented by Hazen. > > (N.B. "WLA" means whatever locally available.) > > Characteristic dashboard travis-ci > > Nicely formatted? yes no > > Git server allowed? any limited to GitHub > > Where tests are run? locally travis-ci server > > Compilers? WLA Many gcc suite versions, clang, etc. > > Platforms? WLA 2.5 year-old Linux, Mac OS X > > The nicely formatted results (e.g., those at > <http://my.cdash.org/index.php?project=PLplot_git> of the dashboard approach > are > a clear advantage for that approach, but I presume the travis-ci developers > will get > on top of this issue with their raw results at some point. > > The fact is lots of developers use GitHub so the extra freedom in this regard > for the > dashboard approach should be considered just a modest advantage for the > dashboard approach. > > The biggest disadvantage of the dashboard approach from the user's > perspective is > ctest chews up lots of local cpu cycles while the advantage for the travis-ci > approach from that same perspective is it chews up lots of cpu cycles on the > travis- > ci server. > > The variety of compilers and compiler versions supported by travis-ci is an > advantage for that approach from the single-users perspective, but this > advantage > decreases considerably if you get many users of PLplot reporting dashboards > for > the wide variety of compilers and compiler versions available to them. > > The local platforms where this dashboard approach should work include > essentially > any platform that supports bash and other Unix commands which are needed by > our tests. Thus, our ctest results (and presumably dashboard submission as > well) > are known to work on Cygwin, MinGW-w64/MSYS2, Mac OS X, and Linux and > should work on most other free and commercial Unices. Nobody has yet got ctest > to work on "bare" > Windows (e.g., MSVC) yet, although theoretically it should work if you used > nmake > and MSVC while putting e.g., the unix tools such as bash on the PATH from say, > your MinGW-w64/MSYS2 platform. Anyhow, even if this issue with ctest on > "bare" > Windows persists, I think there is a clear advantage with platform support > here for > the dashboard approach. > > Note, I have been in contact with the travis-ci support staff about that 2.5 > year-old > Linux limitation (which I consider to be a modest disadvantage now, but it is > on the > cusp of becoming a major disadvantage for our free software needs since we > typically do not support extremely old versions of Linux libraries). From > the reply I > got it appears the travis-ci developers have recently cleared up a bottleneck > in their > software which was making it difficult to update their Linux platforms > available on > their server so it is expected that limitation will be gone reasonably soon > although > there is no ETA yet. > > In sum, I think it is fair to say the two methods complement each other with > advantages and disavantages for each as summarized above. > > Alan > __________________________ > Alan W. Irwin > > Astronomical research affiliation with Department of Physics and Astronomy, > University of Victoria (astrowww.phys.uvic.ca). > > Programming affiliations with the FreeEOS equation-of-state implementation for > stellar interiors (freeeos.sf.net); the Time Ephemerides project > (timeephem.sf.net); > PLplot scientific plotting software package (plplot.sf.net); the libLASi > project > (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the > Linux > Brochure Project (lbproject.sf.net). > __________________________ > > Linux-powered Science > __________________________ > > ------------------------------------------------------------------------------ > _______________________________________________ > Plplot-devel mailing list > Plplot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/plplot-devel DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.
------------------------------------------------------------------------------
_______________________________________________ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
