[Plplot-devel] PLplot Debian package -- spelling issues fixed

[Alan W. Irwin]

On 2017-07-15 11:49+0200 Ole Streicher wrote:


And, finally, there are a few spelling errors found by lintian:
directorys -> directories
Continous -> Continuous
argment -> argument
argments -> arguments


The "directories" spelling issues had already been independently
corrected in this release cycle, but the "continuous" and "argument"
spelling issues were splattered across 22 files in our source tree!
So thanks for reporting those, and I have now fixed them in an
automated way (commit 5fc9108).

I have now reviewed all your posts, and I believe this e-mail
concludes my specific response to topics you have raised so far
(although the C exception handling work that spun off from our
discussion obviously continues due to Phil's efforts to finish his
proof of concept with some input and lots of well-deserved
encouragement from me).

I have very much appreciated your input and help (especially with the
nn and csa licensing issues), and I look forward to more input from
you once you have had a chance to package the git master branch of
PLplot that is soon going to lead to the release of 5.13.0.

Best wishes,

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. Irwin]

Hi Phil:

On 2017-07-23 22:00+0100 Phil Rosenberg wrote:


On 23 July 2017 at 12:27, Alan W. Irwin  wrote:

I suggested in my big e-mail to you early this week (Mon, 17 Jul 2017
14:59:00 -0700 (PDT)) concerning your second iteration a method of doing
this using a list of plstream heap pointers.  Please respond to that
suggestion.


When you said a list I'm not sure I understood. Do you mean a linked
list rather than an array?


No, I was using list in a generic sense so it you store that
information (and also access it) in an array form that would be fine
with me. So basically the idea is whatever you are doing now in
plfreeall for handling plstream heap pointers correctly should also be
done in plmalloc, plcalloc, plrealloc, and plfree so we can 
replace all calls of malloc, calloc, realloc, and free in our

code with calls to the pl* variants of those.





Also please respond to my suggestion (with appropriate code changes
while I handle the CMake part) in that e-mail concerning

#define PL_IMPLEMENT_EXCEPTION_HANDLING and
#ifdef PL_IMPLEMENT_EXCEPTION_HANDLING



Please go back to look at that original e-mail for this sub-topic, but
to summarize what I meant is please implement this preprocessor
instruction idea for iteration 4, and I will
subsequently implement a CMake option to control whether
PL_IMPLEMENT_EXCEPTION_HANDLING is #defined or not.



On 2017-07-22 22:47+0100 Phil Rosenberg wrote:


[in that same e-mail] You asked why every API call required a PLTRY block.
You are correct
that actually they don't. Every API call that may result in a call to
plexit needs a PLTRY block.



I initially didn't understand this explanation, but after _much_ too
long a time considering various scenarios to explain my point of view
the light finally dawned. Exception handling does not allow you to
transfer control to arbitrary places in the code (a stupid assumption
I was making). Instead, it merely allows you to unwind the call graph
from the plexit call (in this case that is the only current use of
PLTHROW in our source code) to return control to a routine potentially
far up the call stack from there such as _any_ of the public API
routines (as you said) that have a possible call to plexit somewhere
lower down on their call graph.


Yes correct, we just unwind the stack to a certain point in the call
sequence (or whatever terminology you choose as you mentioned in your
ps and pps).  This is good, because the user application is still sat
waiting for our API function to return. We don't want to jump to some
specific execution point, we need that function to return and for the
user to find out somehow that there has been an error.


Thanks for that confirmation.  It is a big relief to me to finally move
beyond my previous misconception.





So sorry for my previous noise on this sub-topic, but
now we are finally on the same page in this regard, I have some
further questions and comments about your implementation of
exception handling and how you are using that exception handling
to deal with bad errors.

* If you miss putting a PLTRY and PLCATCH block in one of those public
  API cases where a potential call to plexit is somewhere lower on the
  call graph, then what does the current code do?  Does it simply
  crash (i.e., is that the potential cause of the serious issues I
  have reported to you in my previous post) or do you handle that
  internal error case by, e.g., emitting an "internal error: unhandled
  exception" message followed by an exit?


I think that currently something very odd. The value of
pls->currentjumpbuffer used in PLTHROW would be either out of date or
NULL, so probably some sort of crash. This is why it is important that
every API entry function includes a PLTRY, PLCATCH, PLENDTRY sequence.
Of course we could set things up to check for this scenario, but then
what would we do? Call exit()? I had been trying to think if there was
a C method to check at compile time, but I cannot think of one (there
would be ways in C++ I think). But the API changes so infrequently
that I think just ensuring all API functions include the PLTRY,
PLCATCH, PLENDTRY code is probably okay - open to suggestions though.


Assuming we develop this proof-of-concept into the real deal and merge
this topic to master (and so far I haven't seen any issues in this
proof-of-concept that would preclude reaching that eventual result),
there will be lots of functions where we need to add PLTRY and
PLCATCH blocks.

So given that situation where unhandled exceptions may be a fact of
life for us for some time (and possibly any time in the future when
new topics with e.g., additional externally accessible API are merged
to master) I think we should handle this situation with a specific
error message and calling exit rather than crashing with, e.g., a
segfault.  That conclusion is also based on my impression that is
exactly how C++ deals with the unhandled exception issue. Furthermore,
I think this is fundamentally a more satis

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Phil Rosenberg]

On 23 July 2017 at 12:27, Alan W. Irwin  wrote:
> Hi Phil:
>
> Here is further discussion of what you have said as opposed to the
> error report I sent you in my previous post concerning your 3rd
> iteration.
>
> Do keep in mind the possibility for the future that the memory
> allocation part of your commits are worthwhile in their own right and
> could be finished and landed on master much sooner than the rest of
> your commits that are concerned with implementation of C exception
> handling and use of that for handling our errors.

I'm not sure how useful they really are without exception handling.
Without such handling there is no location in the code where the array
of pointers to the allocated memory is reviewed and in addition the
best response to a failed allocation may not be a call to plexit (but
I guess mostly it would be).

>
> I suggested in my big e-mail to you early this week (Mon, 17 Jul 2017
> 14:59:00 -0700 (PDT)) concerning your second iteration a method of doing
> this using a list of plstream heap pointers.  Please respond to that
> suggestion.

When you said a list I'm not sure I understood. Do you mean a linked
list rather than an array?

>
> Also please respond to my suggestion (with appropriate code changes
> while I handle the CMake part) in that e-mail concerning
>
> #define PL_IMPLEMENT_EXCEPTION_HANDLING and
> #ifdef PL_IMPLEMENT_EXCEPTION_HANDLING
>
> On 2017-07-22 22:47+0100 Phil Rosenberg wrote:
>
>> [in that same e-mail] You asked why every API call required a PLTRY block.
>> You are correct
>> that actually they don't. Every API call that may result in a call to
>> plexit needs a PLTRY block.
>
>
> I initially didn't understand this explanation, but after _much_ too
> long a time considering various scenarios to explain my point of view
> the light finally dawned. Exception handling does not allow you to
> transfer control to arbitrary places in the code (a stupid assumption
> I was making). Instead, it merely allows you to unwind the call graph
> from the plexit call (in this case that is the only current use of
> PLTHROW in our source code) to return control to a routine potentially
> far up the call stack from there such as _any_ of the public API
> routines (as you said) that have a possible call to plexit somewhere
> lower down on their call graph.

Yes correct, we just unwind the stack to a certain point in the call
sequence (or whatever terminology you choose as you mentioned in your
ps and pps). This is good, because the user application is still sat
waiting for our API function to return. We don't want to jump to some
specific execution point, we need that function to return and for the
user to find out somehow that there has been an error.

>
> So sorry for my previous noise on this sub-topic, but
> now we are finally on the same page in this regard, I have some
> further questions and comments about your implementation of
> exception handling and how you are using that exception handling
> to deal with bad errors.
>
> * If you miss putting a PLTRY and PLCATCH block in one of those public
>   API cases where a potential call to plexit is somewhere lower on the
>   call graph, then what does the current code do?  Does it simply
>   crash (i.e., is that the potential cause of the serious issues I
>   have reported to you in my previous post) or do you handle that
>   internal error case by, e.g., emitting an "internal error: unhandled
>   exception" message followed by an exit?

I think that currently something very odd. The value of
pls->currentjumpbuffer used in PLTHROW would be either out of date or
NULL, so probably some sort of crash. This is why it is important that
every API entry function includes a PLTRY, PLCATCH, PLENDTRY sequence.
Of course we could set things up to check for this scenario, but then
what would we do? Call exit()? I had been trying to think if there was
a C method to check at compile time, but I cannot think of one (there
would be ways in C++ I think). But the API changes so infrequently
that I think just ensuring all API functions include the PLTRY,
PLCATCH, PLENDTRY code is probably okay - open to suggestions though.

>
> * What happens when an external application calls a "first" public API
>   which has another "second" public API lower down in its call graph
>   and even further down on the call graph of that second API occurs a
>   potential call to plexit? Then if plexit is called that will PLTHROW
>   control to the catch block of that second API rather than the first.
>   So for this particular call graph case for the first API this second
>   API needs to PLTHROW control to that first PLCATCH block rather than
>   simply returning.  And, of course, since that second API is
>   sometimes called directly from external applications and libraries
>   without anything higher in the call graph with a PLCATCH block, it
>   cannot PLTHROW all the time without sometimes running
>   into the internal error discussed above.  So how are you 

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. Irwin]

On 2017-07-23 06:00-0700 Alan W. Irwin wrote:

P.P.S.  As I am just discovering, one of the huge problems in this
field is terminology. For example, the wikipedia article
 refers immediately to 6
synomyms (ouch!) for call stack with one of those being just plain
"stack" which is the terminology used by the Linux man pages for
setjmp and longjmp.

"Call chain" is not one of those wikipedia-approved synonyms, but I
believe I must have picked up that terminology from another article I
read (where it meant the same thing).  Therefore, since the principal
terminology in wikipedia is "call stack", that is much preferable to
the "call chain" terminology I used below.  But nevertheless, I
thought I did pretty well below considering I wrote that at
6 o'clock in the morning after a long night of analysis without sleep.  :-)

Alan


P.S. I just reviewed the wikipedia article on call graphs, and it
appears I was misusing that term and should have used call chain
instead where a call graph for a given library consists of all the
potential call chains in the library with each one starting at one of
the public API's.

So my understanding now is the PLTHROW macro in plexit unwinds the
particular call chain that called plexit until it reaches the first
routine higher in that call chain where the
the call was made in a PLTRY block and transfer control to the code in the
corresponding PLCATCH block.

@Phil:

Do you agree with this explanation of PLTHROW, PLTRY, and PLCATCH or
does it need further correction?

If that explanation is substantially correct, that one scenario I was
discussing could be described as two possible call chains in the call
graph with one starting at the first public API traversing through the
second public API and eventually traversing to plexit, and the second
starting at the second public API and eventually traversing to plexit
as well.  So the question was does the code need enhancement to deal
with those two different possible call chains where in one case the
PLCATCH block of the second API needs to PLTHROW to unwind the call
chain to the first API PLTRY and transfer control to the corresponding
PLCATCH block, and in the other case the second API catch block needs
simply to return to the calling routine in the external application.

And similarly this change in terminology needs to be applied to my
just-previous discussion of other topics related to PLTHROW, PLTRY,
and PLCATCH.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel



__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. Irwin]

P.S. I just reviewed the wikipedia article on call graphs, and it
appears I was misusing that term and should have used call chain
instead where a call graph for a given library consists of all the
potential call chains in the library with each one starting at one of
the public API's.

So my understanding now is the PLTHROW macro in plexit unwinds the
particular call chain that called plexit until it reaches the first
routine higher in that call chain where the
the call was made in a PLTRY block and transfer control to the code in the
corresponding PLCATCH block.

@Phil:

Do you agree with this explanation of PLTHROW, PLTRY, and PLCATCH or
does it need further correction?

If that explanation is substantially correct, that one scenario I was
discussing could be described as two possible call chains in the call
graph with one starting at the first public API traversing through the
second public API and eventually traversing to plexit, and the second
starting at the second public API and eventually traversing to plexit
as well.  So the question was does the code need enhancement to deal
with those two different possible call chains where in one case the
PLCATCH block of the second API needs to PLTHROW to unwind the call
chain to the first API PLTRY and transfer control to the corresponding
PLCATCH block, and in the other case the second API catch block needs
simply to return to the calling routine in the external application.

And similarly this change in terminology needs to be applied to my
just-previous discussion of other topics related to PLTHROW, PLTRY,
and PLCATCH.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. Irwin]

Hi Phil:

Here is further discussion of what you have said as opposed to the
error report I sent you in my previous post concerning your 3rd
iteration.

Do keep in mind the possibility for the future that the memory
allocation part of your commits are worthwhile in their own right and
could be finished and landed on master much sooner than the rest of
your commits that are concerned with implementation of C exception
handling and use of that for handling our errors.

I suggested in my big e-mail to you early this week (Mon, 17 Jul 2017
14:59:00 -0700 (PDT)) concerning your second iteration a method of doing
this using a list of plstream heap pointers.  Please respond to that
suggestion.

Also please respond to my suggestion (with appropriate code changes
while I handle the CMake part) in that e-mail concerning

#define PL_IMPLEMENT_EXCEPTION_HANDLING and
#ifdef PL_IMPLEMENT_EXCEPTION_HANDLING

On 2017-07-22 22:47+0100 Phil Rosenberg wrote:


[in that same e-mail] You asked why every API call required a PLTRY block. You 
are correct
that actually they don't. Every API call that may result in a call to
plexit needs a PLTRY block.


I initially didn't understand this explanation, but after _much_ too
long a time considering various scenarios to explain my point of view
the light finally dawned. Exception handling does not allow you to
transfer control to arbitrary places in the code (a stupid assumption
I was making). Instead, it merely allows you to unwind the call graph
from the plexit call (in this case that is the only current use of
PLTHROW in our source code) to return control to a routine potentially
far up the call stack from there such as _any_ of the public API
routines (as you said) that have a possible call to plexit somewhere
lower down on their call graph.

So sorry for my previous noise on this sub-topic, but
now we are finally on the same page in this regard, I have some
further questions and comments about your implementation of
exception handling and how you are using that exception handling
to deal with bad errors.

* If you miss putting a PLTRY and PLCATCH block in one of those public
  API cases where a potential call to plexit is somewhere lower on the
  call graph, then what does the current code do?  Does it simply
  crash (i.e., is that the potential cause of the serious issues I
  have reported to you in my previous post) or do you handle that
  internal error case by, e.g., emitting an "internal error: unhandled
  exception" message followed by an exit?

* What happens when an external application calls a "first" public API
  which has another "second" public API lower down in its call graph
  and even further down on the call graph of that second API occurs a
  potential call to plexit? Then if plexit is called that will PLTHROW
  control to the catch block of that second API rather than the first.
  So for this particular call graph case for the first API this second
  API needs to PLTHROW control to that first PLCATCH block rather than
  simply returning.  And, of course, since that second API is
  sometimes called directly from external applications and libraries
  without anything higher in the call graph with a PLCATCH block, it
  cannot PLTHROW all the time without sometimes running
  into the internal error discussed above.  So how are you going to figure out
  when to PLTHROW in a public API PLCATCH block and when not to PLTHROW?
  Or is that case already automatically taken care of by your present code
  (e.g., by each PLTRY macro incrementing a library status flag
  so that the PLCATCH macro can check that flag to see if there
  is a PLCATCH block higher in the call graph that it should PLTHROW
  to)?

* Shouldn't your current plexit call plend (like the master branch
  version does) to shut down the library?  See further discussion
  below concerning this suggestion.

* I also suggest you should implement a PLplot library state variable
  (let's call it plplot_library_status) that is initialized to 0 (good
  status) when the library is initialized but set to 1 (bad status) by
  plexit. Then follow up by implementing a c_plstatus() public API
  that returns the value of library_status and which emits a message
  such as ("library had bad error so it had to be shut down.  To use
  it again you need to initialize the library by calling one of the
  plinit family of routines".) Then an external application could call
  c_plstatus to check on the status of the library after any call to
  our public API and act accordingly.

  Or it could be sloppy about that (i.e., our standard C examples
  might only call plstatus in one example to make sure it works). In
  which case after the library is shutdown (see above) by an internal
  call to plexit there would be a blizzard of further plabort (due to
  plsc->level being 0) and plexit messages after the first one because
  of that library shutdown.  But there would be no call to the dreaded
  exit routine so there is at least the ch

[Plplot-devel] Third proof of concept for the "exception handling" topic

[Alan W. Irwin]

On 2017-07-22 22:47+0100 Phil Rosenberg wrote:


Attached is a new patch series in a tarball. I don't see a huge need
to keep two separate branches any more. In fact I just tried to
disentangle my latest few commits into two branches using git rebase
-i and endured a whole world of pain.


Hi Phil:

Thanks very much for this third version of your exception handler
private topic branch.  It does sound like you prefer to keep your
development environment simpler (at least for now) with just one
branch so I will take that approach as well.

After unpacking the tarball here in

~irwin/Phil.Rosenberg/20170722/exception_handler/

I did the following:

# Create up-to-date master branch
git checkout master
git fetch
git merge --ff-only origin/master
# create new private topic branch from that branch
git checkout -b exception_handler3
# Apply _all_ your commits to that branch
cat ~irwin/Phil.Rosenberg/20170722/exception_handler/*.patch |git am

The result here of that last command was the following:

Applying: Added functions that will record memory allocations in the PLStream
Applying: Part way through plmalloc implementation
Applying: Removed plinitializememorylist and plfreeall calls
Applying: Fix issue with growing memorylist
Applying: Added missing ;
Applying: Added Memory allocation function declarations to plplotP.h
Applying: Added some further memory management functions and check for PLStream 
memory
/home/software/plplot/HEAD/plplot.git/.git/rebase-apply/patch:104: trailing 
whitespace.
//Cause a pointer to no longer be managed by plplot, 
warning: 1 line adds whitespace errors.

Applying: Fixed some problems with memory management and performed style update.
/home/software/plplot/HEAD/plplot.git/.git/rebase-apply/patch:98: trailing 
whitespace.

warning: 1 line adds whitespace errors.
Applying: moved memory management functions from plstrm.h to plplotP.h removing 
duplicate definitions.
Applying: Removed c_usplparseopts function, which was pollution from another 
branch
Applying: Removed use of plmalloc and plfree from plcont.c. This was pollution 
from another branch.
Applying: Added Exception handling macros/functions/variables
Applying: Add some test code for exception handling
Applying: Fixed PLENDTRY/PLTHROW bug
Applying: Remove plexit calls from advancejumpbuffer and plinitialisememorylist
Applying: Added some comments around PLTRY, PLCATCH and PLENDTRY
Applying: Free memory from jumpbuffer stack in plend1
Applying: Added PLTRY blocks to all the api functions in core.c
Applying: Added use of plmalloc to the contour code.
Applying: Restyle only

So all your commits applied cleanly on this fresh and up-to-date topic branch
aside from some extremely minor trailing whitespace issues which should
be easy for me to fix (but I won't do that yet) after the fact
by running the (Unix-tools-only) scripts/remove_trailing_whitespace.sh.

I tested that branch by setting the environment variable
CFLAGS='-g' (to generate line-number information in valgrind
reports), making a fresh configuration of PLplot with
the -DVALGRIND_ALL_TESTS=ON cmake option, and running

make VERBOSE=1 test_c_psc >& test_c_psc.out

Unlike my original good benchmark with this test, the result for this
branch was not clean, i.e. test_c_psc.out (collected in the attached
tarball) ends with an actual valgrind error for example 14.  For
historical reasons examples 14, 17, and 29 are done last so because of
this error for example 14, examples 17 and 29 were not executed, but
all the rest of our standard examples from 00 to 33 (except for 32
which we historically do not test because it has not been propagated
to languages other than C so there is no interesting basis of
comparison) were run and returned success (a 0 exit code). However,
the commands

software@raven> grep -L "0 errors from 0 contexts" 
examples/valgrind.x??c.psc.log
examples/valgrind.x11c.psc.log
examples/valgrind.x14c.psc.log
examples/valgrind.x21c.psc.log
software@raven> grep -L "no leaks are possible" examples/valgrind.x??c.psc.log
examples/valgrind.x14c.psc.log

demostrate there are imperfect valgrind results for examples 11, 14,
and 21 and actually looking at those three files (also collected in
the attached tarball) show those errors are quite serious in nature,
e.g., the initial trouble for all three is an invalid write which is
normally quite bad.  Furthermore, for example 14 that initial error
compounded so badly that valgrind itself exited with an error message
(and non-zero exit code).  I hope these collected valgrind reports
with exact source code line numbers will help you debug all these issues
so we can get back to perfect valgrind reports for the next iteration.

This is already long enough with an attachment for you to respond to
as well so I will place my further response to your additional remarks in a
separate e-mail.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victo