Re: [Plplot-devel] Best git version for Windows?

[Arjen Markus]

Hi Alan,

Never tried it before - I have sofar relied on Cygwin to do the job. I made a 
note to look into this after my holiday next week.

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may 
contain confidential and privileged information. If you are not the intended 
recipient please notify the sender immediately and destroy this message. 
Unauthorized use, disclosure or copying of this message is strictly prohibited. 
The foundation 'Stichting Deltares', which has its seat at Delft, The 
Netherlands, Commercial Registration Number 41146461, is not liable in any way 
whatsoever for consequences and/or damages resulting from the improper, 
incomplete and untimely dispatch, receipt and/or content of this e-mail.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

Re: [Plplot-devel] A mysterious pyc file corruption issue

[Alan W. Irwin]

On 2017-02-21 11:01-0800 Alan W. Irwin wrote:

> Yesterday for the second time in two months an interactive
> comprehensive test failed with a "ValueError: bad marshal data
> (unknown type code)" error for bindings/python/Plframe.py.
[ These errors were so common]
> that the python developers list in 2013 became
> concerned that python would be subject to race conditions when
> generating these files and thus was the author of at least some of
> these corruptions (see discussion thread at
> 
> with the subject line "[Python-Dev] Mysterious Python pyc file
> corruption problems".

I have now been in contact with the OP, Barry Warsaw of python.org, of
that thread who was quite helpful.  For example, Barry told me that
Python is designed so it is frankly impossible for

import Plframe
from Plframe import *

to race (i.e., the first import completely finishes before the second
one starts).  And I cannot find any other cases where Plframe is
imported.  So I think the best bet for explaining this *.pyc
Python-generated file corruption is some unknown Python 2 bug that
does not have anything to do with races.  I got the sense from
what Barry said that he feels Python 3 is now much more reliable than
Python 2.  So this may be another instance of that general idea.

Anyhow, I think the next step is to test whether this corruption
occurs for Python 3.  (And if it does I get the sense that Barry would
be anxious to figure out what that Python 3 bug was.)

@Hazen:

This issue lends lots of additional motivation for making PLplot work
correctly with Python 3.  So please go ahead and push your Python 3
topic as soon as it is in reasonable state, and we can mature it
further (if necessary) from there.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] The new wingdi device has been pushed

[Alan W. Irwin]

On 2017-02-20 00:38-0500 Jim Dishaw wrote:

> I need your help to sort out git.
[]

> This a pure GDI implementation and it works fairly well.  The font
sizes are a bit off, printing causes a core dump, and the busy cursor
does not go away after a resize. I think a may have lost a commit when
I lost the VM.  This driver outputs a lot of debugging messages if
debugging is turned on.  The minimum version of Windows is XP—it will
not work on anything less.  I am not sure if I will be able to
maintain Windows XP compatibility as this driver evolves.  I have not
implemented unicode support.

The unicode support issue is an all-important one in my opinion, but
this does sound like a good start.

> [Because of my current git troubles] I have attached the patches
[for my wingdi development] to this email,
would it be possible for you to apply them?

Yes, pushing your commits was no problem at all (see the recent flurry
of SF commits leading up to and including e639c33).  That last commit
is mine and resulted from running

scripts/remove_trailing_whitespace.sh

and

scripts/style_source.sh

I do not mind doing such cleanups myself, but if you like you
should be able to run those scripts yourself if you have access to a
platform with Unix command-line capabilities (such as Mac OS X,
Cygwin, or MinGW-w64).

Note, I also heavily tested commit e639c33 to assure none of your
changes to common driver-related files interfered with the Linux
device drivers.

So it appears all was well with your series of commits, and I
encourage you to follow up on this good start by continuing with the
plans you discussed to implement access to unicode-aware Windows
system fonts using Uniscribe for the wingdi device and implementing an
additional device that uses Direct2D/DirectWrite to handle text for
Windows platforms that support those more advanced text capabilities.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] Best git version for Windows?

[Alan W. Irwin]

To Jim, Phil, and Arjen:

On 2017-02-20 00:38-0500 Jim Dishaw wrote:

> I need your help to sort out git.  Long story short, I had a disk
corruption on the VM that I was doing my development work and had to
recover my work.  I manage to get my wingdi driver recovered from the
smoking heap.

@Jim:

My sympathies concerning your hardware troubles.  I am especially
sensitive to that issue because I just went through a hardware scare
myself (spent a lot of yesterday running hardware tests when I ran
into the *.pyc corruption issue).  But amazingly this 9-year-old
hardware (with an ASUS motherboard which might be the reason for this
longevity) still passes all hardware tests, and I have concluded (with
a fair amount of confidence) that the *.pyc corruption issue must be
due to some Python bug.  So I plan to keep using this hardware for
a while longer.

> Unfortunately, I appear to be having problems with the git
repository on SourceForge and I am not sure of the cause—I cannot even
clone from SourceForge.

@Jim, Phil, and Arjen:

I used the git SF server just this morning with no issues.  Also, for
the reasons discussed in README.developers you should avoid all gui
versions or "enhanced" versions of git (i.e., try to stick as much as
possible to the real thing).  Bearing those constraints in mind, that
file recommends  for Windows
users, but I just discovered from looking at that site that it has
been obsoleted and msysgit developers now recommend using the "Git for
Windows"  version of git instead.
(I confirmed from that website it considers itself light-weight
[check!] and it does have a command-line version [check!]).  So please
give the command-line version of that project a try, and let us know
whether it works well for you (which would allow us to recommend that
Windows version of git in our README.developers file).

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] We appear to be ready for CMake-3.7.2

[Alan W. Irwin]

I just built CMake-3.7.2 with the bootstrap method and used it for a
comprehensive test of PLplot with no constraints on the tests. The
first time I ran the test I ran into the corrupted Plframe.pyc issue I
recently discussed here.  But once I moved that offending corrupted
Plframe.pyc file out of the way the comprehensive test passed with
flying colours.  So it appears our current master branch code is in
good shape (unless there is something we can do concerning that
intermittent Plframe.py corruption issue) and ready for CMake-3.7.2.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] A mysterious pyc file corruption issue

[Alan W. Irwin]

Yesterday for the second time in two months an interactive
comprehensive test failed with a "ValueError: bad marshal data
(unknown type code)" error for bindings/python/Plframe.py.  This is a
rather common error with python and typically means the associated
*.pyc that is generated by python has been corrupted.  I moved that
corresponding *.pyc file out of the way, and the comprehensive test
(with *.pyc regenerated by python) sailed through afterward without
issues.  For the record, this issue occurred on my Debian Jessie
platform with python version string of

irwin@raven> python --version
Python 2.7.9

There are lots of potential reasons for such *.pyc corruption issues
such as a change in python version and hardware issues, but these
errors are so common that the python developers list in 2013 became
concerned that python would be subject to race conditions when
generating these files and thus was the author of at least some of
these corruptions (see discussion thread at

with the subject line "[Python-Dev] Mysterious Python pyc file
corruption problems".

I did an octal dump of the corrupted file versus the uncorrupted
regenerated one, and as far as I can tell the only difference is
a missing byte in the corrupted file.  (If anyone is interested
I can send those files to you for inspection.)

Yesterday I did do some obvious tests (with memtest, fsck, and git
fsck) of my PC hardware (which is 9 [!] years old, but still going
strong), and all was well.  Furthermore, the above octal dumps showed
no i/o issue with the corrupted file, and the problem always occurs
(so far) with just this particular file.  And these rare errors only
started when I started enabling testing of examples/python/pytkdemo
(our only file that imports PLframe which would generate the *.pyc as
a byproduct of that import) with the test_pytkdemo target. So I am
pretty sure this evidence largely rules out any hardware issue.  And I
have not been fiddling with my python versions, and in any case those
changes should just change a version stamp (at least two bytes) in the
file and not simply remove one byte.

So by a process of elimination, I think this is likely one more
candidate for the mysterious python pyc corruption issue.  However, if
the source of this corruption is a race condition in the python
generation of these files, I believe that would only be an issue if
there are simultaneous attempts to generate this file.  The tests I
run do use parallel builds but the test_pytkdemo target is implemented
with a CMake custom target where there should be no build race
conditions (attempting to build that target twice) unless there is a
bug in either CMake or make.  But if that were the case, we would be
seeing similar errors for our other python test targets, and we don't.
However, if you look at examples/python/pytkdemo, it is interesting
that it imports PLframe in two ways, i.e.

import Plframe
from Plframe import *

This is a fairly common (but sloppy) python idiom for importing both a
namespaced and unnamespaced version of PLframe (because some of our
code uses the namespaced version and some of our code does not).
However, the only way you get a race out of that is if python looks
ahead and starts doing the second import (which would attempt to also
generate a PLframe.pyc file) before the first import was finished, and
I have no idea whether that is a possibility or not.

Anyhow, in the near future I plan to track down all our references to
the version of PLframe that is not namespaced and convert it to the
namespaced version so that second import can be eliminated.  And it
will be interesting to see if that makes this corruption issue
disappear.

Meanwhile, if anyone else can replicate this issue that would even be
stronger evidence it is not due to my hardware.  So if you want to
help out with that, you should run the test_pytkdemo target but only
after touching examples/python/Plframe.py (which would force python to
regenerate the *.pyc file when the test_pytkdemo target is run).  And
you should do this test from time to time under a variety of load
conditions so generating the above error even once may be difficult to
accomplish.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the