[matplotlib-devel] binary installers for python2.6; libpng segfault, MSVCR90.DLL and mingw

2009-06-10 Thread John Hunter
I have uploaded binary installers for python2.5 for win32 to the
website, but python2.6 is a different beast all together.  I have been
trying to make Charlie's support work for mingw32 for windows in
release/win32 work, but have hit a huge wall.  Apparently, mingw
doesn't play nicely with MSVCR90.DLL abd Charlie's hack to simply
remove it from the dll_libraries list doesn't work because _png.pyd
needs these functions and the mingw support is broken.  The dirty
details are at

  http://bugs.python.org/issue3308

  http://www.nabble.com/localtime()-and-MSVCRT9-td18329478.html

numpy went through what looks like a painful experience with some of
these issues detailed at

  http://www.mail-archive.com/numpy-discuss...@scipy.org/msg14552.html

  
http://cournape.wordpress.com/2008/09/02/how-to-embed-a-manifest-into-a-dll-with-mingw-tools-only/

It looks like pygame is going to extreme lengths to work around these
problems, as you can see in this code

   
http://github.com/ab3/pygame-mirror/blob/cf8aee7508194b0be734b080f866c5a24e242cec/msys_link_VC_2008_dlls.py

I've burnt a lot of time chasing false leads before I finally figured
out where the root of the problem is, and it appears to be in the
broken gmtime/localtime support in mingw and MSVCR90.DLL.  I have
narrowed the segfault to png_write_info(png_ptr, info_ptr), and since
I believe png uses time support in the info processing, my guess is
this is where the code is failing, but my attempts to build libpng w/o
time support as a quick workaround

  CFLAGS = -Os -D_ftime=ftime64 -DPNG_NO_READ_tIME -DPNG_NO_WRITE_tIME

have not helped (this was mainly a stab in the dark from poking around
the png headers and src)

There are a variety of solutions offered in the links above ranging
from rebuilding a patched mingw from src to the approach taken by
pygame.  Since the rest of mpl is working fine, I was hoping to simply
hack around the broken part of libpng itself (eg removing the time
calls) but have not succeeded yet.  Unfortunately, I won't have a lot
more time to spend on this in the near future, so I was hoping that
someone with a fresh pair of eyes and a little time could find a
solution.

If you can take a look, to get started check out release/win32 from
the TRUNK and build the sdist from the branch with

  > python setup.py sdist --formats=gztar

Drop the tar.gz into release/win32 and then following the instructions
in release/win32/README.txt

Thanks,
JDH

--
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing 
server and web deployment.
http://p.sf.net/sfu/businessobjects
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel


Re: [matplotlib-devel] binary installers for python2.6; libpng segfault, MSVCR90.DLL and mingw

2009-06-10 Thread John Hunter
On Wed, Jun 10, 2009 at 4:41 PM, John Hunter wrote:

> Drop the tar.gz into release/win32 and then following the instructions
> in release/win32/README.txt

I should add one more detail here.  distutils for python2.6 adds a
dependency to MSVCR90.DLL, and Charlie has a distutils hack in
release/win32/data/setupwin.py to try
and banish the msvcr90.dll which is not on most systems; he sets the
distutils dll_libraries to the empty list::


from distutils import cygwinccompiler

try:
# Python 2.6
# Replace the msvcr func to return an empty list
cygwinccompiler.get_msvcr
cygwinccompiler.get_msvcr = lambda: []

w/o this hack you will get a dll import error rather than a segfault.
You can toggle the hack on and off by changing in the Makefile::


  # on
  ${PYTHON} setupwin.py build_ext -c mingw32 -I ${PY_INCLUDE} -L
${PY_LINKER} bdist_wininst

  # off
  ${PYTHON} setup.py build_ext -c mingw32 -I ${PY_INCLUDE} -L
${PY_LINKER} bdist_wininst

The problem is that the dll cannot be found on the system and
apparently the right solution is to include it in a framework as
described by David in the link I posted

  
http://cournape.wordpress.com/2008/09/02/how-to-embed-a-manifest-into-a-dll-with-mingw-tools-only/

but I think this is something of a red herring because w/ Charlie's
hack _png links against MSVCRT.DLL rather than MSVCR90.DLL but still
segfaults (only on python2.6) on the png_write_info call.  I also
tried forcing 'msvcr71' in the distutils hack which is what python2.5
usesbut to no avail.  When I said I wasted a lot of time chasing down
false leads, most of that was spent on trying various versions of mpl,
numpy, distutils, and the distutils MSVCR hacks, but in the end none
of it mattered.

But for the record I am including some notes I took while
experimenting with different things.  When I write below

  Interestingly, depwalker can find the DLL where PYTHON26.DLL depends
  on, pointing to C:\WINDOWS\WinSxS.VC90.CRT_1fgobbledygook on my box,
  but cannot find it where _PNG.PYD depends on it.

this was before I discovered David's essay on side-by-side assemblies
which explains why the linker finds python's MSVCR90.DLL  but not
_pngs.  What it doesn't explain is why w/ Charlie's hack, when we link
against MSVCRT.DLL or  MSVCR71.DLL, png is still segfaulting, and I
think the answer there is what I pointed to in my first post, the
broken time support in mingw MSVCR90.DLL.  Support for this is in my
note below::

  This time it finds it, but two of the functions are colored red
  indicating a problem: _fstati64 and gmtime.


Notes on the _png segfault and the MSVCR hacks
==

I have been working on the windows installers for mpl 98.5.3.  I am
able to build for python2.5 w/o problems, but when building for
python2.6, I get an DLL local error on any extension code


cd matplotlib-0.98.5.3/build/lib.win32-2.6/matplotlib
> /c/python26/python -c 'import _png'
Traceback (most recent call last):
File "", line 1, in 
ImportError: DLL load failed: The specified procedure could not be found.

when I open _PNG.PYD this file in dependency walker, I see a
dependency on MSVCR90.DLL which is flagged with a question mark "Error
opening file.  The system cannot find the file specified".  All the
*.PYD files have this dependency, including the numpy pyd files, but
most of them have it buried under a PYTHON26.DLL dependency, and
_PNG.PYD has it as a direct dependency (top level).  Interestingly,
depwalker can find the DLL where PYTHON26.DLL depends on, pointing to
C:\WINDOWS\WinSxS.VC90.CRT_1fgobbledygook on my box, but cannot find
it where _PNG.PYD depends on it.


When I look at _PNG.PYD for python2.5 (which is working fine and I've
uploaded the installers) I see a dependency for MSVCR71.DLL located in
C:/windows/system32, and the functions it provides include fclose,
fopen, putc, etc. which libpng apparently needs for file I/O.

As mentioned above, my system *does* have MSVCR90.DLL deep in
C:\WINDOWS\WinSxS.VC90.CRT_1fgobbledygook, and when I move it into
C:\windows\system32 and then reopen _png.pyd in depdendency walker, I
still get an error.  This time it finds it, but two of the functions
are colored red indicating a problem: _fstati64 and gmtime.

Charlie has a distutils hack in release/win32/data/setupwin.py.  Wen I
build the installer using the hacked distutils I can load the DLL file
in mpl either by importing matplotlib._png or in dependency walker.
In depwalker, the msvcrt90.DLL is replaced by MSVCRT.DLL at the top
level _PNG.PYD dependency, but the MSVCR90.DLL is still buried deep
under PYTHON26.DLL.  Unfortunately, use of this module triggers a
segfault.

--
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for exter