[Plplot-devel] Python test results show it is basically working on 32-bit Windows
Here are the Python results I am getting on the MinGW/MSYS/wine 32-bit platform (on my 64-bit hardware): python Missing examples: Differing postscript output : 03 04 05 06 08 09 11 12 14a 15 16 17 18 19 20 21 22 23 25 26 29 33 Missing stdout : Differing stdout: 14 23 These results were obtained from comprehensive_test.sh configured to test just the case of shared libraries for the test_noninteractive target in both the build tree and CMakeified installed examples tree. I believe the Python results above are the first extensive testing reported for Python on a Windows platform. Despite the large number of differences reported above, the actual diff results are not that bad; all of them correspond to a maximum difference of only one unit in the last place as verified by ndiff except for just the following three examples: 17 (large diffs, but no noticable differences can be seen using the gv viewer). 19 (page 3 is completely messed up for the python version which is strange because page 4 [identical to page 3 except for the Baltimore label] looks good). 25 (large diffs, but no noticable differences can be seen using the gv viewer). A closer look at the first few lines of diffs for example 25 showed a polyline drawing issue. Some redundant points were being put in the closed polyline for the Python case. So I attribute at least this part of the large diff to roundoff errors propagating to the choice of whether polylines need an extra point to be closed. That may be the reason for the example 17 differences as well. In general, though from all the many one-unit-in-last-place differences reported above, 32-bit python on Windows appears to have a lot of numerical noise compared to its C counterpart. I have no idea why, or whether that numerical noise problem also exists for all 32-bit Python platforms. The example 14 stdout differences are due to the Python messages appearing in scrambled order, e.g., Enter graphics output file name: Demo of multiple output streams via the psc driver. Running with the second stream as slave to the first. Demo of multiple output streams via the psc driver. Running with the second stream as slave to the first. comes from the first stream and Enter graphics output file name comes from the second stream of that example. I don't have a clue how the messages from the second stream get output first for Python, and vice versa for C, but there it is. The example 23 stdout differences will appear on any 32-bit platform where the python version outputs the FCI as 0x8000L while the C version drops the L suffix. In sum, I believe there are mundane explanations for most of the above diff results so Python is basically working properly on 32-bit Windows. I therefore strongly encourage all our Windows users (and especially our Windows testers) to download and install Python for Windows following the instructions at http://www.miscdebris.net/plplot_wiki/index.php?title=Install_Python so that Python will automatically become part of the Windows build, install, and test of PLplot from then on. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Lotusphere 2011 Register now for Lotusphere 2011 and learn how to connect the dots, take your collaborative environment to the next level, and enter the era of Social Business. http://p.sf.net/sfu/lotusphere-d2d ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Fix for 79-character truncation on plgfnam
Hi Andrew (with one question for Dave): Thanks for contributing to this discussion. On 2010-12-18 13:59- Andrew Ross wrote: I think passing a buffer size is the safest way to go. If you wanted a constant buffer length, then it should be PATH_MAX to ensure any path will fit, but this still has risks associated. Agreed. Note other functions (e.g. getcwd - Linux extension) interpret a null buffer differently, and will allocate a buffer using malloc in this case to allocate a buffer. If the buffer size parameter 0 then the allocated buffer is of that size. If if is zero then a buffer big enough to return the path is allocated. Note getcwd also returns null if the buffer is not big enough and sets errno to ERANGE. I'm not necessarily suggesting this is better, but it an alternative approach. Plus side is that it is easier for the user - allocations are done automatically. Another model for plgnver, plgndev, and plgnfnam is one similar to strlcpy that is discussed by Todd C. Miller and Theo de Raadt at http://www.gratisoft.us/todd/papers/strlcpy.html. That is a most interesting paper to read. I am not necessarily advocating something similar to this model API. I leave that to those more knowledgable in C than I am. However, I am mentioning this paper for further discussion. aside to Dave Can you remember the name of the function you were using as the model for your API suggestion for plgnver, plgndev, and plgnfnam? If so, that will give us a chance to dig into the official documention of that function like we can currently do for getcwd and strlcpy. /aside to Dave Down side for us [of automatic allocation] is that allocated memory in C may be difficult to use in other languages. Unless I misunderstood you, I think that allocation, possible reallocation, and free of a C buffer is a given (which I like) for all approaches discussed so far. For example, this is true for the getcwd Linux extension approach, the strlcpy approach, or Dave's approach. After the C buffer is filled with the character-string information from the PLplot core library, then it is up to each language interface to pass that information on to the language environment in appropriate form. That seems to be the model used in our swig-based language bindings and also for Tcl (if I am following that generated code correctly). My guess is our remaining language bindings follow that model as well. Regardless, they all currently implement plgver, plgdev, and plgfnam without issues so that means it should be possible to adapt those approaches to the plgnver, plgndev, and plgnfnam cases once we finalize the API for those. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Lotusphere 2011 Register now for Lotusphere 2011 and learn how to connect the dots, take your collaborative environment to the next level, and enter the era of Social Business. http://p.sf.net/sfu/lotusphere-d2d ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Tcl issues solved on Windows!
I should have also said that PLplot users on the Windows platform can find directions for installing a Windows version of Tcl at http://www.miscdebris.net/plplot_wiki/index.php?title=Install_Tcl which is linked from http://www.miscdebris.net/plplot_wiki/index.php?title=Specifics_for_various_platforms Of course, that Install_Tcl page was writen from my own wine Windows perspective so Microsoft Windows users are requested to modify that page to add the Microsoft Windows perspective as well. And similarly for the MinGW/MSYS, SWIG, Python, and Lua pages I have recently created that are linked from the Specifics_for_various_platforms page. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Lotusphere 2011 Register now for Lotusphere 2011 and learn how to connect the dots, take your collaborative environment to the next level, and enter the era of Social Business. http://p.sf.net/sfu/lotusphere-d2d ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Fix for 79-character truncation on plgfnam
Hi, Alan, On Dec 18, 2010, at 10:57 AM, Alan W. Irwin wrote: Another model for plgnver, plgndev, and plgnfnam is one similar to strlcpy that is discussed by Todd C. Miller and Theo de Raadt at http://www.gratisoft.us/todd/papers/strlcpy.html. That is a most interesting paper to read. I am not necessarily advocating something similar to this model API. I leave that to those more knowledgable in C than I am. However, I am mentioning this paper for further discussion. It seems like strlcpy is a more convenient alternative to strncpy, but I don't see how this would solve the allocation problem. Wouldn't the user still need to ensure that a large enough buffer has been allocated? If so, the problem of determining how large is large enough still remains. Am I missing something? aside to Dave Can you remember the name of the function you were using as the model for your API suggestion for plgnver, plgndev, and plgnfnam? If so, that will give us a chance to dig into the official documention of that function like we can currently do for getcwd and strlcpy. /aside to Dave OK, digging way back into my mental archives (and googling), I think I first came across this approach in a previous life when I did Windows programming. An example of this approach can be seen in the Windows GetShortPathName function (and probably others as well)... http://msdn.microsoft.com/en-us/library/aa364989(v=vs.85).aspx I mis-remembered the exact details of the return code. The WIN32 functions seem to return the total size needed iff the buffer pointer is NULL and the size is given as 0. This shouldn't really change the fundamentals of what I wrote earlier, however. On 2010-12-18 13:59- Andrew Ross wrote: Down side for us [of automatic allocation] is that allocated memory in C may be difficult to use in other languages. I think automatic allocation by the library function, while not a big drawback, has a few potential points against it. One it that memory allocation is one of the (potentially) more expensive operations, so minimizing it is desirable. I don't think the use cases in question will result in this being a significant factor. The other issue is that once the library has allocated and populated the buffer, language bindings sometimes (typically?) require that the contents be copied into a suitably sized buffer that the scripting language manages. In this case the result is a double allocation (one by the library function, one by the scripting language), a extra strcpy from library buffer to scripting language buffer, and a free of the buffer returned by the library function. Getting the size first, allocating a buffer using he scripting language memory management routines, then passing the pointer to the library routine is, IMHO, more cleanly amenable to scripting language bindings (though I write mine manually so I don't know whether this would apply to SWIG generated bindings). Again, I don't think these factors will be significant for the functions under discussion, so I'll be happy with whatever consensus emerges. Dave -- Lotusphere 2011 Register now for Lotusphere 2011 and learn how to connect the dots, take your collaborative environment to the next level, and enter the era of Social Business. http://p.sf.net/sfu/lotusphere-d2d ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
[Plplot-devel] Lua tests on Windows look good
Here are the Lua diff results I am getting on the MinGW/MSYS/wine 32-bit platform (on my 64-bit hardware): lua Missing examples: 33 Differing postscript output : 04 18 19 26 Missing stdout : Differing stdout: These results were obtained from comprehensive_test.sh configured to test just the case of shared libraries for the test_noninteractive target in the build tree. I got Lua to work on Windows by adopting (revision 11378) the strategy to transform absolute fullpath file names in the leading-slash, single-letter form to the drive-letter form. This is the same strategy that was recently adopted for the Tcl case. I am extremely pleased by the above Windows results which appear to be identical to the Linux results for Lua. Attention to Lua propagation issues for plstring, plstring3, and pllegend and even older Lua propagation issues associated with example 19 should remove the above differences on both Linux and Windows. Note, the good results above were obtained with an install of Lua for Windows which is described in http://www.miscdebris.net/plplot_wiki/index.php?title=Install_Lua. I highly recommend that procedure for all our Windows testers to increase the power of their PLplot tests on Windows. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Lotusphere 2011 Register now for Lotusphere 2011 and learn how to connect the dots, take your collaborative environment to the next level, and enter the era of Social Business. http://p.sf.net/sfu/lotusphere-d2d ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
[Plplot-devel] Windows testing environment
On 2010-12-18 20:15+0100 Werner Smekal wrote: I made an attempt to run comprehensive_test.sh on my Windows system, via win-bash, but it requires basic external programs like dirname and basename that are not present on the average Windows machine. Using msys instead might be an option, but I noticed that CMake on Windows does not like it when there is a Bourne-shell present in the path. Yes, that's a known issue - cmake doesn't want sh.exe in the path, if there is, it wants to use the msys makefiles generator. MSys is also far from complete, since this is just the minimum environment to run the autoconf tools. I would stay away from msys and stick with win-bash. The missing tools (dirname, basename, ...) get from http://gnuwin32.sourceforge.net/ - the shellutils package provides these tools. The cool thing about GnuWin32 is, that these tools work also in standard Windows CLI (without bash/msys/whatever). Actually I use these tools to improve my Windows CLI - this works quite well. Hi Werner: I have changed the subject line to something more appropriate for this thread. I think we will be able to give our Windows testers the choice of either MSYS-bash or win-bash (if that works for all Plplot tests which I have my doubts about, see below) and either gnuwin32 or MSYS. Arjen and I have just proved modern MSYS is complete enough to run all PLplot tests. For example, MSYS does include dirname and basename. I assume from what you have said that gnuwin32 (except for bash where you must use either MSYS-bash or win-bash for PLplot tests) is also complete enough to run the PLplot tests. I suspect the negative feelings you have expressed above about MSYS were established by your extensive experience with older versions of MSYS. Modern MSYS can be downloaded automatically with MinGW in ~5 minutes following the instructions in http://www.miscdebris.net/plplot_wiki/index.php?title=Install_MinGW/MSYS. Also as Arjen and I have recently discovered for Microsoft and wine versions of Windows, comprehensive_test.sh works well now with modern MSYS using the MSYS Makefiles generator. And I am about to generalize scripts/MinGW_MSYS_comprehensive_test.sh so that if the user prefers the MinGW Makefiles generator they can use it with the appropriate path manipulations to momentarily drop MSYS for the cmake invocation. That manipulation is possible with modern MSYS because the automatic installer keeps the MinGW and MSYS locations separate from each other. (Microsoft Windows users will have to test that MinGW Makefiles case for me when MinGW_MSYS_comprehensive_test.sh is generalized. Unlike the MSYS Makefiles case, I currently cannot test the MinGW Makefiles case on wine because certain CMake target dependencies are silently dropped by mingw32-make. mingw32-make uses many more native Windows features than the MSYS version of make, and therefore I assume this issue with mingw32-make on wine is due to some bug in wine's ability to mimic some Windows component used by mingw32-make which will be solved for some future version of wine.) MSYS-bash is easy to install now, and it has many modern bash features that win-bash doesn't have because win-bash development stopped many years ago. We did get win-bash to work for the limited ctest testing environment years ago, but I know the complete testing environment (e.g., including the pkg-config version of the build system for the installed examples) has many modern bash constructs (arrays, for example) which I doubt very much will work for win-bash now. And the situation is only going to get worse as more modern bash constructs creep into our PLplot test environment. Finally, you have expressed a legitimate concern that you prefer the tools you get with GnuWin32 to the (modern) MSYS ones. That should be fine, because all you have to do to assure you are using the GnuWin32 versions is to place the GnuWin32 location higher on the PATH than the MSYS location where MSYS-bash resides. In fact, for the planned change to scripts/MinGW_MSYS_comprehensive_test.sh, the MSYS location will automatically be put last in the PATH. We could also have an option to drop MSYS in the PATH altogether for those still wanting to try win-bash, but I don't think that version of bash will work for comprehensive PLplot testing for the reasons indicated above. Once I am ready, I hope you do test scripts/MinGW_MSYS_comprehensive_test.sh for the MinGW Makefiles case to be sure it works without issues for the combination of GnuWin32 tools (higher than MSYS on the PATH by design of MinGW_MSYS_comprehensive_test.sh) and MSYS-bash (or even win-bash if that actually works for comprehensive testing). Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot
Re: [Plplot-devel] Fix for 79-character truncation on plgfnam
On 2010-12-18 11:34-0800 David MacMahon wrote: Hi, Alan, On Dec 18, 2010, at 10:57 AM, Alan W. Irwin wrote: Another model for plgnver, plgndev, and plgnfnam is one similar to strlcpy that is discussed by Todd C. Miller and Theo de Raadt at http://www.gratisoft.us/todd/papers/strlcpy.html. That is a most interesting paper to read. I am not necessarily advocating something similar to this model API. I leave that to those more knowledgable in C than I am. However, I am mentioning this paper for further discussion. It seems like strlcpy is a more convenient alternative to strncpy, but I don't see how this would solve the allocation problem. Wouldn't the user still need to ensure that a large enough buffer has been allocated? If so, the problem of determining how large is large enough still remains. Am I missing something? See their example 1d where they check for truncation. Instead of taking an error return at that point as they did in their sample code the user could instead realloc if they liked as in your original suggestion. Of course, the real value of the strlcpy model is the thought that has gone into the null termination issue as well as the definition of the size of the buffer that is returned. For example, after much debate about the alternatives they decided to use the snprintf return semantics, that is, the returned length does not include the null terminator which is the same definition as the string length calculated by strlen. N.B. I have now found an additional useful reference for strlcpy which is http://www.manpagez.com/man/3/strlcpy/ aside to Dave Can you remember the name of the function you were using as the model for your API suggestion for plgnver, plgndev, and plgnfnam? If so, that will give us a chance to dig into the official documention of that function like we can currently do for getcwd and strlcpy. /aside to Dave OK, digging way back into my mental archives (and googling), I think I first came across this approach in a previous life when I did Windows programming. An example of this approach can be seen in the Windows GetShortPathName function (and probably others as well)... http://msdn.microsoft.com/en-us/library/aa364989(v=vs.85).aspx I mis-remembered the exact details of the return code. The WIN32 functions seem to return the total size needed iff the buffer pointer is NULL and the size is given as 0. This shouldn't really change the fundamentals of what I wrote earlier, however. Thanks for that reference. However, I have noticed some deficiencies in that documentation of GetShortPathName. There is no mention there of null termination guaranteed in the result which is an important issue covered in the strlcpy documentation. For example, what happens if the buffer receiving the result is too small? Do you get a truncated result with no null terminator or a truncated result of one unit smaller in size followed by a null terminator, or nothing at all in the result? Just because it appears to be better documented on the null-termination guarantee issue, I suggest we adopt the strlcpy model. This gives the user in any language freedom to do anything they like if the returned size (which never counts the null terminator) is = the value of n in the call. For high-level languages like Python where the uses don't have to worry about allocation, they could just call the function again with n chosen to be the previous returned value + 1. For lower-level languages like C, they could realloc the buffer using the correct size, return an error, or just accept the truncated result. Also, I don't think we should use the wrinkle present in GetShortPathName of doing something special for the NULL buffer, size=0 call. That does add a bit of convenience (but is not a necessity) for languages such as C where the buffer is alloced beforehand, but that wrinkle makes no sense for higher level languages such as Python where the correctly sized buffer is created as needed. For example, a typical Python call to plgfnam is fnam = plgfnam() With plgnfnam that would be changed to something like (needed_size, fnam) = plgnfnam(100) if needed_size = 100: # Oops, 100 was too small. fname = plgnfnam(needed_size+1)[1] On the other hand, I would hope the PLplot version or device names are unlikely to be greater than 20 characters in our lifetimes. So for that case, you normally wouldn't bother to check the needed size in Python: version = plgnfnam(20)[1] devname = plgndev(20)[1] Note, even in the unlikely case that the version or device name were greater than 20 characters, the above calls would be perfectly safe and would yield properly truncated results at the 20th character. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar
Re: [Plplot-devel] Fix for 79-character truncation on plgfnam
On Dec 18, 2010, at 6:30 PM, Alan W. Irwin wrote: On 2010-12-18 11:34-0800 David MacMahon wrote: Am I missing something? See their example 1d where they check for truncation. Instead of taking an error return at that point as they did in their sample code the user could instead realloc if they liked as in your original suggestion. Thanks, I did miss something! :-) The strlcpy model seems very similar to the WIN32 convention, only better in that it is more rigorously documented vis-a-vis NUL termination. Just because it appears to be better documented on the null-termination guarantee issue, I suggest we adopt the strlcpy model. This gives the user in any language freedom to do anything they like if the returned size (which never counts the null terminator) is = the value of n in the call. I agree! For high-level languages like Python where the uses don't have to worry about allocation, they could just call the function again with n chosen to be the previous returned value + 1. For high level languages like Python, any (re-)allocation could happen in the bindings and therefore be hidden from the end user. Also, I don't think we should use the wrinkle present in GetShortPathName of doing something special for the NULL buffer, size=0 call. That does add a bit of convenience (but is not a necessity) for languages such as C where the buffer is alloced beforehand, I agree about not treating a NULL pointer as a special case, but strlcpy inherently treats size=0 as a special case (i.e. no NUL terminator). When size=0, strlcpy ignores the pointer and gives the same result as the WIN32 convention (though the WIN32 convention requires the pointer to be NULL). but that wrinkle makes no sense for higher level languages such as Python where the correctly sized buffer is created as needed. For example, a typical Python call to plgfnam is fnam = plgfnam() With plgnfnam that would be changed to something like (needed_size, fnam) = plgnfnam(100) if needed_size = 100: # Oops, 100 was too small. fname = plgnfnam(needed_size+1)[1] This is an example of why I like to hand code language bindings. IMHO, the plgnfnam function, if it is even exposed in the scripting language, should take no arguments and always return a string containing the entire contents. Why make the user provide an initial guess? This requires them to check whether their guess was big enough and try again (without forgetting the +1) if it wasn't. How inconvenient! Why not just do all of that for them inside the bindings for the high-level language's plgfnam call (which would call plgnfnam under the covers)? Dave -- Lotusphere 2011 Register now for Lotusphere 2011 and learn how to connect the dots, take your collaborative environment to the next level, and enter the era of Social Business. http://p.sf.net/sfu/lotusphere-d2d ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
