Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
Hi, Alan, On Mar 14, 2010, at 10:29 , Alan W. Irwin wrote: > That is revision 10869 should be the same as > David's tree that produced his patch I can confirm that r10869 contains all the changes from my patch. Thanks again, Dave -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Mar 13, 2010, at 16:43 , Alan W. Irwin wrote: > I have committed this patch (revision 10864). Thanks, Alan!!! > Thanks, Dave, for all your work on this arbitrary storage of 2D data > API. You're welcome. Thanks for including it! Dave -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Sun, Mar 14, 2010 at 12:29 PM, Alan W. Irwin wrote: > On 2010-03-14 10:06-0700 Alan W. Irwin wrote: > >> On 2010-03-14 09:46-0500 Hezekiah M. Carty wrote: >> >>> The patch or patch application to Subversion seems to be incomplete. >>> cmake fails from a fresh source tree with this error: >>> >>> -- Configuring done >>> CMake Error in src/CMakeLists.txt: >>> Cannot find source file "plf2ops.c". Tried extensions .c .C .c++ .cc >>> .cpp >>> .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx >>> >>> It looks like one of the files was not checked in to Subversion. >> >> Sorry about that. Now fixed in revision 10868. > > I also fixed (revision 10869) an inadvertent inclusion of some of my local > work in plfill.c that was not ready for prime time. My apologies that I > didn't do the original commit exactly consistent with David's patch, but > that should be fixed now. That is revision 10869 should be the same as > David's tree that produced his patch + Hez's recent series of commits to fix > various unrelated issues. > Alan, Thank you for checking in the corrections. Everything looks clean now here as well. Hez -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On 2010-03-14 10:06-0700 Alan W. Irwin wrote: > On 2010-03-14 09:46-0500 Hezekiah M. Carty wrote: > >> The patch or patch application to Subversion seems to be incomplete. >> cmake fails from a fresh source tree with this error: >> >> -- Configuring done >> CMake Error in src/CMakeLists.txt: >> Cannot find source file "plf2ops.c". Tried extensions .c .C .c++ .cc .cpp >> .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx >> >> It looks like one of the files was not checked in to Subversion. > > Sorry about that. Now fixed in revision 10868. I also fixed (revision 10869) an inadvertent inclusion of some of my local work in plfill.c that was not ready for prime time. My apologies that I didn't do the original commit exactly consistent with David's patch, but that should be fixed now. That is revision 10869 should be the same as David's tree that produced his patch + Hez's recent series of commits to fix various unrelated issues. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On 2010-03-14 09:46-0500 Hezekiah M. Carty wrote: > The patch or patch application to Subversion seems to be incomplete. > cmake fails from a fresh source tree with this error: > > -- Configuring done > CMake Error in src/CMakeLists.txt: > Cannot find source file "plf2ops.c". Tried extensions .c .C .c++ .cc .cpp > .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx > > It looks like one of the files was not checked in to Subversion. Sorry about that. Now fixed in revision 10868. __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Sat, Mar 13, 2010 at 7:43 PM, Alan W. Irwin wrote: > Hi Dave (with question for Andrew at the end): > > On 2010-03-12 16:24-0800 David MacMahon wrote: > >> This is a roll-up of all previous PLplot patches related to supporting >> arbitrary storage of 2D user data. This patch is based on (and should >> apply cleanly to) svn/trunk r10859. > > I did some tests of your patch. > > The patched and unpatched PostScript results from the test_diff_psc target > were identical (using cmp -i 170 to compare the 467 (!) files produced by > that target) between the two build directories other than p21.psc where both > plots looked okay, but there were obvious visual differences (axes shifted > slightly around, etc.) > Alan, The patch or patch application to Subversion seems to be incomplete. cmake fails from a fresh source tree with this error: -- Configuring done CMake Error in src/CMakeLists.txt: Cannot find source file "plf2ops.c". Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx It looks like one of the files was not checked in to Subversion. Hez -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
Hi Dave (with question for Andrew at the end): On 2010-03-12 16:24-0800 David MacMahon wrote: > This is a roll-up of all previous PLplot patches related to supporting > arbitrary storage of 2D user data. This patch is based on (and should > apply cleanly to) svn/trunk r10859. I did some tests of your patch. The patched and unpatched PostScript results from the test_diff_psc target were identical (using cmp -i 170 to compare the 467 (!) files produced by that target) between the two build directories other than p21.psc where both plots looked okay, but there were obvious visual differences (axes shifted slightly around, etc.) Note, p21 is a "p" example that runs a high-level octave interface that is known to be problematic (see my previous post where I showed the "p" examples and associated high-level octave interface need lots of TLC). Therefore, I am not surprised p21 gives substantially different results for rounding errors that are presumably only slightly different. We had such propagation of rounding errors in our standard examples for years, and it took quite a bit of effort and some changing of the examples that depended on floating point as opposed to integer comparisons to get rid of most of those issues. Here are the timing results for the patched and unpatched cases. PATCHED softw...@raven> time make -j4 test_diff_psc >& make_test.out real3m23.120s user3m43.878s sys 0m34.386s UNPATCHED softw...@raven> time make -j4 test_diff_psc >& make_test.out real3m27.809s user3m44.350s sys 0m33.274s These times were done for an initially empty build tree so they include both complete build times and run times. Also, certain examples (which may have nothing to do with your patch) dominate the run times. So as discussed previously these results should be taken with a large grain of salt, but there is nothing obviously a lot slower with the patched result and in fact it is slightly (but probably not significantly) faster in total elasped time. Because with this test I could detect no gross efficiency concerns and because the PostScript results were identical (except for p21) I have committed this patch (revision 10864). Thanks, Dave, for all your work on this arbitrary storage of 2D data API. Hi Andrew: Would you please evaluate this revision with the best tests you can think of with regard to your efficiency concerns? If and when you are satisfied on this score (but not before) we should be thinking about propagating these API additions to all languages. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
[Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
This is a roll-up of all previous PLplot patches related to supporting arbitrary storage of 2D user data. This patch is based on (and should apply cleanly to) svn/trunk r10859. Adds support for arbitrary storage of 2D user data. This is very similar to the technique employed by some existing functions (e.g. plfcont and plfshade) that use "evaluator" functions to access 2D user data that is stored in an arbtrary format. The new approach extends the concept of a user-supplied (or predefined) "evaluator" function to a group of user-supplied (or predefined) "operator" functions. The operator functions provide for various operations on the arbitrarily stored 2D data including: get, set, +=, -=, *=, /=, isnan, minmax, and f2eval. To facilitate the passing of an entire family of operator functions (via function pointers), a plf2ops_t structure is defined to contain a pointer to each type of operator function. Predefined operator functions are defined for several common 2D data storage techniques. Variables (of type plf2ops_t) containing function pointers for these operator functions are also defined. New variants of functions that accept 2D data are created. The new variants accept the 2D data as two parameters: a pointer to a plf2ops_t structure containing (pointers to) suitable operator functions and a PLPointer to the actual 2D data store. Existing functions that accept 2D data are modified to simply pass their parameters to the corresponding new variant of the function, along with a pointer to the suitable predefined plf2ops_t stucture of operator function pointers. The list of functions for which new variants are created is: c_plimage, c_plimagefr, c_plmesh, c_plmeshc, c_plot3d, c_plot3dc, c_plot3dcl, c_plshade1, c_plshades, c_plsurf3d, and c_plsurf3dl, and c_plgriddata. The new variants are named the same as their corresponding existing function except that the "c_" prefix is changed to "plf" (e.g. the new variant of c_plmesh is called plfmesh). Adds plfvect declaration to plplot.h and changes the names (and only the names) of some plfvect arguments to make them slightly clearer. In order to maintain backwards API compatibility, this function and the other existing functions that use "evaluator" functions are NOT changed to use the new operator functions. Makes plplot.h and libplplot consistent vis-a-vis pltr0f and pltr2d. Moves the definitions of pltr2f (already declared in plplot.h) from the sccont.c files of the FORTRAN 77 and Fortran 95 bindings into plcont.c. Removes pltr0f declaration from plplot.h. Changes x08c.c to demonstrate use of new support for arbitrary storage of 2D data arrays. Shows how to do surface plots with the following four types of 2D data arrays: 1) PLFLT z[nx][ny]; 2) PLfGrid2 z; 3) PLFLT z[nx*ny]; /* row major order */ 4) PLFLT z[nx*ny]; /* column major order */ --- bindings/f77/sccont.c | 182 - bindings/f95/sccont.c | 182 - examples/c/x08c.c | 45 -- include/plplot.h | 238 ++-- src/CMakeLists.txt|1 + src/plcont.c | 207 +++- src/plf2ops.c | 426 +++ ++ src/plgridd.c | 120 -- src/plimage.c | 49 +-- src/plot3d.c | 212 - src/plshade.c | 81 -- src/plvect.c | 17 +-- 12 files changed, 1200 insertions(+), 560 deletions(-) create mode 100644 src/plf2ops.c 0001-Support-arbitrary-storage-of-2D-user-data.patch.gz Description: GNU Zip compressed data -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Feb 22, 2010, at 10:37 , David MacMahon wrote: > On Feb 22, 2010, at 6:42 , Andrew Ross wrote: > >> I would like to have a thorough comparison of the time difference. >> This should include a "large data" case as well where timings >> might be >> more important. The lena image might be one suitable case. > > As I said earlier, I share this concern. My 100x test was done using > the Lena image, but I agree that further analysis would be reassuring > (or not as the case may be). I have done some further performance testing using x20c (the test using the Lena image). The results are slightly different, but still very encouraging. I think my previous x20c loop tests were not a valid comparison because it turns out I was looping on a call to plimage, which I had not yet changed to call the plf2ops version named "plfimge". I had created plfimagefr, but I had not created plfimage. So I think my big x20c loop test was not really exercising new code. I have since created plfimage (patch forthcoming) and modified x20c.c to loop 2000 times on each of the two calls to plimage that plot only Lena's eyes (one call plots only the eyes in their normal size/ position, the other call plots them "zoomed in"). This "subset" plotting involves copying parts of the 2D data array so it definitely makes use of the new plf2ops code. I ran the modified x20c test three times using an unpatched libplplot with NO plf2ops stuff added and three times using a patched libplplot with the plf2ops stuff added. Here are the results from my (now old) MacBook Pro laptop... WithOUT plf2ops --- 2m 22.1s 2m 21.8s 2m 21.9s WITH plf2ops 2m 23.2s 2m 23.1s 2m 23.2s So the plf2ops version took about 1.2 seconds longer than the non- plf2ops version or approximately a 1.5% performance hit. It would be great is someone else could repeat this test and corroborate (or not) the results. Dave -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Feb 22, 2010, at 10:37 , David MacMahon wrote: > The patch under discussion should maintain 100% backwards > compatibility; any case where it doesn't is a bug! I found a rather embarrassing oversight: it seems that I moved the converted implementation of c_plgriddata into plfgriddata, but I forgot to make c_plgriddata call plfgriddata! In addition to leaving plgriddata with an empty body, I also forgot to add plfgriddata to plplot.h. I'll be sending out an incremental patch to fix this. Sorry for any inconvenience, Dave -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
Hi, Andrew, Thanks for your review and thoughtful comments! Here are some replies. On Feb 22, 2010, at 6:42 , Andrew Ross wrote: > 1) Using structures of function pointers in C is clearly the way to > implement this, but it might give us headaches for some languages > where > either function pointers or structures are not available. I thought about this point and hoped to alleviate any problems by not exposing the actual structures themselves. Instead I provide functions that return pointers to said structures. Languages that do not support function pointers or structures should at least be able to map the "pointer to structure" return values of these functions to integers. > 2) There are quite a few callback functions implemented. In most cases > we only need a get / set, although I can see that having the other > operators is (slightly) more efficient in that only one call is > required. > Does this make a noticeable difference? Even for functions that require only set or get, I chose to pass in a pointer to the structure to keep the API consistent. Functions that accept a pointer to a "plf2ops" structure yet use only one function pointer from that structure can simply declare a local function pointer and initialize it from the desired member of the structure. > 3) Talking of efficiency, I worry that this introduces a large > additional > level of complexity for a rather specialist set of cases where odd > data > storage methods are used. I am slightly relieved by David's comments, > but I would like to have a thorough comparison of the time difference. > This should include a "large data" case as well where timings might be > more important. The lena image might be one suitable case. As I said earlier, I share this concern. My 100x test was done using the Lena image, but I agree that further analysis would be reassuring (or not as the case may be). > The test > should also multi-language tests to see if not copying large amounts > of data around is quicker than having lots of callbacks. Sounds like a good idea. I think copying/transposing data will lose out because either it requires dynamic allocation in the bindings or puts the onus of copying/transposing the data onto the user. > 4) It terms of applications there are two big uses I can see > a) To simplify the langauge bindings (which is mostly hidden from the >user). > b) To allow the user to use odd data storage methods more > efficiently. I >can think of some interesting uses here like plotting a subset or >subsample of an array. It would be nice to include examples of > this. >This is potentially a much more generally useful outcome of this > from >a user point of view. I completely agree with these points. As I mentioned in an earlier email, I think 4b is a reason to consider implementing something similar for 1D data (e.g. to allow plotting of a single column of a 2D array)! > 5) Given the current mishmash of methods for dealing with 2-d arrays > then I would like to see this implemented consistently for all > functions which take 2-d data. Of course we need to ensure backwards > compatibility, but we should ensure the "new" interface is right from > the start. I tried to make API of new functions as consistent as possible. If you see any places where consistency could be improved, please let me know. The patch under discussion should maintain 100% backwards compatibility; any case where it doesn't is a bug! To maintain backwards compatibility, I did not modify the API of any existing functions. This includes functions, such as plfcont, that already accept a pointer to an "evaluator" function. Those functions are admittedly in an "in between" state. Perhaps new "plf2ops" variants of these functions could be created and the existing forms deprecated. > Sorry this is rather late in the discussion! IMHO, your content more than makes up for any delay! Thanks again, Dave -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Mon, Feb 22, 2010 at 09:37:33AM -0600, Maurice LeBrun wrote: > On Monday, February 22, 2010 at 14:42:57 (+) Andrew Ross writes: > > 3) Talking of efficiency, I worry that this introduces a large additional > > level of complexity for a rather specialist set of cases where odd data > > storage methods are used. I am slightly relieved by David's comments, > > but I would like to have a thorough comparison of the time difference. > > This should include a "large data" case as well where timings might be > > more important. The lena image might be one suitable case. The test > > should also multi-language tests to see if not copying large amounts > > of data around is quicker than having lots of callbacks. > > I see this issue as mostly about program clarity & programmer convenience. > Nothing wrong with a test, but I'll restate (and elaborate upon) my previous > prediction that efficiency is mostly a moot point. Of course those with > scientific programming backgrounds will tend to see function calls associated > with a single data point as somewhat evil, and they'd be right -- in the high > performance arena of say, the central processing loops of a multidimensional > simulation. But that's not the situation here. Aside from the oddball case > of plotting into a memory buffer, at the end of the call chain some i/o will > be performed. That should dwarf the function call overhead. Maurice, You are almost certainly right for the current plplot examples, none of which use large arrays. What I worry about just a little is things like contouring a very large 2-d array where there is a lot of data and a fair bit of array accessing. I'd just like to be satisfied that there is no impact before such a large and intrustive change is made. I'm sure it is just a scientific programming prejudice... Andrew -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Monday, February 22, 2010 at 14:42:57 (+) Andrew Ross writes: > 3) Talking of efficiency, I worry that this introduces a large additional > level of complexity for a rather specialist set of cases where odd data > storage methods are used. I am slightly relieved by David's comments, > but I would like to have a thorough comparison of the time difference. > This should include a "large data" case as well where timings might be > more important. The lena image might be one suitable case. The test > should also multi-language tests to see if not copying large amounts > of data around is quicker than having lots of callbacks. I see this issue as mostly about program clarity & programmer convenience. Nothing wrong with a test, but I'll restate (and elaborate upon) my previous prediction that efficiency is mostly a moot point. Of course those with scientific programming backgrounds will tend to see function calls associated with a single data point as somewhat evil, and they'd be right -- in the high performance arena of say, the central processing loops of a multidimensional simulation. But that's not the situation here. Aside from the oddball case of plotting into a memory buffer, at the end of the call chain some i/o will be performed. That should dwarf the function call overhead. -- Maurice LeBrun -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Tue, Feb 16, 2010 at 09:15:10AM +0100, Arjen Markus wrote: > On 2010-02-16 09:07, Alan W. Irwin wrote: > > On 2010-02-15 14:10-0500 Hazen Babcock wrote: > > > >> David MacMahon wrote: > >>> Adds support for arbitrary storage of 2D user data. This is very > >>> similar to the technique employed by some existing functions (e.g. > >>> plfcont and plfshade) that use "evaluator" functions to access 2D user > >>> data that is stored in an arbtrary format. > > >> Unless anyone objects I will test & commit this change in about a week. > > > > I think we need more time than that. How about two weeks, instead? It > > concerns me there has been no discussion of this API since David sent in the > > patch. If you like it please say so! Of course, if you don't like it, say > > so as well! > > > > I want to emphasize there was consensus on list (lead by Maurice) that an > > API to handle arbitrary 2D data storage was well worth having, and I am > > certainly willing to go along with that major change. However, I think it is > > important for everyone with good C skills here (that let's me out!) to take > > a good look at the overview of this implementation to be sure this new API > > is right for our needs. My point is there is going to be a lot of work > > propagating this to all our languages, updating our standard examples in all > > languages to use the new API, and also documenting the new API. So I ask > > for a thorough review of the new API in the next two weeks by those with > > good C skills. > > > > Hazen, by all means test the patch as well to make sure the implementation > > of the new API actually builds. But you should probably hold off on > > committing until everyone competent has had a chance to review the API to > > make sure this is the 2D array handling we want to have for the forseeable > > PLplot future. > > > > Dave, I thank you for your implementation work. It takes courage to be the > > first to implement something that is going to be thoroughly (I hope) > > reviewed by others, and I thank you for that courage! > > > > I have looked at the patch and it looks like a thorough method to > achieve the goal. However, it is a rather large change as Alan indicates > and I am not sure how it influences the current API. How about adapting > a (C) example for this? This ought to give us some insight as to how > to use this much more flexible scheme and possibly how to propagate it > to the other languages. I agree with Arjen regarding the implementation. Some general points 1) Using structures of function pointers in C is clearly the way to implement this, but it might give us headaches for some languages where either function pointers or structures are not available. 2) There are quite a few callback functions implemented. In most cases we only need a get / set, although I can see that having the other operators is (slightly) more efficient in that only one call is required. Does this make a noticeable difference? 3) Talking of efficiency, I worry that this introduces a large additional level of complexity for a rather specialist set of cases where odd data storage methods are used. I am slightly relieved by David's comments, but I would like to have a thorough comparison of the time difference. This should include a "large data" case as well where timings might be more important. The lena image might be one suitable case. The test should also multi-language tests to see if not copying large amounts of data around is quicker than having lots of callbacks. 4) It terms of applications there are two big uses I can see a) To simplify the langauge bindings (which is mostly hidden from the user). b) To allow the user to use odd data storage methods more efficiently. I can think of some interesting uses here like plotting a subset or subsample of an array. It would be nice to include examples of this. This is potentially a much more generally useful outcome of this from a user point of view. 5) Given the current mishmash of methods for dealing with 2-d arrays then I would like to see this implemented consistently for all functions which take 2-d data. Of course we need to ensure backwards compatibility, but we should ensure the "new" interface is right from the start. Sorry this is rather late in the discussion! Cheers Andrew -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On 2010-02-16 11:53-0800 David MacMahon wrote: >> However, I strongly second Arjen's plea to at least convert >> one example (example 16?) to using the new 2D array API directly so that >> all >> the issues of porting it to the various languages (including perl/PDL) can >> be worked out. > > Since the plshade functions have not changed dramatically, I think examples 8 > (plot3d), 20 (plimage), and/or 21 (plgriddata) might be better choices to > "port" to the new API. Examining how the "post-patch" versions of the > existing functions call into their new variants would show how these changes > would look. I would be happy to do this for one of these examples (requests > please!), but it might be instructive for others to try it as well; it should > be very straightforward. Thanks for that offer. Just to be definite let's pick example 8. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On Feb 16, 2010, at 9:24 , Alan W. Irwin wrote: > From the description, it sounds like the current 2D array API has been > reimplemented as a wrapper for the Dave's new 2D array API. So > just testing > the normal examples with the current 2D array API should indirectly > exercise > the new API. Yes. To be very clear, my intent is that this patch NOT change the API of any existing function. If anyone finds that it does, please let me know and I will fix it. The patch should only add new things (structures, functions, etc.) and move pltr2f() from the f77 and f95 bindings into libplplot proper. It turns out that there are really two "current" 2D array APIs. Some of the PLplot functions already support the concept of arbitrary storage of 2D data (e.g functions relating to plcont, plshade, plvect). My patch neither changes the API of these functions nor introduces new variants of them. These existing functions still accept a single function pointer for *accessing* (aka "evaluating") the arbitrarily stored 2D data rather than a pointer to a structure of function pointers for *operating* on said data (accessing the data is one of several supported "operations"). Other PLplot functions that work with 2D data arrays, such as plplot3d, plsurf, and plmesh, current;y (i.e. pre-patch) require that the 2D data be stored in a particular form. My patch does not change the API of these functions, but it does introduce new variants of them that accept a 2D data array as a PLPointer plus a pointer to a structure of function pointers for operating on the data. The patch also replaces the implementation of the existing functions to forward their arguments to the new corresponding variant with a suitable pointer to a set of operator functions. In theory, the existing functions could be written as preprocessor macros that invoke the new variants, but obviously that would not work for existing code that expects the existing functions to exist in the library as callable symbols. The newly created variants are really modified and renamed versions of the existing functions. Other than the 2D data operations, the implementations themselves should remain unchanged. > However, I strongly second Arjen's plea to at least convert > one example (example 16?) to using the new 2D array API directly so > that all > the issues of porting it to the various languages (including perl/ > PDL) can > be worked out. Since the plshade functions have not changed dramatically, I think examples 8 (plot3d), 20 (plimage), and/or 21 (plgriddata) might be better choices to "port" to the new API. Examining how the "post- patch" versions of the existing functions call into their new variants would show how these changes would look. I would be happy to do this for one of these examples (requests please!), but it might be instructive for others to try it as well; it should be very straightforward. My primary concern about the patch is the impact it will have (if any) on the performance of calls involving 2D data arrays. I added a loop to x20c.c such that it calls into plimage 100 times. I ran it with "pre-patch" and "post-patch" builds using the null driver and found no significant performance change, so I have some confidence that it does not make things horribly worse, but I still worry (a little) about potential negative impacts to performance. Thanks, Dave -- SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
Hi, Alan, On Feb 16, 2010, at 0:07 , Alan W. Irwin wrote: > Dave, I thank you for your implementation work. It takes courage to > be the > first to implement something that is going to be thoroughly (I hope) > reviewed by others, and I thank you for that courage! At the risk of sounding like a mutual "thank you" fest, thank you for your leadership in overseeing and contributing to the ongoing development of PLplot. It is somewhat bittersweet that my specific motivation for supporting the arbitrary storage of 2D data has been nullified (see other pltr0f/ pltr2f thread) yet my desire for this capability remains. IOW, I don't need/require this patch anymore but I still think it adds value and is well worthwhile. :-) Dave -- SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On 2010-02-16 08:53-0700 Doug Hunt wrote: > Hi Alan, all: Would this change require re-writing the perl interface to the > 2D functions? Eventually. > Would this be required right away, or would there be backwards > compatible variants left around to ease the transition? No and yes. > > It might be useful to know which demos are affected by this. > None directly at this time. >From the description, it sounds like the current 2D array API has been reimplemented as a wrapper for the Dave's new 2D array API. So just testing the normal examples with the current 2D array API should indirectly exercise the new API. However, I strongly second Arjen's plea to at least convert one example (example 16?) to using the new 2D array API directly so that all the issues of porting it to the various languages (including perl/PDL) can be worked out. As it happens, there is a similar issue with Hazen's new getter/setter API. We need a C example that uses it to test whether we can propagate that new API to all our languages without difficulties. The truth is until our standard examples use any new API directly, that new API is largely ignored both by developers here and users. As we gain experience with both Dave's new 2D array API, and Hazen's new getter/setter API, we may find that some (but probably not all) of the old API should be deprecated. Of course, long in the future we will want to remove all our deprecated API, but that should be discussed thoroughly at the time before we actually do it since libraries and applications that depend on PLplot will all be affected. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
Hi Alan, all: Would this change require re-writing the perl interface to the 2D functions? Would this be required right away, or would there be backwards compatible variants left around to ease the transition? It might be useful to know which demos are affected by this. Thanks, Doug Hunt dh...@ucar.edu Software Engineer UCAR - COSMIC, Tel. (303) 497-2611 On Tue, 16 Feb 2010, Alan W. Irwin wrote: > On 2010-02-15 14:10-0500 Hazen Babcock wrote: > >> David MacMahon wrote: >>> Adds support for arbitrary storage of 2D user data. This is very >>> similar to the technique employed by some existing functions (e.g. >>> plfcont and plfshade) that use "evaluator" functions to access 2D user >>> data that is stored in an arbtrary format. The new approach extends >>> the concept of a user-supplied (or predefined) "evaluator" function to a >>> group of user-supplied (or predefined) "operator" functions. The >>> operator functions provide for various operations on the arbitrarily >>> stored 2D data including: get, set, +=, -=, *=, /=, isnan, and minmax. >>> >>> To facilitate the passing of an entire family of operator functions (via >>> function pointers), a plf2ops_t structure is defined to contain a >>> pointer to each type of operator function. Predefined operator >>> functions are defined for several common 2D data storage techniques. >>> Variables (of type plf2ops_t) containing function pointers for these >>> operator functions are also defined. >>> >>> New variants of functions that accept 2D data are created. The new >>> variants accept the 2D data as two parameters: a pointer to a plf2ops_t >>> structure containing (pointers to) suitable operator functions and a >>> PLPointer to the actual 2D data store. Existing functions that accept >>> 2D data are modified to simply pass their parameters to the >>> corresponding new variant of the function, along with a pointer to the >>> suitable predefined plf2ops_t stucture of operator function pointers. >>> >>> The list of functions for which new variants are created is: >>> c_plimagefr, c_plmesh, c_plmeshc, c_plot3d, c_plot3dc, c_plot3dcl, >>> c_plsurf3d, and c_plsurf3dl, and c_plgriddata. The new variants are >>> named the same as their corresponding existing function except that the >>> "c_" prefix is changed to "plf" (e.g. the new variant of c_plmesh is >>> called plfmesh). >>> >>> Adds plfvect declaration to plplot.h and changes the names (and only the >>> names) of some plfvect arguments to make them slightly clearer. In >>> order to maintain backwards API compatibility, this function and the >>> other existing functions that use "evaluator" functions are NOT changed >>> to use the new operator functions. >>> >>> Makes plplot.h and libplplot consistent vis-a-vis pltr0f and pltr2d. >>> Moves the definitions of pltr2f (already declared in plplot.h) from the >>> sccont.c files of the FORTRAN 77 and Fortran 95 bindings into plcont.c. >>> Removes pltr0f declaration from plplot.h. >>> --- >>> bindings/f77/sccont.c | 182 -- >>> bindings/f95/sccont.c | 182 -- >>> include/plplot.h | 196 +++-- >>> src/CMakeLists.txt|1 + >>> src/plcont.c | 207 +- >>> src/plf2ops.c | 397 >>> + >>> src/plgridd.c | 119 --- >>> src/plimage.c | 16 ++- >>> src/plot3d.c | 212 +-- >>> src/plshade.c |8 +- >>> src/plvect.c | 17 +-- >>> 11 files changed, 1004 insertions(+), 533 deletions(-) >>> create mode 100644 src/plf2ops.c >> >> Unless anyone objects I will test & commit this change in about a week. > > I think we need more time than that. How about two weeks, instead? It > concerns me there has been no discussion of this API since David sent in the > patch. If you like it please say so! Of course, if you don't like it, say > so as well! > > I want to emphasize there was consensus on list (lead by Maurice) that an > API to handle arbitrary 2D data storage was well worth having, and I am > certainly willing to go along with that major change. However, I think it is > important for everyone with good C skills here (that let's me out!) to take > a good look at the overview of this implementation to be sure this new API > is right for our needs. My point is there is going to be a lot of work > propagating this to all our languages, updating our standard examples in all > languages to use the new API, and also documenting the new API. So I ask > for a thorough review of the new API in the next two weeks by those with > good C skills. > > Hazen, by all means test the patch as well to make sure the implementation > of the new API actually builds. But you should probably hold off on > committing until everyone competent has had a chance to review the API to > make sure this is the 2D array handling we want to have for th
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On 2010-02-16 09:07, Alan W. Irwin wrote: > On 2010-02-15 14:10-0500 Hazen Babcock wrote: > >> David MacMahon wrote: >>> Adds support for arbitrary storage of 2D user data. This is very >>> similar to the technique employed by some existing functions (e.g. >>> plfcont and plfshade) that use "evaluator" functions to access 2D user >>> data that is stored in an arbtrary format. >> Unless anyone objects I will test & commit this change in about a week. > > I think we need more time than that. How about two weeks, instead? It > concerns me there has been no discussion of this API since David sent in the > patch. If you like it please say so! Of course, if you don't like it, say > so as well! > > I want to emphasize there was consensus on list (lead by Maurice) that an > API to handle arbitrary 2D data storage was well worth having, and I am > certainly willing to go along with that major change. However, I think it is > important for everyone with good C skills here (that let's me out!) to take > a good look at the overview of this implementation to be sure this new API > is right for our needs. My point is there is going to be a lot of work > propagating this to all our languages, updating our standard examples in all > languages to use the new API, and also documenting the new API. So I ask > for a thorough review of the new API in the next two weeks by those with > good C skills. > > Hazen, by all means test the patch as well to make sure the implementation > of the new API actually builds. But you should probably hold off on > committing until everyone competent has had a chance to review the API to > make sure this is the 2D array handling we want to have for the forseeable > PLplot future. > > Dave, I thank you for your implementation work. It takes courage to be the > first to implement something that is going to be thoroughly (I hope) > reviewed by others, and I thank you for that courage! > I have looked at the patch and it looks like a thorough method to achieve the goal. However, it is a rather large change as Alan indicates and I am not sure how it influences the current API. How about adapting a (C) example for this? This ought to give us some insight as to how to use this much more flexible scheme and possibly how to propagate it to the other languages. Regards, Arjen -- SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
On 2010-02-15 14:10-0500 Hazen Babcock wrote: > David MacMahon wrote: >> Adds support for arbitrary storage of 2D user data. This is very >> similar to the technique employed by some existing functions (e.g. >> plfcont and plfshade) that use "evaluator" functions to access 2D user >> data that is stored in an arbtrary format. The new approach extends >> the concept of a user-supplied (or predefined) "evaluator" function to a >> group of user-supplied (or predefined) "operator" functions. The >> operator functions provide for various operations on the arbitrarily >> stored 2D data including: get, set, +=, -=, *=, /=, isnan, and minmax. >> >> To facilitate the passing of an entire family of operator functions (via >> function pointers), a plf2ops_t structure is defined to contain a >> pointer to each type of operator function. Predefined operator >> functions are defined for several common 2D data storage techniques. >> Variables (of type plf2ops_t) containing function pointers for these >> operator functions are also defined. >> >> New variants of functions that accept 2D data are created. The new >> variants accept the 2D data as two parameters: a pointer to a plf2ops_t >> structure containing (pointers to) suitable operator functions and a >> PLPointer to the actual 2D data store. Existing functions that accept >> 2D data are modified to simply pass their parameters to the >> corresponding new variant of the function, along with a pointer to the >> suitable predefined plf2ops_t stucture of operator function pointers. >> >> The list of functions for which new variants are created is: >> c_plimagefr, c_plmesh, c_plmeshc, c_plot3d, c_plot3dc, c_plot3dcl, >> c_plsurf3d, and c_plsurf3dl, and c_plgriddata. The new variants are >> named the same as their corresponding existing function except that the >> "c_" prefix is changed to "plf" (e.g. the new variant of c_plmesh is >> called plfmesh). >> >> Adds plfvect declaration to plplot.h and changes the names (and only the >> names) of some plfvect arguments to make them slightly clearer. In >> order to maintain backwards API compatibility, this function and the >> other existing functions that use "evaluator" functions are NOT changed >> to use the new operator functions. >> >> Makes plplot.h and libplplot consistent vis-a-vis pltr0f and pltr2d. >> Moves the definitions of pltr2f (already declared in plplot.h) from the >> sccont.c files of the FORTRAN 77 and Fortran 95 bindings into plcont.c. >> Removes pltr0f declaration from plplot.h. >> --- >> bindings/f77/sccont.c | 182 -- >> bindings/f95/sccont.c | 182 -- >> include/plplot.h | 196 +++-- >> src/CMakeLists.txt|1 + >> src/plcont.c | 207 +- >> src/plf2ops.c | 397 >> + >> src/plgridd.c | 119 --- >> src/plimage.c | 16 ++- >> src/plot3d.c | 212 +-- >> src/plshade.c |8 +- >> src/plvect.c | 17 +-- >> 11 files changed, 1004 insertions(+), 533 deletions(-) >> create mode 100644 src/plf2ops.c > > Unless anyone objects I will test & commit this change in about a week. I think we need more time than that. How about two weeks, instead? It concerns me there has been no discussion of this API since David sent in the patch. If you like it please say so! Of course, if you don't like it, say so as well! I want to emphasize there was consensus on list (lead by Maurice) that an API to handle arbitrary 2D data storage was well worth having, and I am certainly willing to go along with that major change. However, I think it is important for everyone with good C skills here (that let's me out!) to take a good look at the overview of this implementation to be sure this new API is right for our needs. My point is there is going to be a lot of work propagating this to all our languages, updating our standard examples in all languages to use the new API, and also documenting the new API. So I ask for a thorough review of the new API in the next two weeks by those with good C skills. Hazen, by all means test the patch as well to make sure the implementation of the new API actually builds. But you should probably hold off on committing until everyone competent has had a chance to review the API to make sure this is the 2D array handling we want to have for the forseeable PLplot future. Dave, I thank you for your implementation work. It takes courage to be the first to implement something that is going to be thoroughly (I hope) reviewed by others, and I thank you for that courage! Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net)
Re: [Plplot-devel] [PATCH] Support arbitrary storage of 2D user data
David MacMahon wrote: > Adds support for arbitrary storage of 2D user data. This is very > similar to the technique employed by some existing functions (e.g. > plfcont and plfshade) that use "evaluator" functions to access 2D user > data that is stored in an arbtrary format. The new approach extends > the concept of a user-supplied (or predefined) "evaluator" function to a > group of user-supplied (or predefined) "operator" functions. The > operator functions provide for various operations on the arbitrarily > stored 2D data including: get, set, +=, -=, *=, /=, isnan, and minmax. > > To facilitate the passing of an entire family of operator functions (via > function pointers), a plf2ops_t structure is defined to contain a > pointer to each type of operator function. Predefined operator > functions are defined for several common 2D data storage techniques. > Variables (of type plf2ops_t) containing function pointers for these > operator functions are also defined. > > New variants of functions that accept 2D data are created. The new > variants accept the 2D data as two parameters: a pointer to a plf2ops_t > structure containing (pointers to) suitable operator functions and a > PLPointer to the actual 2D data store. Existing functions that accept > 2D data are modified to simply pass their parameters to the > corresponding new variant of the function, along with a pointer to the > suitable predefined plf2ops_t stucture of operator function pointers. > > The list of functions for which new variants are created is: > c_plimagefr, c_plmesh, c_plmeshc, c_plot3d, c_plot3dc, c_plot3dcl, > c_plsurf3d, and c_plsurf3dl, and c_plgriddata. The new variants are > named the same as their corresponding existing function except that the > "c_" prefix is changed to "plf" (e.g. the new variant of c_plmesh is > called plfmesh). > > Adds plfvect declaration to plplot.h and changes the names (and only the > names) of some plfvect arguments to make them slightly clearer. In > order to maintain backwards API compatibility, this function and the > other existing functions that use "evaluator" functions are NOT changed > to use the new operator functions. > > Makes plplot.h and libplplot consistent vis-a-vis pltr0f and pltr2d. > Moves the definitions of pltr2f (already declared in plplot.h) from the > sccont.c files of the FORTRAN 77 and Fortran 95 bindings into plcont.c. > Removes pltr0f declaration from plplot.h. > --- > bindings/f77/sccont.c | 182 -- > bindings/f95/sccont.c | 182 -- > include/plplot.h | 196 +++-- > src/CMakeLists.txt|1 + > src/plcont.c | 207 +- > src/plf2ops.c | 397 > + > src/plgridd.c | 119 --- > src/plimage.c | 16 ++- > src/plot3d.c | 212 +-- > src/plshade.c |8 +- > src/plvect.c | 17 +-- > 11 files changed, 1004 insertions(+), 533 deletions(-) > create mode 100644 src/plf2ops.c Unless anyone objects I will test & commit this change in about a week. -Hazen -- SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
