To Phil and Andrew: @Andrew: I am addressing you directly (in addition to Phil) because I have questions for you below concerning whether you can replicate my weird timing results for wxwidgets and also concerning the issue of shared memory leaks on Linux.
On 2015-07-03 13:53+0100 Phil Rosenberg wrote: > Hi Alan > So I think you should now find the wxPLViewer speed has significantly > improved. The item I have dealt with now was that there is a timer at > the OS level (initiated by wxWidgets) which sends a message to the > viewer which initiates checking for new commands from the console app. > At this point I used to check for a single message and return after > dealing with it. However it seems that for whatever reason (different > OS, different wxWidgets version?) on some systems the minimum time > between the timer calls was quite long. This meant that when we were > dealing with text which required a lot of small transmissions to the > viewer there was a huge overhead. I have now modified things so that > on every timer event I check for new transmissions at least 100 times > with a 1 millisecond sleep between each. This means that for many > small transmissions in a short time we remove most of the overhead. > > On my CentOS machine the execution time reported by time x08c -dev > wxWidgets -np has dropped from 40s to about 2 s. Note that this only > times the console part, not the viewer, but still we have clearly > moved to something I would call acceptable. > > Note that there will always be a significant overhead for the > transmission so this will never compete with Cairo, but this is > unavoidable because wxWidgets cannot be run in a library without > running it as a separate thread and PlPlot isn't thread safe. Note > also that when using the wxWidgets driver via the wxWidgets binding > (I.e. from within a wxWidgets app) this all becomes irrelevant as the > viewer is not used. > > Anyway I hope this fix closes the issue and you are happy Alan. @Phil: The timing results for current master tip (commit id 5e74b6a6, "Modification to previous wxPLViewer optimisation") are much improved on first execution of examples. So big congratulations on that result! However, an important issue still remains as shown by the following repeated timing results for both xcairo (as a typical benchmark) and wxwidgets: software@raven> time examples/c/x00c -dev xcairo -np real 0m0.168s user 0m0.008s sys 0m0.016s software@raven> time examples/c/x00c -dev xcairo -np real 0m0.129s user 0m0.020s sys 0m0.004s software@raven> time examples/c/x00c -dev xcairo -np real 0m0.114s user 0m0.020s sys 0m0.008s software@raven> time examples/c/x00c -dev wxwidgets -np real 0m0.365s user 0m0.008s sys 0m0.012s software@raven> time examples/c/x00c -dev wxwidgets -np real 0m1.231s user 0m0.008s sys 0m0.012s software@raven> time examples/c/x00c -dev wxwidgets -np real 0m5.587s user 0m0.008s sys 0m0.012s So the xcairo case has subsequent time results that are up to 1.5 times faster (in real time which is the most important component) then the first execution of the example while the wxwidgets case has subsequent time results that are up to 15 (!) times slower. The initial fast wxwidgers result seems guaranteed if you wait a minute or so since your last attempt to use wxwidgets. I mostly tested the above wxwidgets time results with the -np option (for convenience), but I also tried the test a few times without -np and got the fast, slow, slow,... pattern for that case as well. Also, I usually get a consistent fast, slow, slow,.... pattern for wxwidgets, but just now I tried it again, and I got the fast result for something like 10 times in a row, then the slow result. So the wxwidgets inconsistency in time results is sometimes inconsistent. :-) The above results for the xcairo case are quite typical of all non-wxwidgets cases I have ever investigated before, where real time results are significantly shorter on second and subsequent execution because of the well-known effect on Linux of the system caching all reasonably small files in memory to improve small file access times for subsequent use. The very unusual much longer times that often but not always immediately appear on second and subsequent executions that occur for the wxwidgets case above strongly suggest to me a system resource is not being properly released by the first execution of the example so getting that resource back again often takes a lot of extra time on the second and subsequent runs. I brought up this issue with you off list much earlier this week, but you did not respond at that time, but I hope you do that now. The first step in your response should be to try the above bash commands for at least the wxwidgets case on your Windows and Linux platforms. to (a) see if this issue also occurs on Windows, and (b) see if this issue occurs on all Linux systems accessible to you. @Andrew: could you please try the above time tests as well on your Linux platforms? @Both: If the issue is a Linux resource which I am chronically short of because of my prior wxwidgets testing and a shared memory leak (see discussion below) in wxwidgets, then you might not be able to replicate the issue on Linux until you do sufficient wxwidgets testing. My Linux up time is currently 42 days, and if I reboot (which I won't do lightly because there are two users on the system and it is a bit of a pain for us to reset all desktops the way we like them), I might find the issue goes away for a while. @Phil: If the issue is a Linux-only one, then an obvious candidate for a system resource grabber is the shm_open calls in drivers/wxwidgets_comms.cpp that helps to allow shared memory access between the wxwidgets device driver and wxPLViewer. I have just now read the shm_overview and shm_open/shm_unlink man pages, and it appears that shm_unlink must be called by _both_ the wxwidgets device driver and wxPLViewer to properly release the shared memory. So a code review for those two cases to make sure that _always_ happens (i.e., there is no shared memory leak) is indicated. @Andrew: will you comment further here please about whether we should be concerned about possible shared memory leaks? I now realize, for example, that both Phil's and my calling the IPC method that is used "shared memory" is likely not specific enough because it does not distinguish the old-fashioned shmget method (which we do not use) with the modern mmap-based method that we do use. See <http://stackoverflow.com/questions/5656530/how-to-use-shared-memory-with-linux-in-c> for comments on this important "shared-memory" distinction. Furthermore, from the discussion at <http://stackoverflow.com/questions/22691621/how-to-avoid-shared-memory-leaks> it appears the mmap-based method cannot actually leak memory. Is that your interpretation of those remarks as well? @Phil: One thing I noticed from reading those man pages is it is suggested the name used for shm_open should always start with a "/" for the most portable results. It appears our current wxwidgets code does not follow that suggestion so this may affect Unix platforms that are not Linux. So I suggest you will want to address that issue (even though it has nothing to do with timing. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ ------------------------------------------------------------------------------ Don't Limit Your Business. Reach for the Cloud. GigeNET's Cloud Solutions provide you with the tools and support that you need to offload your IT needs and focus on growing your business. Configured For All Businesses. Start Your Cloud Today. https://www.gigenetcloud.com/ _______________________________________________ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
