Hi Alan - this is now slightly out of context as I found it in my drafts folder unsent. I can confirm similar odd timing results on my Centos machine. I did have a couple of exit() calls in wxPLViewer, which to be honest was just lazy of me. On was used to close the app when -np was used. I have now replaced these both with clean exit methods. this ensures all destructors are called and I can now say that at least as far as I can tell on Windows the shared memory is _always_ released. It is certainly released when the window is closed using -np. This is on the viewer side. I presume the use of -np still gives a clean exit on the console side and doesn't invoke any exit calls anywhere?
Unfortunately the odd timings still persist. Note that the application only asks for 1 MB of shared memory so even if not freed I wouldn't have thought it would slow anything down really after just one run. I haven't got time right now to set up a decent debugging environment on my Centos machine, but I will try to double check the shared memory is released on that system as soon as I can. Phil On 3 July 2015 at 20:32, Alan W. Irwin <ir...@beluga.phys.uvic.ca> wrote: > To Phil and Andrew: > > @Andrew: I am addressing you directly (in addition to Phil) because I > have questions for you below concerning whether you can replicate my > weird timing results for wxwidgets and also concerning the issue of > shared memory leaks on Linux. > > > On 2015-07-03 13:53+0100 Phil Rosenberg wrote: > >> Hi Alan >> So I think you should now find the wxPLViewer speed has significantly >> improved. The item I have dealt with now was that there is a timer at >> the OS level (initiated by wxWidgets) which sends a message to the >> viewer which initiates checking for new commands from the console app. >> At this point I used to check for a single message and return after >> dealing with it. However it seems that for whatever reason (different >> OS, different wxWidgets version?) on some systems the minimum time >> between the timer calls was quite long. This meant that when we were >> dealing with text which required a lot of small transmissions to the >> viewer there was a huge overhead. I have now modified things so that >> on every timer event I check for new transmissions at least 100 times >> with a 1 millisecond sleep between each. This means that for many >> small transmissions in a short time we remove most of the overhead. >> >> On my CentOS machine the execution time reported by time x08c -dev >> wxWidgets -np has dropped from 40s to about 2 s. Note that this only >> times the console part, not the viewer, but still we have clearly >> moved to something I would call acceptable. >> >> Note that there will always be a significant overhead for the >> transmission so this will never compete with Cairo, but this is >> unavoidable because wxWidgets cannot be run in a library without >> running it as a separate thread and PlPlot isn't thread safe. Note >> also that when using the wxWidgets driver via the wxWidgets binding >> (I.e. from within a wxWidgets app) this all becomes irrelevant as the >> viewer is not used. >> >> Anyway I hope this fix closes the issue and you are happy Alan. > > > @Phil: > > The timing results for current master tip (commit id 5e74b6a6, > "Modification to previous wxPLViewer optimisation") are much improved > on first execution of examples. So big congratulations on that result! > > However, an important issue still remains as shown by the following > repeated timing results for both xcairo (as a typical benchmark) and > wxwidgets: > > software@raven> time examples/c/x00c -dev xcairo -np > > real 0m0.168s > user 0m0.008s > sys 0m0.016s > software@raven> time examples/c/x00c -dev xcairo -np > > real 0m0.129s > user 0m0.020s > sys 0m0.004s > software@raven> time examples/c/x00c -dev xcairo -np > > real 0m0.114s > user 0m0.020s > sys 0m0.008s > > software@raven> time examples/c/x00c -dev wxwidgets -np > > real 0m0.365s > user 0m0.008s > sys 0m0.012s > software@raven> time examples/c/x00c -dev wxwidgets -np > > real 0m1.231s > user 0m0.008s > sys 0m0.012s > software@raven> time examples/c/x00c -dev wxwidgets -np > > real 0m5.587s > user 0m0.008s > sys 0m0.012s > > So the xcairo case has subsequent time results that are up to 1.5 > times faster (in real time which is the most important component) then > the first execution of the example while the wxwidgets case has > subsequent time results that are up to 15 (!) times slower. The > initial fast wxwidgers result seems guaranteed if you wait a minute or > so since your last attempt to use wxwidgets. I mostly tested the > above wxwidgets time results with the -np option (for convenience), > but I also tried the test a few times without -np and got the fast, > slow, slow,... pattern for that case as well. Also, I usually get a > consistent fast, slow, slow,.... pattern for wxwidgets, but just now I > tried it again, and I got the fast result for something like 10 times > in a row, then the slow result. So the wxwidgets inconsistency in time > results is sometimes inconsistent. :-) > > The above results for the xcairo case are quite typical of all > non-wxwidgets cases I have ever investigated before, where real time > results are significantly shorter on second and subsequent > execution because of the well-known effect on Linux of the system > caching all reasonably small files in memory to improve small file > access times for subsequent use. > > The very unusual much longer times that often but not always > immediately appear on second and subsequent executions that occur for > the wxwidgets case above strongly suggest to me a system resource is > not being properly released by the first execution of the example so > getting that resource back again often takes a lot of extra time on > the second and subsequent runs. > > I brought up this issue with you off list much earlier this week, but > you did not respond at that time, but I hope you do that now. The > first step in your response should be to try the above bash commands > for at least the wxwidgets case on your Windows and Linux platforms. > to (a) see if this issue also occurs on Windows, and (b) see if this > issue occurs on all Linux systems accessible to you. > > @Andrew: could you please try the above time tests as well > on your Linux platforms? > > @Both: > > If the issue is a Linux resource which I am chronically short of > because of my prior wxwidgets testing and a shared memory leak (see > discussion below) in wxwidgets, then you might not be able to > replicate the issue on Linux until you do sufficient wxwidgets > testing. My Linux up time is currently 42 days, and if I reboot > (which I won't do lightly because there are two users on the system > and it is a bit of a pain for us to reset all desktops the way we like > them), I might find the issue goes away for a while. > > @Phil: > > If the issue is a Linux-only one, then an obvious candidate for a > system resource grabber is the shm_open calls in > drivers/wxwidgets_comms.cpp that helps to allow shared memory access > between the wxwidgets device driver and wxPLViewer. I have just now > read the shm_overview and shm_open/shm_unlink man pages, and it > appears that shm_unlink must be called by _both_ the wxwidgets device > driver and wxPLViewer to properly release the shared memory. So a > code review for those two cases to make sure that _always_ happens > (i.e., there is no shared memory leak) is indicated. > > @Andrew: will you comment further here please about whether we should > be concerned about possible shared memory leaks? I now realize, for > example, that both Phil's and my calling the IPC method that is used > "shared memory" is likely not specific enough because it does not > distinguish the old-fashioned shmget method (which we do not use) with > the modern mmap-based method that we do use. See > <http://stackoverflow.com/questions/5656530/how-to-use-shared-memory-with-linux-in-c> > for comments on this important "shared-memory" distinction. Furthermore, > from the discussion at > <http://stackoverflow.com/questions/22691621/how-to-avoid-shared-memory-leaks> > it appears the mmap-based method cannot actually leak memory. Is that > your interpretation of those remarks as well? > > @Phil: > > One thing I noticed from reading those man pages is it is suggested > the name used for shm_open should always start with a "/" for the most > portable results. It appears our current wxwidgets code does not > follow that suggestion so this may affect Unix platforms that are not > Linux. So I suggest you will want to address that issue (even though > it has nothing to do with timing. > > > Alan > __________________________ > Alan W. Irwin > > Astronomical research affiliation with Department of Physics and Astronomy, > University of Victoria (astrowww.phys.uvic.ca). > > Programming affiliations with the FreeEOS equation-of-state > implementation for stellar interiors (freeeos.sf.net); the Time > Ephemerides project (timeephem.sf.net); PLplot scientific plotting > software package (plplot.sf.net); the libLASi project > (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); > and the Linux Brochure Project (lbproject.sf.net). > __________________________ > > Linux-powered Science > __________________________ ------------------------------------------------------------------------------ Don't Limit Your Business. Reach for the Cloud. GigeNET's Cloud Solutions provide you with the tools and support that you need to offload your IT needs and focus on growing your business. Configured For All Businesses. Start Your Cloud Today. https://www.gigenetcloud.com/ _______________________________________________ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
