Hi Alan. Just a quick reply, sorry for top posting, but I'm replying on my 
phone.

The text dominated case idle issues doesn't surprise me. As I said this 
requires 2 way comms, which is does involve both executables sitting in while 
sleep loops. I think I can fix this and I will look to do so asap. The line 
issue surprises me a bit. I will try to look at the cause.

Phil

-----Original Message-----
From: "Alan W. Irwin" <ir...@beluga.phys.uvic.ca>
Sent: ‎02/‎01/‎2016 23:17
To: "Phil Rosenberg" <p.d.rosenb...@gmail.com>
Cc: "PLplot development list" <Plplot-devel@lists.sourceforge.net>
Subject: RE: Linux wxwidgets idle issue not solved for text- or 
line-dominatedplots

On 2016-01-02 16:28-0000 Phil Rosenberg wrote:

> I have reproduced your timings on windows with the command
> time ./x20.exe -dev wxwidgets -np. This gives me a similar ~23s real
time and a fraction of a second user and sys time. Note that this is
run from Cygwin bash shell, but x20.exe is built using visual studio.

Example 20 is not the one I measured since that is a special case in
many ways.  Instead, I ran the test_c_wxwidgets target which runs a
list of C examples for

-dev wxwidgets -np

That list of examples is given in
plplot_test/test_c_interactive.sh.in. For my previous tests I locally
modified that list here to be "01 04 08 14 16 24 30" for all devices,
but you can locally modify it to be any list of examples you want. For
example, you could modify it to "01 01 01 01 01 01 .... " to get good
timing numbers for a short example such as example 1, see below. Also,
your system will have very different speed than mine.  So what is
important is not comparisons with my numbers, but instead what
fraction of time is spent with _both_ example and wxPLViewer
application being idle (i.e., both in wait mode) compared to the
fraction of time when one of them is using the cpu.  This "idle" issue
is a substantial problem for the Linux case, and it is currently
unknown whether that is a substantial problem for the Windows case
until you do the equivalent of the Linux experiments I did below.

Note, I have now concluded you cannot reliably estimate the fraction
of time in the idle state using the Linux "time" command (or
equivalent Windows command-line app) because it only gathers time
statistics for the example it is given and obviously cannot account,
(say) for separate processes such as wxPLViewer. (So that makes its
user and system cpu estimates too small because they do not include
the wxPLViewer numbers for those, but the real time is reliable (as
also measured by a nearby clock) because wxPLViewer exits before the example
exits.)

So I redid the above test for the above list of examples, and found
the overall time spent on -dev wxwidgets + wxPLViewer is still an
order of magnitude larger than for the -dev qtwidget case (29 seconds
versus 3.6 seconds).  Some of that could be that the Linux wxwidgets
library might just be slower that the Linux Qt library. But I think
that ratio is unlikely to be such a large factor because ultimately
both libraries simply organize calls to the X library which does all
the real work on Linux.  Furthermore, while I was running those
comparisons I looked at a GUI of overall cpu usage, and for -dev
qtwidget, the cpu usage was 100 per cent during the time of the test.
However, for the combination of -dev wxwidgets + wxPLViewer the total
cpu usage varied markedly from example to example. So from this result
it appears that some examples have a mix of PLplot commands that don't
trigger idle issues on Linux, but many others do.

So I looked further at GUI cpu usage results (with the GUI expanded to
the whole page so I could look at results with good resolution) for
repeated (i.e., 01 01 01, etc.) examples for -dev wxwidgets with the
following results for average total cpu usage percentage (these
numbers are roughly good to +/- 05 per cent since I am using an
average value estimated by eye from a graph and subtracting a ~5 per
cent baseline for what the average graph level looks like when the
example is not running.  I presume this background level I subtract is
due to the GUI itself taking some CPU time to display the results as a
continously upgraded graph).  Also, note I have two cpus and for most
cases the plot showed the OS distributed the load 50-50 between them
so the numbers below refer to the average individual cpu usage for
each cpu.  i.e., 100 per cent minus the numbers tabulated below is
what I state in the 3rd column which is the fraction of the time the
entire system was completely idle (everything in wait state) while
running these PLplot examples.

example cpu %  idle %   dominant type of graphics

00      30 %   70 %     lines
01      30 %   70 %     lines
02      30 %   70 %     lines
03      25 %   75 %     lines
04      25 %   75 %     lines
05      20 %   80 %     lines
06      20 %   80 %     text
07      15 %   85 %     text
08      90 %   10 %     fills
15      40 %   60 %     lines
16      75 %   25 %     fills

For most of these examples I also looked at detailed cpu usage graphs
for -dev xwin and -dev qtwidget and they were all similar, i.e., above
90 per cent cpu usage with essentially no idle issue at all.

So it is pretty clear from the above wxwidgets Linux results there is
a severe idle issue for text, still fairly severe for lines, and
essentially no issue at all for fills (or else the issue is swamped by
some other wxwidgets fill inefficiency since in the case of repeated example 8,
-dev xwin, qtwidget, and qtwidgets took a total real time of 6, 5, and
35 seconds.  And the corresponding numbers for repeated example 16 were 9, 7,
and 26 seconds.  Note also that the example 20 you have been looking
at also may not have a perceptible "idle" issue because that example
is virtually all fills as well.

Assuming you can get access to a Windows GUI that plots total cpu usage as a
function of time, could you run similar tests to the above for a
selection of text, line, and fill examples for -dev wxwidgets and -dev
wingcc?  Typical ratios of total time taken for wxwidgets versus wingcc
for some/all of the examples would be useful as well.

> I tried quite hard to locate the source of the delay. There are a
few sleep calls while I have to wait for the viewer.

It only takes one sleep call in a loop to make both example and
wxwidgets idle a large fraction of the time....

> I tried the following tests to try to isolate the problem. With
every wxWidget driver callback returning immediately so the driver
does nothing, I still get a real time of ~5s. So even when the
wxWidgets driver does nothing there is still some unaccounted for
time.

Example 20, has some overhead due to reading in the Lena file so that
is an atypical example I would avoid from that regard.  Also, it
apparently is fill-dominated so if you are trying to deal with the
idle issue, that is not the example to use from my result about
that fill-dominated examples don't have perceptible idle issues.

> If I allow initialisation to occur the real time jumps to ~12
seconds. This presumably involves some time waiting for the viewer to
initialise and for the shared memory to be allocated.

> Allowing everything other than getting the size of text ups the time to 
> around 15s.

> Enabling getting text size ups the total to 23s again.

> So on my system I seem to have the following approximate timing breakdown
> 5s plplot core
> 7s wxViewer initialisation
> 8s getting text size
> 3s everything else wx related
>

> The reason getting text size takes so long I that it requests the
size from the viewer and that two way comms takes a lot of time.
However, I have just realised how I can do that without the comma, so
I should be able to eliminate that. The rest, well I'm not sure.

Disabling parts of the wxwidgets device as you have done is a useful
idea, but you have to break down the real time numbers above into the
part that is cpu and the part that is idle time to make this idea
truly effective.  So something like the experiments I did above for
the Linux case is required in addition to your disabling parts of
wxwidgets. If it turns out such experiments show there are no
wxwidgets idle issues on Windows, then that narrows down what you have
to look at to just the code that is run only for the Linux case.

So to summarize this, my recent experiments show that text- and
line-dominated examples have a pretty severe idle issue on Linux while
fill-dominated examples show no perceptible idle issue.  So the
highest priority question at the moment is whether or not there is a
similar idle issue on Windows. Once we definitely know that, then it
makes it much easier for us to decide on the best strategy for
tracking down and fixing the idle issue.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
------------------------------------------------------------------------------
_______________________________________________
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel

Reply via email to