Re: [JPP-Devel] A new ThreadQueue
Okay, I've commit it. The new thread Larry started recently will be a better place for the other issue. Thanks for all your patience! :) Kind regards, Sascha Michaël Michaud schrieb: if you both vote for a commit, then commit (from my perspective Michael would be the other person for an approvement - but probably he is busy - and we both trust you both :o) Yes definetely. I hardly followed your interesting discussion about ThreadQueue but I'm sure you have good reasons to do this change and I'm grateful if you could improve this dark part of the code. Michaël Let's start a new thread that works on rendering changes. I'm open to all solutions, but some of them may take a little longer to incorporate than others. When he is ready, we should evaluate Landon's pluggable rendering system to see if it can be part of the solution. I'm afraid I never really understood what class of problems it was targeted at solving. ;) ..an exercise??? stefan I'll test the less radical ZoomToSelectedItemsPlugIn solution. We may need to make the RenderManager repaint timer duration get and set-able so that we can synchronize better. I'll also take a look at your proposed rendering system changes, but I would also like to investigate a different solution first. I'll report back soon. regards, Larry On 6/20/07, *Sascha L. Teichmann* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi! Larry Becker schrieb: Thanks Stefan. We appreciate other voices in what was becoming a dialog. We're talking about a very small technical detail here so nobody who is only concerned in the 'big plot' is willing to follow. My fault. I should talk more about brand new, cool, and hyper-ultra features for the end users ... I agree with your two problems as stated. 1 - We cannot determine exactly when a rendering has ended (successfully or not). 2 - We have a javax,swing.Timer in RenderingManager that triggers a repaint any x ms. First of all let me state that we are not talking about 'A new ThreadQueue' any more. The problems we discussing now exist with the old and the new one. See below. My main goal in this thread was to bring on a new ThreadQueue. Are we through with this? If yes, I will commit it. I'm not sure that changing the repaint timer setting from 1000 to 400 affects performance significantly. My benchmarks show a 3% increase in render time for burlulc. I'm not sure if that will be true for all platforms and CPUs. There could be some advantage from my Dell's P4 hyper-threading here. I'll have to do more tests on a single CPU to be sure. I did drop the time to 40 ms and got a 22% increase. At that level, Jon's comment about too little happening each repaint rang true. Sorry Larry, setting the the value to 400ms does not solve the problem on my lower end machine. The flash is regularly omitted if I zoom to smaller scales. All the tweaking you are suggesting seems to scale with your horse power (only). A 400 ms repaint timer doesn't solve the problem, but it makes it bearable for zoom to selected. I can't speak to how this affects the printing plugins. For databases and WMSLayers, it isn't the solution. Exactly. And you may be supprised: there are people out there who want to print WMS layers too. Ask Jukka! Are the printing plugins working around this problem now by long delays or what? The Print/Layout plug-in offers an option to set a wait time for this kind of layers with a comment like Sorry dear user, we aren't capable to figure out how long it may take to print this layer. Try several value. Do some nested intervals. Do a little trial and error. IMHO this isn't very cool. Same for zooming. To avoid expensive data import into the Print/Layout plug-in we offer the opportunity to insert previews (screen shots) of the map. Later when the real print is done the data needs to be extracted from the GIS. This involves a zooming to a bookmarked position. Zooming is not reliable. So we have the same waiting options here with the same cheesy comment Sorry, dear user ... I can live with this. As I said a long time ago at the very beginning of this thread: I can explain this to customers, I'm able proof the quirks and I'm also able to show possible solutions. The real interesting question (to me) is: Is it possible in OpenJUMP to correct little design flaws like this? Even if we have to bend the original design a bit? I'll have a keen look at Landons pluggable renderer stuff ... I'm trying to assess the impact of not fixing this issue right now. I don't want to minimize the problem, but I see other more pressing
Re: [JPP-Devel] A new ThreadQueue
Hi Sascha, I agree that the new ThreadQueue is a go. I vote for commit. Let's start a new thread that works on rendering changes. I'm open to all solutions, but some of them may take a little longer to incorporate than others. When he is ready, we should evaluate Landon's pluggable rendering system to see if it can be part of the solution. I'm afraid I never really understood what class of problems it was targeted at solving. I'll test the less radical ZoomToSelectedItemsPlugIn solution. We may need to make the RenderManager repaint timer duration get and set-able so that we can synchronize better. I'll also take a look at your proposed rendering system changes, but I would also like to investigate a different solution first. I'll report back soon. regards, Larry On 6/20/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi! Larry Becker schrieb: Thanks Stefan. We appreciate other voices in what was becoming a dialog. We're talking about a very small technical detail here so nobody who is only concerned in the 'big plot' is willing to follow. My fault. I should talk more about brand new, cool, and hyper-ultra features for the end users ... I agree with your two problems as stated. 1 - We cannot determine exactly when a rendering has ended (successfully or not). 2 - We have a javax,swing.Timer in RenderingManager that triggers a repaint any x ms. First of all let me state that we are not talking about 'A new ThreadQueue' any more. The problems we discussing now exist with the old and the new one. See below. My main goal in this thread was to bring on a new ThreadQueue. Are we through with this? If yes, I will commit it. I'm not sure that changing the repaint timer setting from 1000 to 400 affects performance significantly. My benchmarks show a 3% increase in render time for burlulc. I'm not sure if that will be true for all platforms and CPUs. There could be some advantage from my Dell's P4 hyper-threading here. I'll have to do more tests on a single CPU to be sure. I did drop the time to 40 ms and got a 22% increase. At that level, Jon's comment about too little happening each repaint rang true. Sorry Larry, setting the the value to 400ms does not solve the problem on my lower end machine. The flash is regularly omitted if I zoom to smaller scales. All the tweaking you are suggesting seems to scale with your horse power (only). A 400 ms repaint timer doesn't solve the problem, but it makes it bearable for zoom to selected. I can't speak to how this affects the printing plugins. For databases and WMSLayers, it isn't the solution. Exactly. And you may be supprised: there are people out there who want to print WMS layers too. Ask Jukka! Are the printing plugins working around this problem now by long delays or what? The Print/Layout plug-in offers an option to set a wait time for this kind of layers with a comment like Sorry dear user, we aren't capable to figure out how long it may take to print this layer. Try several value. Do some nested intervals. Do a little trial and error. IMHO this isn't very cool. Same for zooming. To avoid expensive data import into the Print/Layout plug-in we offer the opportunity to insert previews (screen shots) of the map. Later when the real print is done the data needs to be extracted from the GIS. This involves a zooming to a bookmarked position. Zooming is not reliable. So we have the same waiting options here with the same cheesy comment Sorry, dear user ... I can live with this. As I said a long time ago at the very beginning of this thread: I can explain this to customers, I'm able proof the quirks and I'm also able to show possible solutions. The real interesting question (to me) is: Is it possible in OpenJUMP to correct little design flaws like this? Even if we have to bend the original design a bit? I'll have a keen look at Landons pluggable renderer stuff ... I'm trying to assess the impact of not fixing this issue right now. I don't want to minimize the problem, but I see other more pressing issues of speed and usability that need my attention right now. I need to work on the speed of Task Loading, and I found a new issue today working with very large datasets. It takes a _very_ long time to delete 50,000 objects. I see big gains with minimal effort expended in these areas. This is fine to me and I appreciate this. I have other things for the 'big plot'(tm) in the pipe too (the improved WMS support e.g). I'm the last who wants to throttle you with unimportant things. :-) So Sascha, what is your other suggestion for a less radical solution? Larry, the modifications I'd suggested were very easy to implement. I removed the Timer in the RenderingManager, adjusted the Renderers and made the RenderingManager a RenderingMaster which is able to track the rendering. Under [1] you find the according src tree. I left out the docs/libs/dlls and so on. If
Re: [JPP-Devel] A new ThreadQueue
Hei Sascha and Larry, i really appreciate your conversation (dialog?) because they are done i a way that aims to be clear .. although i don't understand anytime the indeept things, i think i could learn a lot without looking on code but back to business.. Larry Becker schrieb: Hi Sascha, I agree that the new ThreadQueue is a go. I vote for commit. if you both vote for a commit, then commit (from my perspective Michael would be the other person for an approvement - but probably he is busy - and we both trust you both :o) Let's start a new thread that works on rendering changes. I'm open to all solutions, but some of them may take a little longer to incorporate than others. When he is ready, we should evaluate Landon's pluggable rendering system to see if it can be part of the solution. I'm afraid I never really understood what class of problems it was targeted at solving. ;) ..an exercise??? stefan I'll test the less radical ZoomToSelectedItemsPlugIn solution. We may need to make the RenderManager repaint timer duration get and set-able so that we can synchronize better. I'll also take a look at your proposed rendering system changes, but I would also like to investigate a different solution first. I'll report back soon. regards, Larry On 6/20/07, *Sascha L. Teichmann* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi! Larry Becker schrieb: Thanks Stefan. We appreciate other voices in what was becoming a dialog. We're talking about a very small technical detail here so nobody who is only concerned in the 'big plot' is willing to follow. My fault. I should talk more about brand new, cool, and hyper-ultra features for the end users ... I agree with your two problems as stated. 1 - We cannot determine exactly when a rendering has ended (successfully or not). 2 - We have a javax,swing.Timer in RenderingManager that triggers a repaint any x ms. First of all let me state that we are not talking about 'A new ThreadQueue' any more. The problems we discussing now exist with the old and the new one. See below. My main goal in this thread was to bring on a new ThreadQueue. Are we through with this? If yes, I will commit it. I'm not sure that changing the repaint timer setting from 1000 to 400 affects performance significantly. My benchmarks show a 3% increase in render time for burlulc. I'm not sure if that will be true for all platforms and CPUs. There could be some advantage from my Dell's P4 hyper-threading here. I'll have to do more tests on a single CPU to be sure. I did drop the time to 40 ms and got a 22% increase. At that level, Jon's comment about too little happening each repaint rang true. Sorry Larry, setting the the value to 400ms does not solve the problem on my lower end machine. The flash is regularly omitted if I zoom to smaller scales. All the tweaking you are suggesting seems to scale with your horse power (only). A 400 ms repaint timer doesn't solve the problem, but it makes it bearable for zoom to selected. I can't speak to how this affects the printing plugins. For databases and WMSLayers, it isn't the solution. Exactly. And you may be supprised: there are people out there who want to print WMS layers too. Ask Jukka! Are the printing plugins working around this problem now by long delays or what? The Print/Layout plug-in offers an option to set a wait time for this kind of layers with a comment like Sorry dear user, we aren't capable to figure out how long it may take to print this layer. Try several value. Do some nested intervals. Do a little trial and error. IMHO this isn't very cool. Same for zooming. To avoid expensive data import into the Print/Layout plug-in we offer the opportunity to insert previews (screen shots) of the map. Later when the real print is done the data needs to be extracted from the GIS. This involves a zooming to a bookmarked position. Zooming is not reliable. So we have the same waiting options here with the same cheesy comment Sorry, dear user ... I can live with this. As I said a long time ago at the very beginning of this thread: I can explain this to customers, I'm able proof the quirks and I'm also able to show possible solutions. The real interesting question (to me) is: Is it possible in OpenJUMP to correct little design flaws like this? Even if we have to bend the original design a bit? I'll have a keen look at Landons pluggable renderer stuff ... I'm trying to assess the impact of not fixing this issue right now. I don't want to minimize the problem, but I
Re: [JPP-Devel] A new ThreadQueue
if you both vote for a commit, then commit (from my perspective Michael would be the other person for an approvement - but probably he is busy - and we both trust you both :o) Yes definetely. I hardly followed your interesting discussion about ThreadQueue but I'm sure you have good reasons to do this change and I'm grateful if you could improve this dark part of the code. Michaël Let's start a new thread that works on rendering changes. I'm open to all solutions, but some of them may take a little longer to incorporate than others. When he is ready, we should evaluate Landon's pluggable rendering system to see if it can be part of the solution. I'm afraid I never really understood what class of problems it was targeted at solving. ;) ..an exercise??? stefan I'll test the less radical ZoomToSelectedItemsPlugIn solution. We may need to make the RenderManager repaint timer duration get and set-able so that we can synchronize better. I'll also take a look at your proposed rendering system changes, but I would also like to investigate a different solution first. I'll report back soon. regards, Larry On 6/20/07, *Sascha L. Teichmann* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi! Larry Becker schrieb: Thanks Stefan. We appreciate other voices in what was becoming a dialog. We're talking about a very small technical detail here so nobody who is only concerned in the 'big plot' is willing to follow. My fault. I should talk more about brand new, cool, and hyper-ultra features for the end users ... I agree with your two problems as stated. 1 - We cannot determine exactly when a rendering has ended (successfully or not). 2 - We have a javax,swing.Timer in RenderingManager that triggers a repaint any x ms. First of all let me state that we are not talking about 'A new ThreadQueue' any more. The problems we discussing now exist with the old and the new one. See below. My main goal in this thread was to bring on a new ThreadQueue. Are we through with this? If yes, I will commit it. I'm not sure that changing the repaint timer setting from 1000 to 400 affects performance significantly. My benchmarks show a 3% increase in render time for burlulc. I'm not sure if that will be true for all platforms and CPUs. There could be some advantage from my Dell's P4 hyper-threading here. I'll have to do more tests on a single CPU to be sure. I did drop the time to 40 ms and got a 22% increase. At that level, Jon's comment about too little happening each repaint rang true. Sorry Larry, setting the the value to 400ms does not solve the problem on my lower end machine. The flash is regularly omitted if I zoom to smaller scales. All the tweaking you are suggesting seems to scale with your horse power (only). A 400 ms repaint timer doesn't solve the problem, but it makes it bearable for zoom to selected. I can't speak to how this affects the printing plugins. For databases and WMSLayers, it isn't the solution. Exactly. And you may be supprised: there are people out there who want to print WMS layers too. Ask Jukka! Are the printing plugins working around this problem now by long delays or what? The Print/Layout plug-in offers an option to set a wait time for this kind of layers with a comment like Sorry dear user, we aren't capable to figure out how long it may take to print this layer. Try several value. Do some nested intervals. Do a little trial and error. IMHO this isn't very cool. Same for zooming. To avoid expensive data import into the Print/Layout plug-in we offer the opportunity to insert previews (screen shots) of the map. Later when the real print is done the data needs to be extracted from the GIS. This involves a zooming to a bookmarked position. Zooming is not reliable. So we have the same waiting options here with the same cheesy comment Sorry, dear user ... I can live with this. As I said a long time ago at the very beginning of this thread: I can explain this to customers, I'm able proof the quirks and I'm also able to show possible solutions. The real interesting question (to me) is: Is it possible in OpenJUMP to correct little design flaws like this? Even if we have to bend the original design a bit? I'll have a keen look at Landons pluggable renderer stuff ... I'm trying to assess the impact of not fixing this issue right now. I don't want to minimize the problem, but I see other more pressing issues of speed and usability that need my attention right now. I need to work on the speed of Task Loading, and I found a new issue today working with very large datasets. It
Re: [JPP-Devel] A new ThreadQueue
except the ones from pirol I am not really aware of other renderers...
but compatibility is (sometimes unfortunately) one of our goals
stefan
Sascha L. Teichmann schrieb:
Hi Larry,
this all fits together very well and I have
a couple of comments/ideas. This is most radical one:
Let me switch to software designer mode for a moment.
(This is what my company is paying me for besides working
with legacy code... ;-)
We have actually two problems:
1 - We cannot determine exactly when a rendering
has ended (successfully or not).
This is important for e.g. printing and zooming+flashing.
The 'wakeup' Runnable trick only works if all jobs are
processed in a serial manner. It fails when DB or WMS layers
are involved because these are processed truly parallel.
What we really want is some kind of a reliable barrier
mechanism that triggers certain things after a rendering.
(unlocking a wait in the printing case and flashing in case of
zoom to selection)
2 - We have a javax,swing.Timer in RenderingManager
that triggers a repaint any x ms.
To what value we should set x?
A x too large makes the the GUI feels sluggish. x choosen too
small costs a lot of performance. I've got interesting profiling
numbers from JProbe that told me that the copyTo() stuff can
take up to 25% of the rendering time when you set x too low.
If we ask how we have to set x, we're asking the wrong question.
The question should state:
Has a renderer made enough progress to do a repaint?
If the on screen differences between two repaints would be too
small, why should we waste time to do a redraw? Only a significant
change (say a quarter of features drawn more) is worth this expense.
With a fixed scheduled timer you cannot test this efficiently.
This has to be done in cooperation with the particular renderer.
How to address these issues?
In my software designer dreamland i would propose the following:
1 - Remove the fixed scheduled timer
2 - Change the Renderer interface (Gulp!)
I would introduce an inner interface 'RenderingMaster':
interface RenderingMaster {
void doRepaint(Renderer thisRenderer);
void renderingDone(Renderer thisRenderer, boolean success);
}
and change the signature of createRunnable() to:
Runnable createRunnable(RenderingMaster master);
The idea of this extra callback should be pretty obvious.
Each time a renderer has produced a significant part of
output it calls master.doRepaint(this). On the implementors
side of RenderingMaster we can do a repaint.
If a renderer finished its job it calls
master.renderingDone(this, true). (false after canceling)
On the implementors side of RenderingMaster we can do some
reference counting and kick some listeners when all jobs are
done.
From designer dreamland back to the reality of legacy code:
I would estimate about 1-2 days of work to refactor the core code
to use this new pattern + some testing.
The real problem arises from the fact that there is a sky
full of JUMPs and a lot of non-core Renderers out there.
This modification would be a heavy one and I
don't really dare to think about the consequences ...
Things like mouse wheel zooming may profit from this change too.
So I don't want to wipe off this idea.
Any comments?
Regards,
Sascha
PS: If you don't like it all I have some 'workaround' ideas too ...
Larry Becker schrieb:
Hi Sascha,
I have figured out what is different about rendering timing in
SkyJUMP and OpenJump. The randomly delayed drawing in OpenJump is
caused by the repaintTimer in RenderingManager. In OpenJump and JUMP it
is set to 1 second, and in SkyJUMP it is set to 400 ms. This makes
SkyJUMP rendering more responsive to support Mouse Wheel zooming,
although still somewhat random.
When the number of items withing the current window for a given layer
falls below the maxFeatures threshold, the
simpleFeatureCollectionRenderer is used, otherwise
imageCachingFeatureCollectionRenderer is used. Image Caching implies
that we have to wait for the repaintTimer to expire before all of the
results of render operations are guaranteed to be copied on screen.
When image Caching is disabled, drawing is synchronized but unresponsive
because nothing appears on the screen until the end of the process.
I'm not sure we can do anything about the repaintTimer's apparent
randomness. The whole point of having a repaint timer is to be
unsynchronized with the layer rendering process. Setting the timer
value to 400 ms seems to make it responsive enough for the
ZoomToSelectedItemsPlugIn. We'll need to do that anyway when mouse
wheel zooming in ported over.
regards,
Larry
On 6/18/07, *Larry Becker* [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] wrote:
Sascha,
I
Re: [JPP-Devel] A new ThreadQueue
Thanks Stefan. We appreciate other voices in what was becoming a dialog.
Hi Sascha,
I agree with your two problems as stated.
1 - We cannot determine exactly when a rendering
has ended (successfully or not).
2 - We have a javax,swing.Timer in RenderingManager
that triggers a repaint any x ms.
I'm not sure that changing the repaint timer setting from 1000 to 400
affects performance significantly. My benchmarks show a 3% increase in
render time for burlulc. I'm not sure if that will be true for all
platforms and CPUs. There could be some advantage from my Dell's P4
hyper-threading here. I'll have to do more tests on a single CPU to be
sure. I did drop the time to 40 ms and got a 22% increase. At that level,
Jon's comment about too little happening each repaint rang true.
A 400 ms repaint timer doesn't solve the problem, but it makes it bearable
for zoom to selected. I can't speak to how this affects the printing
plugins. For databases and WMSLayers, it isn't the solution. Are the
printing plugins working around this problem now by long delays or what?
I'm trying to assess the impact of not fixing this issue right now. I don't
want to minimize the problem, but I see other more pressing issues of speed
and usability that need my attention right now. I need to work on the speed
of Task Loading, and I found a new issue today working with very large
datasets. It takes a _very_ long time to delete 50,000 objects. I see big
gains with minimal effort expended in these areas.
So Sascha, what is your other suggestion for a less radical solution?
regards,
Larry
On 6/20/07, Stefan Steiniger [EMAIL PROTECTED] wrote:
except the ones from pirol I am not really aware of other renderers...
but compatibility is (sometimes unfortunately) one of our goals
stefan
Sascha L. Teichmann schrieb:
Hi Larry,
this all fits together very well and I have
a couple of comments/ideas. This is most radical one:
Let me switch to software designer mode for a moment.
(This is what my company is paying me for besides working
with legacy code... ;-)
We have actually two problems:
1 - We cannot determine exactly when a rendering
has ended (successfully or not).
This is important for e.g. printing and zooming+flashing.
The 'wakeup' Runnable trick only works if all jobs are
processed in a serial manner. It fails when DB or WMS layers
are involved because these are processed truly parallel.
What we really want is some kind of a reliable barrier
mechanism that triggers certain things after a rendering.
(unlocking a wait in the printing case and flashing in case of
zoom to selection)
2 - We have a javax,swing.Timer in RenderingManager
that triggers a repaint any x ms.
To what value we should set x?
A x too large makes the the GUI feels sluggish. x choosen too
small costs a lot of performance. I've got interesting profiling
numbers from JProbe that told me that the copyTo() stuff can
take up to 25% of the rendering time when you set x too low.
If we ask how we have to set x, we're asking the wrong question.
The question should state:
Has a renderer made enough progress to do a repaint?
If the on screen differences between two repaints would be too
small, why should we waste time to do a redraw? Only a significant
change (say a quarter of features drawn more) is worth this expense.
With a fixed scheduled timer you cannot test this efficiently.
This has to be done in cooperation with the particular renderer.
How to address these issues?
In my software designer dreamland i would propose the following:
1 - Remove the fixed scheduled timer
2 - Change the Renderer interface (Gulp!)
I would introduce an inner interface 'RenderingMaster':
interface RenderingMaster {
void doRepaint(Renderer thisRenderer);
void renderingDone(Renderer thisRenderer, boolean success);
}
and change the signature of createRunnable() to:
Runnable createRunnable(RenderingMaster master);
The idea of this extra callback should be pretty obvious.
Each time a renderer has produced a significant part of
output it calls master.doRepaint(this). On the implementors
side of RenderingMaster we can do a repaint.
If a renderer finished its job it calls
master.renderingDone(this, true). (false after canceling)
On the implementors side of RenderingMaster we can do some
reference counting and kick some listeners when all jobs are
done.
From designer dreamland back to the reality of legacy code:
I would estimate about 1-2 days of work to refactor the core code
to use this new pattern + some testing.
The real problem arises from the fact that there is a sky
full of JUMPs and a lot of non-core Renderers out there.
This modification would be a heavy one and I
don't really dare to think about
Re: [JPP-Devel] A new ThreadQueue
Hi! Larry Becker schrieb: Thanks Stefan. We appreciate other voices in what was becoming a dialog. We're talking about a very small technical detail here so nobody who is only concerned in the 'big plot' is willing to follow. My fault. I should talk more about brand new, cool, and hyper-ultra features for the end users ... I agree with your two problems as stated. 1 - We cannot determine exactly when a rendering has ended (successfully or not). 2 - We have a javax,swing.Timer in RenderingManager that triggers a repaint any x ms. First of all let me state that we are not talking about 'A new ThreadQueue' any more. The problems we discussing now exist with the old and the new one. See below. My main goal in this thread was to bring on a new ThreadQueue. Are we through with this? If yes, I will commit it. I'm not sure that changing the repaint timer setting from 1000 to 400 affects performance significantly. My benchmarks show a 3% increase in render time for burlulc. I'm not sure if that will be true for all platforms and CPUs. There could be some advantage from my Dell's P4 hyper-threading here. I'll have to do more tests on a single CPU to be sure. I did drop the time to 40 ms and got a 22% increase. At that level, Jon's comment about too little happening each repaint rang true. Sorry Larry, setting the the value to 400ms does not solve the problem on my lower end machine. The flash is regularly omitted if I zoom to smaller scales. All the tweaking you are suggesting seems to scale with your horse power (only). A 400 ms repaint timer doesn't solve the problem, but it makes it bearable for zoom to selected. I can't speak to how this affects the printing plugins. For databases and WMSLayers, it isn't the solution. Exactly. And you may be supprised: there are people out there who want to print WMS layers too. Ask Jukka! Are the printing plugins working around this problem now by long delays or what? The Print/Layout plug-in offers an option to set a wait time for this kind of layers with a comment like Sorry dear user, we aren't capable to figure out how long it may take to print this layer. Try several value. Do some nested intervals. Do a little trial and error. IMHO this isn't very cool. Same for zooming. To avoid expensive data import into the Print/Layout plug-in we offer the opportunity to insert previews (screen shots) of the map. Later when the real print is done the data needs to be extracted from the GIS. This involves a zooming to a bookmarked position. Zooming is not reliable. So we have the same waiting options here with the same cheesy comment Sorry, dear user ... I can live with this. As I said a long time ago at the very beginning of this thread: I can explain this to customers, I'm able proof the quirks and I'm also able to show possible solutions. The real interesting question (to me) is: Is it possible in OpenJUMP to correct little design flaws like this? Even if we have to bend the original design a bit? I'll have a keen look at Landons pluggable renderer stuff ... I'm trying to assess the impact of not fixing this issue right now. I don't want to minimize the problem, but I see other more pressing issues of speed and usability that need my attention right now. I need to work on the speed of Task Loading, and I found a new issue today working with very large datasets. It takes a _very_ long time to delete 50,000 objects. I see big gains with minimal effort expended in these areas. This is fine to me and I appreciate this. I have other things for the 'big plot'(tm) in the pipe too (the improved WMS support e.g). I'm the last who wants to throttle you with unimportant things. :-) So Sascha, what is your other suggestion for a less radical solution? Larry, the modifications I'd suggested were very easy to implement. I removed the Timer in the RenderingManager, adjusted the Renderers and made the RenderingManager a RenderingMaster which is able to track the rendering. Under [1] you find the according src tree. I left out the docs/libs/dlls and so on. If one is interested look at Renderer, RenderingManager, ImageCachingRenderer, ImageCachingFeatureCollectionRenderer and ZoomToSelectedItemsPlugIn. Now the less radical solution: Start a new thread in ZoomToSelectedItemsPlugIn and let it sleep for 400ms before you do the flash. Something like this has to be done in all cases because the ThreadQueue tells you about the end of a rendering thread and not if the panel is ready for a flash. This has to do with repaint()/copyTo() stuff which has to be done afterwards and takes some (machine dependent) time too. regards, Sascha [1] http://intevation.de/~teichmann/openjump/openjump-src-renderer-mod.tar.bz2 - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take
Re: [JPP-Devel] A new ThreadQueue
Hi Sascha,
I have figured out what is different about rendering timing in SkyJUMP
and OpenJump. The randomly delayed drawing in OpenJump is caused by the
repaintTimer in RenderingManager. In OpenJump and JUMP it is set to 1
second, and in SkyJUMP it is set to 400 ms. This makes SkyJUMP rendering
more responsive to support Mouse Wheel zooming, although still somewhat
random.
When the number of items withing the current window for a given layer
falls below the maxFeatures threshold, the simpleFeatureCollectionRenderer
is used, otherwise imageCachingFeatureCollectionRenderer is used. Image
Caching implies that we have to wait for the repaintTimer to expire before
all of the results of render operations are guaranteed to be copied on
screen. When image Caching is disabled, drawing is synchronized but
unresponsive because nothing appears on the screen until the end of the
process.
I'm not sure we can do anything about the repaintTimer's apparent
randomness. The whole point of having a repaint timer is to be
unsynchronized with the layer rendering process. Setting the timer value to
400 ms seems to make it responsive enough for the
ZoomToSelectedItemsPlugIn. We'll need to do that anyway when mouse wheel
zooming in ported over.
regards,
Larry
On 6/18/07, Larry Becker [EMAIL PROTECTED] wrote:
Sascha,
I replaced the ThreadQueue.Listener with the following code:
panel.getRenderingManager().getDefaultRendererThreadQueue().add(
new Runnable() {
public void run() {
try {
flash(geometries, panel);
} catch (NoninvertibleTransformException e) {};
}
});
This works great in SkyJUMP where I also used it to fix my refresh timer
and ZoomRealtime, however although it is better than the Listener in
OpenJump, it still occasionally flashes first and then zooms. Clearly there
is something wrong, but it is not in your ThreadQueue code. I'll look some
more tomorrow.
regards,
Larry
On 6/18/07, Larry Becker [EMAIL PROTECTED] wrote:
Sascha,
Don't you have the same effects with the original one?
I begin to see... I can reproduce flash problems easily in JUMP
and OpenJump, but not in SkyJUMP. That explains why we are both
surprised. I have no idea why there is a difference, but I will
investigate further.
regards,
Larry
On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Larry,
_exactly_ this the thread lottery we are playing with the
assumption that no running threads means there no more rendering jobs!
I get the same irritating behavior with the original ThreadQueue.
I put an System.err.println(flash!) into the listener of
the zoom plug-in. Sometimes it gets printed before the display
is in the right 'mood' to display a flash. Result: no visible
flash or only a shortened variant.
Don't you have the same effects with the original one?
I have!
Register a println Listener yourself to the ThreadQueue
and be surprised how often it is called.
The zoom plug-in builds on this assumption the even more venturous
assumption that the zoom is done when there no more running threads.
It does not take into account that the hole repaint()/erase()/copyTo()
stuff also takes some time. The invokeLater() call does not make it
more predictable.
Let us compare the TQs:
1) Original TQ:
public void add(final Runnable runnable) {
queuedRunnables.add(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
setRunningThreads(getRunningThreads() - 1);
processQueue();
}
}
});
processQueue();
}
private void setRunningThreads(int runningThreads) {
this.runningThreads = runningThreads;
if (runningThreads == 0) { fireAllRunningThreadsFinished(); }
}
The defaultRenderingThread has only one Thread running.
- runningThreads == 1 during try block of new Runnable.run().
setRunningThread() in the finally sets it to zero
- listeners get there kick.
This means that after each and every job the listeners get kicked.
2) New TQ: (in worker thread's run())
for (;;) {
// unimportant part
try {
runnable.run();
}
catch (Exception e) {
e.printStackTrace();
}
boolean lastRunningThread;
synchronized (runningThreads) {
lastRunningThread = runningThreads[0] == 1;
}
if (lastRunningThread) {
fireAllRunningThreadsFinished();
}
}
The defaultRenderingThread has only one Thread running.
- runningThreads[0] == 1
after the try block lastRunningThread is set to true
if runningThreads[0] == 1. This is always fulfilled for
the defaultRenderingThread.
- listeners get there kick.
Re: [JPP-Devel] A new ThreadQueue
Hi Larry,
this all fits together very well and I have
a couple of comments/ideas. This is most radical one:
Let me switch to software designer mode for a moment.
(This is what my company is paying me for besides working
with legacy code... ;-)
We have actually two problems:
1 - We cannot determine exactly when a rendering
has ended (successfully or not).
This is important for e.g. printing and zooming+flashing.
The 'wakeup' Runnable trick only works if all jobs are
processed in a serial manner. It fails when DB or WMS layers
are involved because these are processed truly parallel.
What we really want is some kind of a reliable barrier
mechanism that triggers certain things after a rendering.
(unlocking a wait in the printing case and flashing in case of
zoom to selection)
2 - We have a javax,swing.Timer in RenderingManager
that triggers a repaint any x ms.
To what value we should set x?
A x too large makes the the GUI feels sluggish. x choosen too
small costs a lot of performance. I've got interesting profiling
numbers from JProbe that told me that the copyTo() stuff can
take up to 25% of the rendering time when you set x too low.
If we ask how we have to set x, we're asking the wrong question.
The question should state:
Has a renderer made enough progress to do a repaint?
If the on screen differences between two repaints would be too
small, why should we waste time to do a redraw? Only a significant
change (say a quarter of features drawn more) is worth this expense.
With a fixed scheduled timer you cannot test this efficiently.
This has to be done in cooperation with the particular renderer.
How to address these issues?
In my software designer dreamland i would propose the following:
1 - Remove the fixed scheduled timer
2 - Change the Renderer interface (Gulp!)
I would introduce an inner interface 'RenderingMaster':
interface RenderingMaster {
void doRepaint(Renderer thisRenderer);
void renderingDone(Renderer thisRenderer, boolean success);
}
and change the signature of createRunnable() to:
Runnable createRunnable(RenderingMaster master);
The idea of this extra callback should be pretty obvious.
Each time a renderer has produced a significant part of
output it calls master.doRepaint(this). On the implementors
side of RenderingMaster we can do a repaint.
If a renderer finished its job it calls
master.renderingDone(this, true). (false after canceling)
On the implementors side of RenderingMaster we can do some
reference counting and kick some listeners when all jobs are
done.
From designer dreamland back to the reality of legacy code:
I would estimate about 1-2 days of work to refactor the core code
to use this new pattern + some testing.
The real problem arises from the fact that there is a sky
full of JUMPs and a lot of non-core Renderers out there.
This modification would be a heavy one and I
don't really dare to think about the consequences ...
Things like mouse wheel zooming may profit from this change too.
So I don't want to wipe off this idea.
Any comments?
Regards,
Sascha
PS: If you don't like it all I have some 'workaround' ideas too ...
Larry Becker schrieb:
Hi Sascha,
I have figured out what is different about rendering timing in
SkyJUMP and OpenJump. The randomly delayed drawing in OpenJump is
caused by the repaintTimer in RenderingManager. In OpenJump and JUMP it
is set to 1 second, and in SkyJUMP it is set to 400 ms. This makes
SkyJUMP rendering more responsive to support Mouse Wheel zooming,
although still somewhat random.
When the number of items withing the current window for a given layer
falls below the maxFeatures threshold, the
simpleFeatureCollectionRenderer is used, otherwise
imageCachingFeatureCollectionRenderer is used. Image Caching implies
that we have to wait for the repaintTimer to expire before all of the
results of render operations are guaranteed to be copied on screen.
When image Caching is disabled, drawing is synchronized but unresponsive
because nothing appears on the screen until the end of the process.
I'm not sure we can do anything about the repaintTimer's apparent
randomness. The whole point of having a repaint timer is to be
unsynchronized with the layer rendering process. Setting the timer
value to 400 ms seems to make it responsive enough for the
ZoomToSelectedItemsPlugIn. We'll need to do that anyway when mouse
wheel zooming in ported over.
regards,
Larry
On 6/18/07, *Larry Becker* [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] wrote:
Sascha,
I replaced the ThreadQueue.Listener with the following code:
panel.getRenderingManager().getDefaultRendererThreadQueue().add(
new Runnable() {
public void run() {
try {
Re: [JPP-Devel] A new ThreadQueue
Sascha,
I tried one second, and it feels slow. When I am arrowing through a
selection of records in View/Edit Attributes it makes me wait for the
flash before I move on. Really, this is becoming an issue of
compromising the interactivity of the application to achieve some
theoretical benefit that can't be seen or measured.
How about 400 ms? That is about the average reaction time.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Take zoom/panning as an example:
When I zoom to a certain level I often do some zooming
or panning within a few seconds again because the
viewport is not adjust the way I really want it to be.
I agree with you in principle and therefore the 'pooling' of
threads in the new ThreadQueue is only light version:
just keeping a running thread a little alive.
For say one second instead of five?!
- Sascha
Larry Becker schrieb:
Thread pooling may be important for servers, but it doesn't seem to be
a performance factor in JUMP. If no new jobs are being added in 50
ms, the cpu is probably idle anyway.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Not if you want to do thread pooling.
The real problem: How can I get a notification
when a zoom is done?
The ZoomToSelectedItemsPlugIn ThreadQueue code looks
like a workaround due to lack of a real possibility
to get informed when the zoom is done.
I will have a look at this problem.
regards,
Sascha
Larry Becker schrieb:
I cut the WORKER_STAY_ALIVE_TIME to 50 ms and the flash now works. 50
ms is an eternity in CPU time anyway.
regards,
Larry
On 6/15/07, Larry Becker [EMAIL PROTECTED] wrote:
Thanks for finding that Listener use in ZoomToSelectedItemsPlugIn. I
tried it and it doesn't flash anymore.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
In core the
com.vividsolutions.jump.workbench.ui.zoom.ZoomToSelectedItemsPlugIn
uses the ThreadQueue.Listener interface. But the code looks like it can
cope with the 'slightly' shifted semantic.
I would vote for stick a @deprecated tag (+ some explanations)
to the respective methods and to the interface because code
that uses these mechanisms is possibly not aware of the
sophisticated difference between 'no running threads left'
and 'all jobs done'.
- Sascha
Larry Becker schrieb:
Hi Sascha,
Adding a 'wakeup' Runnable works great and is easier than using the
listener anyway.
By the way, I couldn't find any other code using the Listener
interface, but I suppose there could be a Plug-in somewhere that does.
We should probably leave it out anyway since if we are leaving it in
for compatibility, it should actually be compatible. The easiest way
to find out if it is really needed by anyone but me might be to leave
it out and see if it breaks anyone's compile. The same applies to
getRunningThreads since it doesn't behave exactly like it did before
either.
Anyway, thanks for the help. I'm able to determine the end of
rendering much more accurately now.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this
Re: [JPP-Devel] A new ThreadQueue
Larry,
there is probably somebody out there (younger than us)
how says that 400ms feels slow too.
I've thought a bit about the compromise and came to the
conclusion that we don't need a make a compromise here.
We have simply to restore the behavior of the
original TheadQueue. The original one fires the
Listeners when the running threads went down to zero.
We can do the same when we're in the situation that we
are the last remaining thread with our job done.
In this case the number of running threads is
one but this measure wasn't reliable in the old
ThreadQueue too. So it doesn't matter.
But in difference to the original we keep the
worker thread alive afterwards instead of killing it.
Find attached a new version of the ThreadQueue that
implements this behavior.
regards,
Sascha
Larry Becker schrieb:
Sascha,
I tried one second, and it feels slow. When I am arrowing through a
selection of records in View/Edit Attributes it makes me wait for the
flash before I move on. Really, this is becoming an issue of
compromising the interactivity of the application to achieve some
theoretical benefit that can't be seen or measured.
How about 400 ms? That is about the average reaction time.
regards,
Larry Becker
/*
* The Unified Mapping Platform (JUMP) is an extensible, interactive GUI
* for visualizing and manipulating spatial features with geometry and attributes.
*
* Copyright (C) 2003 Vivid Solutions
* Copyright (C) 2007 Intevation GmbH
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
* Suite #1A
* 2328 Government Street
* Victoria BC V8T 5G5
* Canada
*
* (250)385-6040
* www.vividsolutions.com
*/
package com.vividsolutions.jump.workbench.ui.renderer;
import java.util.LinkedList;
import java.util.ArrayList;
/**
* This thread queue executes at maximum N Runnables in parallel
* were N is a given number of worker threads that should be used.
* If N threads are running and busy each further incoming
* Runnable is queued until one of the threads has finished its current job.
* If a worker thread becomes idle (no more job in the queue)
* it is hold alive for 5 seconds. If during this period of time
* no new Runnable is enqueued the worker thread dies.
*
* @author Sascha L. Teichmann ([EMAIL PROTECTED])
*/
public class ThreadQueue
{
/** The time a worker thread stays alive if idle */
public static final long WORKER_STAY_ALIVE_TIME = 5000L;
/**
* Worker thread. Fetches Runnable from the surrounding
* ThreadQueue instance.
*/
protected class Worker
extends Thread
{
public void run() {
try {
for (;;) {
Runnable runnable;
synchronized (queuedRunnables) {
if (queuedRunnables.isEmpty()) {
++waitingThreads;
try {
queuedRunnables.wait(WORKER_STAY_ALIVE_TIME);
}
catch (InterruptedException ie) {
}
finally {
--waitingThreads;
}
// if still empty - die!
if (queuedRunnables.isEmpty())
break;
}
if (disposed)
break;
runnable = (Runnable)queuedRunnables.remove();
} // synchronized queuedRunnables
try {
runnable.run();
}
catch (Exception e) {
e.printStackTrace();
}
// check if we are the last of the mohicans ...
boolean lastRunningThread;
synchronized (runningThreads) {
lastRunningThread = runningThreads[0] == 1;
}
if (lastRunningThread) {
fireAllRunningThreadsFinished();
}
} // for (;;)
}
finally { // guarantee that counter goes down
synchronized (runningThreads) {
--runningThreads[0];
}
}
}
} // class Worker
/**
* If the number of running threads goes down to zero
* implementations of this interface are able to be informed.
*/
public interface Listener {
void allRunningThreadsFinished();
} // interface Listener
/** Number of running threads */
protected int [] runningThreads = new int[1];
/** max. Number of threads running parallel */
protected int maxRunningThreads;
/** Number of threads that are currently idle */
protected int waitingThreads;
/** The queue of Runnables jobs waiting to be run */
protected LinkedList queuedRunnables;
/** Singals that the ThreadQueue is going to quit */
protected boolean disposed;
/** List
Re: [JPP-Devel] A new ThreadQueue
Sascha, Thanks for your patience. I like the idea of preserving the original behavior, however this version doesn't seem to flash consistently. Sometimes it doesn't flash, sometimes it does. regards, Larry On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Larry, there is probably somebody out there (younger than us) how says that 400ms feels slow too. I've thought a bit about the compromise and came to the conclusion that we don't need a make a compromise here. We have simply to restore the behavior of the original TheadQueue. The original one fires the Listeners when the running threads went down to zero. We can do the same when we're in the situation that we are the last remaining thread with our job done. In this case the number of running threads is one but this measure wasn't reliable in the old ThreadQueue too. So it doesn't matter. But in difference to the original we keep the worker thread alive afterwards instead of killing it. Find attached a new version of the ThreadQueue that implements this behavior. regards, Sascha Larry Becker schrieb: Sascha, I tried one second, and it feels slow. When I am arrowing through a selection of records in View/Edit Attributes it makes me wait for the flash before I move on. Really, this is becoming an issue of compromising the interactivity of the application to achieve some theoretical benefit that can't be seen or measured. How about 400 ms? That is about the average reaction time. regards, Larry Becker - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel -- http://amusingprogrammer.blogspot.com/ - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] A new ThreadQueue
Larry,
_exactly_ this the thread lottery we are playing with the
assumption that no running threads means there no more rendering jobs!
I get the same irritating behavior with the original ThreadQueue.
I put an System.err.println(flash!) into the listener of
the zoom plug-in. Sometimes it gets printed before the display
is in the right 'mood' to display a flash. Result: no visible
flash or only a shortened variant.
Don't you have the same effects with the original one?
I have!
Register a println Listener yourself to the ThreadQueue
and be surprised how often it is called.
The zoom plug-in builds on this assumption the even more venturous
assumption that the zoom is done when there no more running threads.
It does not take into account that the hole repaint()/erase()/copyTo()
stuff also takes some time. The invokeLater() call does not make it
more predictable.
Let us compare the TQs:
1) Original TQ:
public void add(final Runnable runnable) {
queuedRunnables.add(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
setRunningThreads(getRunningThreads() - 1);
processQueue();
}
}
});
processQueue();
}
private void setRunningThreads(int runningThreads) {
this.runningThreads = runningThreads;
if (runningThreads == 0) { fireAllRunningThreadsFinished(); }
}
The defaultRenderingThread has only one Thread running.
- runningThreads == 1 during try block of new Runnable.run().
setRunningThread() in the finally sets it to zero
- listeners get there kick.
This means that after each and every job the listeners get kicked.
2) New TQ: (in worker thread's run())
for (;;) {
// unimportant part
try {
runnable.run();
}
catch (Exception e) {
e.printStackTrace();
}
boolean lastRunningThread;
synchronized (runningThreads) {
lastRunningThread = runningThreads[0] == 1;
}
if (lastRunningThread) {
fireAllRunningThreadsFinished();
}
}
The defaultRenderingThread has only one Thread running.
- runningThreads[0] == 1
after the try block lastRunningThread is set to true
if runningThreads[0] == 1. This is always fulfilled for
the defaultRenderingThread.
- listeners get there kick.
This means that after each and every job the listeners get kicked.
= Same behavior for 1) and 2)
Maybe I bore you a bit by repeating it.
Regards,
Sascha
Larry Becker schrieb:
Sascha,
Thanks for your patience. I like the idea of preserving the
original behavior, however this version doesn't seem to flash
consistently. Sometimes it doesn't flash, sometimes it does.
regards,
Larry
On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Larry,
there is probably somebody out there (younger than us)
how says that 400ms feels slow too.
I've thought a bit about the compromise and came to the
conclusion that we don't need a make a compromise here.
We have simply to restore the behavior of the
original TheadQueue. The original one fires the
Listeners when the running threads went down to zero.
We can do the same when we're in the situation that we
are the last remaining thread with our job done.
In this case the number of running threads is
one but this measure wasn't reliable in the old
ThreadQueue too. So it doesn't matter.
But in difference to the original we keep the
worker thread alive afterwards instead of killing it.
Find attached a new version of the ThreadQueue that
implements this behavior.
regards,
Sascha
Larry Becker schrieb:
Sascha,
I tried one second, and it feels slow. When I am arrowing through a
selection of records in View/Edit Attributes it makes me wait for the
flash before I move on. Really, this is becoming an issue of
compromising the interactivity of the application to achieve some
theoretical benefit that can't be seen or measured.
How about 400 ms? That is about the average reaction time.
regards,
Larry Becker
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
Re: [JPP-Devel] A new ThreadQueue
Sascha,
Don't you have the same effects with the original one?
I begin to see... I can reproduce flash problems easily in JUMP
and OpenJump, but not in SkyJUMP. That explains why we are both
surprised. I have no idea why there is a difference, but I will
investigate further.
regards,
Larry
On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Larry,
_exactly_ this the thread lottery we are playing with the
assumption that no running threads means there no more rendering jobs!
I get the same irritating behavior with the original ThreadQueue.
I put an System.err.println(flash!) into the listener of
the zoom plug-in. Sometimes it gets printed before the display
is in the right 'mood' to display a flash. Result: no visible
flash or only a shortened variant.
Don't you have the same effects with the original one?
I have!
Register a println Listener yourself to the ThreadQueue
and be surprised how often it is called.
The zoom plug-in builds on this assumption the even more venturous
assumption that the zoom is done when there no more running threads.
It does not take into account that the hole repaint()/erase()/copyTo()
stuff also takes some time. The invokeLater() call does not make it
more predictable.
Let us compare the TQs:
1) Original TQ:
public void add(final Runnable runnable) {
queuedRunnables.add(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
setRunningThreads(getRunningThreads() - 1);
processQueue();
}
}
});
processQueue();
}
private void setRunningThreads(int runningThreads) {
this.runningThreads = runningThreads;
if (runningThreads == 0) { fireAllRunningThreadsFinished(); }
}
The defaultRenderingThread has only one Thread running.
- runningThreads == 1 during try block of new Runnable.run().
setRunningThread() in the finally sets it to zero
- listeners get there kick.
This means that after each and every job the listeners get kicked.
2) New TQ: (in worker thread's run())
for (;;) {
// unimportant part
try {
runnable.run();
}
catch (Exception e) {
e.printStackTrace();
}
boolean lastRunningThread;
synchronized (runningThreads) {
lastRunningThread = runningThreads[0] == 1;
}
if (lastRunningThread) {
fireAllRunningThreadsFinished();
}
}
The defaultRenderingThread has only one Thread running.
- runningThreads[0] == 1
after the try block lastRunningThread is set to true
if runningThreads[0] == 1. This is always fulfilled for
the defaultRenderingThread.
- listeners get there kick.
This means that after each and every job the listeners get kicked.
= Same behavior for 1) and 2)
Maybe I bore you a bit by repeating it.
Regards,
Sascha
Larry Becker schrieb:
Sascha,
Thanks for your patience. I like the idea of preserving the
original behavior, however this version doesn't seem to flash
consistently. Sometimes it doesn't flash, sometimes it does.
regards,
Larry
On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Larry,
there is probably somebody out there (younger than us)
how says that 400ms feels slow too.
I've thought a bit about the compromise and came to the
conclusion that we don't need a make a compromise here.
We have simply to restore the behavior of the
original TheadQueue. The original one fires the
Listeners when the running threads went down to zero.
We can do the same when we're in the situation that we
are the last remaining thread with our job done.
In this case the number of running threads is
one but this measure wasn't reliable in the old
ThreadQueue too. So it doesn't matter.
But in difference to the original we keep the
worker thread alive afterwards instead of killing it.
Find attached a new version of the ThreadQueue that
implements this behavior.
regards,
Sascha
Larry Becker schrieb:
Sascha,
I tried one second, and it feels slow. When I am arrowing through a
selection of records in View/Edit Attributes it makes me wait for the
flash before I move on. Really, this is becoming an issue of
compromising the interactivity of the application to achieve some
theoretical benefit that can't be seen or measured.
How about 400 ms? That is about the average reaction time.
regards,
Larry Becker
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
Re: [JPP-Devel] A new ThreadQueue
Sascha,
I replaced the ThreadQueue.Listener with the following code:
panel.getRenderingManager().getDefaultRendererThreadQueue().add(
new Runnable() {
public void run() {
try {
flash(geometries, panel);
} catch (NoninvertibleTransformException e) {};
}
});
This works great in SkyJUMP where I also used it to fix my refresh timer and
ZoomRealtime, however although it is better than the Listener in OpenJump,
it still occasionally flashes first and then zooms. Clearly there is
something wrong, but it is not in your ThreadQueue code. I'll look some
more tomorrow.
regards,
Larry
On 6/18/07, Larry Becker [EMAIL PROTECTED] wrote:
Sascha,
Don't you have the same effects with the original one?
I begin to see... I can reproduce flash problems easily in JUMP
and OpenJump, but not in SkyJUMP. That explains why we are both
surprised. I have no idea why there is a difference, but I will
investigate further.
regards,
Larry
On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Larry,
_exactly_ this the thread lottery we are playing with the
assumption that no running threads means there no more rendering jobs!
I get the same irritating behavior with the original ThreadQueue.
I put an System.err.println(flash!) into the listener of
the zoom plug-in. Sometimes it gets printed before the display
is in the right 'mood' to display a flash. Result: no visible
flash or only a shortened variant.
Don't you have the same effects with the original one?
I have!
Register a println Listener yourself to the ThreadQueue
and be surprised how often it is called.
The zoom plug-in builds on this assumption the even more venturous
assumption that the zoom is done when there no more running threads.
It does not take into account that the hole repaint()/erase()/copyTo()
stuff also takes some time. The invokeLater() call does not make it
more predictable.
Let us compare the TQs:
1) Original TQ:
public void add(final Runnable runnable) {
queuedRunnables.add(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
setRunningThreads(getRunningThreads() - 1);
processQueue();
}
}
});
processQueue();
}
private void setRunningThreads(int runningThreads) {
this.runningThreads = runningThreads;
if (runningThreads == 0) { fireAllRunningThreadsFinished(); }
}
The defaultRenderingThread has only one Thread running.
- runningThreads == 1 during try block of new Runnable.run().
setRunningThread() in the finally sets it to zero
- listeners get there kick.
This means that after each and every job the listeners get kicked.
2) New TQ: (in worker thread's run())
for (;;) {
// unimportant part
try {
runnable.run();
}
catch (Exception e) {
e.printStackTrace();
}
boolean lastRunningThread;
synchronized (runningThreads) {
lastRunningThread = runningThreads[0] == 1;
}
if (lastRunningThread) {
fireAllRunningThreadsFinished();
}
}
The defaultRenderingThread has only one Thread running.
- runningThreads[0] == 1
after the try block lastRunningThread is set to true
if runningThreads[0] == 1. This is always fulfilled for
the defaultRenderingThread.
- listeners get there kick.
This means that after each and every job the listeners get kicked.
= Same behavior for 1) and 2)
Maybe I bore you a bit by repeating it.
Regards,
Sascha
Larry Becker schrieb:
Sascha,
Thanks for your patience. I like the idea of preserving the
original behavior, however this version doesn't seem to flash
consistently. Sometimes it doesn't flash, sometimes it does.
regards,
Larry
On 6/18/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Larry,
there is probably somebody out there (younger than us)
how says that 400ms feels slow too.
I've thought a bit about the compromise and came to the
conclusion that we don't need a make a compromise here.
We have simply to restore the behavior of the
original TheadQueue. The original one fires the
Listeners when the running threads went down to zero.
We can do the same when we're in the situation that we
are the last remaining thread with our job done.
In this case the number of running threads is
one but this measure wasn't reliable in the old
ThreadQueue too. So it doesn't matter.
But in difference to the original we keep the
worker thread alive afterwards instead of killing it.
Find attached a new version of the ThreadQueue that
implements this behavior.
regards,
Sascha
Larry Becker schrieb:
Sascha,
I tried one second, and it feels slow. When I am arrowing through
a
Re: [JPP-Devel] A new ThreadQueue
Sascha, I have reached a point where I need some help with the new ThreadQueue implementation. I have modified my code that calls getRunningThreads to use the Listener with the allRunningThreadsFinished method instead. This was much cleaner and worked fine until I replaced ThreadQueue with your version. The problem I seem to be having is that the method doesn't fire until after the 5 second WORKER_STAY_ALIVE_TIME has expired. Can you explain what I should be doing or modify the code so that it fires when there are no more jobs waiting? I have attached the version of ThreadQueue.java that I have been using in case it is outdated. thanks, Larry Becker -- http://amusingprogrammer.blogspot.com/ ThreadQueue.java Description: Binary data - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] A new ThreadQueue
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this only works on
the defaultRenderingThreadQueue and therefore
its only an interim solution. The
WMS/DB ThreadQueue is private and a true
parallel ThreadQueue.
My goal is to add a renderAllSynchronized()
to RenderingManager that used the above
Runnable/unlock trick. The secret WMS/DB
will be eliminated (or better the default one
and the WMS/DB ThreadQueue will become the
default). This is doable with the
RunnableArrayList class I posted a while
ago, which simulates the single thread
default ThreadQueue. But one step after
the other ...
Regards, Sascha
Larry Becker schrieb:
Sascha,
I have reached a point where I need some help with the new
ThreadQueue implementation. I have modified my code that calls
getRunningThreads to use the Listener with the
allRunningThreadsFinished method instead. This was much cleaner and
worked fine until I replaced ThreadQueue with your version. The
problem I seem to be having is that the method doesn't fire until
after the 5 second WORKER_STAY_ALIVE_TIME has expired. Can you
explain what I should be doing or modify the code so that it fires
when there are no more jobs waiting?
I have attached the version of ThreadQueue.java that I have been using
in case it is outdated.
thanks,
Larry Becker
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
/*
* The Unified Mapping Platform (JUMP) is an extensible, interactive GUI
* for visualizing and manipulating spatial features with geometry and attributes.
*
* Copyright (C) 2003 Vivid Solutions
* Copyright (C) 2007 Intevation GmbH
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
* Suite #1A
* 2328 Government Street
* Victoria BC V8T 5G5
* Canada
*
* (250)385-6040
* www.vividsolutions.com
*/
package com.vividsolutions.jump.workbench.ui.renderer;
import java.util.LinkedList;
import java.util.ArrayList;
/**
* This thread queue executes at maximum N Runnables in parallel
* were N is a given number of worker threads that should be used.
* If N threads are running and busy each further incoming
* Runnable is queued until one of the threads has finished its
Re: [JPP-Devel] A new ThreadQueue
In core the
com.vividsolutions.jump.workbench.ui.zoom.ZoomToSelectedItemsPlugIn
uses the ThreadQueue.Listener interface. But the code looks like it can
cope with the 'slightly' shifted semantic.
I would vote for stick a @deprecated tag (+ some explanations)
to the respective methods and to the interface because code
that uses these mechanisms is possibly not aware of the
sophisticated difference between 'no running threads left'
and 'all jobs done'.
- Sascha
Larry Becker schrieb:
Hi Sascha,
Adding a 'wakeup' Runnable works great and is easier than using the
listener anyway.
By the way, I couldn't find any other code using the Listener
interface, but I suppose there could be a Plug-in somewhere that does.
We should probably leave it out anyway since if we are leaving it in
for compatibility, it should actually be compatible. The easiest way
to find out if it is really needed by anyone but me might be to leave
it out and see if it breaks anyone's compile. The same applies to
getRunningThreads since it doesn't behave exactly like it did before
either.
Anyway, thanks for the help. I'm able to determine the end of
rendering much more accurately now.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this only works on
the defaultRenderingThreadQueue and therefore
its only an interim solution. The
WMS/DB ThreadQueue is private and a true
parallel ThreadQueue.
My goal is to add a renderAllSynchronized()
to RenderingManager that used the above
Runnable/unlock trick. The secret WMS/DB
will be eliminated (or better the default one
and the WMS/DB ThreadQueue will become the
default). This is doable with the
RunnableArrayList class I posted a while
ago, which simulates the single thread
default ThreadQueue. But one step after
the other ...
Regards, Sascha
Larry Becker schrieb:
Sascha,
I have reached a point where I need some help with the new
ThreadQueue implementation. I have modified my code that calls
getRunningThreads to use the Listener with the
allRunningThreadsFinished method instead. This was much cleaner and
worked fine until I replaced ThreadQueue with your version. The
problem I seem to be having is that the method doesn't fire until
after the 5 second WORKER_STAY_ALIVE_TIME has expired. Can you
explain what I should be doing or modify the code so that it fires
when there are no more jobs waiting?
I have attached the version of ThreadQueue.java that I have been using
in case it is outdated.
thanks,
Larry Becker
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
-
This SF.net email is sponsored by DB2
Re: [JPP-Devel] A new ThreadQueue
Thanks for finding that Listener use in ZoomToSelectedItemsPlugIn. I
tried it and it doesn't flash anymore.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
In core the
com.vividsolutions.jump.workbench.ui.zoom.ZoomToSelectedItemsPlugIn
uses the ThreadQueue.Listener interface. But the code looks like it can
cope with the 'slightly' shifted semantic.
I would vote for stick a @deprecated tag (+ some explanations)
to the respective methods and to the interface because code
that uses these mechanisms is possibly not aware of the
sophisticated difference between 'no running threads left'
and 'all jobs done'.
- Sascha
Larry Becker schrieb:
Hi Sascha,
Adding a 'wakeup' Runnable works great and is easier than using the
listener anyway.
By the way, I couldn't find any other code using the Listener
interface, but I suppose there could be a Plug-in somewhere that does.
We should probably leave it out anyway since if we are leaving it in
for compatibility, it should actually be compatible. The easiest way
to find out if it is really needed by anyone but me might be to leave
it out and see if it breaks anyone's compile. The same applies to
getRunningThreads since it doesn't behave exactly like it did before
either.
Anyway, thanks for the help. I'm able to determine the end of
rendering much more accurately now.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this only works on
the defaultRenderingThreadQueue and therefore
its only an interim solution. The
WMS/DB ThreadQueue is private and a true
parallel ThreadQueue.
My goal is to add a renderAllSynchronized()
to RenderingManager that used the above
Runnable/unlock trick. The secret WMS/DB
will be eliminated (or better the default one
and the WMS/DB ThreadQueue will become the
default). This is doable with the
RunnableArrayList class I posted a while
ago, which simulates the single thread
default ThreadQueue. But one step after
the other ...
Regards, Sascha
Larry Becker schrieb:
Sascha,
I have reached a point where I need some help with the new
ThreadQueue implementation. I have modified my code that calls
getRunningThreads to use the Listener with the
allRunningThreadsFinished method instead. This was much cleaner and
worked fine until I replaced ThreadQueue with your version. The
problem I seem to be having is that the method doesn't fire until
after the 5 second WORKER_STAY_ALIVE_TIME has expired. Can you
explain what I should be doing or modify the code so that it fires
when there are no more jobs waiting?
I have attached the version of ThreadQueue.java that I have been using
in case it is outdated.
thanks,
Larry Becker
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
Re: [JPP-Devel] A new ThreadQueue
I cut the WORKER_STAY_ALIVE_TIME to 50 ms and the flash now works. 50
ms is an eternity in CPU time anyway.
regards,
Larry
On 6/15/07, Larry Becker [EMAIL PROTECTED] wrote:
Thanks for finding that Listener use in ZoomToSelectedItemsPlugIn. I
tried it and it doesn't flash anymore.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
In core the
com.vividsolutions.jump.workbench.ui.zoom.ZoomToSelectedItemsPlugIn
uses the ThreadQueue.Listener interface. But the code looks like it can
cope with the 'slightly' shifted semantic.
I would vote for stick a @deprecated tag (+ some explanations)
to the respective methods and to the interface because code
that uses these mechanisms is possibly not aware of the
sophisticated difference between 'no running threads left'
and 'all jobs done'.
- Sascha
Larry Becker schrieb:
Hi Sascha,
Adding a 'wakeup' Runnable works great and is easier than using the
listener anyway.
By the way, I couldn't find any other code using the Listener
interface, but I suppose there could be a Plug-in somewhere that does.
We should probably leave it out anyway since if we are leaving it in
for compatibility, it should actually be compatible. The easiest way
to find out if it is really needed by anyone but me might be to leave
it out and see if it breaks anyone's compile. The same applies to
getRunningThreads since it doesn't behave exactly like it did before
either.
Anyway, thanks for the help. I'm able to determine the end of
rendering much more accurately now.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this only works on
the defaultRenderingThreadQueue and therefore
its only an interim solution. The
WMS/DB ThreadQueue is private and a true
parallel ThreadQueue.
My goal is to add a renderAllSynchronized()
to RenderingManager that used the above
Runnable/unlock trick. The secret WMS/DB
will be eliminated (or better the default one
and the WMS/DB ThreadQueue will become the
default). This is doable with the
RunnableArrayList class I posted a while
ago, which simulates the single thread
default ThreadQueue. But one step after
the other ...
Regards, Sascha
Larry Becker schrieb:
Sascha,
I have reached a point where I need some help with the new
ThreadQueue implementation. I have modified my code that calls
getRunningThreads to use the Listener with the
allRunningThreadsFinished method instead. This was much cleaner and
worked fine until I replaced ThreadQueue with your version. The
problem I seem to be having is that the method doesn't fire until
after the 5 second WORKER_STAY_ALIVE_TIME has expired. Can you
explain what I should be doing or modify the code so that it fires
when there are no more jobs waiting?
I have attached the version of ThreadQueue.java that I have been using
in case it is outdated.
thanks,
Larry Becker
Re: [JPP-Devel] A new ThreadQueue
Not if you want to do thread pooling.
The real problem: How can I get a notification
when a zoom is done?
The ZoomToSelectedItemsPlugIn ThreadQueue code looks
like a workaround due to lack of a real possibility
to get informed when the zoom is done.
I will have a look at this problem.
regards,
Sascha
Larry Becker schrieb:
I cut the WORKER_STAY_ALIVE_TIME to 50 ms and the flash now works. 50
ms is an eternity in CPU time anyway.
regards,
Larry
On 6/15/07, Larry Becker [EMAIL PROTECTED] wrote:
Thanks for finding that Listener use in ZoomToSelectedItemsPlugIn. I
tried it and it doesn't flash anymore.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
In core the
com.vividsolutions.jump.workbench.ui.zoom.ZoomToSelectedItemsPlugIn
uses the ThreadQueue.Listener interface. But the code looks like it can
cope with the 'slightly' shifted semantic.
I would vote for stick a @deprecated tag (+ some explanations)
to the respective methods and to the interface because code
that uses these mechanisms is possibly not aware of the
sophisticated difference between 'no running threads left'
and 'all jobs done'.
- Sascha
Larry Becker schrieb:
Hi Sascha,
Adding a 'wakeup' Runnable works great and is easier than using the
listener anyway.
By the way, I couldn't find any other code using the Listener
interface, but I suppose there could be a Plug-in somewhere that does.
We should probably leave it out anyway since if we are leaving it in
for compatibility, it should actually be compatible. The easiest way
to find out if it is really needed by anyone but me might be to leave
it out and see if it breaks anyone's compile. The same applies to
getRunningThreads since it doesn't behave exactly like it did before
either.
Anyway, thanks for the help. I'm able to determine the end of
rendering much more accurately now.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this only works on
the defaultRenderingThreadQueue and therefore
its only an interim solution. The
WMS/DB ThreadQueue is private and a true
parallel ThreadQueue.
My goal is to add a renderAllSynchronized()
to RenderingManager that used the above
Runnable/unlock trick. The secret WMS/DB
will be eliminated (or better the default one
and the WMS/DB ThreadQueue will become the
default). This is doable with the
RunnableArrayList class I posted a while
ago, which simulates the single thread
default ThreadQueue. But one step after
the other ...
Regards, Sascha
Larry Becker schrieb:
Sascha,
I have reached a point where I need some help with the new
ThreadQueue implementation. I have modified my code that calls
getRunningThreads to use the Listener with the
allRunningThreadsFinished method instead. This was much cleaner and
worked fine until I replaced ThreadQueue with your version. The
problem I seem to be having is that the method doesn't fire until
after the 5 second WORKER_STAY_ALIVE_TIME has expired. Can you
explain what I should be doing or modify the code so that it fires
when there are no more jobs waiting?
I have attached the version of ThreadQueue.java that I have been using
in case it is outdated.
thanks,
Larry Becker
Re: [JPP-Devel] A new ThreadQueue
Thread pooling may be important for servers, but it doesn't seem to be
a performance factor in JUMP. If no new jobs are being added in 50
ms, the cpu is probably idle anyway.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Not if you want to do thread pooling.
The real problem: How can I get a notification
when a zoom is done?
The ZoomToSelectedItemsPlugIn ThreadQueue code looks
like a workaround due to lack of a real possibility
to get informed when the zoom is done.
I will have a look at this problem.
regards,
Sascha
Larry Becker schrieb:
I cut the WORKER_STAY_ALIVE_TIME to 50 ms and the flash now works. 50
ms is an eternity in CPU time anyway.
regards,
Larry
On 6/15/07, Larry Becker [EMAIL PROTECTED] wrote:
Thanks for finding that Listener use in ZoomToSelectedItemsPlugIn. I
tried it and it doesn't flash anymore.
regards,
Larry
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
In core the
com.vividsolutions.jump.workbench.ui.zoom.ZoomToSelectedItemsPlugIn
uses the ThreadQueue.Listener interface. But the code looks like it can
cope with the 'slightly' shifted semantic.
I would vote for stick a @deprecated tag (+ some explanations)
to the respective methods and to the interface because code
that uses these mechanisms is possibly not aware of the
sophisticated difference between 'no running threads left'
and 'all jobs done'.
- Sascha
Larry Becker schrieb:
Hi Sascha,
Adding a 'wakeup' Runnable works great and is easier than using the
listener anyway.
By the way, I couldn't find any other code using the Listener
interface, but I suppose there could be a Plug-in somewhere that does.
We should probably leave it out anyway since if we are leaving it in
for compatibility, it should actually be compatible. The easiest way
to find out if it is really needed by anyone but me might be to leave
it out and see if it breaks anyone's compile. The same applies to
getRunningThreads since it doesn't behave exactly like it did before
either.
Anyway, thanks for the help. I'm able to determine the end of
rendering much more accurately now.
regards,
Larry Becker
On 6/15/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hello Larry,
the version of the ThreadQueue is a bit outdated.
The version you have has no getRunningThreads() method.
This is need for compatibility. And there was a bug in
remove(Listener) which is fixed by now.
I attach the current ThreadQueue.
Now to your problem:
The Listeners are in for compatibility and you are right
they get there kick when the WORKER_STAY_ALIVE_TIME has
expired. But once again: These Listeners _do_ _not_ help
you! You want to know when the last job has ended, not
when there are no more running Threads. We discussed this
already.
I would suggest the following solution on your side
to archive the desired effect:
code
Graphics2D destination = ... // from outer space
LayerViewPanel layerViewPanel = ... // from outer space
RenderingManager renderingManager =
layerViewPanel.getRenderingManager();
ThreadQueue defaultThreadQueue =
renderingManager.getDefaultRendererThreadQueue();
// add all the Renderer Runnables to the ThreadQueue
renderingManager.renderAll();
final boolean [] locked = { true };
// because defaultThreadQueue does its jobs
// one after each other append a 'wakeup' Runnable ...
defaultThreadQueue.add(new Runnable() {
public void run() {
synchronized (locked) {
locked[0] = false;
locked.notify();
}
}
});
try {
synchronized (locked) {
// you could simply write
// while (locked[0]) locked.wait();
// but following is more defensive
int count = 0;
while (locked[0] count++ 10)
locked.wait(1L);
}
}
catch (InterruptedException ie) {
}
renderingManager.copyTo(destination);
/code
But as I said earlier, this only works on
the defaultRenderingThreadQueue and therefore
its only an interim solution. The
WMS/DB ThreadQueue is private and a true
parallel ThreadQueue.
My goal is to add a renderAllSynchronized()
to RenderingManager that used the above
Runnable/unlock trick. The secret WMS/DB
will be eliminated (or better the default one
and the WMS/DB ThreadQueue will become the
default). This is doable with the
RunnableArrayList class I posted a while
ago, which simulates the single thread
default ThreadQueue. But one step after
the other ...
Regards, Sascha
Larry Becker schrieb:
Sascha,
I have reached a point where I need some help with the new
ThreadQueue implementation. I have modified my code that calls
getRunningThreads to use the Listener with the
allRunningThreadsFinished method instead. This was much cleaner and
worked fine until I replaced ThreadQueue with your version.
Re: [JPP-Devel] A new ThreadQueue ...
Hi Larry,
Larry Becker schrieb:
Hi Sascha,
Only potentially, but practically speaking my code is blocking the
GUI thread, and there is nothing running to put more jobs on the
queue. The while loop in processQueue will keep maxRunningThreads
running untill the queue begins to empty. When is empty, there are no
more render jobs to process.
This is wrong. Say you have enqueued three jobs.
The defaultRendering ThreadQueue is limited to one Thread running
so all threads are working one after each other. After the first
job has endend, the one and only Thread brings the counter to zero
because this if the only thread working on the jobs. But there are
two more in the queue. It is not important if the GUI thread is
blocked. There are two jobs more to do. And more problems arrive
if you have jobs that are _not_ in the default rendering ThreadQueue
like WMS. They are rendered in truly parallel way with a second
thread. Your 'while (default queue has running threads) sleep(time)'
loop don't test this second queue.
BTW: Didn't you mention that you place a
try { ... } catch (ArrayIndexOutOfBoundsException ex) { ...}
workaround in your SkyJUMP code? Have you ever thought about
the reasons? Looking at the code of OJ I didn't find this
'improvement' ...
This has been an interesting discussion, but we could go back and
forth like this forever. I admit that you know Threads very well, and
that your design appears to be better. I'm simply saying that JUMP
ThreadQueue is not as bad as you are making it out to be, and that it
has been working perfectly fine for years now. Could it be improved?
Of course, Jon's comments show he was constantly seeking improvements
to it. I am just cautious about replacing a critical section of code
without adequate testing. Give me a well tested bad design over an
untested good design any day.
I totally agree with you. Never touch a running system. But the system
is not running very well here. Nobody wrote code in the past that needs
precise reliability in this thread issues. This has changed.
And as you say: It's a critical portion of the code. For this kinds
of code I demand 'nearly' (*) mathematical correctness. And talking
about business: I have a customer here how stumbled into this concrete
thread trap. Now I can show him my analysis and my enhancements and how
the issue is able to be fixed. I can send him a patch set against OJ to
make his private copy work. But I prefer to fix the problem at the root
and discuss it with the OJ team.
Don't get me wrong, I do appreciate your hard work and contribution.
I just need a little more time to test.
This why are talking about and I'm willing to give you this time. :-)
regards,
Larry
(*) Having in mind Knuth's famous quotation about correctness proves and
tests.
Regards,
Sascha
On 5/30/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hi Larry,
the magic word here is 'critical region'
Have a look a the finally block.
Assume the setRunningThreads(getRunningThreads() - 1) call
was executed and the counter went to zero. This is done in Thread A.
And now directly comes a context switch to your code
in Thread B. Your if statement thinks that there are no
jobs any more - Your loop ends. Context switch back to Thread A.
Now the processQueue() call is executed and potentially a
new thread is started because the queue is not empty.
Counter is zero but queue not empty.
Thread programming is delicate, isn't it?
- Sascha
Larry Becker schrieb:
Hi Sascha,
I do not see the difference. It seems to me that when a task is
finished, the runningThreads count is decremented as in:
try {
runnable.run();
} finally {
setRunningThreads(getRunningThreads() - 1);
processQueue();
}
So it would seem to me that when the count reaches zero that there are
no more jobs.
regards,
Larry
On 5/30/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hi Larry,
The method is named runningThreads(). This was my
mistake. I will add add get getRunningThreads() to
reestablish compatibility. Thanks for this hint.
BTW: You're code:
while (threadQueue.getRunningThreads() 0) Thread.sleep(200);
tests if there are running _threads_ left. This do not tell
you if there are _jobs_ to to! What you really want to know
is when the last job is done. See the difference?
The new RenderingManager will have a new method to archive
this. It will switch the thread queue to single threaded mode
and enqueues a special Runnable at the end. Then the calling
thread is lock with a monitor. If the special Runnable is
executed the monitor is unlocked and the calling thread continues.
This is what you want.
Switching to single thread mode has the advantage to keep
the correct z order of the layers. This is essential for
printing because you don't want to have any internal repaint()
operations. This
Re: [JPP-Devel] A new ThreadQueue ...
OK Sascha, you win. I see your point. Your logic is inescapable
about the defaultRendering ThreadQueue. My only defense is that my
code will not produce the correct raster without the while (default
queue has running threads) check. WMS Layers (only) will be missing.
Regarding the ArrayIndexOutOfBoundsException - that only occurred when
I introduced mouse wheel zooming which has the effect of overloading
the event queue with zoom events. Of course, the potential was there
for it to happen before, it was just very unlikely.
I do appreciate your mathematical rigor, and arguing with you served
the very useful purpose (for me) of getting to know you better. I
hope to argue with you again on some other topic. ;-) I find
arguments among programmers to be a good way of vetting a design, and
I never take it personally. I hope you don't either.
regards,
Larry
On 5/31/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hi Larry,
Larry Becker schrieb:
Hi Sascha,
Only potentially, but practically speaking my code is blocking the
GUI thread, and there is nothing running to put more jobs on the
queue. The while loop in processQueue will keep maxRunningThreads
running untill the queue begins to empty. When is empty, there are no
more render jobs to process.
This is wrong. Say you have enqueued three jobs.
The defaultRendering ThreadQueue is limited to one Thread running
so all threads are working one after each other. After the first
job has endend, the one and only Thread brings the counter to zero
because this if the only thread working on the jobs. But there are
two more in the queue. It is not important if the GUI thread is
blocked. There are two jobs more to do. And more problems arrive
if you have jobs that are _not_ in the default rendering ThreadQueue
like WMS. They are rendered in truly parallel way with a second
thread. Your 'while (default queue has running threads) sleep(time)'
loop don't test this second queue.
BTW: Didn't you mention that you place a
try { ... } catch (ArrayIndexOutOfBoundsException ex) { ...}
workaround in your SkyJUMP code? Have you ever thought about
the reasons? Looking at the code of OJ I didn't find this
'improvement' ...
This has been an interesting discussion, but we could go back and
forth like this forever. I admit that you know Threads very well, and
that your design appears to be better. I'm simply saying that JUMP
ThreadQueue is not as bad as you are making it out to be, and that it
has been working perfectly fine for years now. Could it be improved?
Of course, Jon's comments show he was constantly seeking improvements
to it. I am just cautious about replacing a critical section of code
without adequate testing. Give me a well tested bad design over an
untested good design any day.
I totally agree with you. Never touch a running system. But the system
is not running very well here. Nobody wrote code in the past that needs
precise reliability in this thread issues. This has changed.
And as you say: It's a critical portion of the code. For this kinds
of code I demand 'nearly' (*) mathematical correctness. And talking
about business: I have a customer here how stumbled into this concrete
thread trap. Now I can show him my analysis and my enhancements and how
the issue is able to be fixed. I can send him a patch set against OJ to
make his private copy work. But I prefer to fix the problem at the root
and discuss it with the OJ team.
Don't get me wrong, I do appreciate your hard work and contribution.
I just need a little more time to test.
This why are talking about and I'm willing to give you this time. :-)
regards,
Larry
(*) Having in mind Knuth's famous quotation about correctness proves and
tests.
Regards,
Sascha
On 5/30/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hi Larry,
the magic word here is 'critical region'
Have a look a the finally block.
Assume the setRunningThreads(getRunningThreads() - 1) call
was executed and the counter went to zero. This is done in Thread A.
And now directly comes a context switch to your code
in Thread B. Your if statement thinks that there are no
jobs any more - Your loop ends. Context switch back to Thread A.
Now the processQueue() call is executed and potentially a
new thread is started because the queue is not empty.
Counter is zero but queue not empty.
Thread programming is delicate, isn't it?
- Sascha
Larry Becker schrieb:
Hi Sascha,
I do not see the difference. It seems to me that when a task is
finished, the runningThreads count is decremented as in:
try {
runnable.run();
} finally {
setRunningThreads(getRunningThreads() - 1);
processQueue();
}
So it would seem to me that when the count reaches zero that there are
no more jobs.
regards,
Re: [JPP-Devel] A new ThreadQueue ...
It doesn't look like I'm going to have time to test the new ThreadQueue anytime soon. I did plug it in long enough to determine that it broke my code that called getRunningThreads() since that method is no longer present. I didn't have time to look for the new method that I should use instead. I should be able to get back to it in a couple of weeks (I'm instructing a class in our software next week). regards, Larry On 5/30/07, Michaël Michaud [EMAIL PROTECTED] wrote: Hei Saschachaël i got positive feedback from my colleaque on the changes of the threading stuff. If Michael aggrees too, you can commit As far as I'm concerned, I am really pleased to see Sascha's contribution to the core. OJ really needs contributions from skilled programmers :-) . Michaël stefan Sascha L. Teichmann schrieb: Hi! Stefan Steiniger schrieb: Similar to Landon I have never touched threading nor heared any lectures or read books on it. Thus .. i dont know if it turns out to be good or bad. But i know that making things more clean is good objective. Thus, i support the changes. It would be good when we finally change the code, that you also put into the code some docs or references where you outline why somethings has changed. (actually I currently would opt to leave the old code as comment and not to remove entirely) The ThreadQueue is JavaDoc'ed but I can add some design notes as well. Keeping old things as comments negates the existence of version control systems. CVS is exactly the time machine you for this. As I pointed out earlier having a ChangeLog would be nice to document motivations and backgrounds for a change. Simply generating a ChangeLog out of the commit messages is convenient but it does not uncover the true potential of such a 'log' file. Due to my lack of knowledge i forwarded your email to a colleague who developed a web-service based on JUMP (which runs on a server, see [1]), an ajax client and recently did some multiprocessor experiments for the web-services processing. I look forward to his comments. I hope 1) he will look on your proposal and 2) you can wait some days until he, Larry and Michael got an opinion. @Michael: You ask what to change. For the moment just replace ThreadQueue.java with one I've sent last. The first patches are addressing the problem with ThreadQueue limited to one Thread at a time (defaultRenderingThreadQueue in RenderingManger). You can apply these patches too, but they are not needed any longer. With next patch I'll send the this second thread queue will be removed from RenderingManager entirely. What to test? Doing your day-to-day is the best thing you can do. The most important thing is that the new one behaves exactly like the old one. The magic word is compatibility. I don't want to break the show. Use plug-ins, use many layers, use less layer, use db and WMS layers. Do all the fancy stuff you like to do. If the OJ freezes or something other looks odd just tell me. :-) - Sascha Michaël Michaud schrieb: Sascha L. Teichmann a écrit : Really back to topic: Find attached a replacement for ThreadQueue [1]. To use it just overwrite the original one. Hi Sascha : I think that trying to have a cleaner and more simple code is an excellent goal, and I'd like to help, but I'm not sure I can understand all these thread issues. If you tell me exactly which classes I must replace (on ly ThreadQueue or also the pieces of code from your previous mail) and what kind of tests I should do (rendering different kind of layers ? mixing different kind of layers), I'll try to make some more tests on my desktop. Thanks, for the hard work Michaël This one works for the real parallel case of layer rendering too. Each time a thread finished executing its Runnable it looks into the queue if they are more jobs to do. This prevents unnecessary thread creations/shutdowns. If the queue is empty the worker thread is kept alive for 5 seconds waiting for new jobs. This results in a kind of thread pooling. @Larry: I've isolated my implementation and the OJ ThreadQueue and done a synthetic benchmark with a larger number of jobs (10,000+). My implementation works about two orders faster than the OJ one. But this is of little meaning because OJ only has to handle a number of jobs equal the number of layers. This will hardly hit 10,000+ ;-) But again: My mission is improve the structure not primarily the speed. I've tested the new ThreadQueue to some extent but I'am male(tm) after all ... potentially making mistakes. It would be very kind if someone test it too. My next step will be some clean up in the RenderingManager [2]. I'am not sure that it is really needed to have two ThreadQueues there. The effect of the single tread one can be easily simulated with a data structure like the RunnableArrayList which I've posted already. Any
Re: [JPP-Devel] A new ThreadQueue ...
SS, Sunburned Surveyor schrieb: Sascha, You wrote: As I pointed out earlier having a ChangeLog would be nice to document motivations and backgrounds for a change. Simply generating a ChangeLog out of the commit messages is convenient but it does not uncover the true potential of such a 'log' file. I agree, and I use small text files to store a change log for each Java class in my own programming work. Perhaps I similar system would meet your needs. If you have any ideas for a change log that we can use in the OpenJUMP CVS throw it out and we can consider it. As long as the format of the change log was simple and consistent I would have no problem with this. Tell us what you had in mind for a change log and we will see what Stefan and the others think. The format of a ChangeLog file is pretty simple and pretty standard. To see what I mean look a the ChangeLog of the Print/Layout[1] or WFS[2] plug-ins. Most of entries are pretty trivial, short and technical. A second level ChangeLog called Changes.txt summarizes the changes for the end users. It also helps the release managers to keep track of what is done. A TODO.txt file is also very helpful. A lot of open source projects are using this scheme (with variations). - Sascha [1] http://jump-pilot.cvs.sourceforge.net/*checkout*/jump-pilot/PrintLayoutPlugin/ChangeLog?revision=1.186 [2] http://jump-pilot.cvs.sourceforge.net/*checkout*/jump-pilot/WFSPlugin/ChangeLog?revision=1.47 - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] A new ThreadQueue ...
Sascha L. Teichmann a écrit : Really back to topic: Find attached a replacement for ThreadQueue [1]. To use it just overwrite the original one. Hi Sascha : I think that trying to have a cleaner and more simple code is an excellent goal, and I'd like to help, but I'm not sure I can understand all these thread issues. If you tell me exactly which classes I must replace (on ly ThreadQueue or also the pieces of code from your previous mail) and what kind of tests I should do (rendering different kind of layers ? mixing different kind of layers), I'll try to make some more tests on my desktop. Thanks, for the hard work Michaël This one works for the real parallel case of layer rendering too. Each time a thread finished executing its Runnable it looks into the queue if they are more jobs to do. This prevents unnecessary thread creations/shutdowns. If the queue is empty the worker thread is kept alive for 5 seconds waiting for new jobs. This results in a kind of thread pooling. @Larry: I've isolated my implementation and the OJ ThreadQueue and done a synthetic benchmark with a larger number of jobs (10,000+). My implementation works about two orders faster than the OJ one. But this is of little meaning because OJ only has to handle a number of jobs equal the number of layers. This will hardly hit 10,000+ ;-) But again: My mission is improve the structure not primarily the speed. I've tested the new ThreadQueue to some extent but I'am male(tm) after all ... potentially making mistakes. It would be very kind if someone test it too. My next step will be some clean up in the RenderingManager [2]. I'am not sure that it is really needed to have two ThreadQueues there. The effect of the single tread one can be easily simulated with a data structure like the RunnableArrayList which I've posted already. Any comments? Yours, Sascha [1] com.vividsolutions.jump.workbench.ui.renderer.ThreadQueue [2] com.vividsolutions.jump.workbench.ui.renderer.RenderingManager Sunburned Surveyor schrieb: Sascha, Please accept my sincerest aopologies. I'm afriad my American ignorance of other cultures is more than just a little obvious at times. I believe I have made the same mistake with Jan. :] Please be patient with me as I learn the details of cultures across the Pacific and Atalantic Oceans! The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: TNX, but for the records 'he' would be more suited in my case. 'Sascha' is basically a Russian term of endearment for the boys name 'Alexander' but it's also used as a girls name. BTW: 'Jan' is a girls name in the US too, isn't? ;-) - Sascha Sunburned Surveyor schrieb: Sascha and Larry, I must admit that I am way over my head here. I haven't done much thread programming in Java. (Hopefully Stefan has!) Sascha wrote: My primary goal is to simplify the threading code to make it more reliable in terms of time. This seems like an admirable goal to me. If Larry, or a similar programmer of his experience, agrees that this changes would be beneficial, I say we give Sascha a shot at it. It sounds like she has considered her changes carefully. Just my two cents. The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to carefully refactor the system doing little patches step by step not to break it. Refactoring the ThreadQueue with it's 'flaws' seems a to be a good starting point to me. One reason for this change is the fact that you are able to figure out if the default thread runs empty. But there is no way to figure out when the last thread ends. That are different things. A mechanism for this is planned. Sorry, if I've shadowed my true intentions to much. Maybe I should discuss more before I send patches. ;-) Back to the technical side: The patch needs some testing but I don't expect too much performance improvement. A less intrusive alternative to bind the Runnables that go to the default ThreadQueue into one thread is to create a container which self is Runnable (see e.g. RunnableArrayList attached) and put them into an instance of this class. This container is put into multiRendererThreadQueue as a Runnable. With this modification the defaultRendererThreadQueue can be removed (multiRendererThreadQueue renamed to defaultRendererThreadQueue). Only an idea ... I'm in discussion mode now. ;-) - Sascha Larry Becker schrieb: Hi
Re: [JPP-Devel] A new ThreadQueue ...
[1]: http://www.ixserve.de/pub_whatiswebgen.php (note the links to papers dont work but they are available in the following repository:) i.e.: http://www.geo.unizh.ch/publications/degen/autocarto2005_burghardt_final.pdf btw. Moritz Neun (my colleague) is currently writing up his thesis. so by September there is some more stuff on it (other papers already exists too and are submitted to journals) Very interesting... Is there any relation between M. Neun thesis and the jump-based wps-client available at https://incubator.52north.org/twiki/bin/view/Processing/52nWebProcessingService ? Michaël Michaël Michaud schrieb: Sascha L. Teichmann a écrit : Really back to topic: Find attached a replacement for ThreadQueue [1]. To use it just overwrite the original one. Hi Sascha : I think that trying to have a cleaner and more simple code is an excellent goal, and I'd like to help, but I'm not sure I can understand all these thread issues. If you tell me exactly which classes I must replace (on ly ThreadQueue or also the pieces of code from your previous mail) and what kind of tests I should do (rendering different kind of layers ? mixing different kind of layers), I'll try to make some more tests on my desktop. Thanks, for the hard work Michaël This one works for the real parallel case of layer rendering too. Each time a thread finished executing its Runnable it looks into the queue if they are more jobs to do. This prevents unnecessary thread creations/shutdowns. If the queue is empty the worker thread is kept alive for 5 seconds waiting for new jobs. This results in a kind of thread pooling. @Larry: I've isolated my implementation and the OJ ThreadQueue and done a synthetic benchmark with a larger number of jobs (10,000+). My implementation works about two orders faster than the OJ one. But this is of little meaning because OJ only has to handle a number of jobs equal the number of layers. This will hardly hit 10,000+ ;-) But again: My mission is improve the structure not primarily the speed. I've tested the new ThreadQueue to some extent but I'am male(tm) after all ... potentially making mistakes. It would be very kind if someone test it too. My next step will be some clean up in the RenderingManager [2]. I'am not sure that it is really needed to have two ThreadQueues there. The effect of the single tread one can be easily simulated with a data structure like the RunnableArrayList which I've posted already. Any comments? Yours, Sascha [1] com.vividsolutions.jump.workbench.ui.renderer.ThreadQueue [2] com.vividsolutions.jump.workbench.ui.renderer.RenderingManager Sunburned Surveyor schrieb: Sascha, Please accept my sincerest aopologies. I'm afriad my American ignorance of other cultures is more than just a little obvious at times. I believe I have made the same mistake with Jan. :] Please be patient with me as I learn the details of cultures across the Pacific and Atalantic Oceans! The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: TNX, but for the records 'he' would be more suited in my case. 'Sascha' is basically a Russian term of endearment for the boys name 'Alexander' but it's also used as a girls name. BTW: 'Jan' is a girls name in the US too, isn't? ;-) - Sascha Sunburned Surveyor schrieb: Sascha and Larry, I must admit that I am way over my head here. I haven't done much thread programming in Java. (Hopefully Stefan has!) Sascha wrote: My primary goal is to simplify the threading code to make it more reliable in terms of time. This seems like an admirable goal to me. If Larry, or a similar programmer of his experience, agrees that this changes would be beneficial, I say we give Sascha a shot at it. It sounds like she has considered her changes carefully. Just my two cents. The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to carefully refactor the system doing little patches step by step not to break it. Refactoring the ThreadQueue with it's 'flaws' seems a to be a good starting point to me. One reason for this change is the fact that you are able to figure out if the default thread runs empty. But there is no way to figure out when the last thread ends. That are different things. A mechanism for this is planned. Sorry, if I've shadowed my true intentions to much. Maybe
Re: [JPP-Devel] A new ThreadQueue ...
mhm.. more or less yes/not ;o) we know that guy (Theodor Förster) who works on that wps services. He does a thesis at ITC enschede and attends the same workshops as Moritz because the goals are somehow similar. but thanx for the link. I need to study it (until know i did only know two conference articles from that guy.) stefan Michaël Michaud schrieb: [1]: http://www.ixserve.de/pub_whatiswebgen.php (note the links to papers dont work but they are available in the following repository:) i.e.: http://www.geo.unizh.ch/publications/degen/autocarto2005_burghardt_final.pdf btw. Moritz Neun (my colleague) is currently writing up his thesis. so by September there is some more stuff on it (other papers already exists too and are submitted to journals) Very interesting... Is there any relation between M. Neun thesis and the jump-based wps-client available at https://incubator.52north.org/twiki/bin/view/Processing/52nWebProcessingService ? Michaël Michaël Michaud schrieb: Sascha L. Teichmann a écrit : Really back to topic: Find attached a replacement for ThreadQueue [1]. To use it just overwrite the original one. Hi Sascha : I think that trying to have a cleaner and more simple code is an excellent goal, and I'd like to help, but I'm not sure I can understand all these thread issues. If you tell me exactly which classes I must replace (on ly ThreadQueue or also the pieces of code from your previous mail) and what kind of tests I should do (rendering different kind of layers ? mixing different kind of layers), I'll try to make some more tests on my desktop. Thanks, for the hard work Michaël This one works for the real parallel case of layer rendering too. Each time a thread finished executing its Runnable it looks into the queue if they are more jobs to do. This prevents unnecessary thread creations/shutdowns. If the queue is empty the worker thread is kept alive for 5 seconds waiting for new jobs. This results in a kind of thread pooling. @Larry: I've isolated my implementation and the OJ ThreadQueue and done a synthetic benchmark with a larger number of jobs (10,000+). My implementation works about two orders faster than the OJ one. But this is of little meaning because OJ only has to handle a number of jobs equal the number of layers. This will hardly hit 10,000+ ;-) But again: My mission is improve the structure not primarily the speed. I've tested the new ThreadQueue to some extent but I'am male(tm) after all ... potentially making mistakes. It would be very kind if someone test it too. My next step will be some clean up in the RenderingManager [2]. I'am not sure that it is really needed to have two ThreadQueues there. The effect of the single tread one can be easily simulated with a data structure like the RunnableArrayList which I've posted already. Any comments? Yours, Sascha [1] com.vividsolutions.jump.workbench.ui.renderer.ThreadQueue [2] com.vividsolutions.jump.workbench.ui.renderer.RenderingManager Sunburned Surveyor schrieb: Sascha, Please accept my sincerest aopologies. I'm afriad my American ignorance of other cultures is more than just a little obvious at times. I believe I have made the same mistake with Jan. :] Please be patient with me as I learn the details of cultures across the Pacific and Atalantic Oceans! The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: TNX, but for the records 'he' would be more suited in my case. 'Sascha' is basically a Russian term of endearment for the boys name 'Alexander' but it's also used as a girls name. BTW: 'Jan' is a girls name in the US too, isn't? ;-) - Sascha Sunburned Surveyor schrieb: Sascha and Larry, I must admit that I am way over my head here. I haven't done much thread programming in Java. (Hopefully Stefan has!) Sascha wrote: My primary goal is to simplify the threading code to make it more reliable in terms of time. This seems like an admirable goal to me. If Larry, or a similar programmer of his experience, agrees that this changes would be beneficial, I say we give Sascha a shot at it. It sounds like she has considered her changes carefully. Just my two cents. The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to
Re: [JPP-Devel] A new ThreadQueue ...
SS, no problem ... apologies accepted! :-) ..and I wont tell Jan's wife about it. Back to topic: I'am fully aware of possible side effects that I may trigger with my changing request. It's a bit of a pity that OJ doesn't have a real devel branch to put in stuff that is a little 'experimental'. This may lead to another discussion ... hopefully in another thread. But as a professional FLOSS software engineer with customers (and J.) looking at my fingers I'll always try to be very careful with my doings. ;-) Really back to topic: Find attached a replacement for ThreadQueue [1]. To use it just overwrite the original one. This one works for the real parallel case of layer rendering too. Each time a thread finished executing its Runnable it looks into the queue if they are more jobs to do. This prevents unnecessary thread creations/shutdowns. If the queue is empty the worker thread is kept alive for 5 seconds waiting for new jobs. This results in a kind of thread pooling. @Larry: I've isolated my implementation and the OJ ThreadQueue and done a synthetic benchmark with a larger number of jobs (10,000+). My implementation works about two orders faster than the OJ one. But this is of little meaning because OJ only has to handle a number of jobs equal the number of layers. This will hardly hit 10,000+ ;-) But again: My mission is improve the structure not primarily the speed. I've tested the new ThreadQueue to some extent but I'am male(tm) after all ... potentially making mistakes. It would be very kind if someone test it too. My next step will be some clean up in the RenderingManager [2]. I'am not sure that it is really needed to have two ThreadQueues there. The effect of the single tread one can be easily simulated with a data structure like the RunnableArrayList which I've posted already. Any comments? Yours, Sascha [1] com.vividsolutions.jump.workbench.ui.renderer.ThreadQueue [2] com.vividsolutions.jump.workbench.ui.renderer.RenderingManager Sunburned Surveyor schrieb: Sascha, Please accept my sincerest aopologies. I'm afriad my American ignorance of other cultures is more than just a little obvious at times. I believe I have made the same mistake with Jan. :] Please be patient with me as I learn the details of cultures across the Pacific and Atalantic Oceans! The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: TNX, but for the records 'he' would be more suited in my case. 'Sascha' is basically a Russian term of endearment for the boys name 'Alexander' but it's also used as a girls name. BTW: 'Jan' is a girls name in the US too, isn't? ;-) - Sascha Sunburned Surveyor schrieb: Sascha and Larry, I must admit that I am way over my head here. I haven't done much thread programming in Java. (Hopefully Stefan has!) Sascha wrote: My primary goal is to simplify the threading code to make it more reliable in terms of time. This seems like an admirable goal to me. If Larry, or a similar programmer of his experience, agrees that this changes would be beneficial, I say we give Sascha a shot at it. It sounds like she has considered her changes carefully. Just my two cents. The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to carefully refactor the system doing little patches step by step not to break it. Refactoring the ThreadQueue with it's 'flaws' seems a to be a good starting point to me. One reason for this change is the fact that you are able to figure out if the default thread runs empty. But there is no way to figure out when the last thread ends. That are different things. A mechanism for this is planned. Sorry, if I've shadowed my true intentions to much. Maybe I should discuss more before I send patches. ;-) Back to the technical side: The patch needs some testing but I don't expect too much performance improvement. A less intrusive alternative to bind the Runnables that go to the default ThreadQueue into one thread is to create a container which self is Runnable (see e.g. RunnableArrayList attached) and put them into an instance of this class. This container is put into multiRendererThreadQueue as a Runnable. With this modification the defaultRendererThreadQueue can be removed (multiRendererThreadQueue renamed to defaultRendererThreadQueue). Only an idea ... I'm in discussion mode now. ;-) - Sascha Larry Becker
Re: [JPP-Devel] A new ThreadQueue ...
Hi Sascha,
I read your comments and look at your code with interest. It appears to
be an improved ThreadQueue implementation, but will require a lot of testing
to verify. Before I invest this time, I would like to know what problem it
is solving. I see your dislikes a - e, but these are not really problems,
only architectural critiques. Have you done any benchmarks that show that
the new SingleThreadQueue speeds up rendering? Your logical argument that
it should be more efficient is persuasive, but I have been surprised by
Java before.
respectfully,
Larry Becker
On 5/23/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote:
Hi together,
as some of you may already know i have my dislikes against
ThreadQueue [1] (Hi, Larry!) see my mail [2]
a - It forks a new thread for any Runnable it processes.
b - It has an ugly busy wait loop inside.
c - The event listener for empty queue fires to often.
d - The default ThreadQueue is some kind of thread serializer.
e - The DB/WMS ThreadQueue has no public access.
Now I've written a sub class of ThreadQueue: SingleThreadQueue
(see attachment). This one deals with the issues a, b and d.
I also attached a patch against RenderingManager [3] to handle e.
The new class (to be placed in package
com.vividsolutions.jump.workbench.ui.renderer) is a drop-in
replacement for the default ThreadQueue in RenderingManager.
Not for the ThreadQueue that handles the DB/WMS layers.
Because Jon limited the number of parallel threads in default
queue to 1 I see no reason why to fork a new thread for each
Runnable it processes. Thread creation/shutdown is fairly
expensive. Instead a single background thread is started
which processes the Runnables one by one. If the thread
is idle for 30 secs it shuts itself down. If you have a lot
of (non-WMS/BB) layers this should improve performance
and save some resources. The processing itself is done
with a monitor (synchronized/wait/notify) so there is no
busy wait any more.
The DB/WMS ThreadQueue (real parallel threads) is left untouched
for the moment. Depending on my personal schedule I will send
a patch against this one too. Preliminary code with thread pooling
exists but it needs a bit more testing.
Find attached the new class and patches against RenderingManager and
the old ThreadQueue to bring it to work.
Comments are very welcome. :-)
Kind regrads,
Sascha
[1] com.vividsolutions.jump.workbench.ui.renderer.ThreadQueue
[2]
http://sourceforge.net/mailarchive/message.php?msg_name=4653389E.6000706%40intevation.de
[3] com.vividsolutions.jump.workbench.ui.renderer.RenderingManager
Index:
./src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java
===
RCS file:
/cvsroot/jump-pilot/openjump/src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java,v
retrieving revision 1.6
diff -u -r1.6 RenderingManager.java
---
./src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java
22 May 2007 18:47:12 - 1.6
+++
./src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java
24 May 2007 04:10:30 -
@@ -77,7 +77,7 @@
* non-database layers in parallel. In fact, it will make the GUI
less
* responsive. [Jon Aquino]
*/
- private ThreadQueue defaultRendererThreadQueue = new
ThreadQueue(1);
+ private ThreadQueue defaultRendererThreadQueue = new
SingleThreadQueue();
/**
* WMS and database processing are done on the server side, so
allow these
@@ -294,6 +294,10 @@
return defaultRendererThreadQueue;
}
+ public ThreadQueue getMultiRendererThreadQueue() {
+ return multiRendererThreadQueue;
+ }
+
public void dispose() {
repaintTimer.stop();
defaultRendererThreadQueue.dispose();
@@ -334,4 +338,4 @@
contentIDToRendererMap.remove(contentID);
}
-}
\ No newline at end of file
+}
Index:
./src/com/vividsolutions/jump/workbench/ui/renderer/ThreadQueue.java
===
RCS file:
/cvsroot/jump-pilot/openjump/src/com/vividsolutions/jump/workbench/ui/renderer/ThreadQueue.java,v
retrieving revision 1.1
diff -u -r1.1 ThreadQueue.java
---
./src/com/vividsolutions/jump/workbench/ui/renderer/ThreadQueue.java16
Jun 2005 22:50:38 - 1.1
+++
./src/com/vividsolutions/jump/workbench/ui/renderer/ThreadQueue.java24
May 2007 04:09:15 -
@@ -47,6 +47,10 @@
private Vector queuedRunnables = new Vector();
private int maxRunningThreads;
+ public ThreadQueue() {
+ this(1);
+ }
+
public ThreadQueue(final int maxRunningThreads) {
this.maxRunningThreads = maxRunningThreads;
}
@@ -95,7 +99,7 @@
public static interface Listener {
public void allRunningThreadsFinished();
}
-private void fireAllRunningThreadsFinished() {
+
Re: [JPP-Devel] A new ThreadQueue ...
Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to carefully refactor the system doing little patches step by step not to break it. Refactoring the ThreadQueue with it's 'flaws' seems a to be a good starting point to me. One reason for this change is the fact that you are able to figure out if the default thread runs empty. But there is no way to figure out when the last thread ends. That are different things. A mechanism for this is planned. Sorry, if I've shadowed my true intentions to much. Maybe I should discuss more before I send patches. ;-) Back to the technical side: The patch needs some testing but I don't expect too much performance improvement. A less intrusive alternative to bind the Runnables that go to the default ThreadQueue into one thread is to create a container which self is Runnable (see e.g. RunnableArrayList attached) and put them into an instance of this class. This container is put into multiRendererThreadQueue as a Runnable. With this modification the defaultRendererThreadQueue can be removed (multiRendererThreadQueue renamed to defaultRendererThreadQueue). Only an idea ... I'm in discussion mode now. ;-) - Sascha Larry Becker schrieb: Hi Sascha, I read your comments and look at your code with interest. It appears to be an improved ThreadQueue implementation, but will require a lot of testing to verify. Before I invest this time, I would like to know what problem it is solving. I see your dislikes a - e, but these are not really problems, only architectural critiques. Have you done any benchmarks that show that the new SingleThreadQueue speeds up rendering? Your logical argument that it should be more efficient is persuasive, but I have been surprised by Java before. respectfully, Larry Becker On 5/23/07, *Sascha L. Teichmann* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi together, as some of you may already know i have my dislikes against ThreadQueue [1] (Hi, Larry!) see my mail [2] a - It forks a new thread for any Runnable it processes. b - It has an ugly busy wait loop inside. c - The event listener for empty queue fires to often. d - The default ThreadQueue is some kind of thread serializer. e - The DB/WMS ThreadQueue has no public access. Now I've written a sub class of ThreadQueue: SingleThreadQueue (see attachment). This one deals with the issues a, b and d. I also attached a patch against RenderingManager [3] to handle e. The new class (to be placed in package com.vividsolutions.jump.workbench.ui.renderer) is a drop-in replacement for the default ThreadQueue in RenderingManager. Not for the ThreadQueue that handles the DB/WMS layers. Because Jon limited the number of parallel threads in default queue to 1 I see no reason why to fork a new thread for each Runnable it processes. Thread creation/shutdown is fairly expensive. Instead a single background thread is started which processes the Runnables one by one. If the thread is idle for 30 secs it shuts itself down. If you have a lot of (non-WMS/BB) layers this should improve performance and save some resources. The processing itself is done with a monitor (synchronized/wait/notify) so there is no busy wait any more. The DB/WMS ThreadQueue (real parallel threads) is left untouched for the moment. Depending on my personal schedule I will send a patch against this one too. Preliminary code with thread pooling exists but it needs a bit more testing. Find attached the new class and patches against RenderingManager and the old ThreadQueue to bring it to work. Comments are very welcome. :-) Kind regrads, Sascha [1] com.vividsolutions.jump.workbench.ui.renderer.ThreadQueue [2] http://sourceforge.net/mailarchive/message.php?msg_name=4653389E.6000706%40intevation.de [3] com.vividsolutions.jump.workbench.ui.renderer.RenderingManager Index: ./src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java === RCS file: /cvsroot/jump-pilot/openjump/src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java,v retrieving revision 1.6 diff -u - r1.6 RenderingManager.java --- ./src/com/vividsolutions/jump/workbench/ui/renderer/RenderingManager.java 22
Re: [JPP-Devel] A new ThreadQueue ...
TNX, but for the records 'he' would be more suited in my case. 'Sascha' is basically a Russian term of endearment for the boys name 'Alexander' but it's also used as a girls name. BTW: 'Jan' is a girls name in the US too, isn't? ;-) - Sascha Sunburned Surveyor schrieb: Sascha and Larry, I must admit that I am way over my head here. I haven't done much thread programming in Java. (Hopefully Stefan has!) Sascha wrote: My primary goal is to simplify the threading code to make it more reliable in terms of time. This seems like an admirable goal to me. If Larry, or a similar programmer of his experience, agrees that this changes would be beneficial, I say we give Sascha a shot at it. It sounds like she has considered her changes carefully. Just my two cents. The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to carefully refactor the system doing little patches step by step not to break it. Refactoring the ThreadQueue with it's 'flaws' seems a to be a good starting point to me. One reason for this change is the fact that you are able to figure out if the default thread runs empty. But there is no way to figure out when the last thread ends. That are different things. A mechanism for this is planned. Sorry, if I've shadowed my true intentions to much. Maybe I should discuss more before I send patches. ;-) Back to the technical side: The patch needs some testing but I don't expect too much performance improvement. A less intrusive alternative to bind the Runnables that go to the default ThreadQueue into one thread is to create a container which self is Runnable (see e.g. RunnableArrayList attached) and put them into an instance of this class. This container is put into multiRendererThreadQueue as a Runnable. With this modification the defaultRendererThreadQueue can be removed (multiRendererThreadQueue renamed to defaultRendererThreadQueue). Only an idea ... I'm in discussion mode now. ;-) - Sascha Larry Becker schrieb: Hi Sascha, I read your comments and look at your code with interest. It appears to be an improved ThreadQueue implementation, but will require a lot of testing to verify. Before I invest this time, I would like to know what problem it is solving. I see your dislikes a - e, but these are not really problems, only architectural critiques. Have you done any benchmarks that show that the new SingleThreadQueue speeds up rendering? Your logical argument that it should be more efficient is persuasive, but I have been surprised by Java before. respectfully, Larry Becker On 5/23/07, *Sascha L. Teichmann* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi together, as some of you may already know i have my dislikes against ThreadQueue [1] (Hi, Larry!) see my mail [2] a - It forks a new thread for any Runnable it processes. b - It has an ugly busy wait loop inside. c - The event listener for empty queue fires to often. d - The default ThreadQueue is some kind of thread serializer. e - The DB/WMS ThreadQueue has no public access. Now I've written a sub class of ThreadQueue: SingleThreadQueue (see attachment). This one deals with the issues a, b and d. I also attached a patch against RenderingManager [3] to handle e. The new class (to be placed in package com.vividsolutions.jump.workbench.ui.renderer) is a drop-in replacement for the default ThreadQueue in RenderingManager. Not for the ThreadQueue that handles the DB/WMS layers. Because Jon limited the number of parallel threads in default queue to 1 I see no reason why to fork a new thread for each Runnable it processes. Thread creation/shutdown is fairly expensive. Instead a single background thread is started which processes the Runnables one by one. If the thread is idle for 30 secs it shuts itself down. If you have a lot of (non-WMS/BB) layers this should improve performance and save some resources. The processing itself is done with a monitor (synchronized/wait/notify) so there is no busy wait any more. The DB/WMS ThreadQueue (real parallel threads) is left untouched for the moment. Depending on my personal schedule I will send a patch against this one too. Preliminary code with thread pooling exists but it needs a bit more testing.
Re: [JPP-Devel] A new ThreadQueue ...
Sascha, Please accept my sincerest aopologies. I'm afriad my American ignorance of other cultures is more than just a little obvious at times. I believe I have made the same mistake with Jan. :] Please be patient with me as I learn the details of cultures across the Pacific and Atalantic Oceans! The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: TNX, but for the records 'he' would be more suited in my case. 'Sascha' is basically a Russian term of endearment for the boys name 'Alexander' but it's also used as a girls name. BTW: 'Jan' is a girls name in the US too, isn't? ;-) - Sascha Sunburned Surveyor schrieb: Sascha and Larry, I must admit that I am way over my head here. I haven't done much thread programming in Java. (Hopefully Stefan has!) Sascha wrote: My primary goal is to simplify the threading code to make it more reliable in terms of time. This seems like an admirable goal to me. If Larry, or a similar programmer of his experience, agrees that this changes would be beneficial, I say we give Sascha a shot at it. It sounds like she has considered her changes carefully. Just my two cents. The Sunburned Surveyor On 5/24/07, Sascha L. Teichmann [EMAIL PROTECTED] wrote: Hi Larry, short answer first: No, I don't have any benchmarks, yet. The long answer: My primary goal is to simplify the threading code to make it more reliable in terms of time. Gaining performance improvements would be a neat side effect. Background: Multi-threading may be fine for slow layers which arrives later on the screen but for exporting the data (to print/layout e.g) it would be nice to have them arriving one after the other. My final goal is to have a simple switch between the normal and the serial mode. To archive that I try to carefully refactor the system doing little patches step by step not to break it. Refactoring the ThreadQueue with it's 'flaws' seems a to be a good starting point to me. One reason for this change is the fact that you are able to figure out if the default thread runs empty. But there is no way to figure out when the last thread ends. That are different things. A mechanism for this is planned. Sorry, if I've shadowed my true intentions to much. Maybe I should discuss more before I send patches. ;-) Back to the technical side: The patch needs some testing but I don't expect too much performance improvement. A less intrusive alternative to bind the Runnables that go to the default ThreadQueue into one thread is to create a container which self is Runnable (see e.g. RunnableArrayList attached) and put them into an instance of this class. This container is put into multiRendererThreadQueue as a Runnable. With this modification the defaultRendererThreadQueue can be removed (multiRendererThreadQueue renamed to defaultRendererThreadQueue). Only an idea ... I'm in discussion mode now. ;-) - Sascha Larry Becker schrieb: Hi Sascha, I read your comments and look at your code with interest. It appears to be an improved ThreadQueue implementation, but will require a lot of testing to verify. Before I invest this time, I would like to know what problem it is solving. I see your dislikes a - e, but these are not really problems, only architectural critiques. Have you done any benchmarks that show that the new SingleThreadQueue speeds up rendering? Your logical argument that it should be more efficient is persuasive, but I have been surprised by Java before. respectfully, Larry Becker On 5/23/07, *Sascha L. Teichmann* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi together, as some of you may already know i have my dislikes against ThreadQueue [1] (Hi, Larry!) see my mail [2] a - It forks a new thread for any Runnable it processes. b - It has an ugly busy wait loop inside. c - The event listener for empty queue fires to often. d - The default ThreadQueue is some kind of thread serializer. e - The DB/WMS ThreadQueue has no public access. Now I've written a sub class of ThreadQueue: SingleThreadQueue (see attachment). This one deals with the issues a, b and d. I also attached a patch against RenderingManager [3] to handle e. The new class (to be placed in package com.vividsolutions.jump.workbench.ui.renderer) is a drop-in replacement for the default ThreadQueue in RenderingManager. Not for the ThreadQueue that handles the DB/WMS layers. Because Jon limited the number of parallel threads in default queue to 1 I see no reason why to fork a new thread for each Runnable it processes. Thread creation/shutdown is fairly expensive. Instead a single background thread is started which processes the Runnables one by one. If the thread is idle for 30 secs it shuts itself
