Forgot to reply all.
On Aug 14, 2010, at 10:28 AM, Justin Deoliveira <[email protected]>
wrote:
Hi Andrea,
Thanks for the feedback, comments inline.
On Sat, Aug 14, 2010 at 2:06 AM, Andrea Aime <[email protected]>
wrote:
On Fri, Aug 13, 2010 at 5:02 PM, Justin Deoliveira <[email protected]
> wrote:
> Hi all,
> I would like permission to add a monitoring community module to
svn. I have
> done some work documenting what exactly the module does and a bit
about its
> design here:
> http://geoserver.org/display/GEOS/Monitoring
> The code I would like to commit was originally a prototype
developed by
> Andrea that I have been trying to move forward.
Hi there,
the development is sure exciting, I'm pretty sure everybody with a
GeoServer in production would like to know where the requests come
from, what are the most requests layers, and who's the damn user that
is using 99.5% of your computing resources :-p (especially the
latter!).
Some comments about the wiki page:
- the "response" table has a "start time - The time stamp the response
to the request started.". Isn't that already part of the request
stats?
Yeah, when thinking about the model originally i thought it might be
nice to track when the response started vs when the request
started... maybe to weed out those requests that are taking a long
time to process vs long time to encode the response. But yeah, not
sure how useful that will be. I have not added that to the module
yet. It is implemented as it originally was, request start time and
total time
- when you say "http interface" you mean a REST one (RESTful or REST
like)?
Yes :) I dare not call it a "REST interface" and risk getting called
out for misusing the term.
One thing I find is missing is how to deal with the statistics
gathering overhead.
During the workflow there are three distintinct points that take quite
a bit of time to be carried out:
- reverse DNS lookup of the client IP (might even time out, which
would take various seconds to happen)
- saving/updating the request information on a database (especially if
it's sitting on another machine/is busy, which is likely the case in a
cluster setup)
- gathering the geo-location from the IP
All of these should be performed in a separate thread, other than the
request one, to make sure the server still have resources to respond.
Even doing them at the end of the request cycle would not cut it
because that would keep the thread from responding to other requests
and could badlly interact with the control-flow extension making the
latter think the request is still in progress.
Yeah, this is one thing I have added to the original code. Basically
the monitor maintains a ThreadPoolExectorService to be used for post
processing operations such as this. Seems to work well enough. What
are your thoughts on fixed size thread pool vs dynamic/cached one?
A system with the requests threads feeding into a queue and a thread
group reading from the queue and performing the long operations is
probably better (with more than one thread doing the long operations
since those often involve network communications)
Another thing I have added and am experimenting with is doing all
database transactions in a separate thread as well. However in order
to do this some serialization is required which you don't really
want to mix up database transactions that should be executed
serially. So I added (actually looked it up online and found someone
with the same problem ) a sort of pipelining executor service.
Basically it is a map where the key is some id (in this case the
request id) and the values are synchronized queues of tasks to
perform. Then a single thread continually scans the map for jobs to
execute. When it finds one it delegates it out to a thread pool
executor service. When that task is complete the next job in that
same queue can execute.
Anyways, this is something i implemented only for a specific monitor
dao (not the hibernate one), so it is easily disabled.
My 2 cents
Cheers
Andrea
PS: oh, btw, +1 on adding the community module of course
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
------------------------------------------------------------------------------
This SF.net email is sponsored by
Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
