Hi guys,
I'm interested to know whether either of you came up with any further
leads on this issue. I'm experiencing a similar thing, using version 2.3.5.
My instance of GeoServer is primarily used to serve tiles of
weather-related GeoTIFF (relatively small in size) layers which are
created and aged dynamically using the REST API. Authentication is
required (implemented using "data security" rules on workspaces) and is
provided with each request by means of HTTP header proxy authentication.
Under light to moderate load GeoServer operates normally for anywhere
between minutes and hours before OGC service requests suddenly hang,
despite the Tomcat being responsive to other requests.
A thread dump shows many threads in the WAITING state stemming from
org.geoserver.security.auth.LRUAuthenticationCacheImpl.
Regards,
Andrew
On 20/08/2013 17:11, Eric Smets wrote:
Hello,
We have had this issue with a version 2.2.0 after upgrading the major
cause of the huge number of threads disappeared,
but we still have the issue that from time to time a timer thread was
using 100% CPU.
What we saw is that often (always) the CPU issue appears after an
outOf memory problem, than an other issue caused this.
In our case this happend after a java.lang.OutOfMemoryError: Java heap
space on a rendering task, we are still looking
what could trigger this issue.
Regards,
Eric
---
Op 19/08/2013 22:33, Mike Grogan schreef:
This looks like it may be getting stuck in a TimerTask
in LRUAuthenticationCacheImpl.java that is searching for expired
authentication tokens. The stack from the process utilizing 100% CPU
is consistently:
"Timer-0" daemon prio=10 tid=0x00007f57495e1800 nid=0x6d0e runnable
[0x00007f5743546000]
java.lang.Thread.State: RUNNABLE
at
org.geoserver.security.auth.LRUAuthenticationCacheImpl$1.run(LRUAuthenticationCacheImpl.java:66)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
In this case, my process id was 27918, which matches the hex 6d0e
above. I see this consistently after tomcat restart, though,
obviously, the PID/nid changes. Each time, though, the trace for
PID/nid returns the TimerTask above.
Looking at the source, this is a TimerTask for adding authentication
tokens to be removed.
So, Andrea, I know you and I just discussed gwc and security not
working well together earlier. I do still have auth on my data
layers ... and am considering some workarounds outside of geoserver
to help me.
Do you think this issue goes away once I remove all the
authentication to the data layers?
Even if it does, could there be cause for concern outside of my GWC
services? Could I have stumbled upon a race condition or similar
that might impact other things? Could this impact any geoserver
layers, etc., requiring authentication even outside of GWC?
Thanks,
Mike Grogan
On Mon, Aug 19, 2013 at 2:32 PM, Mike Grogan
<d.michael.gro...@gmail.com <mailto:d.michael.gro...@gmail.com>> wrote:
I have gotten a thread dump with "kill -3". Care to take a look
at it?
- Mike
On Mon, Aug 19, 2013 at 2:19 PM, Andrea Aime
<andrea.a...@geo-solutions.it
<mailto:andrea.a...@geo-solutions.it>> wrote:
On Mon, Aug 19, 2013 at 7:52 PM, Mike Grogan
<d.michael.gro...@gmail.com
<mailto:d.michael.gro...@gmail.com>> wrote:
Sorry for # of questions from me recently ... trying to
get my first geoserver instance running (and one question
resulted in legitimate bug report) ...
I am now having an issue with GWC ... specifically using
the gmaps service ... taking a thread to 100% CPU use and
hanging ... with no more response from the service. Any
pointers toward debugging this?
This seems to happen randomly ... sometimes it will
happen quickly after a tomcat restart ... sometimes I
won't see it again for a few hours. Not a huge load on
server. Just me in development zooming and panning the
maps, requesting tiles from the gmaps gwc service.
jstack can't even seem to walk the stack in the hung
thread. I get a "Error occurred during stack walking:
java.lang.NullPointerException" from jstack on the
specific PID
Hmm... not sure, but have you tried sending sigquit instead?
Quoting from
http://www.crazysquirrel.com/computing/java/basics/java-thread-dump.jspx
-----------------------------------------
kill -QUIT process_id
There process_id is the process number of the running Java
process. The thread dump will be sent to wherever standard
out is redirected too.
-----------------------------------------
Cheers
Andrea
--
==
Our support, Your Success! Visit
http://opensdi.geo-solutions.it for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313 <tel:%2B39%200584%20962313>
fax: +39 0584 1660272 <tel:%2B39%200584%201660272>
mob: +39 339 8844549 <tel:%2B39%20%C2%A0339%208844549>
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
Eric smetseric.sm...@fks.be
FKS bvba - Formal and Knowledge Systemshttp://www.fks.be/
Schampbergstraat 32 Tel: ++32-(0)11-21 49 11
B-3511 Hasselt Fax: ++32-(0)11-22 04 19
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
