Re: Notification strategy for OutOfMemoryError

2014-01-24 Thread Bill Davidson
On 1/23/2014 5:21 PM, Christopher Schultz wrote: >If you'd care to post your code to either the list or onto the wiki, I'm >sure it would be useful to someone. Feel free to trim-out huge sections >of the code and say "make this fit your environment", etc. if you don't >want to show everyone how ba

Re: Notification strategy for OutOfMemoryError

2014-01-23 Thread Howard W. Smith, Jr.
On Thu, Jan 23, 2014 at 8:21 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > Glad to see my thoughts were useful. If you'd care to post your code > to either the list or onto the wiki, I'm sure it would be useful to > someone. > +1 I love it when others share code, and thanks for

Re: Notification strategy for OutOfMemoryError

2014-01-23 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bill, On 1/23/14, 8:08 PM, Bill Davidson wrote: > On 12/11/2013 11:42 PM, André Warnier wrote: >> The original issue of the OP was to be notified ASAP when an OOM >> occurs. And he indicated that an OOM resulted in a message in the >> logs. So, "som

Re: Notification strategy for OutOfMemoryError

2014-01-23 Thread Bill Davidson
On 12/11/2013 11:42 PM, André Warnier wrote: The original issue of the OP was to be notified ASAP when an OOM occurs. And he indicated that an OOM resulted in a message in the logs. So, "something" is already catching the OOM exception, to write this line in the logs. On the other hand, there i

Re: Notification strategy for OutOfMemoryError

2013-12-12 Thread André Warnier
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 12/12/13, 2:42 AM, André Warnier wrote: The original issue of the OP was to be notified ASAP when an OOM occurs. And he indicated that an OOM resulted in a message in the logs. So, "something" is already catch

Re: Notification strategy for OutOfMemoryError

2013-12-12 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 12/12/13, 2:42 AM, André Warnier wrote: > The original issue of the OP was to be notified ASAP when an OOM > occurs. And he indicated that an OOM resulted in a message in the > logs. So, "something" is already catching the OOM exception, t

Re: Notification strategy for OutOfMemoryError

2013-12-11 Thread André Warnier
Guys, just wondering.. The original issue of the OP was to be notified ASAP when an OOM occurs. And he indicated that an OOM resulted in a message in the logs. So, "something" is already catching the OOM exception, to write this line in the logs. On the other hand, there is ample literature av

Re: Notification strategy for OutOfMemoryError

2013-12-11 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bill, On 12/11/13, 12:53 PM, Bill Davidson wrote: > On 12/11/2013 7:14 AM, Christopher Schultz wrote: >> 3. cath IOException in a filter and set an application attribute. Check this attribute from your monitor. I've been consi

Re: Notification strategy for OutOfMemoryError

2013-12-11 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bill, On 12/11/13, 12:53 PM, Bill Davidson wrote: > On 12/11/2013 7:14 AM, Christopher Schultz wrote: >> 3. cath IOException in a filter and set an application attribute. Check this attribute from your monitor. I've been consi

Re: Notification strategy for OutOfMemoryError

2013-12-11 Thread Bill Davidson
On 12/11/2013 7:14 AM, Christopher Schultz wrote: 3. cath IOException in a filter and set an application attribute. Check this attribute from your monitor. I've been considering doing this, because I can rig it so that the error handler does not actually require any memory to run. The problem

Re: Notification strategy for OutOfMemoryError

2013-12-11 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bill, On 12/9/13, 8:20 PM, Bill Davidson wrote: > On 12/9/2013 3:12 PM, Christopher Schultz wrote: >> >> Was it a transient error, or a chronic condition? A single thread >> can, for instance, spew objects into its stack or eden space >> exhausting

Re: Notification strategy for OutOfMemoryError

2013-12-10 Thread Bill Davidson
On 12/9/2013 5:20 PM, Bill Davidson wrote: On 12/9/2013 3:12 PM, Christopher Schultz wrote: 1. Use -XX:OnOutOfMemoryError="cmd args;cmd args" Rig this to email you, register a passive-check data point with your monitoring server, etc. Just remember that OOMEs happen for a number of reasons. Yo

Re: Notification strategy for OutOfMemoryError

2013-12-10 Thread André Warnier
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bill, On 12/9/13, 5:38 PM, Bill Davidson wrote: Last week, one of my servers got an OutOfMemoryError at approximately 1:21pm. :( It's worth pointing out that this is not a trivial issue. My monitoring software whic

Re: Notification strategy for OutOfMemoryError

2013-12-09 Thread Bill Davidson
On 12/9/2013 3:12 PM, Christopher Schultz wrote: Was it a transient error, or a chronic condition? A single thread can, for instance, spew objects into its stack or eden space exhausting memory but, when that thread hits the OOME, all those objects are freed which basically recovers from the sit

Re: Notification strategy for OutOfMemoryError

2013-12-09 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bill, On 12/9/13, 5:38 PM, Bill Davidson wrote: > Last week, one of my servers got an OutOfMemoryError at > approximately 1:21pm. :( It's worth pointing out that this is not a trivial issue. > My monitoring software which does a heart beat check

Notification strategy for OutOfMemoryError

2013-12-09 Thread Bill Davidson
Last week, one of my servers got an OutOfMemoryError at approximately 1:21pm. My monitoring software which does a heart beat check once per minute did not notice until 3:01pm. Heart beat kept working for over an hour and a half. During that time my high capacity high availablity 24/7 applicatio