Limping on after an OOME is never a good idea. Our strategy for handling exhaustion of the in-memory store will be to set a byte-limit on the store and keep a count of messages sizes added/removed. The JVM will be allocated enough heap to hold such a store, so we can cleanly detect exhaustion of the store, without encountering OOME. OOME may still occurr in other places, for example, unbounded buffers used by mina, but if it does, its a bug and something in need of fixing to make things more robust (for example, use a bounded buffer and block when full).
On 18/05/07, Marnie McCormack <[EMAIL PROTECTED]> wrote:
Also meant to mention that I have concerns around our existing handling of OutOfMemory which imho we shouldn't try to handle and survive. However, I know lots of our users feel differently about that and would prefer the broker to limp on ..... I think it's highly dubious to try and 'manage' and OoM situation using queue limits etc. But I know others differ here ! Bfn, Marnie On 5/18/07, Marnie McCormack <[EMAIL PROTECTED]> wrote: > > Hi Rupert & Arnaud, > > Just been thinking about how we might sensibly go about making any changes > to method signatures as backwards compatible as possible, I guess only > really in terms of any changes which ripple through to the client code - > which I'm guessing some will ? > > For example, I'm thinking that replacing some Exceptions with Runtimes > will cause compilation failures/warnings for our clients - unless we really > don't have changes to the exposed API on the client side ? And are we saying > that all Runtimes should be/will be handled as a System.exit() case ? > > The only other area I'm aware of that might cause our existing users > issues is around the handling of undeliverable messages (i.e. where we're > using the immediate flag to detect a 'no consumers on this queue' scenario). > I know we've already got a user problem here in terms of the M2 broker > behaving differently from it's predecessors. (I know it's documented locally > Rupert though probably not in Apache JIRAs ?) > > A related, but more general problem, is the correct use of exception > listener. It's not too clear what exceptions ought to always go to te > listener and which should not. This seems like a minor issue, but since the > listener interface does not expose any exceptions it gets tricky in that it > forces the user's client code to handle to exception within the listener > method(s) which is not very flexible imho. > > I'm a little rusty (for obvious - at least in person reasons) so apologies > if I've missed a change/beat :-) > > Hth, > Bfn, > Marnie > > On 5/18/07, Arnaud Simon <[EMAIL PROTECTED]> wrote: > > > > Rupert, > > > > > We know there are some things about the existing exception handling > > that are > > > not right. In particular when the broker encounters unrecoverable > > > conditions, it will often go limping on in an unknown state, when > > really it > > > should simply terminate the JVM. > > > > +1 > > > > > There is also some dodgy logging code, spread throughout the codebase, > > that > > > causes exceptions to be logged mutliple times, in constructors, on > > creation, > > > in catch and rethrow scenarios and so on; I've mailed about this > > before. I > > > had to write a document explaining to sys admins what to do when they > > see > > > various log messages, so as a side to this I have compiled a list of > > todo's > > > to eliminate the dodgy logging. Logging will be done where exceptions > > are > > > handled and never where they are created (at least logging above the > > debug > > > level, which is for devs only and can occur anywhere we like). > > > > I agree with that, I'll be careful now as I may have logged my > > exceptions when throwing them. > > > > > Some thoughts: > > > > > .... > > > So, I think its correct that you have made message store throw a > > checked > > > QueueDoesntExistException, if a method handler looks up a non-existant > > > queue, it might translate that into a 404. Other code that calls that > > > method, for example during start-up and configuration, will deal with > > that > > > exception in its own way, and not as a 404, since it doesn't go > > through the > > > protocol. > > > > I agree 100% with what you are saying. I have introduced > > InternalErrorException because it should be translated into an error > > code. This is part of the dtx API. But however for consistency and for > > avoiding this exception being abused we should use a Runtime one. > > > > > One thing I'm keen on is making exceptions only have one constructor, > > and > > > passing in optional arguments as nulls. Its not a huge deal, its just > > that I > > > like to Javadoc stuff and its a pain to write three lots of Javadoc > > and > > > contructors for such trivial things as exceptions. I never understand > > why it > > > is that exceptions in particular seem to end up with huge numbers of > > > different constructors for every permutation of options. AMQException > > has 4 > > > and thats after I got rid of another 3! On that note, I think a bit of > > > javadoc on all exception classes explaining the conditions under which > > the > > > exception may be thrown, is extremely usefull. > > > > +1 > > > > As I don't want to walk on your toes we should coordinate. One way of > > doing it is for you to define the new hierarchy. I can then go through > > my code for: > > 1) removing useless logging statements > > 2) use your Exception hierarchy > > > > What do you think? BTW I have already written a JDBC store that uses the > > same exception strategy than before. I suggest that we check it in as it > > is. I'll update the code later based on your exception hierarchy. > > > > Arnaud > > > > > > > > > > >
