sorry for the delay in reply. was dealing with a family issue that I needed to prioritize...
On Wed, Oct 21, 2015 at 6:52 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > Right off the top, can't you use INDIVIDUAL_ACK here, rather than > committing transactions? That seems like the ideal mode to let you choose > which messages to ack without having to ack all the ones up to a certain > point. > > I thought about that. We had moved to sessions to avoid over-indexing because our tasks create more messages and this way I can bulk commit them as one unit. But maybe if I just deal with the "at least once" semantics while the transactions aren't combined I'll just execute a message at least once. But there might be a failure scenario where we execute the second message hundreds of times where if it was a transaction this could be avoided. > > Also, I'm curious about how a 30-second message with a prefetch size of 1 > results in a 5-minute latency; why isn't that 2 * 30 seconds = 1 minute? > > It's because I have one connection per thread per server. So if we have 10 servers, each thread has ten sessions. and if prefetch is 1 then that means I prefetch 10 total messages. If each message takes 30 seconds to execute that thread will take a while to handle all ten. This leads to significant latency. I pushed some code last week to instrument this and our average latency right now is 3-5 minutes between prefetching a message and servicing a message. Fortunately there's a timestamp added on prefetch so I can just take the current time that I am executing the message/task and then subtract the prefetch time to compute the latency. Kevin -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts>
