Dropping slow AMQP consumer behaviour

2014-12-23 Thread Marcel Meulemans
I have a question about what the expected behaviour of dropping a slow AMQP
consumer is/should be. I am using 5.11-SNAPSHOT. The behaviour I am
observing is as follows:

- I have an AMQP (proton-c) client that is consuming messages from ActiveMQ
from a specific queue via a specific AMQP link.
- The client is on a different machine from ActiveMQ
- While device is correctly consuming messages turn off the wifi on the
AMQP machine (there are still messages in this queue).
- Obviously communication stops.
- Turn back on the wifi.
- ActiveMQ will have dropped the slow consumer: 2014-12-22 11:57:07,414 |
INFO  | aborting slow consumer: ID:ubuntu-hp-54642-1418997390497-2:16:0:0
for destination:queue://somequeue |
org.apache.activemq.broker.region.policy.AbortSlowConsumerStrategy |
ActiveMQ Broker[localhost] Scheduler
- Both computers still consider the TCP connection to be active (the
connection is still listed also by ActiveMQ
- On the client nothing happens: no messages are consumed anymore
(obviously), AMQP link is open on remote and local side.

Currently I seem to be unable to recover from this situation because the
client does not know ActiveMQ has drop it as a consumer. I expected
ActiveMQ to close the AMQP link, or maybe even the session, after which
recovering would be fairly trivial.

Any ideas?

-- 
Marcel


Re: leveldb issue

2014-12-23 Thread Christian Grassi
Hi Kevin,
Do you have any update on this issue?
It looks to me that this config is not really production ready.
What do you think?

Is someone of the development team having a look?

Chris

Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com ha
scritto:


Re: Dropping slow AMQP consumer behaviour

2014-12-23 Thread Tim Bain
Are you using a non-zero wireFormat.maxInactivityDuration?  This type of
situation is exactly what it's intended to detect.
On Dec 23, 2014 2:51 AM, Marcel Meulemans m.meulem...@tkhinnovations.com
wrote:

 I have a question about what the expected behaviour of dropping a slow AMQP
 consumer is/should be. I am using 5.11-SNAPSHOT. The behaviour I am
 observing is as follows:

 - I have an AMQP (proton-c) client that is consuming messages from ActiveMQ
 from a specific queue via a specific AMQP link.
 - The client is on a different machine from ActiveMQ
 - While device is correctly consuming messages turn off the wifi on the
 AMQP machine (there are still messages in this queue).
 - Obviously communication stops.
 - Turn back on the wifi.
 - ActiveMQ will have dropped the slow consumer: 2014-12-22 11:57:07,414 |
 INFO  | aborting slow consumer: ID:ubuntu-hp-54642-1418997390497-2:16:0:0
 for destination:queue://somequeue |
 org.apache.activemq.broker.region.policy.AbortSlowConsumerStrategy |
 ActiveMQ Broker[localhost] Scheduler
 - Both computers still consider the TCP connection to be active (the
 connection is still listed also by ActiveMQ
 - On the client nothing happens: no messages are consumed anymore
 (obviously), AMQP link is open on remote and local side.

 Currently I seem to be unable to recover from this situation because the
 client does not know ActiveMQ has drop it as a consumer. I expected
 ActiveMQ to close the AMQP link, or maybe even the session, after which
 recovering would be fairly trivial.

 Any ideas?

 --
 Marcel



Re: leveldb issue

2014-12-23 Thread Tim Bain
Is there a bug report in Jira for it, and has someone been able to identify
a configuration and set of steps to reliably reproduce it?  If not, the
odds are good that no one is actively investigating it, so doing those
things would be the first step towards getting the problem fixed.
On Dec 23, 2014 5:47 AM, Christian Grassi christiangra...@gmail.com
wrote:

 Hi Kevin,
 Do you have any update on this issue?
 It looks to me that this config is not really production ready.
 What do you think?

 Is someone of the development team having a look?

 Chris

 Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com ha
 scritto:



Re: Dropping slow AMQP consumer behaviour

2014-12-23 Thread Marcel Meulemans
 Are you using a non-zero wireFormat.maxInactivityDuration?  This type of
 situation is exactly what it's intended to detect.

I don't think the AMQP protocol converter implements an
InactivityMonitor a.t.m. and I won't help I think. The problem is not
that there is no communication between the two parties (the
communication is only interrupted briefly), the problem is that the
client side does not know that ActiveMQ has dropped him as a consumer
of a particular queue.

-- 
Marcel


Not Getting to First Base

2014-12-23 Thread tlwtheq
Hi,
I'm having trouble getting activemq to start at all,

Error message is:

Error: Could not find or load main class Files

Note that I both my PATH and CLASSPATH variables are pointing to the
requisite jars.

So what else am I missing?

Thanks,
Tara



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Not-Getting-to-First-Base-tp4689208.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: leveldb issue

2014-12-23 Thread Christian Grassi
the jira is the folowing
https://issues.apache.org/jira/browse/AMQ-5300

It was opened on july.
I can reproduce it always, with the configuration in my first mail.

Christian

On Tue, Dec 23, 2014 at 3:54 PM, Tim Bain tb...@alumni.duke.edu wrote:

 Is there a bug report in Jira for it, and has someone been able to identify
 a configuration and set of steps to reliably reproduce it?  If not, the
 odds are good that no one is actively investigating it, so doing those
 things would be the first step towards getting the problem fixed.
 On Dec 23, 2014 5:47 AM, Christian Grassi christiangra...@gmail.com
 wrote:

  Hi Kevin,
  Do you have any update on this issue?
  It looks to me that this config is not really production ready.
  What do you think?
 
  Is someone of the development team having a look?
 
  Chris
 
  Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com ha
  scritto:
 



Re: Dropping slow AMQP consumer behaviour

2014-12-23 Thread Tim Bain
I'd have thought that if the broker no longer considered the consumer to be
active, it would have stopped sending keep-alive packets, which would then
cause the consumer to abort, but I haven't looked at that code in any depth.

Have you tried setting abortConnection=true on the
AbortSlowConsumerStrategy, to have the broker close the TCP connection when
it aborts the consumer, rather than just sending a request to the consumer
to ask it nicely to abort?  I'm not sure if it will change the behavior,
but it's worth a shot.

On Tue, Dec 23, 2014 at 8:27 AM, Marcel Meulemans 
m.meulem...@tkhinnovations.com wrote:

  Are you using a non-zero wireFormat.maxInactivityDuration?  This type of
  situation is exactly what it's intended to detect.

 I don't think the AMQP protocol converter implements an
 InactivityMonitor a.t.m. and I won't help I think. The problem is not
 that there is no communication between the two parties (the
 communication is only interrupted briefly), the problem is that the
 client side does not know that ActiveMQ has dropped him as a consumer
 of a particular queue.

 --
 Marcel



Re: Not Getting to First Base

2014-12-23 Thread Tim Bain
Are you running Windows with a space in one of your directory names?  The
first two search results I got when I Googled for your error message relate
to problems starting from a path with a space.

On Tue, Dec 23, 2014 at 8:28 AM, tlwtheq t...@shadowfirehosting.com wrote:

 Hi,
 I'm having trouble getting activemq to start at all,

 Error message is:

 Error: Could not find or load main class Files

 Note that I both my PATH and CLASSPATH variables are pointing to the
 requisite jars.

 So what else am I missing?

 Thanks,
 Tara



 --
 View this message in context:
 http://activemq.2283324.n4.nabble.com/Not-Getting-to-First-Base-tp4689208.html
 Sent from the ActiveMQ - User mailing list archive at Nabble.com.



Re: leveldb issue

2014-12-23 Thread Tim Bain
I was going to tell you that since you have a configuration that reliably
reproduces the problem, and it's not described in that JIRA (which
describes symptoms but no means of reproducing the problem), you should
update it to provide the configuration so that someone can investigate.
But looking more closely at the description of AMQ-5300, it doesn't sound
like what you're seeing; that bug is very specifically about deleting the
index files for LevelDB and then restarting the broker, whereas you haven't
described doing either of those things.  (Or maybe you did and just left it
out of your description?)  But unless you really did delete the index, your
issue is a separate bug (though the two might turn out to have the same
root cause once someone investigates) and should be submitted as a new JIRA.

On Tue, Dec 23, 2014 at 8:45 AM, Christian Grassi christiangra...@gmail.com
 wrote:

 the jira is the folowing
 https://issues.apache.org/jira/browse/AMQ-5300

 It was opened on july.
 I can reproduce it always, with the configuration in my first mail.

 Christian

 On Tue, Dec 23, 2014 at 3:54 PM, Tim Bain tb...@alumni.duke.edu wrote:

  Is there a bug report in Jira for it, and has someone been able to
 identify
  a configuration and set of steps to reliably reproduce it?  If not, the
  odds are good that no one is actively investigating it, so doing those
  things would be the first step towards getting the problem fixed.
  On Dec 23, 2014 5:47 AM, Christian Grassi christiangra...@gmail.com
  wrote:
 
   Hi Kevin,
   Do you have any update on this issue?
   It looks to me that this config is not really production ready.
   What do you think?
  
   Is someone of the development team having a look?
  
   Chris
  
   Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com ha
   scritto:
  
 



Re: Diagnosing long term ActiveMQ memory leaks?

2014-12-23 Thread Tim Bain
BTW, you never did answer the question about which GC strategy you're
using, and it occurred to me that if you're using CMS, lots of full GCs
that don't actually reclaim much memory after a long time being up is the
classic failure scenario for CMS.  It happens when Old Gen gets fragmented,
which in turn happens because CMS is a non-compacting GC strategy in Old
Gen.  If you're using CMS and seeing continual full GCs, you should look at
whether G1 GC would be better for your needs.

On Mon, Dec 22, 2014 at 11:03 PM, Kevin Burton bur...@spinn3r.com wrote:

 Great feedback.  Thanks btw. I’m working on getting up better JMX monitors
 so I can track memory here more aggressively.  Bumping up memory by 1.5G
 temporarily fixed the problem. However, it seems correlated to the number
 of connections.  So I suspect I’ll just hit this again in the next few
 weeks.

 By that time I plan to have better JMX monitors in place to resolve this.

 On Sat, Dec 20, 2014 at 10:28 PM, Tim Bain tb...@alumni.duke.edu wrote:
 
  What JVM are you using, and what GC strategy with which options?  And for
  that matter, what broker version?
 
  With Hotspot 7u21 and G1GC while running a long-running performance
 stress
  test I've observed that Old Gen use increases over time (despite the fact
  that G1GC is supposed to collect Old Gen during its normal collection
  operations), and GCs against Old Gen happen semi-continually after Old
 Gen
  hits a certain memory threshold.  However, unlike what you're observing,
 1)
  the GCs I saw were Old Gen GCs but not full GCs (G1 allows GCing Old Gen
  during incremental GCs), 2) the broker remains responsive with reasonable
  pause times close to my target, and 3) once Old Gen hits the 90%
 threshold
  that forces a full GC, that full GC is able to successfully collect
 nearly
  all of the Old Gen memory.  My conclusion from that was that although
  objects were being promoted to Old Gen (and I tried unsuccessfully to
  prevent that from occurring, see
 
 
 http://activemq.2283324.n4.nabble.com/Potential-Bug-in-Master-Slave-with-Replicated-LevelDB-Store-td4686450.html
  ),
  nearly all of them were unreachable by the time a full GC actually
  occurred.
 
  So if you're seeing continual full GCs (not just Old Gen GCs if you're
  using G1) that don't actually free any Old Gen memory, then what you're
  seeing different behavior than I saw, and it means that the objects in
 Old
  Gen are still reachable.  One possible reason for that would be messages
  still being held in destinations waiting to be consumed; look for queues
  without consumers (especially DLQs), as well as durable subscribers that
  are offline.  If you're certain that's not the case, maybe you can post
  some of the results of analyzing the heap snapshot so that people who
 know
  the codebase better could see if anything jumps out?
 
  On Sat, Dec 20, 2014 at 1:51 PM, Kevin Burton bur...@spinn3r.com
 wrote:
 
   I’m trying to diagnose a long term memory leak with ActiveMQ.
  
   Basically , my app runs fine for about a week or so, then goes to 100%
  CPU
   doing continually full GCs back to back.
  
   No work is done during that period.
  
   I have a large number of sessions to the AMQ box, but things are fine
 on
   startup.
  
   It’s entirely possible that y app isn’t releasing resources, but I”m
  trying
   to figure out the best way to track that down.
  
   I’m using org.apache.activemq.UseDedicatedTaskRunner=false so that
 thread
   pools are used.  Which apparently can cause a bit of wasted memory.
  
   I have a heap snapshot.  I loaded that into the Eclipse Memory Analyzer
  and
   didn’t see any obvious candidates but of course I’m not an expert on
 the
   ActiveMQ code base.
  
   Are there any solid JMX counters I can track during this process?
 Number
   of sessions? etc.
  
   --
  
   Founder/CEO Spinn3r.com
   Location: *San Francisco, CA*
   blog: http://burtonator.wordpress.com
   … or check out my Google+ profile
   https://plus.google.com/102718274791889610666/posts
   http://spinn3r.com
  
 


 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



Replicated LevelDB

2014-12-23 Thread James Green
Ref http://activemq.apache.org/replicated-leveldb-store.html

Is this page actually detailing that Zookeeper will maintain one master
BROKER and that writes through this broker will be replicated to slave
brokers?

I read - interpreted - the URL and title as being about leveldb data being
replicated, and that perhaps the broker software would know nothing about
the storage. But reading the article further perhaps it's far more reaching
than the title infers...

James


Re: Not Getting to First Base

2014-12-23 Thread tlwtheq
Yes, I am running with path name
C:\Program Files (x86)\apache-activemq-5.10.0\bin\activemq  .

How else would I execute?  I know Windows used to have nicknames for such
directories
as Program Files (x86) but it doesn't seem to have that now.   (Or does
it?  Pardon me,
I'm not a really bid proponent of Windows. Would rather be trying this on a
Linux box,
but I don't have that option.  :)  )

Even using an Explorer window and clicking on the activemq.bat yields the
same error.



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Not-Getting-to-First-Base-tp4689208p4689219.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: Not Getting to First Base

2014-12-23 Thread Tim Bain
As I understand it, extended Windows paths always have an eight-character
short version formed by the first N characters (usually 6) plus ~ and one
or more digits.  Usually Program Files was PROGRA~1 (you can confirm
that by doing dir C:\PROGRA~1 and seeing if it matches; if not, try
PROGRA~2, PROGRA~3, etc. till you find it), so you could try specifying the
path that way.

Tim

On Tue, Dec 23, 2014 at 11:09 AM, tlwtheq t...@shadowfirehosting.com
wrote:

 Yes, I am running with path name
 C:\Program Files (x86)\apache-activemq-5.10.0\bin\activemq  .

 How else would I execute?  I know Windows used to have nicknames for such
 directories
 as Program Files (x86) but it doesn't seem to have that now.   (Or does
 it?  Pardon me,
 I'm not a really bid proponent of Windows. Would rather be trying this on a
 Linux box,
 but I don't have that option.  :)  )

 Even using an Explorer window and clicking on the activemq.bat yields the
 same error.



 --
 View this message in context:
 http://activemq.2283324.n4.nabble.com/Not-Getting-to-First-Base-tp4689208p4689219.html
 Sent from the ActiveMQ - User mailing list archive at Nabble.com.



ActiveMQ 5.10.0 and microsoft_jdbc_driver_4_1_for_sql_server

2014-12-23 Thread erstanl
So I recently updated to jre7 and have noticed a strange problem between jre7
and the microsoft_jdbc_driver_4_0_for_sql_server driver. Looks like a known
issue, resulting in the recommendation to update to
microsoft_jdbc_driver_4_1_for_sql_server. However, it would appear that 5.10
does not support this newer driver:

2014-12-18 14:59:03,347 | INFO  | Database adapter driver override not found
for : [microsoft_jdbc_driver_4_1_for_sql_server].  Will use default
implementation. | org.apache.activemq.store.jdbc.JDBCPersistenceAdapter |
WrapperSimpleAppMain
2014-12-18 14:59:03,631 | INFO  | Database lock driver override not found
for : [microsoft_jdbc_driver_4_1_for_sql_server].  Will use default
implementation. | org.apache.activemq.store.jdbc.JDBCPersistenceAdapter |
WrapperSimpleAppMain

It then sleeps for a period, and tries again to no avail.

I have seen posts from others stating that if I change from:

persistenceAdapter
jdbcPersistenceAdapter dataSource=#mssql-ds
createTablesOnStartup=true/
/persistenceAdapter
 
to:

persistenceAdapter
jdbcPersistenceAdapter dataSource=#mssql-ds
createTablesOnStartup=true
adapter
transact-jdbc-adapter /
/adapter
databaseLocker
transact-database-locker/
/databaseLocker
/jdbcPersistenceAdapter
/persistenceAdapter

It works. And it does in my case as well. I would like to better understand:

A) Why am I not able to use the microsoft_jdbc_driver_4_1_for_sql_server
driver without changing the config?
B) What is the second XML snippet actually telling ActiveMQ to do?

Thanks in advance.



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-5-10-0-and-microsoft-jdbc-driver-4-1-for-sql-server-tp4689221.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: leveldb issue

2014-12-23 Thread Kevin Burton
I’m still working on it and trying to figure it out.. it might take me some
time to reproduce it as it seems to have a few issues that might be
compounding the issue.

Bumping up memory definitely seemed to help resolve the issue.

I think what would be ideal is for me to write a producer/consumer and then
continually restart activemq+leveldb while it’s executing to see what
happens.

My their is that there’s data loss.  But I’m not sure if it’s also due to
my GC issue I was seeing before.

On Tue, Dec 23, 2014 at 4:46 AM, Christian Grassi christiangra...@gmail.com
 wrote:

 Hi Kevin,
 Do you have any update on this issue?
 It looks to me that this config is not really production ready.
 What do you think?

 Is someone of the development team having a look?

 Chris

 Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com ha
 scritto:




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: leveldb issue

2014-12-23 Thread Kevin Burton
  that bug is very specifically about deleting the
index files for LevelDB and then restarting the broker, whereas you haven't
described doing either of those things.

Yes.. I agree as well.  That’s why I didn’t follow up on that thread.  I’m
concerned about the error message though.

On Tue, Dec 23, 2014 at 8:50 AM, Tim Bain tb...@alumni.duke.edu wrote:

 I was going to tell you that since you have a configuration that reliably
 reproduces the problem, and it's not described in that JIRA (which
 describes symptoms but no means of reproducing the problem), you should
 update it to provide the configuration so that someone can investigate.
 But looking more closely at the description of AMQ-5300, it doesn't sound
 like what you're seeing; that bug is very specifically about deleting the
 index files for LevelDB and then restarting the broker, whereas you haven't
 described doing either of those things.  (Or maybe you did and just left it
 out of your description?)  But unless you really did delete the index, your
 issue is a separate bug (though the two might turn out to have the same
 root cause once someone investigates) and should be submitted as a new
 JIRA.

 On Tue, Dec 23, 2014 at 8:45 AM, Christian Grassi 
 christiangra...@gmail.com
  wrote:

  the jira is the folowing
  https://issues.apache.org/jira/browse/AMQ-5300
 
  It was opened on july.
  I can reproduce it always, with the configuration in my first mail.
 
  Christian
 
  On Tue, Dec 23, 2014 at 3:54 PM, Tim Bain tb...@alumni.duke.edu wrote:
 
   Is there a bug report in Jira for it, and has someone been able to
  identify
   a configuration and set of steps to reliably reproduce it?  If not, the
   odds are good that no one is actively investigating it, so doing those
   things would be the first step towards getting the problem fixed.
   On Dec 23, 2014 5:47 AM, Christian Grassi christiangra...@gmail.com
 
   wrote:
  
Hi Kevin,
Do you have any update on this issue?
It looks to me that this config is not really production ready.
What do you think?
   
Is someone of the development team having a look?
   
Chris
   
Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com ha
scritto:
   
  
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Diagnosing long term ActiveMQ memory leaks?

2014-12-23 Thread Kevin Burton
The default GC which I think is still parallel.  But I should explicitly
set it.  I’m working on getting JMX monitors up so that I can track
ActiveMQ counters but also GC state.

I can’t get a reliable and short term failure to occur so right now I’ve
focused on mitigating the issue and easily rebuilding my queue when it
crashes.

On Tue, Dec 23, 2014 at 8:55 AM, Tim Bain tb...@alumni.duke.edu wrote:

 BTW, you never did answer the question about which GC strategy you're
 using, and it occurred to me that if you're using CMS, lots of full GCs
 that don't actually reclaim much memory after a long time being up is the
 classic failure scenario for CMS.  It happens when Old Gen gets fragmented,
 which in turn happens because CMS is a non-compacting GC strategy in Old
 Gen.  If you're using CMS and seeing continual full GCs, you should look at
 whether G1 GC would be better for your needs.

 On Mon, Dec 22, 2014 at 11:03 PM, Kevin Burton bur...@spinn3r.com wrote:

  Great feedback.  Thanks btw. I’m working on getting up better JMX
 monitors
  so I can track memory here more aggressively.  Bumping up memory by 1.5G
  temporarily fixed the problem. However, it seems correlated to the number
  of connections.  So I suspect I’ll just hit this again in the next few
  weeks.
 
  By that time I plan to have better JMX monitors in place to resolve this.
 
  On Sat, Dec 20, 2014 at 10:28 PM, Tim Bain tb...@alumni.duke.edu
 wrote:
  
   What JVM are you using, and what GC strategy with which options?  And
 for
   that matter, what broker version?
  
   With Hotspot 7u21 and G1GC while running a long-running performance
  stress
   test I've observed that Old Gen use increases over time (despite the
 fact
   that G1GC is supposed to collect Old Gen during its normal collection
   operations), and GCs against Old Gen happen semi-continually after Old
  Gen
   hits a certain memory threshold.  However, unlike what you're
 observing,
  1)
   the GCs I saw were Old Gen GCs but not full GCs (G1 allows GCing Old
 Gen
   during incremental GCs), 2) the broker remains responsive with
 reasonable
   pause times close to my target, and 3) once Old Gen hits the 90%
  threshold
   that forces a full GC, that full GC is able to successfully collect
  nearly
   all of the Old Gen memory.  My conclusion from that was that although
   objects were being promoted to Old Gen (and I tried unsuccessfully to
   prevent that from occurring, see
  
  
 
 http://activemq.2283324.n4.nabble.com/Potential-Bug-in-Master-Slave-with-Replicated-LevelDB-Store-td4686450.html
   ),
   nearly all of them were unreachable by the time a full GC actually
   occurred.
  
   So if you're seeing continual full GCs (not just Old Gen GCs if you're
   using G1) that don't actually free any Old Gen memory, then what you're
   seeing different behavior than I saw, and it means that the objects in
  Old
   Gen are still reachable.  One possible reason for that would be
 messages
   still being held in destinations waiting to be consumed; look for
 queues
   without consumers (especially DLQs), as well as durable subscribers
 that
   are offline.  If you're certain that's not the case, maybe you can post
   some of the results of analyzing the heap snapshot so that people who
  know
   the codebase better could see if anything jumps out?
  
   On Sat, Dec 20, 2014 at 1:51 PM, Kevin Burton bur...@spinn3r.com
  wrote:
  
I’m trying to diagnose a long term memory leak with ActiveMQ.
   
Basically , my app runs fine for about a week or so, then goes to
 100%
   CPU
doing continually full GCs back to back.
   
No work is done during that period.
   
I have a large number of sessions to the AMQ box, but things are fine
  on
startup.
   
It’s entirely possible that y app isn’t releasing resources, but I”m
   trying
to figure out the best way to track that down.
   
I’m using org.apache.activemq.UseDedicatedTaskRunner=false so that
  thread
pools are used.  Which apparently can cause a bit of wasted memory.
   
I have a heap snapshot.  I loaded that into the Eclipse Memory
 Analyzer
   and
didn’t see any obvious candidates but of course I’m not an expert on
  the
ActiveMQ code base.
   
Are there any solid JMX counters I can track during this process?
  Number
of sessions? etc.
   
--
   
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com
   
  
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile

Re: leveldb issue

2014-12-23 Thread Tim Bain
Based on what Christian has said, it sounds like he's got a configuration
that reliably reproduces his problem whereas you're not at that point yet,
so let's have him create a JIRA bug for that so we don't lose the details
about how to produce it.  As you investigate, hopefully you'll be able
update the JIRA to give more details on what's actually going on, ideally
with a proposed patch if your investigation leads you to one.  (Thanks for
investigating to that depth, BTW.)

On Tue, Dec 23, 2014 at 4:13 PM, Kevin Burton bur...@spinn3r.com wrote:

   that bug is very specifically about deleting the
 index files for LevelDB and then restarting the broker, whereas you haven't
 described doing either of those things.

 Yes.. I agree as well.  That’s why I didn’t follow up on that thread.  I’m
 concerned about the error message though.

 On Tue, Dec 23, 2014 at 8:50 AM, Tim Bain tb...@alumni.duke.edu wrote:

  I was going to tell you that since you have a configuration that reliably
  reproduces the problem, and it's not described in that JIRA (which
  describes symptoms but no means of reproducing the problem), you should
  update it to provide the configuration so that someone can investigate.
  But looking more closely at the description of AMQ-5300, it doesn't sound
  like what you're seeing; that bug is very specifically about deleting the
  index files for LevelDB and then restarting the broker, whereas you
 haven't
  described doing either of those things.  (Or maybe you did and just left
 it
  out of your description?)  But unless you really did delete the index,
 your
  issue is a separate bug (though the two might turn out to have the same
  root cause once someone investigates) and should be submitted as a new
  JIRA.
 
  On Tue, Dec 23, 2014 at 8:45 AM, Christian Grassi 
  christiangra...@gmail.com
   wrote:
 
   the jira is the folowing
   https://issues.apache.org/jira/browse/AMQ-5300
  
   It was opened on july.
   I can reproduce it always, with the configuration in my first mail.
  
   Christian
  
   On Tue, Dec 23, 2014 at 3:54 PM, Tim Bain tb...@alumni.duke.edu
 wrote:
  
Is there a bug report in Jira for it, and has someone been able to
   identify
a configuration and set of steps to reliably reproduce it?  If not,
 the
odds are good that no one is actively investigating it, so doing
 those
things would be the first step towards getting the problem fixed.
On Dec 23, 2014 5:47 AM, Christian Grassi 
 christiangra...@gmail.com
  
wrote:
   
 Hi Kevin,
 Do you have any update on this issue?
 It looks to me that this config is not really production ready.
 What do you think?

 Is someone of the development team having a look?

 Chris

 Il giorno gio 18 dic 2014 21:50 Kevin Burton bur...@spinn3r.com
 ha
 scritto:

   
  
 



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com