[jira] [Created] (AMQ-5461) Locker is not released after starting fails

2014-11-27 Thread Shi Lei (JIRA)
Shi Lei created AMQ-5461:


 Summary: Locker is not released after starting fails 
 Key: AMQ-5461
 URL: https://issues.apache.org/jira/browse/AMQ-5461
 Project: ActiveMQ
  Issue Type: Bug
  Components: Broker
Affects Versions: 5.10.0
 Environment: JDK7, Windows 7
Reporter: Shi Lei



How to produce, 

1, Configure activemq 


  


  


  

  

  


  

  

2, start another program, ex. tomcat to listen at port 8093

3, BrokerService brokerService = 
BrokerFactory.createBroker("xbean:ActiveMQ.xml");
brokerService.start();

4, Check table activemq_lock
you can see activemq_lock.broker_name is "testBroker"
and activemq_lock.time is updated every 5 seconds

But brokerService.isStarted() is false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (AMQ-5435) Persistence Adapter Starting Thread is still alive after stopping a slave broker with lease database locker

2014-11-27 Thread Shi Lei (JIRA)

[ 
https://issues.apache.org/jira/browse/AMQ-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228075#comment-14228075
 ] 

Shi Lei commented on AMQ-5435:
--

This is the stop method
public void stop() throws Exception {
if (stopped.compareAndSet(false, true)) {
stopping.set(true);
ServiceStopper stopper = new ServiceStopper();
try {
doStop(stopper);
} catch (Exception e) {
stopper.onException(this, e);
} finally {
postStop(stopper);
}
stopped.set(true);
started.set(false);
stopping.set(false);
for(ServiceListener l:this.serviceListeners) {
l.stopped(this);
}
stopper.throwFirstException();
}
}

The problem is that after being stopped, stopping is false,

So LeaseDatabaseLocker.doStart will still in loop after the locker is stopped.

> Persistence Adapter Starting Thread is still alive after stopping a slave 
> broker with lease database locker
> ---
>
> Key: AMQ-5435
> URL: https://issues.apache.org/jira/browse/AMQ-5435
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker
>Affects Versions: 5.10.0
> Environment: Windows, JDK7
>Reporter: Shi Lei
> Attachments: patch2.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> I am using jdbc master/slave with lease database lock.
> 
> I found if I call broker.stop to stop a slave broker service (which means it 
> tries to get a lease locker and has not got yet), its Persistence Adapter 
> Starting Thread is still alive. If I create and start a new broker in the 
> same java VM, there will be 2 Persistence Adapter Starting Threads inside the 
> same java VM. At this time, if the master broker is down, the stopped broker 
> will get the database lease locker, but somehow it cannot start broker. Now I 
> have 2 broker service in the same VM. One has got the locker, but cannot 
> start broker, the other one is still requesting the locker.
> The root cause is that after stopping broker, 
> LeaseDatabaseLocker.isStopping() is false, LeaseDatabaseLocker.isStopped() is 
> true,
> In LeaseDatabaseLocker.doStart
> {code}
>  while (!isStopping()) {
> Connection connection = null;
> PreparedStatement statement = null;
> try {
> connection = getConnection();
> initTimeDiff(connection);
> statement = connection.prepareStatement(sql);
> setQueryTimeout(statement);
> now = System.currentTimeMillis() + diffFromCurrentTime;
> statement.setString(1, getLeaseHolderId());
> statement.setLong(2, now + lockAcquireSleepInterval);
> statement.setLong(3, now);
> int result = statement.executeUpdate();
> if (result == 1) {
> // we got the lease, verify we still have it
> if (keepAlive()) {
> break;
> }
> }
> reportLeasOwnerShipAndDuration(connection);
> } catch (Exception e) {
> LOG.debug(getLeaseHolderId() + " lease acquire failure: "+ e, 
> e);
> if (isStopping()) {
> throw new Exception(
> "Cannot start broker as being asked to shut down. 
> "
> + "Interrupted attempt to acquire lock: "
> + e, e);
> }
> if (handleStartException) {
> 
> lockable.getBrokerService().handleIOException(IOExceptionSupport.create(e));
> }
> } finally {
> close(statement);
> close(connection);
> }
> LOG.info(getLeaseHolderId() + " failed to acquire lease.  
> Sleeping for " + lockAcquireSleepInterval + " milli(s) before trying 
> again...");
> TimeUnit.MILLISECONDS.sleep(lockAcquireSleepInterval);
> }
> if (isStopping()) {
> throw new RuntimeException(getLeaseHolderId() + " failing lease 
> acquire due to stop");
> }
> LOG.info(getLeaseHolderId() + ", becoming master with lease expiry " 
> + new Date(now) + " on dataSource: " + dataSource);
> }
> {code}
> I think we should replace isStopping() with isStopping() or isStopped().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5447) Memory Leak after shutdown embeded broker with JDBC persistence

2014-11-21 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5447:
-
Attachment: 2.png

path to root

> Memory Leak after shutdown embeded broker with JDBC persistence
> ---
>
> Key: AMQ-5447
> URL: https://issues.apache.org/jira/browse/AMQ-5447
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker, Message Store
>Affects Versions: 5.10.0
> Environment: Windows7, JDK7
>Reporter: Shi Lei
> Attachments: 1.png, 2.png, JDBCPersistenceAdapter.java, patch.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
> Scheduled Task' is still alive.
> Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
> inner class, so the object of JDBCPersistenceAdapter can be reached from the 
> 2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
> instance of BrokerService can be reached from the 2 threads.
> So the stopped brokerService cannot be GC.
> The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
> cleanupTicket without shutdown clockDaemon, that's why the 2 threads are 
> still alive.
> According to 
> http://activemq.apache.org/how-do-i-restart-embedded-broker.html, it's better 
> (more reliable) to instantiate the broker again instead of reuse old broker. 
> So if I restart embeded broker, there will be  1 more BrokerService in 
> memory. I think it's memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5447) Memory Leak after shutdown embeded broker with JDBC persistence

2014-11-21 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5447:
-
Attachment: 1.png

Memory Leak

> Memory Leak after shutdown embeded broker with JDBC persistence
> ---
>
> Key: AMQ-5447
> URL: https://issues.apache.org/jira/browse/AMQ-5447
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker, Message Store
>Affects Versions: 5.10.0
> Environment: Windows7, JDK7
>Reporter: Shi Lei
> Attachments: 1.png, 2.png, JDBCPersistenceAdapter.java, patch.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
> Scheduled Task' is still alive.
> Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
> inner class, so the object of JDBCPersistenceAdapter can be reached from the 
> 2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
> instance of BrokerService can be reached from the 2 threads.
> So the stopped brokerService cannot be GC.
> The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
> cleanupTicket without shutdown clockDaemon, that's why the 2 threads are 
> still alive.
> According to 
> http://activemq.apache.org/how-do-i-restart-embedded-broker.html, it's better 
> (more reliable) to instantiate the broker again instead of reuse old broker. 
> So if I restart embeded broker, there will be  1 more BrokerService in 
> memory. I think it's memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5435) Persistence Adapter Starting Thread is still alive after stopping a slave broker with lease database locker

2014-11-21 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5435:
-
Attachment: patch2.txt

patch

> Persistence Adapter Starting Thread is still alive after stopping a slave 
> broker with lease database locker
> ---
>
> Key: AMQ-5435
> URL: https://issues.apache.org/jira/browse/AMQ-5435
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker
>Affects Versions: 5.10.0
> Environment: Windows, JDK7
>Reporter: Shi Lei
> Attachments: patch2.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> I am using jdbc master/slave with lease database lock.
> 
> I found if I call broker.stop to stop a slave broker service (which means it 
> tries to get a lease locker and has not got yet), its Persistence Adapter 
> Starting Thread is still alive. If I create and start a new broker in the 
> same java VM, there will be 2 Persistence Adapter Starting Threads inside the 
> same java VM. At this time, if the master broker is down, the stopped broker 
> will get the database lease locker, but somehow it cannot start broker. Now I 
> have 2 broker service in the same VM. One has got the locker, but cannot 
> start broker, the other one is still requesting the locker.
> The root cause is that after stopping broker, 
> LeaseDatabaseLocker.isStopping() is false, LeaseDatabaseLocker.isStopped() is 
> true,
> In LeaseDatabaseLocker.doStart
>  while (!isStopping()) {
> Connection connection = null;
> PreparedStatement statement = null;
> try {
> connection = getConnection();
> initTimeDiff(connection);
> statement = connection.prepareStatement(sql);
> setQueryTimeout(statement);
> now = System.currentTimeMillis() + diffFromCurrentTime;
> statement.setString(1, getLeaseHolderId());
> statement.setLong(2, now + lockAcquireSleepInterval);
> statement.setLong(3, now);
> int result = statement.executeUpdate();
> if (result == 1) {
> // we got the lease, verify we still have it
> if (keepAlive()) {
> break;
> }
> }
> reportLeasOwnerShipAndDuration(connection);
> } catch (Exception e) {
> LOG.debug(getLeaseHolderId() + " lease acquire failure: "+ e, 
> e);
> if (isStopping()) {
> throw new Exception(
> "Cannot start broker as being asked to shut down. 
> "
> + "Interrupted attempt to acquire lock: "
> + e, e);
> }
> if (handleStartException) {
> 
> lockable.getBrokerService().handleIOException(IOExceptionSupport.create(e));
> }
> } finally {
> close(statement);
> close(connection);
> }
> LOG.info(getLeaseHolderId() + " failed to acquire lease.  
> Sleeping for " + lockAcquireSleepInterval + " milli(s) before trying 
> again...");
> TimeUnit.MILLISECONDS.sleep(lockAcquireSleepInterval);
> }
> if (isStopping()) {
> throw new RuntimeException(getLeaseHolderId() + " failing lease 
> acquire due to stop");
> }
> LOG.info(getLeaseHolderId() + ", becoming master with lease expiry " 
> + new Date(now) + " on dataSource: " + dataSource);
> }
> I think we should replace isStopping() with isStopping() or isStopped().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5447) Memory Leak after shutdown embeded broker with JDBC persistence

2014-11-21 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5447:
-
Attachment: patch.txt

patch

> Memory Leak after shutdown embeded broker with JDBC persistence
> ---
>
> Key: AMQ-5447
> URL: https://issues.apache.org/jira/browse/AMQ-5447
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker, Message Store
>Affects Versions: 5.10.0
> Environment: Windows7, JDK7
>Reporter: Shi Lei
> Attachments: JDBCPersistenceAdapter.java, patch.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
> Scheduled Task' is still alive.
> Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
> inner class, so the object of JDBCPersistenceAdapter can be reached from the 
> 2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
> instance of BrokerService can be reached from the 2 threads.
> So the stopped brokerService cannot be GC.
> The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
> cleanupTicket without shutdown clockDaemon, that's why the 2 threads are 
> still alive.
> According to 
> http://activemq.apache.org/how-do-i-restart-embedded-broker.html, it's better 
> (more reliable) to instantiate the broker again instead of reuse old broker. 
> So if I restart embeded broker, there will be  1 more BrokerService in 
> memory. I think it's memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5447) Memory Leak after shutdown embeded broker with JDBC persistence

2014-11-20 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5447:
-
Attachment: JDBCPersistenceAdapter.java

patch on 5.10

> Memory Leak after shutdown embeded broker with JDBC persistence
> ---
>
> Key: AMQ-5447
> URL: https://issues.apache.org/jira/browse/AMQ-5447
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker, Message Store
>Affects Versions: 5.10.0
> Environment: Windows7, JDK7
>Reporter: Shi Lei
> Attachments: JDBCPersistenceAdapter.java
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
> Scheduled Task' is still alive.
> Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
> inner class, so the object of JDBCPersistenceAdapter can be reached from the 
> 2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
> instance of BrokerService can be reached from the 2 threads.
> So the stopped brokerService cannot be GC.
> The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
> cleanupTicket without shutdown clockDaemon, that's why the 2 threads are 
> still alive.
> According to 
> http://activemq.apache.org/how-do-i-restart-embedded-broker.html, it's better 
> (more reliable) to instantiate the broker again instead of reuse old broker. 
> So if I restart embeded broker, there will be  1 more BrokerService in 
> memory. I think it's memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5447) Memory Leak after shutdown embeded broker with JDBC persistence

2014-11-20 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5447:
-
Description: 
After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
Scheduled Task' is still alive.
Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
inner class, so the object of JDBCPersistenceAdapter can be reached from the 2 
threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
instance of BrokerService can be reached from the 2 threads.

So the stopped brokerService cannot be GC.

The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
cleanupTicket without shutdown clockDaemon, that's why the 2 threads are still 
alive.

According to http://activemq.apache.org/how-do-i-restart-embedded-broker.html, 
it's better (more reliable) to instantiate the broker again instead of reuse 
old broker. So if I restart embeded broker, there will be  1 more BrokerService 
in memory. I think it's memory leak.

  was:
After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
Scheduled Task' is still alive.
Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
inner class, so the object of JDBCPersistenceAdapter can be referenced from the 
2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
instance of BrokerService can be referenced from the 2 threads.

So the stopped brokerService cannot be GC.

The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
cleanupTicket without shutdown clockDaemon, that's why the 2 threads are still 
alive.

According to http://activemq.apache.org/how-do-i-restart-embedded-broker.html, 
it's better (more reliable) to instantiate the broker again instead of reuse 
old broker. So if I restart embeded broker, there will be  1 more BrokerService 
in memory. I think it's memory leak.


> Memory Leak after shutdown embeded broker with JDBC persistence
> ---
>
> Key: AMQ-5447
> URL: https://issues.apache.org/jira/browse/AMQ-5447
> Project: ActiveMQ
>  Issue Type: Bug
>  Components: Broker, Message Store
>Affects Versions: 5.10.0
> Environment: Windows7, JDK7
>Reporter: Shi Lei
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
> Scheduled Task' is still alive.
> Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
> inner class, so the object of JDBCPersistenceAdapter can be reached from the 
> 2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
> instance of BrokerService can be reached from the 2 threads.
> So the stopped brokerService cannot be GC.
> The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
> cleanupTicket without shutdown clockDaemon, that's why the 2 threads are 
> still alive.
> According to 
> http://activemq.apache.org/how-do-i-restart-embedded-broker.html, it's better 
> (more reliable) to instantiate the broker again instead of reuse old broker. 
> So if I restart embeded broker, there will be  1 more BrokerService in 
> memory. I think it's memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (AMQ-5447) Memory Leak after shutdown embeded broker with JDBC persistence

2014-11-20 Thread Shi Lei (JIRA)
Shi Lei created AMQ-5447:


 Summary: Memory Leak after shutdown embeded broker with JDBC 
persistence
 Key: AMQ-5447
 URL: https://issues.apache.org/jira/browse/AMQ-5447
 Project: ActiveMQ
  Issue Type: Bug
  Components: Broker, Message Store
Affects Versions: 5.10.0
 Environment: Windows7, JDK7
Reporter: Shi Lei


After shutdown embeded activemq broker with JDBC store, 2 'ActiveMQ JDBC PA 
Scheduled Task' is still alive.
Because the 2 thread's Thread factory is object of JDBCPersistenceAdapter's 
inner class, so the object of JDBCPersistenceAdapter can be referenced from the 
2 threads, JDBCPersistenceAdapter has a field point to BrokerService. So the 
instance of BrokerService can be referenced from the 2 threads.

So the stopped brokerService cannot be GC.

The root cause is that when stopping JDBCPersistenceAdapter, only cancelling 
cleanupTicket without shutdown clockDaemon, that's why the 2 threads are still 
alive.

According to http://activemq.apache.org/how-do-i-restart-embedded-broker.html, 
it's better (more reliable) to instantiate the broker again instead of reuse 
old broker. So if I restart embeded broker, there will be  1 more BrokerService 
in memory. I think it's memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (AMQ-5435) Persistence Adapter Starting Thread is still alive after stopping a slave broker with lease database locker

2014-11-12 Thread Shi Lei (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMQ-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shi Lei updated AMQ-5435:
-
Description: 
I am using jdbc master/slave with lease database lock.

I found if I call broker.stop to stop a slave broker service (which means it 
tries to get a lease locker and has not got yet), its Persistence Adapter 
Starting Thread is still alive. If I create and start a new broker in the same 
java VM, there will be 2 Persistence Adapter Starting Threads inside the same 
java VM. At this time, if the master broker is down, the stopped broker will 
get the database lease locker, but somehow it cannot start broker. Now I have 2 
broker service in the same VM. One has got the locker, but cannot start broker, 
the other one is still requesting the locker.

The root cause is that after stopping broker, LeaseDatabaseLocker.isStopping() 
is false, LeaseDatabaseLocker.isStopped() is true,
In LeaseDatabaseLocker.doStart

 while (!isStopping()) {
Connection connection = null;
PreparedStatement statement = null;
try {
connection = getConnection();
initTimeDiff(connection);

statement = connection.prepareStatement(sql);
setQueryTimeout(statement);

now = System.currentTimeMillis() + diffFromCurrentTime;
statement.setString(1, getLeaseHolderId());
statement.setLong(2, now + lockAcquireSleepInterval);
statement.setLong(3, now);

int result = statement.executeUpdate();
if (result == 1) {
// we got the lease, verify we still have it
if (keepAlive()) {
break;
}
}

reportLeasOwnerShipAndDuration(connection);

} catch (Exception e) {
LOG.debug(getLeaseHolderId() + " lease acquire failure: "+ e, 
e);
if (isStopping()) {
throw new Exception(
"Cannot start broker as being asked to shut down. "
+ "Interrupted attempt to acquire lock: "
+ e, e);
}
if (handleStartException) {

lockable.getBrokerService().handleIOException(IOExceptionSupport.create(e));
}
} finally {
close(statement);
close(connection);
}

LOG.info(getLeaseHolderId() + " failed to acquire lease.  Sleeping 
for " + lockAcquireSleepInterval + " milli(s) before trying again...");
TimeUnit.MILLISECONDS.sleep(lockAcquireSleepInterval);
}
if (isStopping()) {
throw new RuntimeException(getLeaseHolderId() + " failing lease 
acquire due to stop");
}

LOG.info(getLeaseHolderId() + ", becoming master with lease expiry " + 
new Date(now) + " on dataSource: " + dataSource);
}

I think we should replace isStopping() with isStopping() or isStopped().


  was:
I am using jdbc master/slave with lease database lock.
I found if I call broker.stop to stop a slave broker service (which means it 
tries to get a lease locker and has not got yet), its Persistence Adapter 
Starting Thread is still alive. If I create and start a new broker in the same 
java VM, there will be 2 Persistence Adapter Starting Threads inside the same 
java VM. At this time, if the master broker is down, the stopped broker will 
get the database lease locker, but somehow it cannot start broker. Now I have 2 
broker service in the same VM. One has got the locker, but cannot start broker, 
the other one is still requesting the locker.

The root cause is that after stopping broker, LeaseDatabaseLocker.isStopping() 
is false, LeaseDatabaseLocker.isStopped() is true,
In LeaseDatabaseLocker.doStart

 while (!isStopping()) {
Connection connection = null;
PreparedStatement statement = null;
try {
connection = getConnection();
initTimeDiff(connection);

statement = connection.prepareStatement(sql);
setQueryTimeout(statement);

now = System.currentTimeMillis() + diffFromCurrentTime;
statement.setString(1, getLeaseHolderId());
statement.setLong(2, now + lockAcquireSleepInterval);
statement.setLong(3, now);

int result = statement.executeUpdate();
if (result == 1) {
// we got the lease, verify we still have it
if (keepAlive()) {
break;
}
}

reportLeasOwnerShipAndDuration(connection);

} catch (Exception e) {
LOG.debug(getLeaseHolderId() + " lease

[jira] [Created] (AMQ-5435) Persistence Adapter Starting Thread is still alive after stopping a slave broker with lease database locker

2014-11-12 Thread Shi Lei (JIRA)
Shi Lei created AMQ-5435:


 Summary: Persistence Adapter Starting Thread is still alive after 
stopping a slave broker with lease database locker
 Key: AMQ-5435
 URL: https://issues.apache.org/jira/browse/AMQ-5435
 Project: ActiveMQ
  Issue Type: Bug
  Components: Broker
Affects Versions: 5.10.0
 Environment: Windows, JDK7
Reporter: Shi Lei


I am using jdbc master/slave with lease database lock.
I found if I call broker.stop to stop a slave broker service (which means it 
tries to get a lease locker and has not got yet), its Persistence Adapter 
Starting Thread is still alive. If I create and start a new broker in the same 
java VM, there will be 2 Persistence Adapter Starting Threads inside the same 
java VM. At this time, if the master broker is down, the stopped broker will 
get the database lease locker, but somehow it cannot start broker. Now I have 2 
broker service in the same VM. One has got the locker, but cannot start broker, 
the other one is still requesting the locker.

The root cause is that after stopping broker, LeaseDatabaseLocker.isStopping() 
is false, LeaseDatabaseLocker.isStopped() is true,
In LeaseDatabaseLocker.doStart

 while (!isStopping()) {
Connection connection = null;
PreparedStatement statement = null;
try {
connection = getConnection();
initTimeDiff(connection);

statement = connection.prepareStatement(sql);
setQueryTimeout(statement);

now = System.currentTimeMillis() + diffFromCurrentTime;
statement.setString(1, getLeaseHolderId());
statement.setLong(2, now + lockAcquireSleepInterval);
statement.setLong(3, now);

int result = statement.executeUpdate();
if (result == 1) {
// we got the lease, verify we still have it
if (keepAlive()) {
break;
}
}

reportLeasOwnerShipAndDuration(connection);

} catch (Exception e) {
LOG.debug(getLeaseHolderId() + " lease acquire failure: "+ e, 
e);
if (isStopping()) {
throw new Exception(
"Cannot start broker as being asked to shut down. "
+ "Interrupted attempt to acquire lock: "
+ e, e);
}
if (handleStartException) {

lockable.getBrokerService().handleIOException(IOExceptionSupport.create(e));
}
} finally {
close(statement);
close(connection);
}

LOG.info(getLeaseHolderId() + " failed to acquire lease.  Sleeping 
for " + lockAcquireSleepInterval + " milli(s) before trying again...");
TimeUnit.MILLISECONDS.sleep(lockAcquireSleepInterval);
}
if (isStopping()) {
throw new RuntimeException(getLeaseHolderId() + " failing lease 
acquire due to stop");
}

LOG.info(getLeaseHolderId() + ", becoming master with lease expiry " + 
new Date(now) + " on dataSource: " + dataSource);
}

I think we should replace isStopping() with isStopping() or isStopped().




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)