Kiran Koneti created CLOUDSTACK-5207: ----------------------------------------
Summary: jdbc.exceptions are observed when both Master 1 and master 2 goes down and comes back. Key: CLOUDSTACK-5207 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5207 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server Affects Versions: 4.3.0 Reporter: Kiran Koneti Priority: Critical Fix For: 4.3.0 1)I have created a replication db with 2 masters say M1 and M2. 2)The CS setup came up and both the db's are in sync. 3)now i made the M1 down and the M2 was made the master and the operations of CS were normal. 4)Now i made the M2 also down and the db of the CS could not be accessed. 5)Later i made the M1 up and then M2.The data sync happened properly but there is an error message which is constantly shown in the MS log and it is as below: "2013-11-20 11:34:50,938 ERROR [c.c.u.d.ConnectionConcierge] (ConnectionConcierge-1:ctx-7c449525) Unable to keep the db connection for LockMaster1 com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed. Connection closed after inability to pick valid new connection during fail-over. at sun.reflect.GeneratedConstructorAccessor25.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:532) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1014) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:988) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:974) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919) at com.mysql.jdbc.LoadBalancingConnectionProxy.invoke(LoadBalancingConnectionProxy.java:532) at com.mysql.jdbc.FailoverConnectionProxy.invoke(FailoverConnectionProxy.java:136) at $Proxy19.prepareStatement(Unknown Source) at org.apache.commons.dbcp.DelegatingConnection.prepareStatement(DelegatingConnection.java:281) at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareStatement(PoolingDataSource.java:313) at com.cloud.utils.db.ConnectionConcierge$ConnectionConciergeManager.testValidity(ConnectionConcierge.java:148) at com.cloud.utils.db.ConnectionConcierge$ConnectionConciergeManager$1.runInContext(ConnectionConcierge.java:211) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679)" -- This message was sent by Atlassian JIRA (v6.1#6144)