Hi, We faced the same issue in ESB latest when clustering enabled, with 4 node cluster, number of connections goes to around 395 and gradually comes down to 20 or less, even with the exception happened(mysql having default max connections 151), it seems it gets stable to 20 or less connections after some time. Created a public jira to track this issue at [1]
[1] - https://wso2.org/jira/browse/REGISTRY-3771 Thanks, On Wed, Jul 13, 2016 at 4:42 PM, Dilini Gunatilake <dili...@wso2.com> wrote: > Hi All, > > I have set up a 4 node cluster for ES 2.1.0 Alpha2 pack with mysql 5.7. > When I started one store node I got an error [3]. > > Then, I checked the connections in the mysql server and found that the > default number of max_connections (151) have been exceeded while the store > node is starting. But initially without starting any ES instances there > were only around 20 connections. So, what we observed was when starting ES > instances substantially high amount of connections will be created and they > will be gradually reduced after sometime. Is this an acceptable behavior? > > Also, we could find the recommended max_connections to configure in some > products [1][2], but couldn't find anything for ES. So, what is the > recommended value for ES? > For the time being we will increase the max connections to 300 and proceed > with testing. > > Your earliest response is highly appreciated. > > [1] https://docs.wso2.com/display/AM200/Tuning+Performance > [2] > https://docs.wso2.com/display/Governance510/Performance+Tuning+Recommendations > > [3] > > [2016-07-13 07:50:40,014] ERROR > {org.wso2.carbon.registry.core.dataaccess.TransactionManager} - Failed to > start new registry transaction. > com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: > Could not create connection to database server. Attempted reconnect 3 > times. Giving up. > at sun.reflect.GeneratedConstructorAccessor107.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:917) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:896) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:885) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860) > at > com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2165) > at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2090) > at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:795) > at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44) > at sun.reflect.GeneratedConstructorAccessor76.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400) > at > com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327) > at > org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278) > at > org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182) > at > org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:701) > at > org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:635) > at > org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:188) > at > org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128) > at > org.wso2.carbon.registry.core.jdbc.dataaccess.JDBCTransactionManager.beginTransaction(JDBCTransactionManager.java:80) > at > org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.beginTransaction(EmbeddedRegistry.java:443) > at > org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.get(EmbeddedRegistry.java:505) > at > org.wso2.carbon.registry.core.session.UserRegistry.getInternal(UserRegistry.java:617) > at > org.wso2.carbon.registry.core.session.UserRegistry.access$400(UserRegistry.java:61) > at > org.wso2.carbon.registry.core.session.UserRegistry$5.run(UserRegistry.java:597) > at > org.wso2.carbon.registry.core.session.UserRegistry$5.run(UserRegistry.java:594) > at java.security.AccessController.doPrivileged(Native Method) > at > org.wso2.carbon.registry.core.session.UserRegistry.get(UserRegistry.java:594) > at > org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler.get(MountHandler.java:399) > at > org.wso2.carbon.registry.core.jdbc.handlers.HandlerManager.get(HandlerManager.java:2444) > at > org.wso2.carbon.registry.core.jdbc.handlers.UserDefinedHandlerManager.get(UserDefinedHandlerManager.java:200) > at > org.wso2.carbon.registry.core.jdbc.handlers.HandlerLifecycleManager.get(HandlerLifecycleManager.java:914) > at > org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.get(EmbeddedRegistry.java:512) > at > org.wso2.carbon.registry.core.caching.CacheBackedRegistry.get(CacheBackedRegistry.java:203) > at > org.wso2.carbon.registry.core.session.UserRegistry.getInternal(UserRegistry.java:617) > at > org.wso2.carbon.registry.core.session.UserRegistry.access$400(UserRegistry.java:61) > at > org.wso2.carbon.registry.core.session.UserRegistry$5.run(UserRegistry.java:597) > at > org.wso2.carbon.registry.core.session.UserRegistry$5.run(UserRegistry.java:594) > at java.security.AccessController.doPrivileged(Native Method) > at > org.wso2.carbon.registry.core.session.UserRegistry.get(UserRegistry.java:594) > at > org.wso2.carbon.registry.indexing.AsyncIndexer$IndexingTask.createIndexDocument(AsyncIndexer.java:242) > at > org.wso2.carbon.registry.indexing.AsyncIndexer$IndexingTask.run(AsyncIndexer.java:210) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: > com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Too > many connections > at sun.reflect.GeneratedConstructorAccessor107.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:917) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871) > at > com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1694) > at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1215) > at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2255) > at > com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2106) > ... 43 more > [2016-07-13 07:50:40,077] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path > "/_system/governance/store/asset_resources/gadget/8c3eda76-ec7f-43e3-bbe8-efa14f85d780/images_thumbnail"could > not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path > /_system/governance/store/asset_resources/gadget/8c3eda76-ec7f-43e3-bbe8-efa14f85d780/images_thumbnail > [2016-07-13 07:50:40,076] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path "/_system/governance/gadgets/admin/Stacked Line > Chart/1.0.0"could not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path /_system/governance/gadgets/admin/Stacked > Line Chart/1.0.0 > [2016-07-13 07:50:40,074] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path > "/_system/governance/store/asset_resources/gadget/57235acc-ad87-41db-84e7-dadc0bcff8d5/images_banner"could > not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path > /_system/governance/store/asset_resources/gadget/57235acc-ad87-41db-84e7-dadc0bcff8d5/images_banner > [2016-07-13 07:50:40,067] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path > "/_system/governance/store/asset_resources/gadget/8c3eda76-ec7f-43e3-bbe8-efa14f85d780/images_banner"could > not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path > /_system/governance/store/asset_resources/gadget/8c3eda76-ec7f-43e3-bbe8-efa14f85d780/images_banner > [2016-07-13 07:50:40,064] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path "/_system/governance/gadgets/admin/WSO2 Carbon Commits > List Discussion/1.0.0"could not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path /_system/governance/gadgets/admin/WSO2 > Carbon Commits List Discussion/1.0.0 > [2016-07-13 07:50:40,062] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path > "/_system/governance/store/asset_resources/site/6b89dd21-0a98-4489-9f0d-fc327ff46bc0/images_thumbnail"could > not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path > /_system/governance/store/asset_resources/site/6b89dd21-0a98-4489-9f0d-fc327ff46bc0/images_thumbnail > [2016-07-13 07:50:40,056] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path > "/_system/governance/store/asset_resources/gadget/4c4f7877-ff39-4379-9ee4-6c1827732ec6/images_thumbnail"could > not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path > /_system/governance/store/asset_resources/gadget/4c4f7877-ff39-4379-9ee4-6c1827732ec6/images_thumbnail > [2016-07-13 07:50:40,079] ERROR > {org.wso2.carbon.registry.indexing.AsyncIndexer} - Error while indexing. > Resource at path > "/_system/governance/store/asset_resources/gadget/57235acc-ad87-41db-84e7-dadc0bcff8d5/images_thumbnail"could > not be > indexedorg.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: > Resource does not exist at path > /_system/governance/store/asset_resources/gadget/57235acc-ad87-41db-84e7-dadc0bcff8d5/images_thumbnail > > > Thank you. > Regards, > > -- > > *Dilini GunatilakeSoftware Engineer - QA Team* > Mobile : +94 (0) 771 162518 > dili...@wso2.com > > _______________________________________________ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Rajith Vitharana Software Engineer, WSO2 Inc. : wso2.com Mobile : +94715883223 Blog : http://lankavitharana.blogspot.com/
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
