Sorry miss clicked. 2) TxnHandler::lock method request new connection when executing this line of code: ConnectionLockIdPair connAndLockId = enqueueLockWithRetry(rqst);
3) After that folowing this stacktrace: - TxnHandler::lock - TxnHandler::checkLockWithRetry - TxnHandler::checkLock In checkLock method we reach this line: handle = getMutexAPI().acquireLock(MUTEX_KEY.CheckLock.name()); 4) acquireLock method requests another connection to DB: dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED); So all in all if i call TxnHandler::lock method in 10 threads same time at first I get all connections to db, that stored in pool, and at acquireLock I will stuck because there's no free connection. Does anyone run into this problem? How can I avoid this problem? Code was taken from here: https://github.com/hortonworks/hive-release/blob/HDP-2.5.0.0-tag/metastore/ src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java I guess the closest branch in apach repo is: https://github.com/apache/hive/blob/branch-2.1/metastore/src/java/org/apache /hadoop/hive/metastore/txn/TxnHandler.java On Tue, Mar 21, 2017 at 12:07 AM, Igor Kuzmenko <[email protected]> wrote: > Hello I'm running Hortonworks data platform 2.5.0.0 with included hive. > I'm using storm hive bolt to load data into my hive. But launching many > hive bolt always leads me to TimeoutException on calling hive metastore. > Metastore logs full of Exception like this: > > 2017-03-15 18:46:12,436 ERROR [pool-5-thread-11]: txn.TxnHandler > (TxnHandler.java:getDbConn(1834)) - There is a problem with a connection > from the pool, retrying(rc=7): Timed out waiting for a free available > connection. (SQLState=08001, ErrorCode=0) > java.sql.SQLException: Timed out waiting for a free available connection. > at com.jolbox.bonecp.DefaultConnectionStrategy.getConnectionInternal( > DefaultConnectionStrategy.java:88) > at com.jolbox.bonecp.AbstractConnectionStrategy.getConnection( > AbstractConnectionStrategy.java:90) > at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:553) > at com.jolbox.bonecp.BoneCPDataSource.getConnection( > BoneCPDataSource.java:131) > at org.apache.hadoop.hive.metastore.txn.TxnHandler. > getDbConn(TxnHandler.java:1827) > at org.apache.hadoop.hive.metastore.txn.TxnHandler.enqueueLockWithRetry( > TxnHandler.java:873) > at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock( > TxnHandler.java:814) > at org.apache.hadoop.hive.metastore.HiveMetaStore$ > HMSHandler.lock(HiveMetaStore.java:5751) > > After looking through code I found out > > 1) TxnHandler class uses connection pool to get db connections and it's > size is 10. > 2) TxnHandler::lock method requset new connection whe executing this line > of code: > >
