[ https://issues.apache.org/jira/browse/HBASE-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Attila Tőkés updated HBASE-13934: --------------------------------- Description: HBase Client get's stuck when I try to execute a PUT operation. {code} Thread [BenchmarkThread-0] (Suspended) owns: BufferedMutatorImpl (id=43) Unsafe.park(boolean, long) line: not available [native method] LockSupport.park(Object) line: 186 AbstractQueuedSynchronizer$ConditionObject.await() line: 2043 ArrayBlockingQueue<E>.take() line: 374 BoundedCompletionService<V>.take() line: 75 ScannerCallableWithReplicas.call(int) line: 190 ScannerCallableWithReplicas.call(int) line: 56 RpcRetryingCaller<T>.callWithoutRetries(RetryingCallable<T>, int) line: 200 ClientSmallReversedScanner.loadCache() line: 211 ClientSmallReversedScanner.next() line: 185 ConnectionManager$HConnectionImplementation.locateRegionInMeta(TableName, byte[], boolean, boolean, int) line: 1200 ConnectionManager$HConnectionImplementation.locateRegion(TableName, byte[], boolean, boolean, int) line: 1109 AsyncProcess.submit(ExecutorService, TableName, List<Row>, boolean, Callback<CResult>, boolean) line: 369 AsyncProcess.submit(TableName, List<Row>, boolean, Callback<CResult>, boolean) line: 320 BufferedMutatorImpl.backgroundFlushCommits(boolean) line: 206 BufferedMutatorImpl.flush() line: 183 HTable.flushCommits() line: 1436 HTable.put(Put) line: 1032 {code} Source code: Connect: {code} this.config = HBaseConfiguration.create(); config.set("hbase.zookeeper.quorum", zookeeperHost); Connection connection = ConnectionFactory.createConnection(config); this.table = connection.getTable(TableName.valueOf(tableName)); {code} Put: {code} final Put put = new Put(Bytes.toBytes(key)); for (Map.Entry<String, String> pair : columnValues.entrySet()) { final String column = pair.getKey(); final String value = pair.getValue(); put.addColumn(columnFamily, Bytes.toBytes(column), Bytes.toBytes(value)); } try { table.put(put); } catch (IOException e) { throw new ClientException("put error", e); } {code} Client log: {code} 17:00:58,193 INFO ZooKeeper:438 - Initiating client connection, connectString=nosql-x64-node-1.local:2181 sessionTimeout=90000 watcher=hconnection-0x3018fc1a0x0, quorum=nosql-x64-node-1.local:2181, baseZNode=/hbase 17:00:58,325 INFO ClientCnxn:975 - Opening socket connection to server 192.168.56.201/192.168.56.201:2181. Will not attempt to authenticate using SASL (unknown error) 17:00:58,329 INFO ClientCnxn:852 - Socket connection established to 192.168.56.201/192.168.56.201:2181, initiating session 17:00:58,346 INFO ClientCnxn:1235 - Session establishment complete on server 192.168.56.201/192.168.56.201:2181, sessionid = 0x14e06dbd6450020, negotiated timeout = 40000 {code} Server's log: {code} 2015-06-18 17:12:28,183 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /192.168.56.1:35002 which had sessionid 0x14e06dbd6450020 2015-06-18 17:12:30,001 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x14e06dbd645001d, timeout of 40000ms exceeded 2015-06-18 17:12:30,002 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x14e06dbd645001d 2015-06-18 17:12:31,078 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.1:35130 2015-06-18 17:12:31,080 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.1:35130 2015-06-18 17:12:31,092 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x14e06dbd6450021 with negotiated timeout 40000 for client /192.168.56.1:35130 {code} Happens both with HBASE running in standalone and distributed mode. Any idea what causing this? HBase version: 1.0.1 (client + server) was: HBase Client get's stuck when I try to execute a PUT operation. {code} Thread [BenchmarkThread-0] (Suspended) owns: BufferedMutatorImpl (id=43) Unsafe.park(boolean, long) line: not available [native method] LockSupport.park(Object) line: 186 AbstractQueuedSynchronizer$ConditionObject.await() line: 2043 ArrayBlockingQueue<E>.take() line: 374 BoundedCompletionService<V>.take() line: 75 ScannerCallableWithReplicas.call(int) line: 190 ScannerCallableWithReplicas.call(int) line: 56 RpcRetryingCaller<T>.callWithoutRetries(RetryingCallable<T>, int) line: 200 ClientSmallReversedScanner.loadCache() line: 211 ClientSmallReversedScanner.next() line: 185 ConnectionManager$HConnectionImplementation.locateRegionInMeta(TableName, byte[], boolean, boolean, int) line: 1200 ConnectionManager$HConnectionImplementation.locateRegion(TableName, byte[], boolean, boolean, int) line: 1109 AsyncProcess.submit(ExecutorService, TableName, List<Row>, boolean, Callback<CResult>, boolean) line: 369 AsyncProcess.submit(TableName, List<Row>, boolean, Callback<CResult>, boolean) line: 320 BufferedMutatorImpl.backgroundFlushCommits(boolean) line: 206 BufferedMutatorImpl.flush() line: 183 HTable.flushCommits() line: 1436 HTable.put(Put) line: 1032 HBaseClient.put(String, Map<String,String>) line: 92 GenericColumnStoreBenchmark$ColumnFamilyBenchmarkTask.doOperation(String) line: 115 GenericColumnStoreBenchmark$ColumnFamilyBenchmarkTask.init(ColumnStoreClient) line: 139 GenericColumnStoreBenchmark$ColumnFamilyBenchmarkTask.init(DatabaseClient) line: 1 MultiThreadedBenchmark$BenchmarkThread.doInit() line: 115 MultiThreadedBenchmark$BenchmarkThread.run() line: 128 {code} Source code: Connect: {code} this.config = HBaseConfiguration.create(); config.set("hbase.zookeeper.quorum", zookeeperHost); Connection connection = ConnectionFactory.createConnection(config); this.table = connection.getTable(TableName.valueOf(tableName)); {code} Put: {code} final Put put = new Put(Bytes.toBytes(key)); for (Map.Entry<String, String> pair : columnValues.entrySet()) { final String column = pair.getKey(); final String value = pair.getValue(); put.addColumn(columnFamily, Bytes.toBytes(column), Bytes.toBytes(value)); } try { table.put(put); } catch (IOException e) { throw new ClientException("put error", e); } {code} Client log: {code} 17:00:58,193 INFO ZooKeeper:438 - Initiating client connection, connectString=nosql-x64-node-1.local:2181 sessionTimeout=90000 watcher=hconnection-0x3018fc1a0x0, quorum=nosql-x64-node-1.local:2181, baseZNode=/hbase 17:00:58,325 INFO ClientCnxn:975 - Opening socket connection to server 192.168.56.201/192.168.56.201:2181. Will not attempt to authenticate using SASL (unknown error) 17:00:58,329 INFO ClientCnxn:852 - Socket connection established to 192.168.56.201/192.168.56.201:2181, initiating session 17:00:58,346 INFO ClientCnxn:1235 - Session establishment complete on server 192.168.56.201/192.168.56.201:2181, sessionid = 0x14e06dbd6450020, negotiated timeout = 40000 {code} Server's log: {code} 2015-06-18 17:12:28,183 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /192.168.56.1:35002 which had sessionid 0x14e06dbd6450020 2015-06-18 17:12:30,001 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x14e06dbd645001d, timeout of 40000ms exceeded 2015-06-18 17:12:30,002 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x14e06dbd645001d 2015-06-18 17:12:31,078 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.1:35130 2015-06-18 17:12:31,080 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.1:35130 2015-06-18 17:12:31,092 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x14e06dbd6450021 with negotiated timeout 40000 for client /192.168.56.1:35130 {code} Happens both with HBASE running in standalone and distributed mode. Any idea what causing this? HBase version: 1.0.1 (client + server) > HBase Client Stuck at ConnectionManager$HConnectionImplementation.locateRegion > ------------------------------------------------------------------------------ > > Key: HBASE-13934 > URL: https://issues.apache.org/jira/browse/HBASE-13934 > Project: HBase > Issue Type: Bug > Components: Client > Reporter: Attila Tőkés > Priority: Blocker > > HBase Client get's stuck when I try to execute a PUT operation. > {code} > Thread [BenchmarkThread-0] (Suspended) > owns: BufferedMutatorImpl (id=43) > Unsafe.park(boolean, long) line: not available [native method] > LockSupport.park(Object) line: 186 > AbstractQueuedSynchronizer$ConditionObject.await() line: 2043 > ArrayBlockingQueue<E>.take() line: 374 > BoundedCompletionService<V>.take() line: 75 > ScannerCallableWithReplicas.call(int) line: 190 > ScannerCallableWithReplicas.call(int) line: 56 > RpcRetryingCaller<T>.callWithoutRetries(RetryingCallable<T>, int) line: > 200 > ClientSmallReversedScanner.loadCache() line: 211 > ClientSmallReversedScanner.next() line: 185 > ConnectionManager$HConnectionImplementation.locateRegionInMeta(TableName, > byte[], boolean, boolean, int) line: 1200 > ConnectionManager$HConnectionImplementation.locateRegion(TableName, > byte[], boolean, boolean, int) line: 1109 > AsyncProcess.submit(ExecutorService, TableName, List<Row>, boolean, > Callback<CResult>, boolean) line: 369 > AsyncProcess.submit(TableName, List<Row>, boolean, Callback<CResult>, > boolean) line: 320 > BufferedMutatorImpl.backgroundFlushCommits(boolean) line: 206 > BufferedMutatorImpl.flush() line: 183 > HTable.flushCommits() line: 1436 > HTable.put(Put) line: 1032 > {code} > Source code: > Connect: > {code} > this.config = HBaseConfiguration.create(); > config.set("hbase.zookeeper.quorum", zookeeperHost); > Connection connection = ConnectionFactory.createConnection(config); > this.table = connection.getTable(TableName.valueOf(tableName)); > {code} > Put: > {code} > final Put put = new Put(Bytes.toBytes(key)); > for (Map.Entry<String, String> pair : columnValues.entrySet()) { > final String column = pair.getKey(); > final String value = pair.getValue(); > put.addColumn(columnFamily, Bytes.toBytes(column), > Bytes.toBytes(value)); > } > try { > table.put(put); > } catch (IOException e) { > throw new ClientException("put error", e); > } > {code} > Client log: > {code} > 17:00:58,193 INFO ZooKeeper:438 - Initiating client connection, > connectString=nosql-x64-node-1.local:2181 sessionTimeout=90000 > watcher=hconnection-0x3018fc1a0x0, quorum=nosql-x64-node-1.local:2181, > baseZNode=/hbase > 17:00:58,325 INFO ClientCnxn:975 - Opening socket connection to server > 192.168.56.201/192.168.56.201:2181. Will not attempt to authenticate using > SASL (unknown error) > 17:00:58,329 INFO ClientCnxn:852 - Socket connection established to > 192.168.56.201/192.168.56.201:2181, initiating session > 17:00:58,346 INFO ClientCnxn:1235 - Session establishment complete on server > 192.168.56.201/192.168.56.201:2181, sessionid = 0x14e06dbd6450020, negotiated > timeout = 40000 > {code} > Server's log: > {code} > 2015-06-18 17:12:28,183 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] > server.NIOServerCnxn: Closed socket connection for client /192.168.56.1:35002 > which had sessionid 0x14e06dbd6450020 > 2015-06-18 17:12:30,001 INFO [SessionTracker] server.ZooKeeperServer: > Expiring session 0x14e06dbd645001d, timeout of 40000ms exceeded > 2015-06-18 17:12:30,002 INFO [ProcessThread(sid:0 cport:-1):] > server.PrepRequestProcessor: Processed session termination for sessionid: > 0x14e06dbd645001d > 2015-06-18 17:12:31,078 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] > server.NIOServerCnxnFactory: Accepted socket connection from > /192.168.56.1:35130 > 2015-06-18 17:12:31,080 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] > server.ZooKeeperServer: Client attempting to establish new session at > /192.168.56.1:35130 > 2015-06-18 17:12:31,092 INFO [SyncThread:0] server.ZooKeeperServer: > Established session 0x14e06dbd6450021 with negotiated timeout 40000 for > client /192.168.56.1:35130 > {code} > Happens both with HBASE running in standalone and distributed mode. > Any idea what causing this? > HBase version: 1.0.1 (client + server) -- This message was sent by Atlassian JIRA (v6.3.4#6332)