[ https://issues.apache.org/jira/browse/HIVE-20042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539265#comment-16539265 ]
Sergey Shelukhin commented on HIVE-20042: ----------------------------------------- cc [~thejas] looks like the continuation from the epic locking patch... maybe the config needs to be cloned, too? > HiveServer2: All operations lock on a Single HiveConf object > ------------------------------------------------------------- > > Key: HIVE-20042 > URL: https://issues.apache.org/jira/browse/HIVE-20042 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Affects Versions: 3.0.0 > Reporter: Gopal V > Priority: Major > Labels: Concurrency > > With the 1000 user test, the session start/tear-down runs only at 100% CPU, > which is due to all threads locking on the same HiveConf object. > OpenSession locks on 0x00000005c091a3a0 > {code} > "HiveServer2-HttpHandler-Pool: Thread-65084" #65084 prio=5 os_prio=0 > tid=0x00000000103bb000 nid=0x4a09 waiting for monitor entry > [0x00007fc1b0987000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418) > - waiting to lock <0x00000005c091a3a0> (a > org.apache.hadoop.hive.conf.HiveConf) > at > org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1437) > at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:4996) > at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5069) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.getUserName(ThriftCLIService.java:424) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:467) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:325) > {code} > GetOperationStatus locks on the same > {code} > "HiveServer2-HttpHandler-Pool: Thread-65082" #65082 prio=5 os_prio=0 > tid=0x00007fc2656be000 nid=0x4a06 waiting for monitor entry > [0x00007fc3159db000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418) > - waiting to lock <0x00000005c091a3a0> (a > org.apache.hadoop.hive.conf.HiveConf) > at > org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1437) > at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:4996) > at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5069) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.GetOperationStatus(ThriftCLIService.java:709) > {code} > Session clean up locks on the same > {code} > "8fd1db09-9f96-49dc-becf-5702826bd4f5 HiveServer2-HttpHandler-Pool: > Thread-64981" #64981 prio=5 os_prio=0 tid=0x000000001d1ab000 nid=0x23d5 > waiting for monitor entry [0x00007fc1b65e3000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418) > - waiting to lock <0x00000005c091a3a0> (a > org.apache.hadoop.hive.conf.HiveConf) > at > org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1177) > at > org.apache.hadoop.conf.Configuration.getTrimmedStringCollection(Configuration.java:2122) > at > org.apache.hadoop.hdfs.DFSUtilClient.getNameServiceIds(DFSUtilClient.java:197) > at org.apache.hadoop.hdfs.HAUtilClient.isLogicalUri(HAUtilClient.java:53) > ... > at > org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:959) > at org.apache.hadoop.hive.ql.Context.clear(Context.java:724) > {code} > Hadoop RPC blocks on the same > {code} > "HiveServer2-HttpHandler-Pool: Thread-59227" #59227 prio=5 os_prio=0 > tid=0x00007fc270aeb800 nid=0x129b waiting for monitor entry > [0x00007fc28b7b5000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418) > - waiting to lock <0x00000005c091a3a0> (a > org.apache.hadoop.hive.conf.HiveConf) > at > org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1177) > at > org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1234) > at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1459) > at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:451) > at org.apache.hadoop.ipc.Client.getConnection(Client.java:1532) > ... > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1580) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) > at org.apache.hadoop.fs.FileSystem.deleteOnExit(FileSystem.java:1677) > at > org.apache.hadoop.hive.ql.session.SessionState.createPath(SessionState.java:791) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)