[ https://issues.apache.org/jira/browse/HIVE-24711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276772#comment-17276772 ]
LinZhongwei commented on HIVE-24711: ------------------------------------ Our hive metastore just enable storage based authorization. And I find these error messages in the hivemetastore.log. Caused by: java.security.AccessControlException: Permission denied: user=hive, access=WRITE, inode="/apps/finance/fdop/fdop_stg/fdop_ft_etl_stg/batch_date=2020-07-07/batch_seq_num=5":gp_fin_fdop_batch:gp_fin_fdop_batch:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:399) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:261) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:193) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1859) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1843) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPathAccess(FSDirectory.java:1793) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAccess(FSNamesystem.java:7804) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.checkAccess(NameNodeRpcServer.java:2217) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.checkAccess(ClientNamenodeProtocolServerSideTranslatorPB.java:1659) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682) at org.apache.hadoop.hive.shims.Hadoop23Shims.wrapAccessException(Hadoop23Shims.java:947) ~[hive-exec-3.1.0.3.1.5.31-1.jar:3.1.0.3.1.5.31-1] at org.apache.hadoop.hive.shims.Hadoop23Shims.checkFileAccess(Hadoop23Shims.java:931) ~[hive-exec-3.1.0.3.1.5.31-1.jar:3.1.0.3.1.5.31-1] at org.apache.hadoop.hive.common.FileUtils.checkFileAccessWithImpersonation(FileUtils.java:402) ~[hive-common-3.1.0.3.1.5.31-1.jar:3.1.0.3.1.5.31-1] at org.apache.hadoop.hive.common.FileUtils.checkFileAccessWithImpersonation(FileUtils.java:370) ~[hive-common-3.1.0.3.1.5.31-1.jar:3.1.0.3.1.5.31-1] > hive metastore memory leak > -------------------------- > > Key: HIVE-24711 > URL: https://issues.apache.org/jira/browse/HIVE-24711 > Project: Hive > Issue Type: Bug > Components: Hive, Metastore > Affects Versions: 3.1.0 > Reporter: LinZhongwei > Priority: Major > > hdp version:3.1.5.31-1 > hive version:3.1.0.3.1.5.31-1 > hadoop version:3.1.1.3.1.5.31-1 > We find that the hive metastore has memory leak if we set > compactor.initiator.on to true. > If we disable the configuration, the memory leak disappear. > How can we resolve this problem? > Even if we set the heap size of hive metastore to 40 GB, after 1 month the > hive metastore service will be down with outofmemory. -- This message was sent by Atlassian Jira (v8.3.4#803005)