[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20404: -- Description: I see these big dirty exceptions in my master log during a long-run Lets clean them up (Are they exceptions I as an operator can actually do something about? Are they 'problems'? Should they be LOG.warn?) {code} 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] cleaner.CleanerChore: Could not delete dir under hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; {} org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) at org.apache.hadoop.ipc.Client.call(Client.java:1435) at org.apache.hadoop.ipc.Client.call(Client.java:1345) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) at com.sun.proxy.$Proxy26.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) at com.sun.proxy.$Proxy27.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) ... {code} Looks like log format is off too... was: I see these big dirty exceptions in my master log during a long-run Lets clean them up (Are they exceptions I as an operator can actually do something about? Are they 'problems'? Should they be LOG.warn?) {code} 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] cleaner.CleanerChore: Could not delete dir under hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; {} org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) at org.apache.hadoo
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Component/s: master > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Attachment: HBASE-20404.0.patch > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-20404.0.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Status: Patch Available (was: In Progress) -v0 - move level from WARN -> DEBUG - update message to give an indication of "okayness" > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-20404.0.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Attachment: HBASE-20404.1.patch > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Status: In Progress (was: Patch Available) > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Attachment: HBASE-20404.2.patch > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch, > HBASE-20404.2.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Status: Patch Available (was: In Progress) -v2 - CleanerChore was incorrectly treating IOE in FileStatus listing as "directory not found" - update javadocs in CleanerChore helper to match FSUtils method it invokes - update CleanerChore to treat IOE in status listing as a failure - check for {{PathIsNotEmptyDirectoryException}} when we expect dirs might not be empty, and only output a small DEBUG message in that case. - for other IOE while deleting, do the full stacktrace but work at INFO instead of WARN. - add tests to cover the above. > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch, > HBASE-20404.2.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Affects Version/s: 1.4.4 > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.4.4, 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch, > HBASE-20404.2.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20404: Resolution: Fixed Fix Version/s: 2.0.1 1.4.4 1.5.0 2.1.0 3.0.0 Status: Resolved (was: Patch Available) pushed to branches-1, branches-2, and master. thanks for the reviews everyone. > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.4.4, 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch, > HBASE-20404.2.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20404: --- Fix Version/s: (was: 1.5.0) > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.4.4, 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.4, 2.0.1 > > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch, > HBASE-20404.2.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[ https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20404: --- Fix Version/s: 1.3.3 > Ugly cleanerchore complaint that dir is not empty > - > > Key: HBASE-20404 > URL: https://issues.apache.org/jira/browse/HBASE-20404 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.4.4, 2.0.0 >Reporter: stack >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.4, 2.0.1 > > Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch, > HBASE-20404.2.patch > > > I see these big dirty exceptions in my master log during a long-run Lets > clean them up (Are they exceptions I as an operator can actually do something > about? Are they 'problems'? Should they be LOG.warn?) > {code} > 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] > cleaner.CleanerChore: Could not delete dir under > hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; > {} > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): > > `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta > is non empty': Directory is not empty > at > org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486) > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489) > at org.apache.hadoop.ipc.Client.call(Client.java:1435) > at org.apache.hadoop.ipc.Client.call(Client.java:1345) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > at com.sun.proxy.$Proxy26.delete(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy27.delete(Unknown Source) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > ... > {code} > Looks like log format is off too... -- This message was sent by Atlassian JIRA (v7.6.3#76005)