[ https://issues.apache.org/jira/browse/HDDS-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nanda kumar updated HDDS-1126: ------------------------------ Target Version/s: 0.4.0 > datanode is trying to qausi-close a container which is already closed > --------------------------------------------------------------------- > > Key: HDDS-1126 > URL: https://issues.apache.org/jira/browse/HDDS-1126 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode > Reporter: Nilotpal Nandi > Assignee: Nanda kumar > Priority: Major > > steps taken : > -------------------- > # created 12 datanodes cluster and running workload on all the nodes > # running failure injection/restart on 1 datanode at a time periodically and > randomly. > > Error seen in ozone.log : > -------------------------- > > {noformat} > 2019-02-18 06:06:32,780 [Datanode State Machine Thread - 0] DEBUG > (DatanodeStateMachine.java:176) - Executing cycle Number : 30 > 2019-02-18 06:06:32,784 [Command processor thread] DEBUG > (CloseContainerCommandHandler.java:71) - Processing Close Container command. > 2019-02-18 06:06:32,785 [Datanode State Machine Thread - 0] DEBUG > (DatanodeStateMachine.java:176) - Executing cycle Number : 31 > 2019-02-18 06:06:32,785 [Command processor thread] ERROR > (CloseContainerCommandHandler.java:118) - Can't close container #37 > org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: > Cannot quasi close container #37 while in CLOSED state. > at > org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.quasiCloseContainer(KeyValueHandler.java:903) > at > org.apache.hadoop.ozone.container.ozoneimpl.ContainerController.quasiCloseContainer(ContainerController.java:93) > at > org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:110) > at > org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93) > at > org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:413) > at java.lang.Thread.run(Thread.java:748) > 2019-02-18 06:06:32,785 [Command processor thread] DEBUG > (CloseContainerCommandHandler.java:71) - Processing Close Container command. > 2019-02-18 06:06:32,788 [Command processor thread] DEBUG > (CloseContainerCommandHandler.java:71) - Processing Close Container command. > 2019-02-18 06:06:32,788 [Datanode State Machine Thread - 0] DEBUG > (DatanodeStateMachine.java:176) - Executing cycle Number : 32 > 2019-02-18 06:06:34,430 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:36,608 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:38,876 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:41,084 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:43,297 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:45,469 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:47,684 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:49,958 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:52,124 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:54,344 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:56,499 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:06:58,764 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:07:00,969 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:07:02,788 [Datanode State Machine Thread - 0] DEBUG > (DatanodeStateMachine.java:176) - Executing cycle Number : 33 > 2019-02-18 06:07:03,240 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > 2019-02-18 06:07:05,486 [main] DEBUG (OzoneClientFactory.java:287) - Using > org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol. > > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org