[
https://issues.apache.org/jira/browse/HDDS-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963214#comment-16963214
]
Anu Engineer commented on HDDS-2383:
------------------------------------
Looks like the SCMCLI was trying to close a container that SCM is also trying
to close. Would it be possible to close a good working container and see if you
see this error again. If so, it is a bug. Thanks
> Closing open container via SCMCli throws exception
> --------------------------------------------------
>
> Key: HDDS-2383
> URL: https://issues.apache.org/jira/browse/HDDS-2383
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
> Reporter: Rajesh Balamohan
> Priority: Major
>
> This was observed in apache master branch.
> Closing the container via {{SCMCli}} throws the following exception, though
> the container ends up getting closed eventually.
> {noformat}
> 2019-10-30 02:44:41,794 INFO
> org.apache.hadoop.hdds.scm.block.SCMBlockDeletingService: Block deletion
> txnID mismatch in datanode 79626ba3-1957-46e5-a8b0-32d7f47fb801 for
> containerID 6. Datanode delete txnID: 0, SCM txnID: 1004
> 2019-10-30 02:44:41,810 INFO
> org.apache.hadoop.hdds.scm.container.IncrementalContainerReportHandler:
> Moving container #4 to CLOSED state, datanode
> 8885d4ba-228a-4fd2-bf5a-831f01594c6c{ip: 10.17.234.37, host:
> vd1327.halxg.cloudera.com, networkLocation: /default-rack, certSerialId:
> null} reported CLOSED replica.
> 2019-10-30 02:44:41,826 INFO
> org.apache.hadoop.hdds.scm.server.SCMClientProtocolServer: Object type
> container id 4 op close new stage complete
> 2019-10-30 02:44:41,826 ERROR
> org.apache.hadoop.hdds.scm.container.ContainerStateManager: Failed to update
> container state #4, reason: invalid state transition from state: CLOSED upon
> event: CLOSE.
> 2019-10-30 02:44:41,826 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 6 on 9860, call Call#3 Retry#0
> org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocol.submitRequest
> from 10.17.234.32:45926
> org.apache.hadoop.hdds.scm.exceptions.SCMException: Failed to update
> container state #4, reason: invalid state transition from state: CLOSED upon
> event: CLOSE.
> at
> org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerState(ContainerStateManager.java:338)
> at
> org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerState(SCMContainerManager.java:326)
> at
> org.apache.hadoop.hdds.scm.server.SCMClientProtocolServer.notifyObjectStageChange(SCMClientProtocolServer.java:388)
> at
> org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.notifyObjectStageChange(StorageContainerLocationProtocolServerSideTranslatorPB.java:303)
> at
> org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.processRequest(StorageContainerLocationProtocolServerSideTranslatorPB.java:158)
> at
> org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB$$Lambda$152/2036820231.apply(Unknown
> Source)
> at
> org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:72)
> at
> org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.submitRequest(StorageContainerLocationProtocolServerSideTranslatorPB.java:112)
> at
> org.apache.hadoop.hdds.protocol.proto.StorageContainerLocationProtocolProtos$StorageContainerLocationProtocolService$2.callBlockingMethod(StorageContainerLocationProtocolProtos.java:30454)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
