[ https://issues.apache.org/jira/browse/HDDS-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jitendra Nath Pandey updated HDDS-1289: --------------------------------------- Priority: Blocker (was: Critical) > get Key failed on SCM restart > ----------------------------- > > Key: HDDS-1289 > URL: https://issues.apache.org/jira/browse/HDDS-1289 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM > Reporter: Nilotpal Nandi > Assignee: Shashikant Banerjee > Priority: Blocker > Attachments: > hadoop-hdfs-scm-ctr-e139-1542663976389-86524-01-000003.log > > > Seeing ContainerNotFoundException in scm log when get key operation tried > after scm restart. > scm.log: > [^hadoop-hdfs-scm-ctr-e139-1542663976389-86524-01-000003.log] > > {noformat} > > > ozone version : > ---------------- > Source code repository g...@github.com:hortonworks/ozone.git -r > 67b7c4fd071b3f557bdb54be2a266b8a611cbce6 > Compiled by jenkins on 2019-03-06T22:02Z > Compiled with protoc 2.5.0 > From source with checksum 65be9a337d178cd3855f5c5a2f111 > Using HDDS 0.4.0.3.0.100.0-348 > Source code repository g...@github.com:hortonworks/ozone.git -r > 67b7c4fd071b3f557bdb54be2a266b8a611cbce6 > Compiled by jenkins on 2019-03-06T22:01Z > Compiled with protoc 2.5.0 > From source with checksum 324109cb3e8b188c1b89dc0b328c3a > root@ctr-e139-1542663976389-86524-01-000006 hdfs# hadoop version > Hadoop 3.1.1.3.0.100.0-348 > Source code repository g...@github.com:hortonworks/hadoop.git -r > 484434b1c2480bdc9314a7ee1ade8a0f4db1758f > Compiled by jenkins on 2019-03-06T22:14Z > Compiled with protoc 2.5.0 > From source with checksum ba6aad94c14256ef3ad8634e3b5086 > This command was run using > /usr/hdp/3.0.100.0-348/hadoop/hadoop-common-3.1.1.3.0.100.0-348.jar > {noformat} > > > > {noformat} > 2019-03-13 17:00:54,348 ERROR container.ContainerReportHandler > (ContainerReportHandler.java:processContainerReplicas(173)) - Received > container report for an unknown container 22 from datanode > 80f046cb-6fe2-4a05-bb67-9bf46f48723b{ip: 172.27.69.155, host: > ctr-e139-1542663976389-86524-01-000005.hwx.site} {} > org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #22 at > org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543) > at > org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230) > at > org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565) > at > org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393) > at > org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51) > at > org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) 2019-03-13 17:00:54,349 ERROR > container.ContainerReportHandler > (ContainerReportHandler.java:processContainerReplicas(173)) - Received > container report for an unknown container 23 from datanode > 80f046cb-6fe2-4a05-bb67-9bf46f48723b{ip: 172.27.69.155, host: > ctr-e139-1542663976389-86524-01-000005.hwx.site} {} > org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #23 at > org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543) > at > org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230) > at > org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565) > at > org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393) > at > org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51) > at > org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) 2019-03-13 17:01:24,230 ERROR > container.ContainerReportHandler > (ContainerReportHandler.java:processContainerReplicas(173)) - Received > container report for an unknown container 22 from datanode > 076fd0d8-ab5f-4fbe-ad10-b71a1ccb19bf{ip: 172.27.39.88, host: > ctr-e139-1542663976389-86524-01-000004.hwx.site} {} > org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #22 at > org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543) > at > org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230) > at > org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565) > at > org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393) > at > org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110) > at > org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51) > at > org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748){noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org