Nilotpal Nandi created HDDS-1131: ------------------------------------ Summary: destroy pipeline failed with PipelineNotFoundException Key: HDDS-1131 URL: https://issues.apache.org/jira/browse/HDDS-1131 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi
steps taken : -------------------- # created 12 datanodes cluster and running workload on all the nodes exceptions seen in scm log ---------------------------------------- {noformat} 2019-02-18 07:17:51,112 INFO org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: destroying pipeline:PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb with group-012343D76ADB:[a40a7b01-a30b-469c-b373-9fcb20a126ed:172.27.54.212:9858, 8c77b16b-8054-49e3-b669-1ff759cfd271:172.27.23.196:9858, 943007c8-4fdd-4926-89e2-2c8c52c05073:172.27.76.72:9858] 2019-02-18 07:17:51,112 INFO org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler: Close container Event triggered for container : #40 2019-02-18 07:17:51,113 INFO org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler: Close container Event triggered for container : #41 2019-02-18 07:17:51,114 INFO org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler: Close container Event triggered for container : #42 2019-02-18 07:22:51,127 WARN org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Pipeline destroy failed for pipeline=PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb dn=a40a7b01-a30b-469c-b373-9fcb20a126ed{ip: 172.27.54.212, host: ctr-e139-1542663976389-62237-01-000007.hwx.site} 2019-02-18 07:22:51,139 WARN org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Pipeline destroy failed for pipeline=PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb dn=8c77b16b-8054-49e3-b669-1ff759cfd271{ip: 172.27.23.196, host: ctr-e139-1542663976389-62237-01-000015.hwx.site} 2019-02-18 07:22:51,149 WARN org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Pipeline destroy failed for pipeline=PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb dn=943007c8-4fdd-4926-89e2-2c8c52c05073{ip: 172.27.76.72, host: ctr-e139-1542663976389-62237-01-000006.hwx.site} 2019-02-18 07:22:51,150 ERROR org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Destroy pipeline failed for pipeline:PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb with group-012343D76ADB:[a40a7b01-a30b-469c-b373-9fcb20a126ed:172.27.54.212:9858, 8c77b16b-8054-49e3-b669-1ff759cfd271:172.27.23.196:9858, 943007c8-4fdd-4926-89e2-2c8c52c05073:172.27.76.72:9858] org.apache.hadoop.hdds.scm.pipeline.PipelineNotFoundException: PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb not found at org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.getPipeline(PipelineStateMap.java:112) at org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.removePipeline(PipelineStateMap.java:247) at org.apache.hadoop.hdds.scm.pipeline.PipelineStateManager.removePipeline(PipelineStateManager.java:90) at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.removePipeline(SCMPipelineManager.java:261) at org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils.destroyPipeline(RatisPipelineUtils.java:103) at org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils.lambda$finalizeAndDestroyPipeline$1(RatisPipelineUtils.java:133) at org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$0(TimeoutScheduler.java:85) at org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$1(TimeoutScheduler.java:104) at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:50) at org.apache.ratis.util.LogUtils$1.run(LogUtils.java:91) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748){noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org