[ https://issues.apache.org/jira/browse/HDDS-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal updated HDDS-3067: -------------------------------- Target Version/s: 0.5.0 > Fix Bug in Scrub Pipeline causing destory pipelines after SCM restart > --------------------------------------------------------------------- > > Key: HDDS-3067 > URL: https://issues.apache.org/jira/browse/HDDS-3067 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM > Reporter: Bharat Viswanadham > Assignee: Bharat Viswanadham > Priority: Blocker > Labels: OMHATest, pull-request-available > Fix For: 0.6.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently, the scrubber is run as part of create pipeline. > When SCM is started, scrubber is coming up and cleaning up all the containers > in SCM. Because when loading pipelines, the pipelineCreationTimeStamp is set > from when the pipeline is created. > > Because of this, below condition is satisfied and destroying all the > pipelines when SCM is restarted. This can be easily reproduced start SCM, > wait for 10 minutes and restart SCM. > > {code:java} > List<Pipeline> needToSrubPipelines = stateManager.getPipelines(type, factor, > Pipeline.PipelineState.ALLOCATED).stream() > .filter(p -> currentTime.toEpochMilli() - p.getCreationTimestamp() > .toEpochMilli() >= pipelineScrubTimeoutInMills) > .collect(Collectors.toList()); > for (Pipeline p : needToSrubPipelines) { > LOG.info("srubbing pipeline: id: " + p.getId().toString() + > " since it stays at ALLOCATED stage for " + > Duration.between(currentTime, p.getCreationTimestamp()).toMinutes() + > " mins."); > finalizeAndDestroyPipeline(p, false); > }{code} > > *Log showing scrubbing of pipeline* > > {code:java} > 2020-02-20 12:42:18,946 [RatisPipelineUtilsThread] INFO > org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: srubbing pipeline: > id: PipelineID=35dff62d-9bfa-449b-b6e8-6f00cc8c1b6e since it stays at > ALLOCATED stage for -1003 mins.{code} > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org