[
https://issues.apache.org/jira/browse/HDDS-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170623#comment-17170623
]
Zheng Huang-Mu commented on HDDS-3724:
--------------------------------------
It's seems to be fixed by [~xyao] in HDDS-4027?
> BackgroundPipelineCreator should not throw Pipeline creation failed exception
> due to insufficient nodes
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-3724
> URL: https://issues.apache.org/jira/browse/HDDS-3724
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Hanisha Koneru
> Priority: Major
>
> BackgroundPipelineCreator runs a scheduler to keep creating pipelines if
> possible. But in a healthy cluster where all the datanodes are part of
> pipelines and there are no more nodes available for creating new pipelines,
> the BackgroundPipelineCreator should not log an error message. In this
> scenario, it is not an error that a pipeline couldn't be created.
> SCM logs were flooded with "Failed to create pipeline" error messages because
> of this:
> {code:java}
> 2020-06-04 11:03:24,229 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed
> due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of
> factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] WARN
> org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline
> creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed
> due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:40,705 [ReplicationMonitor] INFO
> org.apache.hadoop.hdds.scm.container.ReplicationManager: Replication Monitor
> Thread took 0 milliseconds for processing 7 containers.
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of
> factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] WARN
> org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline
> creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed
> due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of
> factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] WARN
> org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline
> creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed
> due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:10:40,700 [ReplicationMonitor] INFO
> org.apache.hadoop.hdds.scm.container.ReplicationManager: Replication Monitor
> Thread took 0 milliseconds for processing 7 containers.
> 2020-06-04 11:11:24,234 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of
> factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:11:24,235 [RatisPipelineUtilsThread] WARN
> org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline
> creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:11:24,235 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed
> due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of
> factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] WARN
> org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline
> creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed
> due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:15:24,242 [RatisPipelineUtilsThread] ERROR
> org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create
> pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of
> factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
