[
https://issues.apache.org/jira/browse/HDFS-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZanderXu updated HDFS-16692:
----------------------------
Description:
When we write some ec data from clients that are not in hdfs cluster, there is
a lot of INFO log output, as blew
{code:shell}
2022-07-26 15:50:40,973 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
2022-07-26 15:50:40,974 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,974 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
2022-07-26 15:50:40,975 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,975 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
2022-07-26 15:50:40,976 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,976 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=19}
2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=3}
{code}
I feel that we should add detailed scope info in this log to show the scope
that we cannot select any good nodes from.
was:
When we writing some ec data from client not in hdfs cluster, there are a large
number of INFO log output, as blew:
{code:shell}
2022-07-26 15:50:40,973 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
2022-07-26 15:50:40,974 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,974 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
2022-07-26 15:50:40,975 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,975 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
2022-07-26 15:50:40,976 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,976 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=19}
2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
(BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas was
chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=3}
{code}
I feel that we should add detailed scope info in this log to show the scope
that we cannot select any good nodes from.
> Add detailed scope info in NotEnoughReplicas Reason logs.
> ---------------------------------------------------------
>
> Key: HDFS-16692
> URL: https://issues.apache.org/jira/browse/HDFS-16692
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When we write some ec data from clients that are not in hdfs cluster, there
> is a lot of INFO log output, as blew
> {code:shell}
> 2022-07-26 15:50:40,973 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
> 2022-07-26 15:50:40,974 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
> 2022-07-26 15:50:40,974 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
> 2022-07-26 15:50:40,975 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
> 2022-07-26 15:50:40,975 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=17}
> 2022-07-26 15:50:40,976 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
> 2022-07-26 15:50:40,976 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
> 2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=18}
> 2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=19}
> 2022-07-26 15:50:40,977 INFO blockmanagement.BlockPlacementPolicy
> (BlockPlacementPolicyDefault.java:chooseRandom(912)) - Not enough replicas
> was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1, TOO_MANY_NODES_ON_RACK=3}
> {code}
> I feel that we should add detailed scope info in this log to show the scope
> that we cannot select any good nodes from.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
