[
https://issues.apache.org/jira/browse/HDDS-17327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng reassigned HDDS-17327:
---------------------------------
Assignee: (was: Siyao Meng)
> Topology-aware read tries an unresponsive nearest datanode first for every
> ReadChunk
> ------------------------------------------------------------------------------------
>
> Key: HDDS-17327
> URL: https://issues.apache.org/jira/browse/HDDS-17327
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: Siyao Meng
> Priority: Major
>
> With {{ozone.network.topology.aware.read}}=true (the default),
> {{XceiverClientGrpc.sortDatanodes}} orders the replicas by distance for every
> ReadChunk, and {{sendCommandWithRetry}} tries them in that order. Nothing
> remembers that a datanode just failed: {{XceiverClientGrpc}} keeps no failed
> datanodes, and the exclusion in {{ContainerProtocolCalls.tryEachDatanode}}
> only lasts for one call. If the nearest datanode hangs (e.g. a stuck disk or
> a long GC pause) instead of refusing connections, every ReadChunk waits
> {{ozone.client.read.timeout}} (30s) before the next replica answers. A
> ReadChunk is sent for each read call within a chunk, so by calculation
> reading a 256MB block with 4MB reads takes up to 64 × 30s, about 32 minutes,
> longer.
> HDDS-16419 removes the nested N × N failover, but keeps the same order for
> each call.
> Proposed fix: keep a set of datanodes that failed recently, with an expiry
> like {{ozone.client.exclude.nodes.expiry.time}} on the write side, and try
> them last.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
