[
https://issues.apache.org/jira/browse/HDDS-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145637#comment-18145637
]
Andrey Yarovoy commented on HDDS-15463:
---------------------------------------
For ratis pipelines we also have problem with balancing. During performance
testing I noticed that 1 or 2 nodes are getting 2x less number of pipelines
comparing to other nodes. At least we have outstanding work for
balancing/rebalancing RATIS pipelines.
As far as I understand we have 2 option for N way replication: send N write to
all the DN and mark write successful when we archive the write quorum. Or write
first replica to one of DNs in the pipeline and let the DN to send out N-1
writes to other DNs in the pipeline and mark the write successful as soon as we
get the write quorum. 1st one would be less latency and more load to client,
the 2nd one adds latency and increase the load to DNs. If we take 2nd approach
we need a design for persistent connection pool between DNs and how to share
these connection.
> Streaming Write Pipeline without Raft
> -------------------------------------
>
> Key: HDDS-15463
> URL: https://issues.apache.org/jira/browse/HDDS-15463
> Project: Apache Ozone
> Issue Type: New Feature
> Components: Ozone Client, Ozone Datanode
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Major
> Labels: pull-request-available
> Attachments: 20260712HDDS-15463 Streaming Write Pipeline without
> Raft.pdf
>
>
> - V1) Raft Pipeline: Use Raft for both WriteChunk and PutBlock
> - V2)Streaming Write Pipeline (HDDS-4454): Use Ratis streaming (RATIS-979)
> for WriteChunk and Raft for PutBlock
> - V3) Streaming Write Pipeline without Raft: Use Ratis streaming for both
> WriteChunk and PutBlock.
> We implement V3 in this JIRA. Will post a design and create subtasks.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]