szetszwo commented on PR #12075:
URL: https://github.com/apache/ozone/pull/12075#issuecomment-7017212746

   > ... double overhead from Raft side because we double the RaftGroup/SM ...
   
   @amaliujia , it won't since the V3 does not uses Raft, i.e. no raft log.  V3 
will just add a small number of objects (probably 1 or 2 per disk).  We should 
share the resource such as threads and also add new confs.  Below are some 
comments from Claude.
   
   Claude: Closed Raft groups: you're right that they're mostly a transient 
cost.
     - Removal: the group is removed, but only when SCM deletes the pipeline: 
after
       ozone.scm.pipeline.destroy.timeout (300s), checked every
       ozone.scm.pipeline.scrub.interval (150s). Until then it lives on every
       datanode of the pipeline. That wait exists for RATIS, so containers can
       close through Raft before the group goes away. CHAINED containers close
       directly, so a CHAINED pipeline could be deleted right after its 
containers
       close, without waiting for the timeout.
     - Storage: fine. A CHAINED group's Raft directory holds only configuration
       entries and metadata, just a few KB.
     - Threads: mostly not shared. Each group's ContainerStateMachine creates 
its
       own pool of hdds.container.ratis.num.container.op.executors (10) threads
       (ContainerStateMachine:303). Ratis adds about 3–5 more per group. Only 
the
       chunk executors are shared. So threads grow with the number of groups, 
open
       plus not yet removed, and deleting CHAINED pipelines promptly keeps that
       bounded. The per-group pool could also become shared later.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to