sergey-safarov opened a new issue, #5811:
URL: https://github.com/apache/couchdb/issues/5811

   ## Summary
   
   I have a large shard which needs to be indexed using a new view. When the 
indexing operation started, the same shard was indexed on several nodes.
   
   <img width="1657" height="1096" alt="Image" 
src="https://github.com/user-attachments/assets/e52abb40-37f5-402f-b868-a7cdd32b346f";
 />
   
   This can be optimised using this logic.
   1) If the shard has X changes, then the required delay is indexing for Y 
milliseconds;
   2) check the other cluster node started indexing for the same shard;
   3) if the shard is already indexed on another node, then skip this shard for 
indexing now.
   
   The same logic applied to all shards. At the end, all nodes are randomly 
distributed shards for indexing.
   On the next view request, the same shard will be indexed on the other node.
   
   ## Desired Behaviour
   
   Nodes distribute the shards indexing task and do not index the same shard on 
several nodes at the same time.
   
   ## Additional context
   
   This allows for distributing CPU load on nodes and makes CPU spikes smoother.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to