psalagnac commented on PR #1729: URL: https://github.com/apache/solr/pull/1729#issuecomment-1610925579
Main issue with your approach is thread starvation in the collection API. I think we have 50 threads dedicated to this in each node. Assuming we do a backup for a very large collection (with much more shards per node than this number of threads), all threads will be either doing a core backup or waiting for another to be done. My understanding is all other operations at the collection API level will be on hold. I don't think throttling should be done at the cost of blocking most of the threads. An other operation which is not an IO consumer (like creating an empty collection) should ideally not be impacted by the large backup. That's why I think throttinlgw should happen at a higher level. When throttinlgw in `BackupCmd`, a single thread is waiting for all others of the backup. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
