murblanc commented on PR #1729: URL: https://github.com/apache/solr/pull/1729#issuecomment-1610372260
I think there are multiple options for throttling: - Is the throttling decision made at the source (CollectionsHandler or Overseer) or on the node actually executing the action? - Are the limits per command execution or overall for a node? (the second option seems better for protecting resources) This proposal is deciding to throttle at the node and takes into account all (backup/restore) commands that need to run on that node. IMO this is too late in the command execution lifecycle. It can lead to starting execution of a very large number of commands and blocking them all waiting for each other. It might be more efficient to block earlier so that commands can complete faster and not compete for resources with newer commands starting execution in an already saturated environment. It can also allow prioritizing one type of command while blocking others (for example if there are restores in the queue they are unblocked first and backups wait for the restores to complete - assuming for example that restores are more urgent than backups). Throttling earlier could also be surfaced back to the client by rejecting new API calls if the system can tell it won't be able to handle them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
