On 03/16/2012 12:18 AM, Jay Dobies wrote:
That has the potential to be really annoying if you have a lot of scheduled syncs. You could be in the middle of some admin operations when one or more scheduled syncs kicks in and basically takes over all of Pulp's processing capabilities. That may be alleviated by suggested usage of off-hours synccing.
PulpDist definitely needs to be able to configure new repos while sync operations are running in the background. (For the initial copy of big trees or when dealing with slow remote servers, PulpDist jobs can take a long time to complete)
And that's just within repo operations. To be delayed from creating a new repo or updating one because I've triggered a handful of consumer operations is also a rough user experience (I say delayed meaning the coordinator isn't the one blocking it, just the sheer lack of open threads in the task pool). That said, I haven't fully thought through what a multiple queue setup would look like. It probably gets really tricky very fast. I just want to make sure we understand how that user experience is going to change now that many more things that previously didn't are now reliant on an open thread in the task pool.
Perhaps it would make sense to have a two-tier task pool? It's similar to your creative math idea, but a bit more formalised.
The basic concept would be to have a "reserved" pool within the larger task pool. Say you had a total task queue size of 10 points, and a reserved weight of 4 points.
Then "background" tasks (like sync jobs or consumer operations) would only be handed over to a task thread if the total weight of the background tasks in progress would remain under the lower non-reserved threshold of 6. "Foreground" tasks (like repo creation and configuration updates) would be allowed up to the full weight of 10 (counting both foreground and background tasks).
That way, the administrators of a particular Pulp installation would have direct control over how they wanted to balance the responsiveness for repo manipulation commands vs content distribution commands.
Cheers, Nick. -- Nick Coghlan Red Hat Engineering Operations, Brisbane _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
