hey, so as we transition to the bucket/shard/sack framework for incoming writes, we've set up locks on the sacks so we only have one process handling any given sack. this allows us to remove the per-metric locking we had previously.
the issue i've notice now is that if we only have per-sack locking, metric based actions can affect sack level processing. for example: scenario 1: 1. delete metric, locks sack to delete single metric, 2. metricd processor attempts to process entire sack but can't, so skips. OR scenario 2: 1. API request passes in 'refresh' param so they want all unaggregated metrics to be processed on demand and returned. 2. API locks 1 or more sacks to process 1 or more metrics 3. metricd processor attempts to process entire sack but can't, so skips. potentially multiple sacks unprocessed in currently cycle. scenario 3 same as scenario 2 but metricd processor locks first, and either blocks API process OR doesn't allow API to guarantee 'all measures processed'. i imagine these scenarios are not critical unless a very large processing interval is defined or if for some unfortunate reason, the metric-based actions are perfectly timed to lock out background processing. alternatively, this could be solved by keeping per-metric locks in addition to per-sack locks. this would effectively double the number of active locks we have so instead of each metricd worker having a single per-sack lock, it will also have a per-metric lock for whatever metric it may be publishing at the time. cheers, -- gord __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev