On Nov 3, 2015 4:29 PM, "Clint Byrum" <cl...@fewbar.com> wrote: > > Excerpts from Boris Pavlovic's message of 2015-11-03 14:20:10 -0800: > > Hi stackers, > > > > Usually such projects like Heat, Tempest, Rally, Scalar, and other tool > > that works with OpenStack are working with resources (e.g. VM, Volumes, > > Images, ..) in the next way: > > > > >>> resource = api.resouce_do_some_stuff() > > >>> while api.resource_get(resource["uuid"]) != expected_status > > >>> sleep(a_bit) > > > > For each async operation they are polling and call many times > > resource_get() which creates significant load on API and DB layers due the > > nature of this request. (Usually getting full information about resources > > produces SQL requests that contains multiple JOINs, e,g for nova vm it's 6 > > joins). > > > > What if we add new API method that will just resturn resource status by > > UUID? Or even just extend get request with the new argument that returns > > only status? > > I like the idea of being able pass in the set of fields you want to > see with each get. In SQL, often times only passing in indexed fields > will allow a query to be entirely serviced by a brief range scan in > the B-tree. For instance, if you have an index on '(UUID, status)', > then this lookup will be a single read from an index in MySQL/MariaDB: > > SELECT status FROM instances WHERE UUID='foo'; > > The explain on this will say 'Using index' and basically you'll just do > a range scan on the UUID portion, and only find one entry, which will > be lightning fast, and return only status since it already has it there > in the index. Maintaining the index is not free, but probably worth it > if your users really do poll this way a lot. > > That said, this is optimizing for polling, and I'm not a huge fan. I'd > much rather see a pub/sub model added to the API, so that users can > simply subscribe to changes in resources, and poll only when a very long > timeout has passed. This will reduce load on API services, databases,
++ this is a much better long term solution if we are investing engineering resources along these lines. > caches, etc. There was a thread some time ago about using Nova's built > in notifications to produce an Atom feed per-project. That seems like > a much more scalable model, as even polling just that super fast query > will still incur quite a bit more cost than a GET with If-Modified-Since > on a single xml file. > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
