Clint, Morgan, I totally agree that the pub/sub model is better approach.
However, there are 2 great things about polling: 1) it's simpler to use than pub/sub (especially in shell) 2) it has really simple implementation & we can get this in OpenStack in few days/weeks What about just supporting both approaches? Best regards, Boris Pavlovic On Wed, Nov 4, 2015 at 9:33 AM, Morgan Fainberg <morgan.fainb...@gmail.com> wrote: > > On Nov 3, 2015 4:29 PM, "Clint Byrum" <cl...@fewbar.com> wrote: > > > > Excerpts from Boris Pavlovic's message of 2015-11-03 14:20:10 -0800: > > > Hi stackers, > > > > > > Usually such projects like Heat, Tempest, Rally, Scalar, and other tool > > > that works with OpenStack are working with resources (e.g. VM, Volumes, > > > Images, ..) in the next way: > > > > > > >>> resource = api.resouce_do_some_stuff() > > > >>> while api.resource_get(resource["uuid"]) != expected_status > > > >>> sleep(a_bit) > > > > > > For each async operation they are polling and call many times > > > resource_get() which creates significant load on API and DB layers due > the > > > nature of this request. (Usually getting full information about > resources > > > produces SQL requests that contains multiple JOINs, e,g for nova vm > it's 6 > > > joins). > > > > > > What if we add new API method that will just resturn resource status by > > > UUID? Or even just extend get request with the new argument that > returns > > > only status? > > > > I like the idea of being able pass in the set of fields you want to > > see with each get. In SQL, often times only passing in indexed fields > > will allow a query to be entirely serviced by a brief range scan in > > the B-tree. For instance, if you have an index on '(UUID, status)', > > then this lookup will be a single read from an index in MySQL/MariaDB: > > > > SELECT status FROM instances WHERE UUID='foo'; > > > > The explain on this will say 'Using index' and basically you'll just do > > a range scan on the UUID portion, and only find one entry, which will > > be lightning fast, and return only status since it already has it there > > in the index. Maintaining the index is not free, but probably worth it > > if your users really do poll this way a lot. > > > > That said, this is optimizing for polling, and I'm not a huge fan. I'd > > much rather see a pub/sub model added to the API, so that users can > > simply subscribe to changes in resources, and poll only when a very long > > timeout has passed. This will reduce load on API services, databases, > > ++ this is a much better long term solution if we are investing > engineering resources along these lines. > > > caches, etc. There was a thread some time ago about using Nova's built > > in notifications to produce an Atom feed per-project. That seems like > > a much more scalable model, as even polling just that super fast query > > will still incur quite a bit more cost than a GET with If-Modified-Since > > on a single xml file. > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
