GitHub user fjros opened a pull request:
https://github.com/apache/libcloud/pull/1058
Handle Vultr API rate-limit
## Handle Vultr API rate-limit
### Description
Vultr API imposes a rate limit of 2 requests/sec (actually they provide
some leeway, but let's stick with the official documentation). A typical
workflow for a libcloud's client that wants to launch a Vultr node could
involve something like the following:
1. driver.list_locations()
2. driver.list_sizes()
3. driver.list_images()
4. driver.list_key_pairs()
5. driver.create_key_pair(name, key)
6. driver.create_node(...)
Even a synchronous client will exceed the rate limit easily.
This PR makes Vultr compute driver handle rate-limiting in two ways:
1. Minimize the likelihood of being rate-limited by not authenticating
requests that don't require an api key. Such requests aren't rate-limited.
2. If hit by the rate limit, retry the request. A `rate_limited` decorator
is in charge of automatically retrying the request after a sleep period when a
503 Service Unavailable response is received.
In order to implement (1), this PR first updates the way the driver
authenticates with Vultr. Now it uses an HTTP header (as described in current
api docs: https://www.vultr.com/api/) instead of a query parameter.
### Status
- done, ready for review
### Checklist (tick everything that applies)
- [ ] [Code
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
(required, can be done after the PR checks)
- [ ] Documentation
- [ ] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
- [ ]
[ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
(required for bigger changes)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fjros/libcloud vultr-rate-limit
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/libcloud/pull/1058.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1058
----
commit 413990ad52b637763dacdf4331eeb2c94ce155d1
Author: Francisco Ros <fj...@doalitic.com>
Date: 2017-05-23T15:01:50Z
Vultr driver authenticates with api key using header instead of query param
This aligns with current api docs: https://www.vultr.com/api/
commit 566dc9fa6f71adc41d07e32f1fd7248352a67f46
Author: Francisco Ros <fj...@doalitic.com>
Date: 2017-05-23T16:10:22Z
Vultr compute driver doesn't authenticate calls that don't need it
This minimizes the likelihood of hitting the rate-limit of Vultr api.
commit 1749cc6589640b78b485ee0f3de89664f0021ef2
Author: Francisco Ros <fj...@doalitic.com>
Date: 2017-05-24T07:00:53Z
Add class ServiceUnavailableError for HTTP 503 return codes
commit cba9b0cbfe193fe5678c8cb22a8230c21c3df600
Author: Francisco Ros <fj...@doalitic.com>
Date: 2017-05-24T07:03:10Z
Retry requests that have been rate-limited in Vultr compute driver
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
