Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package python-requests-ratelimiter for
openSUSE:Factory checked in at 2026-04-25 23:27:56
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-requests-ratelimiter (Old)
and /work/SRC/openSUSE:Factory/.python-requests-ratelimiter.new.11940
(New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-requests-ratelimiter"
Sat Apr 25 23:27:56 2026 rev:2 rq:1349271 version:0.10.0
Changes:
--------
---
/work/SRC/openSUSE:Factory/python-requests-ratelimiter/python-requests-ratelimiter.changes
2026-04-15 20:42:11.666393414 +0200
+++
/work/SRC/openSUSE:Factory/.python-requests-ratelimiter.new.11940/python-requests-ratelimiter.changes
2026-04-25 23:28:14.757646985 +0200
@@ -1,0 +2,11 @@
+Sat Apr 25 19:53:25 UTC 2026 - Dirk Müller <[email protected]>
+
+- update to 0.10.0:
+ * Add max_delay parameter compatible with pyrate-limiter v4
+ * Fix per-host rate-limiting for Redis and Postgres backends
+ * If both per_host=True and a bucket_name is specified, use
+ bucket_name as a bucket prefix
+ * Add warning if a custom Limiter object is passed with
+ per_host=True and no HostBucketFactory
+
+-------------------------------------------------------------------
Old:
----
requests_ratelimiter-0.9.3.tar.gz
New:
----
requests_ratelimiter-0.10.0.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-requests-ratelimiter.spec ++++++
--- /var/tmp/diff_new_pack.2OwxKx/_old 2026-04-25 23:28:15.313669728 +0200
+++ /var/tmp/diff_new_pack.2OwxKx/_new 2026-04-25 23:28:15.313669728 +0200
@@ -17,21 +17,21 @@
Name: python-requests-ratelimiter
-Version: 0.9.3
+Version: 0.10.0
Release: 0
Summary: Easy rate-limiting for python requests
License: MIT
URL: https://github.com/JWCook/requests-ratelimiter
Source:
https://files.pythonhosted.org/packages/source/r/requests_ratelimiter/requests_ratelimiter-%{version}.tar.gz
BuildRequires: %{python_module base >= 3.10}
-BuildRequires: %{python_module pip}
BuildRequires: %{python_module hatchling}
+BuildRequires: %{python_module pip}
+BuildRequires: %{python_module pyrate-limiter >= 4.1 and
%python-pyrate-limiter < 5}
BuildRequires: %{python_module pytest >= 8.3}
BuildRequires: %{python_module pytest-xdist >= 3.1}
BuildRequires: %{python_module requests >= 2.20}
BuildRequires: %{python_module requests-cache >= 1.2.0}
BuildRequires: %{python_module requests-mock >= 1.11}
-BuildRequires: %{python_module pyrate-limiter >= 4.1 and
%python-pyrate-limiter < 5}
Requires: python-requests >= 2.20
Requires: (python-pyrate-limiter >= 4.1 and python-pyrate-limiter < 5)
%python_subpackages
++++++ requests_ratelimiter-0.9.3.tar.gz -> requests_ratelimiter-0.10.0.tar.gz
++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/requests_ratelimiter-0.9.3/PKG-INFO
new/requests_ratelimiter-0.10.0/PKG-INFO
--- old/requests_ratelimiter-0.9.3/PKG-INFO 2020-02-02 01:00:00.000000000
+0100
+++ new/requests_ratelimiter-0.10.0/PKG-INFO 2020-02-02 01:00:00.000000000
+0100
@@ -1,6 +1,6 @@
Metadata-Version: 2.4
Name: requests-ratelimiter
-Version: 0.9.3
+Version: 0.10.0
Summary: Rate-limiting for the requests library
Project-URL: homepage, https://github.com/JWCook/requests-ratelimiter
Project-URL: repository, https://github.com/JWCook/requests-ratelimiter
@@ -149,16 +149,20 @@
<!-- TODO: Section explaining burst rate limit -->
### Advanced Settings
-If you need to define more complex rate limits, you can create a `Limiter`
object instead:
-```python
-from pyrate_limiter import Duration, RequestRate, Limiter
-from requests_ratelimiter import LimiterSession
+If you need to define more complex rate limits, you can use `Limiter` directly.
+Note that it must be used with `HostBucketFactory` if you want per-host
rate-limiting.
-nanocentury_rate = RequestRate(10, Duration.SECOND * 3.156)
-fortnight_rate = RequestRate(1000, Duration.DAY * 14)
-trimonthly_rate = RequestRate(10000, Duration.MONTH * 3)
-limiter = Limiter(nanocentury_rate, fortnight_rate, trimonthly_rate)
+```python
+from pyrate_limiter import Duration, Rate, Limiter
+from requests_ratelimiter import LimiterSession, HostBucketFactory
+nanocentury_rate = Rate(10, Duration.SECOND * 3.156)
+fortnight_rate = Rate(1000, Duration.DAY * 14)
+trimonthly_rate = Rate(10000, Duration.DAY * 90)
+
+# This factory object is required for per-host rate-limiting
+factory = HostBucketFactory(rates=[nanocentury_rate, fortnight_rate,
trimonthly_rate])
+limiter = Limiter(factory)
session = LimiterSession(limiter=limiter)
```
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/requests_ratelimiter-0.9.3/README.md
new/requests_ratelimiter-0.10.0/README.md
--- old/requests_ratelimiter-0.9.3/README.md 2020-02-02 01:00:00.000000000
+0100
+++ new/requests_ratelimiter-0.10.0/README.md 2020-02-02 01:00:00.000000000
+0100
@@ -123,16 +123,20 @@
<!-- TODO: Section explaining burst rate limit -->
### Advanced Settings
-If you need to define more complex rate limits, you can create a `Limiter`
object instead:
+If you need to define more complex rate limits, you can use `Limiter` directly.
+Note that it must be used with `HostBucketFactory` if you want per-host
rate-limiting.
+
```python
-from pyrate_limiter import Duration, RequestRate, Limiter
-from requests_ratelimiter import LimiterSession
+from pyrate_limiter import Duration, Rate, Limiter
+from requests_ratelimiter import LimiterSession, HostBucketFactory
-nanocentury_rate = RequestRate(10, Duration.SECOND * 3.156)
-fortnight_rate = RequestRate(1000, Duration.DAY * 14)
-trimonthly_rate = RequestRate(10000, Duration.MONTH * 3)
-limiter = Limiter(nanocentury_rate, fortnight_rate, trimonthly_rate)
+nanocentury_rate = Rate(10, Duration.SECOND * 3.156)
+fortnight_rate = Rate(1000, Duration.DAY * 14)
+trimonthly_rate = Rate(10000, Duration.DAY * 90)
+# This factory object is required for per-host rate-limiting
+factory = HostBucketFactory(rates=[nanocentury_rate, fortnight_rate,
trimonthly_rate])
+limiter = Limiter(factory)
session = LimiterSession(limiter=limiter)
```
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/requests_ratelimiter-0.9.3/pyproject.toml
new/requests_ratelimiter-0.10.0/pyproject.toml
--- old/requests_ratelimiter-0.9.3/pyproject.toml 2020-02-02
01:00:00.000000000 +0100
+++ new/requests_ratelimiter-0.10.0/pyproject.toml 2020-02-02
01:00:00.000000000 +0100
@@ -1,6 +1,6 @@
[project]
name = 'requests-ratelimiter'
-version = '0.9.3'
+version = '0.10.0'
description = 'Rate-limiting for the requests library'
authors = [{name = 'Jordan Cook'}]
license = 'MIT'
@@ -72,6 +72,11 @@
[tool.mypy]
ignore_missing_imports = true
+[tool.pytest.ini_options]
+markers = [
+ 'integration: marks tests that run against a local HTTP server (deselect
with -m "not integration")',
+]
+
[tool.ruff]
fix = true
unsafe-fixes = true
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/requests_ratelimiter-0.9.3/requests_ratelimiter/buckets.py
new/requests_ratelimiter-0.10.0/requests_ratelimiter/buckets.py
--- old/requests_ratelimiter-0.9.3/requests_ratelimiter/buckets.py
2020-02-02 01:00:00.000000000 +0100
+++ new/requests_ratelimiter-0.10.0/requests_ratelimiter/buckets.py
2020-02-02 01:00:00.000000000 +0100
@@ -1,3 +1,4 @@
+import re
from typing import Dict, Optional, Type
from pyrate_limiter import InMemoryBucket, PostgresBucket, Rate, RedisBucket,
SQLiteBucket
@@ -33,44 +34,41 @@
"""Get or create a bucket for the given item name"""
if item.name not in self.buckets:
# Create new bucket for this name
- bucket = self._create_bucket()
+ bucket = self._create_bucket(item.name)
self.schedule_leak(bucket)
self.buckets[item.name] = bucket
return self.buckets[item.name]
- def _create_bucket(self) -> AbstractBucket:
- """Create a new bucket instance with the configured bucket class"""
+ def _create_bucket(self, name: str) -> AbstractBucket:
+ """Create a new bucket instance, and handle per-host naming for each
supported backend"""
if self.bucket_class == InMemoryBucket:
return InMemoryBucket(self.rates)
elif self.bucket_class == SQLiteBucket:
- kwargs = prepare_sqlite_kwargs(self.bucket_init_kwargs,
self.bucket_name)
+ kwargs = prepare_sqlite_kwargs(self.bucket_init_kwargs, name)
return SQLiteBucket.init_from_file(rates=self.rates, **kwargs)
elif self.bucket_class == RedisBucket:
kwargs = self.bucket_init_kwargs.copy()
- bucket_key = kwargs.pop('bucket_key', self.bucket_name or
'default')
redis = kwargs.pop('redis')
- return RedisBucket.init(rates=self.rates, redis=redis,
bucket_key=bucket_key)
+ bucket_key = kwargs.pop('bucket_key', _sanitize_name(name))
+ return RedisBucket.init(rates=self.rates, redis=redis,
bucket_key=bucket_key, **kwargs)
elif self.bucket_class == PostgresBucket:
kwargs = self.bucket_init_kwargs.copy()
pool = kwargs.pop('pool')
- table = kwargs.pop('table', self.bucket_name or 'default')
+ table = kwargs.pop('table', _sanitize_name(name))
return PostgresBucket(pool=pool, table=table, rates=self.rates)
else:
- # Generic bucket creation - pass rates as first arg
return self.bucket_class(self.rates, **self.bucket_init_kwargs)
def __getitem__(self, name: str) -> AbstractBucket:
"""Dict-like access for backward compatibility with _fill_bucket()
method"""
if name not in self.buckets:
- # Create bucket on access
temp_item = RateItem(name, 0, 1)
return self.get(temp_item)
return self.buckets[name]
def prepare_sqlite_kwargs(bucket_kwargs: Dict, bucket_name: Optional[str] =
None) -> Dict:
- """Prepare SQLiteBucket kwargs for v4 compatibility"""
kwargs = bucket_kwargs.copy()
if 'path' in kwargs:
kwargs['db_path'] = str(kwargs.pop('path'))
@@ -78,8 +76,12 @@
# If bucket_name is specified, use it as the table name to ensure
separation
# This allows multiple sessions with different bucket_names to share a db
file
if bucket_name and 'table' not in kwargs:
- kwargs['table'] = f'bucket_{bucket_name}'
+ kwargs['table'] = f'bucket_{_sanitize_name(bucket_name)}'
# Filter to only supported parameters for SQLiteBucket.init_from_file
supported_params = {'table', 'db_path', 'create_new_table',
'use_file_lock'}
return {k: v for k, v in kwargs.items() if k in supported_params}
+
+
+def _sanitize_name(name: str) -> str:
+ return re.sub(r'[^a-zA-Z0-9_]', '_', name)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/requests_ratelimiter-0.9.3/requests_ratelimiter/requests_ratelimiter.py
new/requests_ratelimiter-0.10.0/requests_ratelimiter/requests_ratelimiter.py
--- old/requests_ratelimiter-0.9.3/requests_ratelimiter/requests_ratelimiter.py
2020-02-02 01:00:00.000000000 +0100
+++
new/requests_ratelimiter-0.10.0/requests_ratelimiter/requests_ratelimiter.py
2020-02-02 01:00:00.000000000 +0100
@@ -7,7 +7,7 @@
from pyrate_limiter import Duration, InMemoryBucket, Limiter, Rate
from pyrate_limiter.abstracts import AbstractBucket, RateItem
-from requests import PreparedRequest, Response, Session
+from requests import PreparedRequest, Response, Session, exceptions
from requests.adapters import HTTPAdapter
from .buckets import HostBucketFactory
@@ -38,6 +38,7 @@
time_function: Optional[Callable[..., float]] = None,
limiter: Optional[Limiter] = None,
per_host: bool = True,
+ max_delay: Optional[float] = None,
limit_statuses: Iterable[int] = (429,),
bucket_name: Optional[str] = None,
**kwargs,
@@ -66,6 +67,11 @@
if limiter:
self.limiter = limiter
self._custom_limiter = True
+ if per_host and not isinstance(limiter.bucket_factory,
HostBucketFactory):
+ logger.warning(
+ 'Custom limiter does not use HostBucketFactory; per-host
rate limiting will '
+ 'not work. Use HostBucketFactory to enable per-host rate
limiting.'
+ )
else:
factory = HostBucketFactory(
rates=rates,
@@ -76,8 +82,7 @@
self.limiter = Limiter(factory, buffer_ms=50)
self._custom_limiter = False
- if kwargs.pop('max_delay', None):
- logger.warning('max_delay is no longer supported')
+ self.max_delay = max_delay
self.limit_statuses = limit_statuses
self.per_host = per_host
self.bucket_name = bucket_name
@@ -91,7 +96,10 @@
def send(self, request: PreparedRequest, **kwargs) -> Response:
"""Send a request with rate-limiting."""
bucket_name = self._bucket_name(request)
- self.limiter.try_acquire(bucket_name, weight=1, blocking=True)
+ timeout = self.max_delay if self.max_delay is not None else -1
+ acquired = self.limiter.try_acquire(bucket_name, weight=1,
blocking=True, timeout=timeout)
+ if not acquired:
+ raise exceptions.Timeout(f'Rate limit not cleared within
max_delay={self.max_delay}s')
response = super().send(request, **kwargs)
if response.status_code in self.limit_statuses:
@@ -101,10 +109,11 @@
def _bucket_name(self, request):
"""Get a bucket name for the given request"""
- if self.bucket_name:
+ if self.per_host:
+ host = urlparse(request.url).netloc
+ return f'{self.bucket_name}:{host}' if self.bucket_name else host
+ elif self.bucket_name:
return self.bucket_name
- elif self.per_host:
- return urlparse(request.url).netloc
else:
return self._default_bucket
@@ -188,16 +197,22 @@
:py:class:`~pyrate_limiter.buckets.redis_bucket.RedisBucket`
bucket_kwargs: Bucket backend keyword arguments
limiter: An existing Limiter object to use instead of the above params
- per_host: Track request rate limits separately for each host
limit_statuses: Alternative HTTP status codes that indicate a rate
limit was exceeded
+ per_host: Track request rate limits separately for each host
+ max_delay: Maximum time (in seconds) to wait for a rate-limited
request. If the rate limit
+ is not cleared within this time, raises
:py:exc:`requests.exceptions.Timeout`.
+ ``None`` = wait indefinitely.
+ bucket_name: Override default bucket name. In per-host mode, this sets
the bucket prefix.
"""
__attrs__ = Session.__attrs__ + [
'limiter',
'limit_statuses',
+ 'max_delay',
'per_host',
'bucket_name',
'_default_bucket',
+ '_custom_limiter',
]
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/requests_ratelimiter-0.9.3/test/test_buckets.py
new/requests_ratelimiter-0.10.0/test/test_buckets.py
--- old/requests_ratelimiter-0.9.3/test/test_buckets.py 1970-01-01
01:00:00.000000000 +0100
+++ new/requests_ratelimiter-0.10.0/test/test_buckets.py 2020-02-02
01:00:00.000000000 +0100
@@ -0,0 +1,136 @@
+from unittest.mock import MagicMock, patch
+
+import pytest
+from pyrate_limiter import InMemoryBucket, PostgresBucket, Rate, RedisBucket
+
+from requests_ratelimiter.buckets import HostBucketFactory,
prepare_sqlite_kwargs
+
+
+def test_in_memory_bucket_creation():
+ factory = HostBucketFactory(rates=[Rate(5, 1000)])
+ item = factory.wrap_item('test_host')
+ factory.get(item)
+ assert 'test_host' in factory.buckets
+
+
+def test_getitem_creates_bucket_on_demand():
+ factory = HostBucketFactory(rates=[Rate(5, 1000)])
+ factory['new_host']
+ assert 'new_host' in factory.buckets
+
+
+def test_getitem_returns_existing_bucket():
+ factory = HostBucketFactory(rates=[Rate(5, 1000)])
+ item = factory.wrap_item('existing_host')
+ bucket1 = factory.get(item)
+ bucket2 = factory['existing_host']
+ assert bucket1 is bucket2
+
+
+def test_wrap_item_uses_bucket_clock():
+ factory = HostBucketFactory(rates=[Rate(5, 1000)])
+ item = factory.wrap_item('test_host')
+ assert item.name == 'test_host'
+ assert item.weight == 1
+ assert item.timestamp > 0
+
+
+def test_wrap_item_custom_weight():
+ factory = HostBucketFactory(rates=[Rate(5, 1000)])
+ item = factory.wrap_item('test_host', weight=3)
+ assert item.weight == 3
+
+
+def test_separate_buckets_per_name():
+ factory = HostBucketFactory(rates=[Rate(5, 1000)])
+ item_a = factory.wrap_item('host_a')
+ item_b = factory.wrap_item('host_b')
+ bucket_a = factory.get(item_a)
+ bucket_b = factory.get(item_b)
+ assert bucket_a is not bucket_b
+ assert len(factory.buckets) == 2
+
+
[email protected](
+ 'extra_init_kwargs, identity, expected_key',
+ [
+ ({}, 'api.example.com', 'api_example_com'),
+ ({'bucket_key': 'override'}, 'api.example.com', 'override'),
+ ({}, 'myapp:api.example.com', 'myapp_api_example_com'),
+ ],
+)
+def test_redis_bucket(extra_init_kwargs, identity, expected_key):
+ mock_redis = MagicMock()
+ mock_redis.script_load.return_value = 'fake_sha1'
+ factory = HostBucketFactory(
+ rates=[Rate(5, 1000)],
+ bucket_class=RedisBucket,
+ bucket_init_kwargs={'redis': mock_redis, **extra_init_kwargs},
+ )
+
+ with patch.object(RedisBucket, 'init', wraps=RedisBucket.init) as
mock_init:
+ factory._create_bucket(identity)
+
+ mock_init.assert_called_once_with(
+ rates=factory.rates, redis=mock_redis, bucket_key=expected_key
+ )
+
+
+def _postgres_init_stub(self, pool, table, rates):
+ self.rates = rates
+ self.failing_rate = None
+
+
[email protected](
+ 'extra_init_kwargs, identity, expected_table',
+ [
+ ({}, 'api.example.com', 'api_example_com'),
+ ({'table': 'override'}, 'api.example.com', 'override'),
+ ({}, 'myapp:api.example.com', 'myapp_api_example_com'),
+ ],
+)
+def test_postgres_bucket(extra_init_kwargs, identity, expected_table):
+ mock_pool = MagicMock()
+ factory = HostBucketFactory(
+ rates=[Rate(5, 1000)],
+ bucket_class=PostgresBucket,
+ bucket_init_kwargs={'pool': mock_pool, **extra_init_kwargs},
+ )
+
+ with patch.object(
+ PostgresBucket,
+ '__init__',
+ autospec=True,
+ return_value=None,
+ side_effect=_postgres_init_stub,
+ ) as mock_init:
+ factory._create_bucket(identity)
+
+ _, kwargs = mock_init.call_args
+ assert kwargs == {'pool': mock_pool, 'table': expected_table, 'rates':
factory.rates}
+
+
+def test_generic_bucket_creation():
+ class CustomBucket(InMemoryBucket):
+ pass
+
+ factory = HostBucketFactory(
+ rates=[Rate(5, 1000)],
+ bucket_class=CustomBucket,
+ )
+ bucket = factory._create_bucket('test_host')
+ assert isinstance(bucket, CustomBucket)
+
+
[email protected](
+ 'kwargs, bucket_name, expected',
+ [
+ ({'path': '/tmp/x.db'}, None, {'db_path': '/tmp/x.db'}),
+ ({}, 'mybucket', {'table': 'bucket_mybucket'}),
+ ({'path': '/tmp/x.db', 'unknown_key': 'val'}, None, {'db_path':
'/tmp/x.db'}),
+ ({'table': 'custom'}, 'mybucket', {'table': 'custom'}),
+ ],
+)
+def test_prepare_sqlite_kwargs(kwargs, bucket_name, expected):
+ result = prepare_sqlite_kwargs(kwargs, bucket_name)
+ assert result == expected
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/requests_ratelimiter-0.9.3/test/test_exports.py
new/requests_ratelimiter-0.10.0/test/test_exports.py
--- old/requests_ratelimiter-0.9.3/test/test_exports.py 2020-02-02
01:00:00.000000000 +0100
+++ new/requests_ratelimiter-0.10.0/test/test_exports.py 2020-02-02
01:00:00.000000000 +0100
@@ -1,8 +1,6 @@
-# ruff: noqa: F403
-# Test pyrate-limiter classes exported from requests_ratelimiter namespace
(only possible in module scope)
-from requests_ratelimiter import *
+import requests_ratelimiter
-# Stub for pytest collection
def test_exports():
- pass
+ for name in requests_ratelimiter.__all__:
+ assert hasattr(requests_ratelimiter, name), f'{name} not found in
requests_ratelimiter'
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/requests_ratelimiter-0.9.3/test/test_integration.py
new/requests_ratelimiter-0.10.0/test/test_integration.py
--- old/requests_ratelimiter-0.9.3/test/test_integration.py 2020-02-02
01:00:00.000000000 +0100
+++ new/requests_ratelimiter-0.10.0/test/test_integration.py 2020-02-02
01:00:00.000000000 +0100
@@ -70,17 +70,14 @@
server.shutdown()
-def _make_session(**kwargs) -> LimiterSession:
- return LimiterSession(**kwargs)
-
-
[email protected]
def test_ratelimit__respects_limit(rate_limit_server):
"""Client should self-throttle so that all requests succeed (no 429s
received).
With per_second=1 and burst=1, the client waits ~1 second between requests,
so 3 sequential requests should take at least 2 seconds total.
"""
- session = _make_session(per_second=1, burst=1)
+ session = LimiterSession(per_second=1, burst=1)
start = time.monotonic()
statuses = [session.get(rate_limit_server).status_code for _ in range(3)]
@@ -90,19 +87,20 @@
assert elapsed >= 2.0, f'Expected >= 2s for 3 requests at 1/s, got:
{elapsed:.2f}s'
[email protected]
def test_ratelimit__server_returns_429(rate_limit_server):
"""When the client bypasses rate-limiting, the server should return 429
responses"""
- # limit_statuses=[] disables the bucket-filling behavior that would
otherwise delay next request
- session = _make_session(per_second=1000, burst=1000, limit_statuses=[])
+ session = LimiterSession(per_second=1000, burst=1000, limit_statuses=[])
statuses = [session.get(rate_limit_server).status_code for _ in range(3)]
assert statuses[0] == 200, f'Expected first request to succeed, got:
{statuses[0]}'
assert 429 in statuses, f'Expected at least one 429, got: {statuses}'
[email protected]
def test_ratelimit__fill_bucket_on_429(rate_limit_server):
"""A 429 response should trigger bucket-filling, delaying the next
request"""
- session = _make_session(per_second=5, burst=5)
+ session = LimiterSession(per_second=5, burst=5)
start = time.monotonic()
statuses = [session.get(rate_limit_server).status_code for _ in range(3)]
elapsed = time.monotonic() - start
@@ -112,10 +110,11 @@
assert elapsed >= 1.0, f'Expected >= 1s delay after 429 (bucket-filling),
got: {elapsed:.2f}s'
[email protected]
def test_ratelimit__per_host_isolation(rate_limit_servers):
"""With per_host=True (default), each host has its own bucket"""
url1, url2 = rate_limit_servers
- session = _make_session(per_second=1, burst=1, per_host=True)
+ session = LimiterSession(per_second=1, burst=1, per_host=True)
# Alternate between the two hosts; neither bucket should fill up
start = time.monotonic()
@@ -129,10 +128,11 @@
assert elapsed < 2.0, f'Expected < 2s with per-host isolation, got:
{elapsed:.2f}s'
[email protected]
def test_ratelimit__per_host_disabled(rate_limit_servers):
"""With per_host=False, all hosts share a single bucket"""
url1, url2 = rate_limit_servers
- session = _make_session(per_second=1, burst=1, per_host=False)
+ session = LimiterSession(per_second=1, burst=1, per_host=False)
# Alternate between two hosts; the shared bucket throttles after the first
request
start = time.monotonic()
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/requests_ratelimiter-0.9.3/test/test_requests_ratelimiter.py
new/requests_ratelimiter-0.10.0/test/test_requests_ratelimiter.py
--- old/requests_ratelimiter-0.9.3/test/test_requests_ratelimiter.py
2020-02-02 01:00:00.000000000 +0100
+++ new/requests_ratelimiter-0.10.0/test/test_requests_ratelimiter.py
2020-02-02 01:00:00.000000000 +0100
@@ -12,17 +12,14 @@
Duration,
InMemoryBucket,
Limiter,
- PostgresBucket,
Rate,
- RedisBucket,
SQLiteBucket,
)
-from requests import PreparedRequest, Session
+from requests import PreparedRequest, Session, exceptions
from requests_cache import CacheMixin
-from requests_ratelimiter import LimiterMixin, LimiterSession
-from requests_ratelimiter.buckets import HostBucketFactory,
prepare_sqlite_kwargs
-from requests_ratelimiter.requests_ratelimiter import _convert_rate
+from requests_ratelimiter import HostBucketFactory, LimiterMixin,
LimiterSession
+from requests_ratelimiter.requests_ratelimiter import _convert_rate,
_get_valid_kwargs
from test.conftest import (
MOCKED_URL,
MOCKED_URL_429,
@@ -51,7 +48,6 @@
lambda: LimiterSession(per_second=5),
lambda: CustomSession(per_second=5, flag=True),
],
- ids=['LimiterSession', 'CustomSession'],
)
def test_rate_limit_enforcement(mock_sleep, session_factory):
session = mount_mock_adapter(session_factory())
@@ -96,6 +92,40 @@
@patch_sleep
+def test_custom_limiter__per_host(mock_sleep):
+ factory = HostBucketFactory(rates=[Rate(5, Duration.SECOND)])
+ limiter = Limiter(factory)
+ session = get_mock_session(limiter=limiter, per_host=True)
+
+ for _ in range(5):
+ session.get(MOCKED_URL)
+ assert mock_sleep.called is False
+
+ # A different host should not be affected
+ session.get(MOCKED_URL_ALT_HOST)
+ assert mock_sleep.called is False
+
+ session.get(MOCKED_URL)
+ assert mock_sleep.called is True
+
+
[email protected](
+ 'session_kwargs, expect_warning',
+ [
+ ({'per_host': True}, True),
+ ({'per_host': False}, False),
+ ],
+)
+def test_custom_limiter__per_host_warning(caplog, session_kwargs,
expect_warning):
+ """Warn when a custom limiter without HostBucketFactory is used with
per_host=True"""
+ bucket = InMemoryBucket([Rate(5, Duration.SECOND)])
+ limiter = Limiter(bucket)
+ with caplog.at_level('WARNING', logger='requests_ratelimiter'):
+ LimiterSession(limiter=limiter, **session_kwargs)
+ assert ('HostBucketFactory' in caplog.text) == expect_warning
+
+
+@patch_sleep
@pytest.mark.parametrize(
'url, session_kwargs, expect_sleep',
[
@@ -138,6 +168,9 @@
(0.5, 1, 1, 2),
(1, 0.5, 2, 1), # 1 req/0.5ms -> 2 req/1ms
(0.001, 1, 1, 1000),
+ (1, 1000, 1, 1000),
+ (100, 1, 100, 1),
+ (10, 10, 10, 10),
],
)
def test_convert_rate(limit, interval, expected_limit, expected_interval):
@@ -148,7 +181,6 @@
@patch_sleep
def test_sqlite_backend(mock_sleep, tmp_path):
- """Check that the SQLite backend works as expected"""
session = get_mock_session(
per_second=5,
bucket_class=SQLiteBucket,
@@ -197,10 +229,7 @@
"""Check that caching integration works as expected"""
class CachedLimiterSession(CacheMixin, LimiterMixin, Session):
- """
- Session class with caching and rate-limiting behavior. Accepts
arguments for both
- LimiterSession and CachedSession.
- """
+ pass
cache_path = tmp_path_factory.mktemp('pytest') / 'cache.db'
ratelimit_path = tmp_path_factory.mktemp('pytest') / 'rate_limit.db'
@@ -224,7 +253,7 @@
assert session.headers is not None
assert session.cookies is not None
assert session.auth is None # Session default
- assert session.hooks is not None
+ assert 'response' in session.hooks
# Tests for lifecycle of bucket factory leaker thread:
@@ -246,7 +275,6 @@
def test_limiter_adapter_close_stops_leaker(limiter_adapter_session: tuple) ->
None:
- """LimiterAdapter.close() stops the Leaker thread."""
session, adapter = limiter_adapter_session
assert adapter.limiter.bucket_factory._leaker is None # no thread before
first request
@@ -260,7 +288,6 @@
def test_limiter_session_close_stops_leaker():
- """LimiterSession.close() stops the Leaker thread spawned on the first
request."""
session = get_mock_session(per_second=5)
assert session.limiter.bucket_factory._leaker is None # no thread before
first request
@@ -274,7 +301,6 @@
def test_limiter_session_context_manager_stops_leaker():
- """Using LimiterSession as a context manager stops the Leaker on
__exit__."""
with get_mock_session(per_second=5) as session:
session.get(MOCKED_URL)
leaker = session.limiter.bucket_factory._leaker
@@ -284,7 +310,6 @@
def test_session_close_cascades_to_limiter_adapter(limiter_adapter_session:
tuple) -> None:
- """Closing a Session cascades to LimiterAdapter.close(), stopping the
Leaker."""
session, adapter = limiter_adapter_session
session.get(MOCKED_URL)
@@ -297,7 +322,6 @@
def test_close_before_any_request_and_idempotent():
- """close() before any request is a safe no-op; calling it twice does not
raise."""
session = LimiterSession(per_second=5)
assert session.limiter.bucket_factory._leaker is None
session.close() # no Leaker was ever created — must not raise
@@ -306,17 +330,15 @@
@patch_sleep
def test_fill_bucket_with_custom_limiter(mock_sleep):
- """_fill_bucket falls back to limiter.buckets() when a custom Limiter is
provided"""
bucket = InMemoryBucket([Rate(5, Duration.SECOND)])
limiter = Limiter(bucket)
- session = get_mock_session(limiter=limiter)
+ session = get_mock_session(limiter=limiter, per_host=False)
session.get(MOCKED_URL_429)
session.get(MOCKED_URL_429)
- assert mock_sleep.called
+ assert mock_sleep.called is True
def test_fill_bucket_no_bucket_logs_warning(caplog):
- """_fill_bucket with no available bucket logs a warning and returns
cleanly"""
mock_limiter = MagicMock()
del mock_limiter.bucket_factory.__getitem__ # no dict-like access
mock_limiter.buckets.return_value = []
@@ -332,42 +354,6 @@
assert 'No buckets available' in caplog.text
-def test_redis_bucket():
- mock_redis = MagicMock()
- mock_redis.script_load.return_value = 'fake_sha1'
- factory = HostBucketFactory(
- rates=[Rate(5, 1000)],
- bucket_class=RedisBucket,
- bucket_init_kwargs={'redis': mock_redis, 'bucket_key': 'test_bucket'},
- )
-
- with patch.object(RedisBucket, 'init', wraps=RedisBucket.init) as
mock_init:
- factory._create_bucket()
-
- mock_init.assert_called_once_with(
- rates=factory.rates, redis=mock_redis, bucket_key='test_bucket'
- )
-
-
-def test_postgres_bucket():
- mock_pool = MagicMock()
- factory = HostBucketFactory(
- rates=[Rate(5, 1000)],
- bucket_class=PostgresBucket,
- bucket_init_kwargs={'pool': mock_pool, 'table': 'test_table'},
- )
-
- with patch.object(PostgresBucket, '__init__', autospec=True,
return_value=None) as mock_init:
- mock_init.side_effect = lambda self, pool, table, rates: (
- setattr(self, 'rates', rates) or setattr(self, 'failing_rate',
None)
- )
- factory._create_bucket()
-
- mock_init.assert_called_once()
- _, kwargs = mock_init.call_args
- assert kwargs == {'pool': mock_pool, 'table': 'test_table', 'rates':
factory.rates}
-
-
@patch_sleep
def test_custom_bucket_class(mock_sleep):
class MyBucket(InMemoryBucket):
@@ -379,39 +365,27 @@
assert mock_sleep.called is False
session.get(MOCKED_URL)
assert mock_sleep.called is True
+ session.close()
-def test_bucket_name_overrides_per_host():
- """Explicit bucket_name takes priority over per_host=True"""
- session = LimiterSession(per_second=5, bucket_name='fixed', per_host=True)
+def test_bucket_name_prefixes_per_host():
+ session = LimiterSession(per_second=5, bucket_name='myapp', per_host=True)
req = PreparedRequest()
req.url = MOCKED_URL
- assert session._bucket_name(req) == 'fixed'
+ assert session._bucket_name(req) == 'myapp:requests-ratelimiter.com'
[email protected](
- 'kwargs, bucket_name, expected',
- [
- ({'path': '/tmp/x.db'}, None, {'db_path': '/tmp/x.db'}),
- ({}, 'mybucket', {'table': 'bucket_mybucket'}),
- ({'path': '/tmp/x.db', 'unknown_key': 'val'}, None, {'db_path':
'/tmp/x.db'}),
- ({'table': 'custom'}, 'mybucket', {'table': 'custom'}), # explicit
table not overridden
- ],
-)
-def test_prepare_sqlite_kwargs(kwargs, bucket_name, expected):
- result = prepare_sqlite_kwargs(kwargs, bucket_name)
- assert result == expected
-
-
-def test_max_delay_logs_warning(caplog):
- with caplog.at_level('WARNING', logger='requests_ratelimiter'):
- LimiterSession(per_second=5, max_delay=10)
- assert 'max_delay' in caplog.text
+@patch_sleep
+def test_max_delay_raises_on_timeout(mock_sleep):
+ session = get_mock_session(per_second=5, max_delay=0.001)
+ for _ in range(5):
+ session.get(MOCKED_URL)
+ with pytest.raises(exceptions.Timeout, match='max_delay'):
+ session.get(MOCKED_URL)
@patch_sleep
def test_burst_allows_consecutive_requests(mock_sleep):
- """burst=3 allows 3 rapid consecutive requests before enforcing per-second
limit"""
session = get_mock_session(per_second=1, burst=3)
for _ in range(3):
session.get(MOCKED_URL)
@@ -427,13 +401,10 @@
(InMemoryBucket, {}), # InMemoryBucket
(SQLiteBucket, None), # SQLiteBucket (will use fixture to provide
kwargs)
],
- ids=['in_memory', 'sqlite'],
)
def test_pickling(mock_sleep, bucket_class, bucket_kwargs, tmp_path):
if bucket_class == SQLiteBucket:
bucket_kwargs = {'path': tmp_path / 'rate_limit.db',
**SQLITE_BUCKET_KWARGS}
- elif bucket_kwargs is None:
- bucket_kwargs = {}
session = get_mock_session(per_second=5, bucket_class=bucket_class,
bucket_kwargs=bucket_kwargs)
unpickled = pickle.loads(pickle.dumps(session))
@@ -449,3 +420,151 @@
assert mock_sleep.called is False
unpickled.get(MOCKED_URL)
assert mock_sleep.called is True
+
+
+def test_pickling_custom_limiter():
+ """Unpickling a session with a custom limiter should preserve
_custom_limiter so that
+ close() does not try to stop a factory it doesn't own."""
+ bucket = InMemoryBucket([Rate(5, Duration.SECOND)])
+ limiter = Limiter(bucket)
+ session = get_mock_session(limiter=limiter, per_host=False)
+ unpickled = pickle.loads(pickle.dumps(session))
+ assert unpickled._custom_limiter is True
+ unpickled.close() # must not raise
+
+
+@patch_sleep
[email protected]('method', ['get', 'post', 'put', 'patch', 'delete',
'head', 'options'])
+def test_rate_limiting_applies_to_all_http_methods(mock_sleep, method):
+ session = get_mock_session(per_second=5)
+ http_method = getattr(session, method)
+ for _ in range(5):
+ http_method(MOCKED_URL)
+ assert mock_sleep.called is False
+ http_method(MOCKED_URL)
+ assert mock_sleep.called is True
+
+
+@patch_sleep
+def test_combined_rate_limits(mock_sleep):
+ session = get_mock_session(per_second=5, per_minute=10)
+
+ for _ in range(5):
+ session.get(MOCKED_URL)
+ assert mock_sleep.called is False
+
+ session.get(MOCKED_URL)
+ assert mock_sleep.called is True
+
+
[email protected](
+ 'kwargs, expected_limit',
+ [
+ ({'per_hour': 100}, 100),
+ ({'per_day': 1000}, 1000),
+ ({'per_month': 10000}, 10000),
+ ],
+)
+def test_rate_limit_period(kwargs, expected_limit):
+ session = LimiterSession(**kwargs)
+ factory = session.limiter.bucket_factory
+ assert len(factory.rates) == 1
+ assert factory.rates[0].limit == expected_limit
+
+
+def test_no_rate_limits_no_limiter():
+ session = LimiterSession()
+ assert session.limiter is not None
+ factory = session.limiter.bucket_factory
+ assert len(factory.rates) == 0
+
+
[email protected](
+ 'url, bucket_name, expected_name',
+ [
+ ('http+mock://example.com/path', None, 'example.com'),
+ ('http+mock://example.com:8080/path', None, 'example.com:8080'),
+ ('http+mock://192.168.1.1/path', None, '192.168.1.1'),
+ ('http+mock://[::1]/path', None, '[::1]'),
+ ('http+mock://[::1]:8080/path', None, '[::1]:8080'),
+ ('http+mock://example.com/path', 'myapp', 'myapp:example.com'),
+ ],
+)
+def test_bucket_name_from_url(url, bucket_name, expected_name):
+ """per_host bucket names are derived from URL netloc, including ports and
IPs"""
+ session = LimiterSession(per_second=5, per_host=True,
bucket_name=bucket_name)
+ req = PreparedRequest()
+ req.url = url
+ assert session._bucket_name(req) == expected_name
+
+
+def test_custom_limiter_close_does_not_stop_factory():
+ bucket = InMemoryBucket([Rate(5, Duration.SECOND)])
+ limiter = Limiter(bucket)
+ session = get_mock_session(limiter=limiter, per_host=False)
+ session.get(MOCKED_URL)
+ session.close()
+
+
+@patch_sleep
+def test_limiter_adapter_per_host(mock_sleep, limiter_adapter_session: tuple)
-> None:
+ session, adapter = limiter_adapter_session
+
+ for _ in range(5):
+ session.get(MOCKED_URL)
+ assert mock_sleep.called is False
+
+ session.get(MOCKED_URL_ALT_HOST)
+ assert mock_sleep.called is False
+
+ session.get(MOCKED_URL)
+ assert mock_sleep.called is True
+
+
+@patch_sleep
+def test_shared_bucket_name_enforces_shared_limit(mock_sleep, tmp_path):
+ ratelimit_path = tmp_path / 'rate_limit.db'
+
+ session_a = get_mock_session(
+ per_second=5,
+ bucket_name='shared',
+ bucket_class=SQLiteBucket,
+ bucket_kwargs={'path': ratelimit_path, **SQLITE_BUCKET_KWARGS},
+ )
+ session_b = get_mock_session(
+ per_second=5,
+ bucket_name='shared',
+ bucket_class=SQLiteBucket,
+ bucket_kwargs={'path': ratelimit_path, **SQLITE_BUCKET_KWARGS},
+ )
+
+ for _ in range(5):
+ session_a.get(MOCKED_URL)
+ assert mock_sleep.called is False
+
+ session_b.get(MOCKED_URL)
+ assert mock_sleep.called is True
+
+
+@patch_sleep
+def test_limit_statuses_multiple_codes(mock_sleep):
+ session = get_mock_session(per_second=5, limit_statuses=[429, 500])
+
+ session.get(MOCKED_URL_500)
+ assert mock_sleep.called is False
+
+ session.get(MOCKED_URL_500)
+ assert mock_sleep.called is True
+
+
[email protected](
+ 'func, kwargs, expected',
+ [
+ (lambda x, y: None, {'x': 1, 'y': 2, 'z': 3}, {'x': 1, 'y': 2}),
+ (lambda x: None, {'x': 1, 'y': 2}, {'x': 1}),
+ (lambda: None, {'x': 1}, {}),
+ (lambda x, y=None: None, {'x': 1, 'y': None}, {'x': 1}),
+ ],
+)
+def test_get_valid_kwargs(func, kwargs, expected):
+ assert _get_valid_kwargs(func, kwargs) == expected
