#35651: The number of Redis connections has been continuously increasing.
-------------------------------------+-------------------------------------
Reporter: gojuukaze | Owner: (none)
Type: Bug | Status: new
Component: Core (Cache system) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by gojuukaze):
* summary: django redis cache not really using connection pooling => The
number of Redis connections has been continuously increasing.
Old description:
> I'm using `django.core.cache.backends.redis.RedisCache` as a backend for
> redis, and I recently observed that the number of Redis connections has
> been continuously increasing.
>
> After debugging, I found that whenever the cache method is called, the
> `RedisCacheClient `class is reinitialized, and in the `__init__` function
> of `RedisCacheClient`, the connection pool is set to empty.
>
> This causes the `_get_connection_pool` method of `RedisCacheClient` to
> always create a new connection pool instead of using the existing one.
>
> {{{
> class RedisCacheClient:
> def __init__(
> self,
> servers,
> serializer=None,
> pool_class=None,
> parser_class=None,
> **options,
> ):
> import redis
>
> self._lib = redis
> self._servers = servers
> self._pools = {} # << === set pools
>
> def _get_connection_pool(self, write):
> index = self._get_connection_pool_index(write)
>
> #
> # self._pools is is always empty.
> #
>
> if index not in self._pools:
> self._pools[index] = self._pool_class.from_url(
> self._servers[index],
> **self._pool_options,
> )
> return self._pools[index]
> }}}
>
> One solution is to put `_pools` outside of `__init__` , for example：
>
> {{{
> class RedisCacheClient:
>
> # init pool
> _pools = {}
>
> def __init__(
> self,
> servers,
> serializer=None,
> pool_class=None,
> parser_class=None,
> **options,
> ):
> import redis
>
> self._lib = redis
> self._servers = servers
> ## self._pools={}
>
> }}}
>
> By the way, I am using Django 5.0.7 and running it in asynchronous mode.
New description:
I'm using `django.core.cache.backends.redis.RedisCache` as a backend for
redis, and I recently observed that the number of Redis connections has
been continuously increasing.
After debugging, I found that whenever the cache method is called, the
`RedisCacheClient `class is reinitialized, and in the `__init__` function
of `RedisCacheClient`, the connection pool is set to empty.
This causes the `_get_connection_pool` method of `RedisCacheClient` to
always create a new connection pool and a new connection instead of using
the existing one.
{{{
class RedisCacheClient:
def __init__(
self,
servers,
serializer=None,
pool_class=None,
parser_class=None,
**options,
):
import redis
self._lib = redis
self._servers = servers
self._pools = {} # << === set pools
def _get_connection_pool(self, write):
index = self._get_connection_pool_index(write)
#
# self._pools is is always empty.
#
if index not in self._pools:
self._pools[index] = self._pool_class.from_url(
self._servers[index],
**self._pool_options,
)
return self._pools[index]
}}}
One solution is to put `_pools` outside of `__init__` , for example：
{{{
class RedisCacheClient:
# init pool
_pools = {}
def __init__(
self,
servers,
serializer=None,
pool_class=None,
parser_class=None,
**options,
):
import redis
self._lib = redis
self._servers = servers
## self._pools={}
}}}
By the way, I am using Django 5.0.7 and running it in asynchronous mode.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/35651#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/010701910d6d8d7b-ba0c9bcd-9be6-40c8-a1ea-010c3685a523-000000%40eu-central-1.amazonses.com.
