Hello everyone. I am seeking some help with Django Channels and Redis.
I have a Django app running on Heroku. This app is rather simple: a Daphne
server, a background worker and a scheduling beat worker (the last two,
using Celery).
I noticed I hit *very often* Redi's MaxClientsError (every 15 minutes or
so) despite using the non-free plan of RedisCloud on Heroku, which has the
following parameters: Memory Size = 100 MB, Dedicated DB = 4, Connections =
256. I was using the free plan for a while with 30 connections, but the
MaxClientsError was popping all the time. Thus I increased to 256
connections, but since I have more bakground tasks, I hit again the error.
In the stacktrace below, you can see the problem occurs within
djangochannelsrestframework code. Line 233, there is this:
for group_name in group_names:
async_to_sync(channel_layer.group_send)(group_name, message)
But we never managed to understand if that was the cause of the error. Is
that way of doing the reason for opening so many connections ? My app is
not a chat. It has only background tasks that discover some data remotely,
and send them through websockets for a kind of "live events RSS".
Wouldn't it be better to use something like:
async2sync = async_to_sync(channel_layer.group_send)
for group_name in group_names:
async2sync(group_name, message)
Thanks a lot in advance!!!
---
My conf is the following:
Django==2.2.*
channels==2.2.*
channels-redis==2.3.*
daphne==2.3.*
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [os.environ.get('REDISCLOUD_URL',
'redis://localhost:6379')],
},
"symmetric_encryption_keys": [SECRET_KEY],
},
}
And the stacktrace:
MaxClientsError: ERR max number of clients reached
File "arcsecond/activities/tasks/archive_datarows.py", line 65, in
parse_archive_latest_rows
target_name=row.target_name)
File "django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "django/db/models/query.py", line 422, in create
obj.save(force_insert=True, using=self.db)
File "django/db/models/base.py", line 741, in save
force_update=force_update, update_fields=update_fields)
File "django/db/models/base.py", line 790, in save_base
update_fields=update_fields, raw=raw, using=using,
File "django/dispatch/dispatcher.py", line 175, in send
for receiver in self._live_receivers(sender)
File "django/dispatch/dispatcher.py", line 175, in <listcomp>
for receiver in self._live_receivers(sender)
File "djangochannelsrestframework/observer/observer.py", line 154, in
post_save_receiver
**kwargs
File "djangochannelsrestframework/observer/observer.py", line 221, in
post_change_receiver
**kwargs
File "djangochannelsrestframework/observer/observer.py", line 233, in
send_messages
async_to_sync(channel_layer.group_send)(group_name, message)
File "asgiref/sync.py", line 79, in __call__
return call_result.result()
File "concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "asgiref/sync.py", line 95, in main_wrap
result = await self.awaitable(*args, **kwargs)
File "channels_redis/core.py", line 559, in group_send
async with self.connection(self.consistent_hash(group)) as connection:
File "channels_redis/core.py", line 742, in __aenter__
self.conn = await self.pool.pop()
File "channels_redis/core.py", line 49, in pop
conns.append(await aioredis.create_redis(**self.host, loop=loop))
File "aioredis/commands/__init__.py", line 178, in create_redis
loop=loop)
File "aioredis/connection.py", line 129, in create_connection
await conn.auth(password)
File "aioredis/util.py", line 48, in wait_ok
res = await fut
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/dfe526ee-cd75-4481-9029-6f423996116d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
