Hi Maxime, Looking at the error logs, it sounds like there may be a clash between celery workers of different tools. Specifically, the log shows that there is a 'celery@cycling-init-bot.worker' ready, but it's a 'celery@algo-news' worker that is using a different time zone. As the Redis server is shared, clients should use a prefix to prevent this from happening.
I'm not too familiar with Celery, but it seems the underlying code allows you to set a KEY_PREFIX through the KOMBU_REDIS_PREFIX environment variable: https://docs.celeryproject.org/projects/kombu/en/stable/_modules/kombu/transport/redis.html Hope this helps! Merlijn On Sat, 27 Feb 2021 at 08:32, maxime delzenne <maxime.delze...@gmail.com> wrote: > Hi, > > Since yesterday evening (round about 22h CET), my celery send me following > error: > > [2021-02-27 07:26:44,782: INFO/MainProcess] Connected to > redis://tools-redis.svc.eqiad.wmflabs:6379/0 > [2021-02-27 07:26:44,867: INFO/MainProcess] mingle: searching for neighbors > [2021-02-27 07:26:45,924: INFO/MainProcess] mingle: sync with 5 nodes > [2021-02-27 07:26:45,926: INFO/MainProcess] mingle: sync complete > [2021-02-27 07:26:45,983: INFO/MainProcess] > celery@cycling-init-bot.worker-56ddc85977-9sf2v ready. > [2021-02-27 07:26:47,749: WARNING/MainProcess] Substantial drift from > celery@algo-news.celery-6444f768f9-w89fb may mean clocks are out of > sync. Current drift is > 3600 seconds. [orig: 2021-02-27 07:26:47.748906 recv: 2021-02-27 > 08:26:47.740345] > > I restarted, but no result. I use TIME_ZONE = 'UTC' in my Django settings. > > Psemdel > _______________________________________________ > Wikimedia Cloud Services mailing list > Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) > https://lists.wikimedia.org/mailman/listinfo/cloud >
_______________________________________________ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
