I think we once tried to abandon Celery for other stuff (kombu, knative)... If there's no reasonable advantage of RQ I see no reason in switching to it. However, rewriting the "Queue worker" can help a lot :)
Also a few days ago one of Celery maintainers expressed interest in contributing to Airflow. I think it's something that may help us. https://github.com/apache/airflow/pull/7754 Tomek On Thu, Mar 26, 2020 at 6:29 PM Daniel Imberman <[email protected]> wrote: > > I mean… we’re not planning it (kind of an “if it’s not broke don’t fix it” > situation), but I don’t think we’re super set on Celery. Would you be > interested in making an AIP to discuss potential benefits? > On Mar 26, 2020, 10:14 AM -0700, Alexandre Vermeerbergen > <[email protected]>, wrote: > > Hi there, > > > > Looks like couple of projects have moved from Celery to RQ: > > * https://pulpproject.org/2018/05/08/pulp3-moving-to-rq/#reasoning > > * https://frappe.io/blog/technology/why-we-moved-from-celery-to-rq > > > > or are planning such move: > > * https://github.com/getredash/redash/issues/4092 > > > > Is such move considered for Airflow? > > > > Kind regards, > > Alexandre
