We prefer ActiveMQ over RabbitMQ because we are used it deploy it in production at scale, in HA way, with the upgrade stuff and so on. Rebuilding all this knowledge with RabbitMQ is an hassle for us...
Le jeu. 26 mars 2020 à 19:30, Jarek Potiuk <jarek.pot...@polidea.com> a écrit : > > Out of curiosity why ActiveMQ rather than RabbitMQ (except that it is an > Apache project that is)? > > J, > > On Thu, Mar 26, 2020 at 6:40 PM Alexandre Vermeerbergen < > avermeerber...@gmail.com> wrote: > > > Hi, > > > > To be honest I found RQ while searching why Airflow isn't currently > > supporting ActiveMQ instead of RabbitMQ. > > > > I found that RabbitMQ come from a dependency of Celery on Kombu; and I > > found an incident in Kombu discussing about ActiveMQ support: > > https://github.com/celery/kombu/issues/548 > > > > I asked if ActiveMQ support was still planned in Kombu, and to my > > surprise I got the hint that some projects have moved from Celery to > > RQ. > > > > Now I realize that RQ means using Redis as the queuing infrastructure, > > which IHMO is not a so good idea when one realize that it takes 6 > > nodes to make Redis HA. > > > > To sum up, I'm fine if Airflow sticks to Celery, and then I'll see if > > there's a way to revive interest in adding ActiveMQ support to Kumbu. > > After all, Airflow and ActiveMQ are Apache projects, so it seems to be > > fair to look for ActiveMQ support for Airflow to get an Apache-based > > stack, isn't it ? > > > > Kind regards, > > Alexandre > > > > Le jeu. 26 mars 2020 à 18:29, Daniel Imberman > > <daniel.imber...@gmail.com> a écrit : > > > > > > I mean… we’re not planning it (kind of an “if it’s not broke don’t fix > > it” situation), but I don’t think we’re super set on Celery. Would you be > > interested in making an AIP to discuss potential benefits? > > > On Mar 26, 2020, 10:14 AM -0700, Alexandre Vermeerbergen < > > avermeerber...@gmail.com>, wrote: > > > > Hi there, > > > > > > > > Looks like couple of projects have moved from Celery to RQ: > > > > * https://pulpproject.org/2018/05/08/pulp3-moving-to-rq/#reasoning > > > > * https://frappe.io/blog/technology/why-we-moved-from-celery-to-rq > > > > > > > > or are planning such move: > > > > * https://github.com/getredash/redash/issues/4092 > > > > > > > > Is such move considered for Airflow? > > > > > > > > Kind regards, > > > > Alexandre > > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/>
