Hi Antony, Do you mean we should use dockerized Airflow instead of installing RPM/DEB packages?
Basically I'm +1 on adding Airflow for the non-containerized users. Regards, 2025年2月21日(金) 17:46 Антон Александров <[email protected]>: > Hi, i think it’s bad idea. > Airflow move very fast. > In my experience better way to start with airflow it’s docker compose, and > after POC move to k8s. > > best regards > Antony Aleksandrov > > > 21 февр. 2025 г., в 10:53, Jialiang Cai <[email protected]> > написал(а): > > > > +1 add Airflow > > > >> On Feb 21, 2025, at 15:51, Kengo Seki <[email protected]> wrote: > >> > >> Hi everyone, > >> > >> As I've submitted BIGTOP-4360 [1], I'd like to propose incorporating > >> Apache Airflow [2] into our stack, > >> because we don't have any job scheduler/workflow orchestrator since > >> we've dropped Oozie from 3.3.0. > >> > >> Rationales in my mind are as follows: > >> > >> * As Astronomer and major cloud vendors provide the managed service of > >> Airflow [3][4][5][6], > >> it is the de-facto standard of open-source workflow orchestrator. > >> > >> * Airflow community is very active [7], so we don't need to be worried > >> about its stagnation > >> for several years at least. > >> > >> * I'm also an Airflow committer, so I can be responsible for > >> maintaining it as a Bigtop component. > >> > >> Any thoughts or comments? > >> > >> [1]: https://issues.apache.org/jira/browse/BIGTOP-4360 > >> [2]: https://airflow.apache.org/ > >> [3]: https://www.astronomer.io/ > >> [4]: https://aws.amazon.com/managed-workflows-for-apache-airflow/ > >> [5]: > https://learn.microsoft.com/en-us/fabric/data-factory/create-apache-airflow-jobs > >> [6]: https://cloud.google.com/composer > >> [7]: https://github.com/apache/airflow/graphs/contributors > >> > >> Kengo Seki <[email protected]> > > > >
