+1 add Airflow
> On Feb 21, 2025, at 15:51, Kengo Seki <[email protected]> wrote:
>
> Hi everyone,
>
> As I've submitted BIGTOP-4360 [1], I'd like to propose incorporating
> Apache Airflow [2] into our stack,
> because we don't have any job scheduler/workflow orchestrator since
> we've dropped Oozie from 3.3.0.
>
> Rationales in my mind are as follows:
>
> * As Astronomer and major cloud vendors provide the managed service of
> Airflow [3][4][5][6],
> it is the de-facto standard of open-source workflow orchestrator.
>
> * Airflow community is very active [7], so we don't need to be worried
> about its stagnation
> for several years at least.
>
> * I'm also an Airflow committer, so I can be responsible for
> maintaining it as a Bigtop component.
>
> Any thoughts or comments?
>
> [1]: https://issues.apache.org/jira/browse/BIGTOP-4360
> [2]: https://airflow.apache.org/
> [3]: https://www.astronomer.io/
> [4]: https://aws.amazon.com/managed-workflows-for-apache-airflow/
> [5]:
> https://learn.microsoft.com/en-us/fabric/data-factory/create-apache-airflow-jobs
> [6]: https://cloud.google.com/composer
> [7]: https://github.com/apache/airflow/graphs/contributors
>
> Kengo Seki <[email protected]>
