+1 too. Agreed with Jarek, this integration will help on-prem users who are moving off HDFS/Oozie with native ofs:// support in Airflow.
Thanks & Regards, Amogh Desai On Sat, Nov 14, 2026 at 6:20 AM Jarek Potiuk <[email protected]> wrote: > Yep., While the HDFS is a little bit "dated" - I think having direct Ozone > support - especially backed by the follow ASF community members, is a good > idea. > > On Thu, Nov 12, 2026 at 11:10 PM v.chaplygin.arenadata.io via dev < > [email protected]> wrote: > > > Hi Airflow community, > > > > I'd like to propose adding a new community provider for Apache Ozone. > > > > It would enter at the incubation stage. In provider.yaml the state > > > > stays not-ready until the first release. It targets Airflow 2.11+ > > > > and Python 3.11+. > > > > Integration overview > > > > -------------------- > > > > Apache Ozone (https://ozone.apache.org/) is an ASF top-level project: a > > > > distributed object store from the Hadoop ecosystem, often deployed next > > > > to HDFS. I work at Arenadata, and we use this integration in our > > > > Airflow fork. > > > > The Amazon provider can talk to Ozone only through its S3 gateway. It > > > > does not handle ofs:// and o3fs:// paths, Ozone volumes, quotas, or > > > > snapshots. This provider does. Filesystem and admin commands go through > > > > the ozone CLI, and Kerberos uses kinit from the connection. It adds an > > > > ozone connection type, hooks, operators, the OzoneKeySensor, > > > > HdfsToOzoneOperator (hadoop distcp) and OzoneBackupOperator for bucket > > > > snapshots. The only Python dependency is apache-airflow. Workers need > > > > the Ozone client, and the DistCp transfer also needs a Hadoop client. > > > > System tests > > > > ------------ > > > > Unit tests in the PR mock the ozone and hadoop CLIs and run without a > > > > cluster. Apache Airflow CI has no Ozone service. We already run > > > > integration tests against the apache/ozone Docker image in our fork. > > > > They need a new Breeze integration, so they are not part of this PR. > > > > We plan to propose that Breeze integration in a follow-up PR. > > > > We are not publishing a public system-test dashboard with this proposal. > > > > Proposed stewards > > > > ----------------- > > > > - Vyacheslav Chaplygin, @slavachy, [email protected] > > > > - Mikhail Zaplatin, @ZMichael3012, [email protected] > > > > - Aleksey Gladkih, @gaxeliy, [email protected] > > > > - Dmitry Bezlutskiy, @Dimitrionian, [email protected] > > > > Sponsoring committer(s) > > > > ----------------------- > > > > - Amogh Desai, @amoghrajesh > > > > Working implementation > > > > ---------------------- > > > > https://github.com/apache/airflow/pull/75978 > > > > Incubation commitment > > > > --------------------- > > > > We commit to: > > > > - Maintaining the provider and responding to issues within a reasonable > > time > > > > - Meeting the incubation health metrics within 6 months > > > > - Participating in quarterly governance updates > > > > Best regards, > > > > Vyacheslav Chaplygin > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
