+1, it will be great to see HF provider around!
Shahar On Tue, Nov 3, 2026, 03:13 Parag Ekbote <[email protected]> wrote: > Hi Airflow community, > > I'd like to propose introducing a new community provider for Hugging Face. > Integration overview > > Hugging Face <https://huggingface.co/> is an open-source AI platform and > community that provides tools and infrastructure for building, sharing and > deploying machine learning models, datasets and applications. Hugging Face > Datasets is widely used in data engineering and machine learning workflows, > and a first-class Airflow provider would make dataset orchestration, > transformation, and publishing simpler. > > The provider initially focuses on Hugging Face Datasets, exposing a > HuggingFaceHook and native operators for loading datasets with revision > pinning, applying user-defined transformations and publishing datasets to > the Hugging Face Hub. > > Future iterations may extend the provider to Hugging Face Models and > Inference Endpoints. > System tests > > The provider integrates with the live Hugging Face Hub. System tests are > included in the PR and exercise dataset loading, transformation and > publishing workflows using the airflow-hf > <https://huggingface.co/airflow-hf> Hugging Face organization. > Proposed stewards > > Parag Ekbote, @ParagEkbote <https://github.com/ParagEkbote>, < > [email protected]> > > Quentin Lhoest, @lhoestq <https://github.com/lhoestq>, < > [email protected]> > Sponsoring committer(s) > > Kaxil Naik, @kaxil <https://github.com/kaxil>, <[email protected]> > Working implementation > <https://github.com/apache/airflow/pull/76317> > > Incubation commitment > > We commit to: > > - > > Maintaining the provider and responding to issues within a reasonable > time. > - > > Meeting the incubation health metrics within six months. > - > > Participating in quarterly governance updates. > > Best regards, > > Parag Ekbote >
