On 2026-11-03 01:12 UTC Parag Ekbote wrote:
> Hi Airflow community,
>
> I'd like to propose introducing a new community provider for Hugging Face.
> Integration overview
>
> Hugging Face <https://huggingface.co/> is an open-source AI platform and
> community that provides tools and infrastructure for building, sharing and
> deploying machine learning models, datasets and applications. Hugging Face
> Datasets is widely used in data engineering and machine learning workflows,
> and a first-class Airflow provider would make dataset orchestration,
> transformation, and publishing simpler.
>
> The provider initially focuses on Hugging Face Datasets, exposing a
> HuggingFaceHook and native operators for loading datasets with revision
> pinning, applying user-defined transformations and publishing datasets to
> the Hugging Face Hub.
>
> Future iterations may extend the provider to Hugging Face Models and
> Inference Endpoints.
> System tests
>
> The provider integrates with the live Hugging Face Hub. System tests are
> included in the PR and exercise dataset loading, transformation and
> publishing workflows using the airflow-hf
> <https://huggingface.co/airflow-hf> Hugging Face organization.
> Proposed stewards
>
> Parag Ekbote, @ParagEkbote <https://github.com/ParagEkbote>, <
> [email protected]>
>
> Quentin Lhoest, @lhoestq <https://github.com/lhoestq>, <
> [email protected]>
> Sponsoring committer(s)
>
> Kaxil Naik, @kaxil <https://github.com/kaxil>, <[email protected]>
> Working implementation
> <https://github.com/apache/airflow/pull/76317>
>
> Incubation commitment
>
> We commit to:
>
> -
>
> Maintaining the provider and responding to issues within a reasonable
> time.
> -
>
> Meeting the incubation health metrics within six months.
> -
>
> Participating in quarterly governance updates.
>
> Best regards,
>
> Parag Ekbote
> +1 Thanks for driving this!
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
