Hi Mihir,
Thanks for raising this. Before we look at a separate "lite" Airflow, it'd
be good to check whether what we ship today is small enough for you. It's a
plain pip install with no Docker needed, and SQLite + LocalExecutor are
already the defaults.
The smallest setup that still schedules Dags is three processes:
pip install "apache-airflow==3.3.2" --constraint
https://raw.githubusercontent.com/apache/airflow/constraints-3.3.2/constraints-3.12.txt
export AIRFLOW__CORE__PARALLELISM=2
airflow db migrate
airflow api-server &
airflow scheduler --skip-serve-logs &
airflow dag-processor &
Swap 3.12 in the constraints URL for your Python version. The triggerer is
only needed for deferrable operators, so you can leave it out; "airflow
standalone" always starts it along with its own supervisor process, which
in my test came to about 350 MB more than running the three processes
yourself. LocalExecutor starts "parallelism" workers up front (default 32),
so set it to what the box actually runs at once. Skipping serve-logs is
fine on one machine, since finished-task logs are read from local disk.
I tried this in an arm64 Linux container (not a real Pi) with a two-task
Dag running every minute. It stayed around 630 MB and ran fine under a 1 GB
memory limit. For comparison, 2.10 with the scheduler and a one-worker
webserver sat around 410 MB and didn't fit in 512 MB either, so 3.x is
roughly 200 MB heavier with the UI running. A Raspberry PI with 2 GB should
be comfortable.
Kaxil
On Wed, 4 Nov 2026 at 23:34, Vikram Koka via dev <[email protected]>
wrote:
> Mihir,
>
> Thank you for taking the time to bring this up!
>
> A lot of the time and energy we spend on the Airflow dev list focuses on
> large enterprise use cases for running Airflow at scale.
> Your motivation as expressed to me earlier today was to use Airflow for
> small use cases at NGOs that need orchestration for their data solutions.
> Their data needs are small yet real, and computing affordability is a key
> requirement.
>
> As an open source project, we absolutely want to meet the needs of this
> audience as well, where feasible.
> The smaller, lightweight execution environment is something that Ephraim
> and Ash are already looking at, but with a different lens, so I will make
> sure to flag this to them as well.
>
> Best regards,
> Vikram
>
>
> On Wed, Nov 4, 2026 at 3:24 PM Mihir Samant <[email protected]> wrote:
>
> > Hi Team,
> >
> > During the recent Apache Airflow Reconnect, I had a brief chat with
> Vikram
> > about an idea to make Airflow more accessible for smaller organizations.
> >
> > *The Core Idea:*
> >
> > Develop a lightweight, streamlined version of Apache Airflow tailored for
> > teams and non-profits with modest data orchestration needs. Many small
> > organizations don't require heavy AI/ML workflows or the infrastructure
> > complexity of Airflow 3.x.
> >
> > *Key Focus Areas:*
> >
> > -
> >
> > *Simplified Scope:* Focus purely on core, standard data processing and
> > pipeline orchestration.
> > -
> >
> > *Easier Deployment:* Offer a fast, non-Docker installation pathway for
> > environments with limited infrastructure or system resources.
> >
> > I believe a lean Airflow profile could expand our adoption across small
> > businesses and community organizations that currently find the full
> > footprint overwhelming.
> >
> > Huge thanks to Vikram for taking the time to listen and discuss this! I’d
> > love to hear the team's thoughts on whether this aligns with any upcoming
> > community or roadmap discussions.
> >
> > Best,
> >
> >
> > *Mihir Samant**519-982-8632 <(519)%20982-8632> <(519)%20982-8632>
> <(519)%20982-8632>*
> >
>
