Hi Jarek, I think it is a good idea to try and integrate mprocs with standalone airflow.
While we are at it, we should try to see if we can somehow wrap around the dependency mprocs needed in a way that users working with standalone airflow do not suffer at all. It can maybe be an inherent dependency or we launch it using subprocess or we even maintain a minimalistic wrapper around mprocs without the npm dependency. My thought immediately goes to golang's embedded binaries which we could probably leverage to design our own using wrapper or similar, maybe check out: https://medium.com/dtoebe/embed-other-binaries-in-golang-binary-3f613314884c All in all, I think it's a good idea if it doesn't make user lives harder than it already is! Thanks & Regards, Amogh Desai On Wed, Dec 24, 2025 at 3:11 AM Natanel <[email protected]> wrote: > Hello Jarek, I think that it is a great idea, it does look better and seems > to be more intuitive to use, when you have 1 screen at a time and you may > change between them quickly and with ease, from experience, it would > significantly improve the ease of use of a standalone airflow instance. > > I think it will work well, and make it simpler to run airflow, though, I > think that Tmux is flexible enough to be able to reach a similar looking UI > with simple and intuitive controls, though it might require a lot of trial > and error, with configuration and community input in order to reach a > simple and intuitive experience for a standalone airflow instance. > > We might want to explore more solutions such as a more comprehensive Tmux > configuration or maybe even Zellij, which is pretty similar to Tmux, and > are both lightweight, and simple to install (though, as Jen said, it will > make running airflow standalone a little harder). > > > I'd say it sounds like a good idea to have it as part of standalone > airflow. One drawback it has that it has a dependency on npm, but also it > has a standalone binary, that Airflow Standalone **could** download from > https://github.com/pvolok/mprocs/releases and run internally. Or maybe > search for or even implement a simple version of it as an alternative in > Python. > > I agree that using the npm version is not ideal, as it will add a > dependency to airflow. > > A minimal version in python could also suffice, though it would take more > time and add more code to maintain, though it would be the lightest weight > option, and using the binary install is probably better, in order to not > rely on npm, especially due to the last seen events on the npm repository > where a lot of malware was pushed to relatively big packages. > > There are a few considerations here, of whether we should bundle a > multiplexer or something similar to ariflow, just like is done with airflow > breeze, another option is always to open multiple new terminal windows or > tabs in the terminal, however, it might be too complex to do so with > different working environments and people using all kinds of different > terminals (and fonts) (i.e having to support windows, macos, xterm, kitty, > alacritty, Konsole, st and more), mprocs seems like a very good candidate, > but having to bundle it with airflow, could make the package heavier (or > the installation more complex), and having an alternative python package > will keep the airflow installation as simple and lightweight for standalone > as possible, the next step should the community decide to implement it in > python, will be research on the api's of different terminals to check if it > is possible to have a simple implementation to cover most of the terminal > emulators people are going to use. > > > Over all, adding such a tool will improve the airflow standalone experience > and benefit a good chunk of airflow users (and contributors) (who may want > to check their dags quickly and locally), and so maybe it is worth it to > make the installation a little harder or require one additional > *optional* dependency > to improve the standalone experience. > > Such an addition would be awesome. > > On Tue, 23 Dec 2025 at 21:20, Dheeraj Turaga <[email protected]> > wrote: > > > Hi Jarek! > > > > I like this idea. We use airflow standalone a lot and more often its hard > > to identify issues when the log is multiplexed onto the screen. Also new > > users who are unaware that airflow drops a .json with a temp account in > > airflow home often find themselves scrambling to quickly copy the auto > > generated credentials before it vanishes into history. > > > > I can see instances where you may want to restart individual services > (dag > > processor, etc) without having to kill the whole standalone run and > launch > > a new one. > > > > Having separate logs for each service and a way to manage each one > > separately would be amazing > > > > I also agree with Jens here as to making the first time standalone run > > working with the most minimum number of commands as possible (pip install > > apache-airflow && airflow standalone) > > > > Adding these via additional options would be nice! > > > > Thanks > > Dheeraj > > > > On Tue, Dec 23, 2025 at 2:42 PM Jarek Potiuk <[email protected]> wrote: > > > > > All good points: > > > > So in standaonle mode we would need to mound one "dummy process" > > > panel giving the key "getting started" information with UI URL and > > > admin user/password at least. > > > > > > Great idea > > > > > > > We should not make it harder for users for first-time user > > > experience. Today after a `pip install apache-airflow` you are > > > potentially good to go > > > > > > Yes. Likely we can leave standalone behaviour working as of today by > > > default > > > > > > > We therefore might consider > > > not naming it `--mprocs` but rather `--interactive` to describe the > > > use case? > > > > > > I really like this flag idea. > > > > > > > On the neutral side though I am not sure if a first time user is > > > "easier" with merged logs to see errors of if a first time user might > be > > > overwhelmed with too many panels to scroll in to find an error not > > > knowing which component maybe raising a stack trace at all. > > > > > > Yep. With `--interactive` flag as non-default this issue would also be > > > handled. > > > > > >
