> mprocs also publishes statically linked binaries Nice!
I think I would best favour building it ourselves or publishing it and using it for the `--interactive` mode because these are the two best options long term. Thanks & Regards, Amogh Desai On Wed, Dec 24, 2025 at 1:36 PM Jarek Potiuk <[email protected]> wrote: > > inherent dependency or we launch it using subprocess or we even maintain > a > > minimalistic wrapper > > around mprocs without the npm dependency. > > My thought immediately goes to golang's embedded binaries > > Actually - (🤦) it's not an NPM app. I looked at the code. It's a > (surprise, surprise) RUST app that has an NPM wrapper for easier > installation only :D - that's what made me think it's npm.. That would > explain why it's so fast and nice :) > > So that is super easy. We don't need to "declare" mprocs as dependency - > when you look here: https://github.com/pvolok/mprocs/releases - mprocs > also > publishes statically linked binaries which does not **require** npm to be > installed: > > So all we need to do when `--interactive` flag is used is to download the > binary for the right architecture, decompress it to temporary location and > run it from there (and we can cache it for later execution)., We could even > copy and publish those binaries some place we control - somewhere in > airflow.apache.org is good for that, similarly as we do with our chart - > so > that we do not have problems with GitHub rate limiting (and our website is > already behind fastly cache, so it will be very fast to download it by > anyone). We could even - for maximum security, build those binaries > ourselves from sources published by mprocs - > https://github.com/pvolok/mprocs/tree/master/scripts has all the scripts > needed to build those. > > J. > > > > On Wed, Dec 24, 2025 at 6:54 AM Amogh Desai <[email protected]> wrote: > > > Hi Jarek, > > > > I think it is a good idea to try and integrate mprocs with standalone > > airflow. > > > > While we are at it, we should try to see if we can somehow wrap around > the > > dependency mprocs > > needed in a way that users working with standalone airflow do not suffer > at > > all. It can maybe be an > > inherent dependency or we launch it using subprocess or we even maintain > a > > minimalistic wrapper > > around mprocs without the npm dependency. > > > > My thought immediately goes to golang's embedded binaries which we could > > probably leverage to > > design our own using wrapper or similar, maybe check out: > > > > > https://medium.com/dtoebe/embed-other-binaries-in-golang-binary-3f613314884c > > > > All in all, I think it's a good idea if it doesn't make user lives harder > > than it already is! > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Wed, Dec 24, 2025 at 3:11 AM Natanel <[email protected]> wrote: > > > > > Hello Jarek, I think that it is a great idea, it does look better and > > seems > > > to be more intuitive to use, when you have 1 screen at a time and you > may > > > change between them quickly and with ease, from experience, it would > > > significantly improve the ease of use of a standalone airflow instance. > > > > > > I think it will work well, and make it simpler to run airflow, though, > I > > > think that Tmux is flexible enough to be able to reach a similar > looking > > UI > > > with simple and intuitive controls, though it might require a lot of > > trial > > > and error, with configuration and community input in order to reach a > > > simple and intuitive experience for a standalone airflow instance. > > > > > > We might want to explore more solutions such as a more comprehensive > Tmux > > > configuration or maybe even Zellij, which is pretty similar to Tmux, > and > > > are both lightweight, and simple to install (though, as Jen said, it > will > > > make running airflow standalone a little harder). > > > > > > > I'd say it sounds like a good idea to have it as part of standalone > > > airflow. One drawback it has that it has a dependency on npm, but also > it > > > has a standalone binary, that Airflow Standalone **could** download > from > > > https://github.com/pvolok/mprocs/releases and run internally. Or maybe > > > search for or even implement a simple version of it as an alternative > in > > > Python. > > > > > > I agree that using the npm version is not ideal, as it will add a > > > dependency to airflow. > > > > > > A minimal version in python could also suffice, though it would take > more > > > time and add more code to maintain, though it would be the lightest > > weight > > > option, and using the binary install is probably better, in order to > not > > > rely on npm, especially due to the last seen events on the npm > repository > > > where a lot of malware was pushed to relatively big packages. > > > > > > There are a few considerations here, of whether we should bundle a > > > multiplexer or something similar to ariflow, just like is done with > > airflow > > > breeze, another option is always to open multiple new terminal windows > or > > > tabs in the terminal, however, it might be too complex to do so with > > > different working environments and people using all kinds of different > > > terminals (and fonts) (i.e having to support windows, macos, xterm, > > kitty, > > > alacritty, Konsole, st and more), mprocs seems like a very good > > candidate, > > > but having to bundle it with airflow, could make the package heavier > (or > > > the installation more complex), and having an alternative python > package > > > will keep the airflow installation as simple and lightweight for > > standalone > > > as possible, the next step should the community decide to implement it > in > > > python, will be research on the api's of different terminals to check > if > > it > > > is possible to have a simple implementation to cover most of the > terminal > > > emulators people are going to use. > > > > > > > > > Over all, adding such a tool will improve the airflow standalone > > experience > > > and benefit a good chunk of airflow users (and contributors) (who may > > want > > > to check their dags quickly and locally), and so maybe it is worth it > to > > > make the installation a little harder or require one additional > > > *optional* dependency > > > to improve the standalone experience. > > > > > > Such an addition would be awesome. > > > > > > On Tue, 23 Dec 2025 at 21:20, Dheeraj Turaga <[email protected]> > > > wrote: > > > > > > > Hi Jarek! > > > > > > > > I like this idea. We use airflow standalone a lot and more often its > > hard > > > > to identify issues when the log is multiplexed onto the screen. Also > > new > > > > users who are unaware that airflow drops a .json with a temp account > in > > > > airflow home often find themselves scrambling to quickly copy the > auto > > > > generated credentials before it vanishes into history. > > > > > > > > I can see instances where you may want to restart individual services > > > (dag > > > > processor, etc) without having to kill the whole standalone run and > > > launch > > > > a new one. > > > > > > > > Having separate logs for each service and a way to manage each one > > > > separately would be amazing > > > > > > > > I also agree with Jens here as to making the first time standalone > run > > > > working with the most minimum number of commands as possible (pip > > install > > > > apache-airflow && airflow standalone) > > > > > > > > Adding these via additional options would be nice! > > > > > > > > Thanks > > > > Dheeraj > > > > > > > > On Tue, Dec 23, 2025 at 2:42 PM Jarek Potiuk <[email protected]> > wrote: > > > > > > > > > All good points: > > > > > > So in standaonle mode we would need to mound one "dummy process" > > > > > panel giving the key "getting started" information with UI URL > > and > > > > > admin user/password at least. > > > > > > > > > > Great idea > > > > > > > > > > > We should not make it harder for users for first-time user > > > > > experience. Today after a `pip install apache-airflow` you are > > > > > potentially good to go > > > > > > > > > > Yes. Likely we can leave standalone behaviour working as of today > by > > > > > default > > > > > > > > > > > We therefore might consider > > > > > not naming it `--mprocs` but rather `--interactive` to describe > > the > > > > > use case? > > > > > > > > > > I really like this flag idea. > > > > > > > > > > > On the neutral side though I am not sure if a first time user is > > > > > "easier" with merged logs to see errors of if a first time user > might > > > be > > > > > overwhelmed with too many panels to scroll in to find an error not > > > > > knowing which component maybe raising a stack trace at all. > > > > > > > > > > Yep. With `--interactive` flag as non-default this issue would also > > be > > > > > handled. > > > > > > > > > > > > > > >
