Hello here,

After trying mprocs for "start-airflow" I think mprocs would be a great
candidate to use for standalone airflow. I think currently one of the
problematic things for standalone airflow is the log multiplexing at the
output and lack of control over the individual processes run as part of
"standalone" airflow execution.

I think `mprocs` solves many of those problems that we have with "airflow
standalone" nicely. It provides a very nice, terminal interface to monitor
and control multiple processes - very user friendly, supporting all modern
terminals, copy&pasting, mouse control for monitoring of running processes,
ability to restart individual processes and it has also a number of
interesting features in the roadmap.

But even today it offers a lot more than we have today in standalone
airflow:

* separate logs for each process with nice navigation
* easy copy&paste
* keyboard shortcuts and navigation to switch and move between panels
* ability to individually restart processes when needed
* ability to autorestart processes when they fail
* ability to run scripts defined in package.json (it's an npm tool)

I'd say it sounds like a good idea to have it as part of standalone
airflow. One drawback it has that it has a dependency on npm, but also it
has a standalone binary, that Airflow Standalone **could** download from
https://github.com/pvolok/mprocs/releases and run internally. Or maybe
search for or even implement simple version of it as alternative in Python.

We could have an `--mprocs/-no-mprocs` flag when starting the standalone
airflow (and decide on default setting, also we could give the user a
choice to run mprocs by default when they run standalone and download the
binary, I think in general it delivers a very good and very user-friendly
(and not even expert-friendly like tmux) interface and "airflow standalone"
would get an immediate usability boost with it. I think the TUI interfaces
of those sort have shown it's power over last years (k9s being the flagship
tool).

I would love to hear your thoughts about it.

J.

Reply via email to