potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1883270253
> I was "naively" testing and miserably failed. Used the current branch, created a new venv, then called pip install -e ., followed by pip install -e .[ssh,docker]. But then calling airflow standalone failed miserably with: > ModuleNotFoundError: No module named 'flask_appbuilder' OK @jscheffl . I **think** I found a good solution that seemed impossible at first - I think we can have cake and eat it too. * the `flask_application_builder` package and other dependencies are not really NEEDED by anything else than webserver.. They are needed for two things only: - to retrieve connection UI widgets from Hooks when providers are discovered - to instantiate `auth_manager` class for webserver Both cases - we only need them for webserver. For scheduler, worker, triggerer and most unit tests, they are not really needed. * in the future we can optimise bootstrapping of Airflow CLI to completely skip those steps - but in case of generally if information provided by the two - provider connection widgets, and auth_manger, is not available, nothing bad happens. Things will work as usual. * so, we can selectively handle import errors on provider's widet retrieval. This way connection's widget information will be missing, but it is only used by Webserver, so it's ok to just raise warnings (there will be a lot of them - for each connection) and continue as usual. The warnings will say that widgets will not be available for hook class X. * similarly when auth_manager is not initialized, we can just raise a warning that "webserver will not work" and continue (unless we start webserver of course in which case we should print detailed exception and exit with error). Eventually, when we do it this way, we will eat cake and have it too. * we do not need to install `fab` packages by default in the `pip install -e .` case, so that "naive" installable installation will be somewhat usable (i.e. airflow CLI will work, ,most of the unit tests will work etc. etc). It will also install considerably faster than `pip install -e ".[devel]"` * however in all the documentation we will present `[devel]` as the **right** way of installing truly complete (but minimal) development environment. It has things like `pytest` - with all extensions we use, 'request_mock', `coverage`, `async` libraries, `black`, `ruff`, `pre-commit`, `mypy` - generally everything you'd need from a complete development environment. That one will also include all pre-installed provider's dependencies (this way we will have generally the same enviornment as after regular airflow installation pulls pre-installed providers - inlcuing fab, sqlite and other pre-installed ones. * similarly all the `hatch` managed environments in our pyproject.toml will have `devel` as default extra I think this is the best approach that nicely combines the consistency of our extras/devel dependencies with the ability of quickly installing `bare` editable airflow with `pure` airflow core requirements only (and serve naive users who will just run `pip install -e .`. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org