-1 from me without a firm plan how we will replace it. I see keeping it and extending to use the new API would ensure that everything the CLI can do locally (i.e. when airflow webserver isn't up yet, with the ) also works over the API with the exception of db utilities.
-ash On 11 August 2020 20:05:56 BST, QP Hou <[email protected]> wrote: >+1 for replacing the existing remote mode client with the open api >based >client. IMO, we don't really have other options here because the >experimental API will be deprecated in the future. > >For OpenAPI based Airflow REST clients, the current plan is to maintain >all >the code gen automation within the main source tree [1], then use it to >populate each individual language specific client repo like the go >client >mentioned earlier. So far, we have the go client completed and >validated to >make sure this development flow will meet our needs. The next step I >think >the community should focus on is getting API auth implemented [2] >before we >move on to generate the python client. How we do API auth could have a >big >impact on client code gen automation, so it is worth waiting for. > >Once we have authentication implemented in both Airflow core and >clients, >we should be all good to start doing version releases for our API >clients. > >That said, adopting open api based clients in the CLI alone won't >address >the issue of CLI depending on full airflow installation. Some of the >cli >commands like `dags test` depend on a full airflow installation by >design. >We will have to either develop a separate CLI intended for remote only >use >or add a flag in the existing cli so it can run in a pure remote mode >where >it would disable loading of code that requires airflow installation >entirely. > >[1]: https://github.com/apache/airflow/tree/master/clients, >[2]: https://github.com/apache/airflow/issues/8112 > >On Tue, Aug 11, 2020 at 11:05 AM Kamil Breguła ><[email protected]> >wrote: > >> Hello, >> >> I think we should remove remote mode in CLI and in-core API Client >> (airflow.api.client package). >> Here is docs about remote mode: >> >> >https://airflow.readthedocs.io/en/latest/usage-cli.html#set-up-connection-to-a-remote-airflow-instance >> >> Since these features were introduced, it has never been actively >developed >> and I don't think it's widely used. At the same time, Apache Airflow >is >> evolving, and this code stands out more and more from the rest. >> >> My main reservations about these features: >> - Remote mode/in-core API Client is rarely used. I asked a few people >and >> none of them used it in production. Does anyone use it? >> - A very small number of commands are available (7 pools command and >2 dags >> command only) >> - Remote mode/API Client depends on experimental REST API. >> - Remote mode/API Client is a handwritten code that is difficult to >> maintain. >> - No documentation for API client >> - Remote mode/API Client has low test coverage. >> - Remote mode does not provide a good level of security, because it >depends >> on experimental API. There is the only authentication, but the >> authenticated user can perform any operation. >> - Requires full Airflow to be installed along with a large number of >> unnecessary dependencies. Some of them are difficult to install in >some >> environments, e.g. setproctitle on Windows >> - Using this client API changes the logger configuration because it >> requires importing the airflow package. >> >> I think this remote mode in CLI is something valuable, but I think we >can >> do it in a different way in the future, e.g. generate a CLI/API >Client >> based on the OpenAPI specification. >> >> Generated API clients can be installed independently of airflow and >will be >> easier to maintain. We already have one API client for golang >implemented >> in this way, so new languages will only be developing this idea. >> - https://github.com/apache/airflow-client-go >> >> I will be happy to discuss the vision of the development of these two >> things. How do we want to develop these two things? >> >> Best regards, >> Kamil Bregula >>
