I’m not sure if we need to vote on this (what is the consequence of -1?), but I don’t mind
+1, binding On 21 February 2019 at 19:58:50, Driesprong, Fokko (fo...@driesprong.frl) wrote: +1 (binding) Would be a great improvement to consolidate the API. Cheers, Fokko Op do 21 feb. 2019 om 19:36 schreef Bas Harenslak < basharens...@godatadriven.com> > A solid API is much needed. Following a standard and using the OpenAPI > spec makes everybody's life easier. > > +1 > > Bas > > > On 21 Feb 2019, at 18:28, Ash Berlin-Taylor <a...@apache.org<mailto: > a...@apache.org>> wrote: > > Dear Airflow community, > > This email calls for a vote to accept Airflow Improvement Proposal 13: > OpenAPI 3 based API definition. > > The vote will last for at least 1 week, and until three +1 (binding) votes > have been cast > > This vote is on the proposal itself, not any specific code or pull > request. A failed vote does not mean the proposal is rejected, just not > accepted at this time. (To reject a proposal entirely is it's own vote) > > This is my +1 (binding) vote. > > My summary of this proposal: Make the API endpoints discoverable and > introspectable by creating an OpenAPI 3 (a.k.a. Swagger) definition of the > current API. > > (OpenAPI is the format that Kubernetes uses to define it's API) > > The original proposal from the wiiki is included in full at the end of > this email (duplicated from > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103088709 > for permanent record) > > Thanks, > Ash > > -- Full Proposal -- > > # Motivation > > Operations performed on airflow server need to be run by a CLI which > either interacts directly with the database or in json mode which performs > a subset of operations via the existing experimental api. This complicates > authentication and interacting with the airflow installation by requiring > the CLI to have a database user, rather than a web user. > > # Requirements > > • Reduce complexity in handling authentication. > > • Provide a layer of abstraction using an industry standard interface to > allow authenticated, remote control for Airflow installations > > • Any API should confirm to existing industry standards > > • API structure should be discoverable > > • API should be versioned to handle any future backwards incompatible > schema changes. > > # Proposal > > Develop an a JSON restful API using the existing Plugin interface, defined > by OpenAPI 3. > > # Implementation > > ## API Definition > > The API will be defined by a YAML OpenAPI 3 definition, which will be > exposed via connexion. For the time being, the API JSON data structures > will follow the existing definitions as defined in > https://airflow.apache.org/api.html. > > This means that client libraries can be autogenerated meaning development > work is focused on the API structure and method handlers themselves, and > not the infrastructure or client libraries. This also opens airflow up to > being easily controlled by many languages and interfaces with minimal work > required on the server side. > > The API base url will have the format > {protocol}://{airflowHost}/api/v{apiVersion}/, where: > > • protocol is one of http or https. > > • airflowHost is AIRFLOW__WEBSERVER__BASE_URL. > > • apiVersion is a integer. > > ## Endpoints > > For the initial work, the existing endpoints should not be modified, so > that a minor release is not required to get this structure into the code > base > > In the interest of maintaining backwards compatibility, the existing API > would be maintained with the same structure at the same endpoints, but > using the OpenAPI codebase until it would be deprecated in future releases. > In parallel to this, an API would be defined at /api/v1 with the Additional > API resources. > > ### Existing > > The base url will prefix all endpoints. There will be significantly more > endpoints to cover airflow functionality, but this list covers the existing > experimental with some extra intermediary endpoints. > > • /dags/{dag_id}/dag_runs > > • /dags/{dag_id}/dag_runs/{execution_date} > > • /test > > • /dags/{dag_id}/tasks/{task_id} > > • /dags/{dag_id}/dag_runs/{execution_date}/tasks/{task_id} > > • /dags/{dag_id}/paused/{paused} > > • /latest_runs > > • /pools > > • /pools/{pool_name} > > ### Additional > > These are suggested endpoints to replace existing ones as more in line > with the concept of resources, rather than actions > > • /dags > > • /dags/{dag_id} > > • /dags/{dag_id}/dag_runs/{execution_date}/tasks > > • /dag_runs > > • Alias to /latest_runs > • /dags/{dag_id}/tasks > > • /healthcheck > > • alias to /test > > • /dags/{dag_id}/status > > • Intended to replace /dags/{dag_id}/paused/{paused} as a restful resource > as opposed to the single field. > > ## Discovery > > To make the API discoverable, connexion provides a self-documenting API UI > with swagger. Further to this, the API structure itself, should be > discoverable implementing a concept such as HATEOAS via HAL. > > Each resource or resource listing response should be defined in a > structure defined in http://stateless.co/hal_specification.html. > > *NOTE:* The exact structure of _links, and _curies are still to be defined. > > ## Proof of concept > > There is an example here of this implementation here > https://github.com/apache/airflow/pull/4640/files > > Important points to note are: > > • Using plugin entrypoint to isolate functionality > https://github.com/apache/airflow/pull/4640/files#diff-2eeaed663bd0d25b7e608891384b7298R416 > > • OpenAPI 3 definition > https://github.com/apache/airflow/pull/4640/files#diff-93e827c54cbc441d84674c814dcae00e > > • Api Plugin and blueprint hooks > https://github.com/apache/airflow/pull/4640/files#diff-5ff8468ade348aeb2ccc273cf3b79550 > > The sample documentation can be seen here: > > • > https://editor.swagger.io/?url=https%3A%2F%2Fraw.githubusercontent.com%2Fdrewsonne%2Fairflow%2Ffeature%2Fflask-restful%2Fairflow%2Fplugin%2Fapi%2Fswagger.yaml > > > # Considerations > > Using a structured definition like OpenAPI may restrict edge cases for > complex, or non-json or non-rest based endpoints in the API. > > Complex authentication methods may also be difficult to implement. The > existing PoC above handles the authentication using the existing api > authentication methods. In future, API keys or OAuth keys may be a better > solution for API access, instead of requiring a session or an OAuth login. > > > >
