Open for a _little_ bit longer than a week, whops! This vote is now closed.
The result is: +1: 9 votes (6 binding, 3 non binding - Abishek: I've corrected yours to be non-binding.) 0: 0 votes -1: 0 votes This AIP is now accepted. > On 24 Feb 2019, at 08:31, Kevin Yang <yrql...@gmail.com> wrote: > > Finally! > > +1 (non-binding) > > On Thu, Feb 21, 2019 at 5:40 PM abhishek sharma <abhioncbr.apa...@gmail.com> > wrote: > >> +1(binding) >> >> >> On Thu, Feb 21, 2019 at 2:09 PM Arthur Wiedmer <arthur.wied...@gmail.com> >> wrote: >> >>> +1 (binding) >>> >>> This is the direction we should be taking. >>> >>> On Thu, Feb 21, 2019 at 9:28 AM Ash Berlin-Taylor <a...@apache.org> >> wrote: >>> >>>> Dear Airflow community, >>>> >>>> This email calls for a vote to accept Airflow Improvement Proposal 13: >>>> OpenAPI 3 based API definition. >>>> >>>> The vote will last for at least 1 week, and until three +1 (binding) >>> votes >>>> have been cast >>>> >>>> This vote is on the proposal itself, not any specific code or pull >>>> request. A failed vote does not mean the proposal is rejected, just not >>>> accepted at this time. (To reject a proposal entirely is it's own vote) >>>> >>>> This is my +1 (binding) vote. >>>> >>>> My summary of this proposal: Make the API endpoints discoverable and >>>> introspectable by creating an OpenAPI 3 (a.k.a. Swagger) definition of >>> the >>>> current API. >>>> >>>> (OpenAPI is the format that Kubernetes uses to define it's API) >>>> >>>> The original proposal from the wiiki is included in full at the end of >>>> this email (duplicated from >>>> >>> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103088709 >>>> for permanent record) >>>> >>>> Thanks, >>>> Ash >>>> >>>> -- Full Proposal -- >>>> >>>> # Motivation >>>> >>>> Operations performed on airflow server need to be run by a CLI which >>>> either interacts directly with the database or in json mode which >>> performs >>>> a subset of operations via the existing experimental api. This >>> complicates >>>> authentication and interacting with the airflow installation by >> requiring >>>> the CLI to have a database user, rather than a web user. >>>> >>>> # Requirements >>>> >>>> • Reduce complexity in handling authentication. >>>> >>>> • Provide a layer of abstraction using an industry standard >>>> interface to allow authenticated, remote control for Airflow >>> installations >>>> >>>> • Any API should confirm to existing industry standards >>>> >>>> • API structure should be discoverable >>>> >>>> • API should be versioned to handle any future backwards >>>> incompatible schema changes. >>>> >>>> # Proposal >>>> >>>> Develop an a JSON restful API using the existing Plugin interface, >>> defined >>>> by OpenAPI 3. >>>> >>>> # Implementation >>>> >>>> ## API Definition >>>> >>>> The API will be defined by a YAML OpenAPI 3 definition, which will be >>>> exposed via connexion. For the time being, the API JSON data structures >>>> will follow the existing definitions as defined in >>>> https://airflow.apache.org/api.html. >>>> >>>> This means that client libraries can be autogenerated meaning >> development >>>> work is focused on the API structure and method handlers themselves, >> and >>>> not the infrastructure or client libraries. This also opens airflow up >> to >>>> being easily controlled by many languages and interfaces with minimal >>> work >>>> required on the server side. >>>> >>>> The API base url will have the format >>>> {protocol}://{airflowHost}/api/v{apiVersion}/, where: >>>> >>>> • protocol is one of http or https. >>>> >>>> • airflowHost is AIRFLOW__WEBSERVER__BASE_URL. >>>> >>>> • apiVersion is a integer. >>>> >>>> ## Endpoints >>>> >>>> For the initial work, the existing endpoints should not be modified, so >>>> that a minor release is not required to get this structure into the >> code >>>> base >>>> >>>> In the interest of maintaining backwards compatibility, the existing >> API >>>> would be maintained with the same structure at the same endpoints, but >>>> using the OpenAPI codebase until it would be deprecated in future >>> releases. >>>> In parallel to this, an API would be defined at /api/v1 with the >>> Additional >>>> API resources. >>>> >>>> ### Existing >>>> >>>> The base url will prefix all endpoints. There will be significantly >> more >>>> endpoints to cover airflow functionality, but this list covers the >>> existing >>>> experimental with some extra intermediary endpoints. >>>> >>>> • /dags/{dag_id}/dag_runs >>>> >>>> • /dags/{dag_id}/dag_runs/{execution_date} >>>> >>>> • /test >>>> >>>> • /dags/{dag_id}/tasks/{task_id} >>>> >>>> • /dags/{dag_id}/dag_runs/{execution_date}/tasks/{task_id} >>>> >>>> • /dags/{dag_id}/paused/{paused} >>>> >>>> • /latest_runs >>>> >>>> • /pools >>>> >>>> • /pools/{pool_name} >>>> >>>> ### Additional >>>> >>>> These are suggested endpoints to replace existing ones as more in line >>>> with the concept of resources, rather than actions >>>> >>>> • /dags >>>> >>>> • /dags/{dag_id} >>>> >>>> • /dags/{dag_id}/dag_runs/{execution_date}/tasks >>>> >>>> • /dag_runs >>>> >>>> • Alias to /latest_runs >>>> • /dags/{dag_id}/tasks >>>> >>>> • /healthcheck >>>> >>>> • alias to /test >>>> >>>> • /dags/{dag_id}/status >>>> >>>> • Intended to replace /dags/{dag_id}/paused/{paused} >> as a >>>> restful resource as opposed to the single field. >>>> >>>> ## Discovery >>>> >>>> To make the API discoverable, connexion provides a self-documenting API >>> UI >>>> with swagger. Further to this, the API structure itself, should be >>>> discoverable implementing a concept such as HATEOAS via HAL. >>>> >>>> Each resource or resource listing response should be defined in a >>>> structure defined in http://stateless.co/hal_specification.html. >>>> >>>> *NOTE:* The exact structure of _links, and _curies are still to be >>> defined. >>>> >>>> ## Proof of concept >>>> >>>> There is an example here of this implementation here >>>> https://github.com/apache/airflow/pull/4640/files >>>> >>>> Important points to note are: >>>> >>>> • Using plugin entrypoint to isolate functionality >>>> >>> >> https://github.com/apache/airflow/pull/4640/files#diff-2eeaed663bd0d25b7e608891384b7298R416 >>>> >>>> • OpenAPI 3 definition >>>> >>> >> https://github.com/apache/airflow/pull/4640/files#diff-93e827c54cbc441d84674c814dcae00e >>>> >>>> • Api Plugin and blueprint hooks >>>> >>> >> https://github.com/apache/airflow/pull/4640/files#diff-5ff8468ade348aeb2ccc273cf3b79550 >>>> >>>> The sample documentation can be seen here: >>>> >>>> • >>>> >>> >> https://editor.swagger.io/?url=https%3A%2F%2Fraw.githubusercontent.com%2Fdrewsonne%2Fairflow%2Ffeature%2Fflask-restful%2Fairflow%2Fplugin%2Fapi%2Fswagger.yaml >>>> >>>> >>>> # Considerations >>>> >>>> Using a structured definition like OpenAPI may restrict edge cases for >>>> complex, or non-json or non-rest based endpoints in the API. >>>> >>>> Complex authentication methods may also be difficult to implement. The >>>> existing PoC above handles the authentication using the existing api >>>> authentication methods. In future, API keys or OAuth keys may be a >> better >>>> solution for API access, instead of requiring a session or an OAuth >>> login. >>>> >>>> >>>> >>> >>
