[
https://issues.apache.org/jira/browse/FLINK-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135238#comment-16135238
]
ASF GitHub Bot commented on FLINK-7040:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/4569
[FLINK-7040] [REST] Add basics for REST communication
## What is the purpose of the change
This PR implements the fundamentals for generic Client/Server REST
communication that will be used for the client/cluster communication, the
WebRuntimeMonitor and queryable state.
#### Endpoints
`Endpoints` are the main runtime component that have to be started before
any communication can happen. Their primary purpose is setting up the
underlying netty stack.
The `RestClientEndpoint` is a fully-functional class that provides an
asynchronous API for sending requests/receiving responses based around
`CompleteFutures`. Requests are sent in a synchronous fashion; a new request is
only sent out after a response was received for the previous request (for
simplicity).
The `RestServerEndpoint` is an abstract class that is very similar to the
`WebRuntimeMonitor`. Implementations have to implement a single method
`abstract Collection<? extends AbstractRestHandler<?, ?>>
initializeHandlers();` that returns a collection of handlers that should be
registered.
#### Messages
To send a request the client accepts 3 arguments:
* a `MessageHeaders`
* a `RequestBody`
* a `ParameterMapper`
`RequestBodies` represent the http message body, and will be converted to
JSON using jackson-databind.
`ParameterMappers` are used to assemble the final url, including query and
path parameters
`MessageHeaders` are stateless objects that define a link between requests
and responses as well as provide meta-data about the communication for this
particular pair. Headers have generic type arguments for requests and responses
and provide some level of type-safeness; if the client and server use the same
headers class the request/response type, url, http status codes etc. are all
well-defined.
Essentially, headers provide a tight coupling between handlers, clients,
requests and responses (provided that implementations don't define arbitrary
headers on the fly).
For example, to define the communication for submitting a job one would
define a `JobSubmitRequestBody` that contains the serialized job graph, a
`JobSubmitResponseBody` containing the URL to track the job status and a
`JobSubmitHeaders` class that define the HTTP method (POST), the url (e.g
`/submit`, the response http status code (ACCEPTED).
#### Handlers
Someone has to deal with requests and send out responses, which is where
the `AbstractRestHandler` comes into play. This is an abstract class that
manages all the netty stuff, and, just like the `MessageHeaders`, has generic
type arguments for a `RequestBody` and `ResponseBody`. Only a single method
must be implemented, which accepts a request and returns a response: `abstract
CompletableFuture<HandlerResponse<P>> handleRequest(@Nonnull HandlerRequest<R>
request);`
A `HandlerRequest` contains a `RequestBody` and maps for path/query
parameters.
A `HandlerResponse` contains either a `ResponseBody`, or an `ErrorResponse`.
## Brief change log
- Add `MessageHeaders`, `Request-/ResponseBody` for modeling messages
- Add `AbstractRestHandler`, `HandlerRequest/-Response` for message
processing
- Add `RestClient-/-ServerEndpoint` that setup netty
## Verifying this change
This change added tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink rest_client_final
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4569.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4569
----
commit 6085639ad5438228ff56d66a4988cf52cbe850b2
Author: zentol <[email protected]>
Date: 2017-08-16T13:17:45Z
[FLINK-7040] [rest] Add basics for REST communication
commit af95d0729595d1c707ef3041b8fbdbc21c0d0d4a
Author: zentol <[email protected]>
Date: 2017-08-21T09:53:13Z
Add better error message for get requests with a body
commit 5a7a8c2877f2b25cc6281462007a5ff03de40781
Author: zentol <[email protected]>
Date: 2017-08-21T12:40:09Z
Consistent error message for 404
commit bc3b4afd075182cc17b1cb9343215a04b17e5253
Author: zentol <[email protected]>
Date: 2017-08-21T13:00:28Z
Rework resolve URL generation
commit efbcb82076d01f176d8fff903cfcba461591fedc
Author: zentol <[email protected]>
Date: 2017-08-21T13:03:27Z
Rework handler registration
----
> Flip-6 client-cluster communication
> -----------------------------------
>
> Key: FLINK-7040
> URL: https://issues.apache.org/jira/browse/FLINK-7040
> Project: Flink
> Issue Type: New Feature
> Components: Cluster Management, Mesos
> Reporter: Till Rohrmann
> Assignee: Chesnay Schepler
> Priority: Critical
> Labels: flip-6
>
> With the new Flip-6 architecture, the client will communicate with the
> cluster in a RESTful manner.
> The cluster shall support the following REST calls:
> * List jobs (GET): Get list of all running jobs on the cluster
> * Submit job (POST): Submit a job to the cluster (only supported in session
> mode)
> * Lookup job leader (GET): Gets the JM leader for the given job
> * Get job status (GET): Get the status of an executed job (and maybe the
> JobExecutionResult)
> * Cancel job (PUT): Cancel the given job
> * Stop job (PUT): Stops the given job
> * Take savepoint (POST): Take savepoint for given job (How to return the
> savepoint under which the savepoint was stored? Maybe always having to
> specify a path)
> * Get KV state (GET): Gets the KV state for the given job and key (Queryable
> state)
> * Poll/subscribe to notifications for job (GET, WebSocket): Polls new
> notifications from the execution of the given job/Opens WebSocket to receive
> notifications
> The first four REST calls will be served by the REST endpoint running in the
> application master/cluster entrypoint. The other calls will be served by a
> REST endpoint running along side to the JobManager.
> Detailed information about different implementations and their pros and cons
> can be found in this document:
> https://docs.google.com/document/d/1eIX6FS9stwraRdSUgRSuLXC1sL7NAmxtuqIXe_jSi-k/edit?usp=sharing
> The implementation will most likely be Netty based.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
