[ https://issues.apache.org/jira/browse/FLINK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308146#comment-16308146 ]
ASF GitHub Bot commented on FLINK-8119: --------------------------------------- GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5234 [FLINK-8119] [flip6] Wire correct Flip6 components in Flip6YarnClusterDescriptor ## What is the purpose of the change Let the Flip6YarnClusterDescriptor create a RestClusterClient as ClusterClient. Moreover, this commit makes the YarnResourceManager register under the REST port at Yarn. This PR is based on #5233. ## Brief change log - Implement unsupported methods in `RestClusterClient` - Register `YarnResourceManager` under REST port at Yarn - Let `Flip6YarnClusterDescriptor` return a `RestClusterClient` ## Verifying this change - Tested manually ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) You can merge this pull request into a Git repository by running: $ git pull https://github.com/tillrohrmann/flink implementRestClusterClientMethods Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5234.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5234 ---- commit c48324702b0ebff3fd7a16e480ead90fc4c9a30b Author: Till Rohrmann <trohrmann@...> Date: 2017-12-07T12:57:24Z [FLINK-8328] [flip6] Move Yarn ApplicationStatus polling out of YarnClusterClient Introduce YarnApplicationStatusMonitor which does the Yarn ApplicationStatus polling in the FlinkYarnSessionCli. This decouples the YarnClusterClient from the actual communication with Yarn and, thus, gives a better separation of concerns. commit 783bd1daf36b34a74cbe718a52a1043ba38d5a44 Author: Till Rohrmann <trohrmann@...> Date: 2017-12-20T15:43:21Z [FLINK-8329] [flip6] Move YarnClient to AbstractYarnClusterDescriptor Moves the YarnClient from the YarnClusterClient to the AbstractYarnClusterDescriptor. This makes the latter responsible for the lifecycle management of the client and gives a better separation of concerns. commit 192adb786a48d19b71e797355500652d51de6296 Author: Till Rohrmann <trohrmann@...> Date: 2017-12-18T17:59:30Z [FLINK-8332] [flip6] Move savepoint dispose into ClusterClient Move the savepoint disposal logic from the CliFrontend into the ClusterClient. This gives a better separation of concerns and allows the CliFrontend to be used with different ClusterClient implementations. commit a73b6e2850d4b2445a835914ba570d1057e59dfb Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T06:42:18Z [FLINK-8333] [flip6] Separate deployment options from command options This commit separates the parsing of command options and deployment options into two steps. This makes it easier to make the CustomCommandLines non-static. Moreover, this commit moves the CliFrontend into the cli sub package. commit 57df8f9e1c5f4015cd6543ded43849a71003ef36 Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T06:59:34Z [FLINK-8338] [flip6] Make CustomCommandLines non static in CliFrontend This commit changes how CustomCommandLines are registered at the CliFrontend. Henceforth, the CliFrontend is initialized with the set of CustomCommandLines instead of registering them statically. This improves maintainability and testability. commit 2ee8059b949b4a9c2e165f3e78716ae531e5c0e7 Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T08:22:12Z [FLINK-8339] [flip6] Let CustomCommandLine return ClusterDescriptor Instead of directly retrieving or deploying a Flink cluster, the CustomCommandLine now only returns a ClusterDescriptor which can be used for these operations. This disentangles the ClusterDescriptor and the CustomCommandLine a bit better supporting a proper lifecycle management of the former. commit 9ff62c7cfadc977d02f414ee26fae6b76a568eda Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T08:55:13Z [FLINK-8340] [flip6] Remove passing of Configuration to CustomCommandLine Since the Configuration does not change over the lifetime of a CustomCommandLine, we can safely pass it as a constructor argument instead of method argument. commit 2b6fda525fb737a76b055510f8081003010237ed Author: Till Rohrmann <trohrmann@...> Date: 2017-12-20T15:32:18Z [FLINK-8341] [flip6] Remove not needed options from CommandLineOptions commit 17755e5273364ccfc740d6b4a31b148d582d8cfa Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T09:19:34Z [FLINK-8342] [flip6] Remove generic type parameter from ClusterDescriptor commit 6e740f08c0998514c52f21a8bfeacfc3b748df27 Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T09:35:37Z [FLINK-8343] [flip6] Remove Yarn specific commands from YarnClusterDescriptor Remove Yarn specific commands from YarnClusterDescriptor. This is a preparational step to make the FlinkYarnSessionCli work with the Flip-6 RestClusterClient. commit 7435269fa3d50e8a358831cfa678f47f875afb73 Author: Till Rohrmann <trohrmann@...> Date: 2017-12-29T14:12:46Z [hotfix] Add help command to FlinkYarnSessionCli commit a1f6e76ddb4d9295b016b581f8b2dce0fa0c74e7 Author: Till Rohrmann <trohrmann@...> Date: 2018-01-02T13:33:43Z [FLINK-8347] [flip6] Make cluster id used by ClusterDescriptor typesafe The ClusterDescriptor uses a typed cluster id for the ClusterClient retrieval. Moreover, the ClusterClient and the CustomCommandLine are typed accordingly. commit c9487a12dc70e45817d37d9765ba76d01d3e06ed Author: Till Rohrmann <trohrmann@...> Date: 2017-12-29T14:29:06Z [FLINK-8348] [flip6] Print help for DefaultCLI commit f328a2044a6dccf7e9e0e80b8be9645cefa535f8 Author: Till Rohrmann <trohrmann@...> Date: 2017-12-27T11:08:34Z [FLINK-8119] [flip6] Wire correct Flip6 components in Flip6YarnClusterDescriptor Let the Flip6YarnClusterDescriptor create a RestClusterClient as ClusterClient. Moreover, this commit makes the YarnResourceManager register under the REST port at Yarn. commit 639d015fc1c1d046a1fa2e33aabcaef1cf52a191 Author: Till Rohrmann <trohrmann@...> Date: 2017-12-29T16:08:45Z [hotfix] Log failure message only if Yarn application truly failed ---- > Cannot submit jobs to YARN Session in FLIP-6 mode > ------------------------------------------------- > > Key: FLINK-8119 > URL: https://issues.apache.org/jira/browse/FLINK-8119 > Project: Flink > Issue Type: Sub-task > Components: Client > Affects Versions: 1.5.0 > Reporter: Gary Yao > Assignee: Till Rohrmann > Priority: Blocker > Labels: flip-6 > Fix For: 1.5.0 > > > Cannot submit jobs to YARN Session in FLIP-6 mode because > {{FlinkYarnSessionCli}} becomes the _active_ CLI (should be > {{Flip6DefaultCLI}}). > *Steps to reproduce* > # Build Flink 1.5 {{101fef7397128b0aba23221481ab86f62b18118f}} > # {{bin/yarn-session.sh -flip6 -d -n 1 -s 1 -jm 1024 -tm 1024}} > # {{bin/flink run -flip6 ./examples/streaming/WordCount.jar}} > # Verify that the job will not run. -- This message was sent by Atlassian JIRA (v6.4.14#64029)