Hi, The link to FAQ (https://ci.apache.org/projects/flink/flink-docs-release-0.10/faq.html <https://ci.apache.org/projects/flink/flink-docs-release-0.10/faq.html>) is on the yarn setup 0.10 documentation page (https://ci.apache.org/projects/flink/flink-docs-release-0.10/setup/yarn_setup.html <https://ci.apache.org/projects/flink/flink-docs-release-0.10/setup/yarn_setup.html>) described in this sentence: If you have troubles using the Flink YARN client, have a look in the FAQ section <https://ci.apache.org/projects/flink/flink-docs-release-0.10/faq.html>.
Is the scheduling features considered for next releases? Thank you. Best regards, Ovidiu > On 20 Nov 2015, at 11:59, Robert Metzger <rmetz...@apache.org> wrote: > > Hi Ovidiu, > > you can submit multiple programs to a running Flink cluster (or a YARN > session). Flink does currently not have any queuing mechanism. > The JobManager will reject a program if there are not enough free resources > for it. If there are enough resources for multiple programs, they'll run > concurrently. > Note that Flink is not starting separate JVMs for the programs, so if one > program is doing a System.exit(0), it is killing the entire JVM, including > other running programs. > > You can start as many YARN sessions (or single jobs to YARN) as you have > resources available on the cluster. The resource allocation is up to the > scheduler you've configured in YARN. > > In general, we recommend to start a YARN session per program. You can also > directly submit a Flink program to YARN. > > Where did you find the link to the FAQ? The link on the front page is > working: http://flink.apache.org/faq.html <http://flink.apache.org/faq.html> > > > > On Fri, Nov 20, 2015 at 11:41 AM, Ovidiu-Cristian MARCU > <ovidiu-cristian.ma...@inria.fr <mailto:ovidiu-cristian.ma...@inria.fr>> > wrote: > Hi, > > I am currently interested in experimenting on Flink over Hadoop YARN. > I am documenting from the documentation we have here: > https://ci.apache.org/projects/flink/flink-docs-release-0.10/setup/yarn_setup.html > > <https://ci.apache.org/projects/flink/flink-docs-release-0.10/setup/yarn_setup.html> > > There is a subsection Start Flink Session which states the following: A > session will start all required Flink services (JobManager and TaskManagers) > so that you can submit programs to the cluster. Note that you can run > multiple programs per session. > > Can you be more precise regarding the multiple programs per session? If I > submit multiple programs concurently what will happen (can I?)? Maybe they > will run in a FIFO fashion or what should I expect? > > The internals section specify that users can execute multiple Flink Yarn > sessions in parallel. This is great, this invites to static partitioning of > resources in order to run multiple applications concurrently. Do you support > a fair scheduler similar to what Spark claims it has? > > There is FAQ section > (https://ci.apache.org/projects/flink/flink-docs-release-0.10/faq.html > <https://ci.apache.org/projects/flink/flink-docs-release-0.10/faq.html>) > resource that is missing, can this be updated? > > Thank you. > > Best regards, > Ovidiu > >
