Hi Theofilos, Assuming you have the FlinkYarnCluster after the call to deploy(). You can get the JobManager address using the
InetSocketAddress address = cluster.getJobManagerAddress(); Then create a Configuration with this address: Configuration config = new Configuration(); config.setString(ConfigConstants.JOB_MANAGER_IPC_ADDRESS_KEY, address.getHostName()); config.setInteger(ConfigConstants.JOB_MANAGER_IPC_PORT_KEY, address.getPort()); Then the client: Client client = new Client(config); Then use it to submit jobs blocking/detached, e.g. client.runBlocking(...); client.runDetached(...); Cheers, Max On Tue, Apr 19, 2016 at 11:25 PM, Theofilos Kakantousis <t...@kth.se> wrote: > Hi Max, > > Thank you for your reply. Exactly, I want to setup the Yarn cluster and > submit a job through code and not using cmd client. > I had done what you suggested, I used part of the deploy method to write my > own code that starts up the cluster which seems to be working fine. > > Could you point me to some examples how to use the Client you mention? > > Cheers, > Theofilos > > > On 2016-04-19 16:35, Maximilian Michels wrote: >> >> Hi Theofilos, >> >> I'm not sure whether I understand correctly what you are trying to do. >> I'm assuming you don't want to use the command-line client. >> >> You can setup the Yarn cluster in your code manually using the >> FlinkYarnClient class. The deploy() method will give you a >> FlinkYarnCluster which you can use to connect to the deployed cluster. >> Then get the JobManager address and use the Client class to submit >> Flink jobs to the cluster. I have to warn you that these classes are >> subject to change in Flink 1.1.0 and above. >> >> Let me know if the procedure works for you. >> >> Cheers, >> Max >> >> On Tue, Apr 19, 2016 at 2:37 PM, Theofilos Kakantousis <t...@kth.se> >> wrote: >>> >>> Hi everyone, >>> >>> I'm using Flink 0.10.1 and hadoop 2.4.0 to implement a client that >>> submits a >>> flink application to Yarn. To keep it simple I use the >>> ConnectedComponents >>> app from flink examples. >>> >>> I set the required properties (Resources, AM ContainerLaunchContext etc.) >>> on >>> the YARN client interface. What happens is the JobManager and TaskManager >>> processes start and based on the logs containers are running but the >>> actual >>> application does not start. I'm probably missing the proper way to pass >>> parameters to the ApplicationMaster and it cannot pick up the application >>> it >>> needs to run. Anyone knows where I could get some info on how to pass >>> runtime params to the AppMaster? >>> >>> The ApplicationMaster launchcontainer script includes the following: >>> exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx1024M >>> org.apache.flink.yarn.ApplicationMaster -c >>> org.apache.flink.examples.java.graph.ConnectedComponents 1> >>> /tmp/stdOut5237161854714899800 2> /tmp/stdErr606502839107545371 " >>> >>> Thank you, >>> Theofilos >>> >
