Hi Vinay, I was referring to setting up an deploying a YARN cluster directly through a Java program, instead of using the command-line interface. When you do that, you typically construct a YarnClusterDescriptor and parameterize it, then you call deploy() to deploy the cluster and create a YarnClusterClient for interaction.
-Max On Fri, Nov 18, 2016 at 12:20 PM, vinay patil <[email protected]> wrote: > Hi Max, > > When I provided the config file using --yarnship option , it worked. Thank > you. > > However I did not get this : " it's better to add it to the ship files in > the > cluster descriptor" Can you please tell me how to do this ? > > Regards, > Vinay Patil > > On Fri, Nov 18, 2016 at 4:12 AM, Maximilian Michels [via Apache Flink User > Mailing List archive.] <[hidden email]> wrote: >> >> The JVM only accepts Jar files in the classpath. You will have to load >> your custom files from the working directory of the node where the lib >> directory is shipped. >> >> By the way, the /lib directory is meant for Jar files. If you want to >> ship a custom file, it's better to add it to the ship files in the >> cluster descriptor or use the --yarnship option from the command-line. >> >> -Max >> >> >> On Thu, Nov 17, 2016 at 5:37 AM, vinay patil <[hidden email]> wrote: >> >> > Hi Robert, >> > >> > Sorry for the late response. >> > >> > I checked the TM and JM logs but am not able to see that file added in >> > the >> > classpath. >> > >> > I have deployed Flink on EMR, and placed the yaml file in >> > /usr/lib/flink/lib >> > folder. >> > >> > What am I doing wrong here ? >> > >> > Regards, >> > Vinay Patil >> > >> > On Wed, Oct 26, 2016 at 1:25 PM, rmetzger0 [via Apache Flink User >> > Mailing >> > List archive.] <[hidden email]> wrote: >> >> >> >> >> Hi Vinay, >> >> >> >> the JobManager and TaskManager logs contain the classpath used when >> >> starting a container on YARN. Can you check if the yaml file is in the >> >> classpath? >> >> >> >> On Tue, Oct 25, 2016 at 8:28 AM, vinay patil <[hidden email]> wrote: >> >>> >> >>> Hi Max, >> >>> >> >>> As discussed here , I have put my yaml file in the flink lib >> >>> directory, >> >>> but >> >>> still I am not able to get this file from classpath. >> >>> >> >>> I am using Flink 1.1.1 and cfg4j to load the file from classpath. >> >>> >> >>> Running the job on YARN in EMR using the below command: >> >>> >> >>> ./bin/flink run <jar_name> >> >>> >> >>> Can you please let me know what I am doing wrong here. >> >>> >> >>> Regards, >> >>> Vinay Patil >> >>> >> >>> >> >>> >> >>> -- >> >>> View this message in context: >> >>> >> >>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p9702.html >> >>> Sent from the Apache Flink User Mailing List archive. mailing list >> >>> archive at Nabble.com. >> >> >> >> >> >> >> >> >> >> ________________________________ >> >> If you reply to this email, your message will be added to the >> >> discussion >> >> below: >> >> >> >> >> >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p9719.html >> >> To start a new topic under Apache Flink User Mailing List archive., >> >> email >> >> [hidden email] >> >> To unsubscribe from Apache Flink User Mailing List archive., click >> >> here. >> >> NAML >> > >> > >> > >> > ________________________________ >> > View this message in context: Re: add FLINK_LIB_DIR to classpath on yarn >> > -> >> > add properties file to class path on yarn >> > >> > Sent from the Apache Flink User Mailing List archive. mailing list >> > archive >> > at Nabble.com. >> >> >> ________________________________ >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p10187.html >> To start a new topic under Apache Flink User Mailing List archive., email >> [hidden email] >> To unsubscribe from Apache Flink User Mailing List archive., click here. >> NAML > > > > ________________________________ > View this message in context: Re: add FLINK_LIB_DIR to classpath on yarn -> > add properties file to class path on yarn > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com.
