OK, got that, thank you for your solution, it worked for me with --yarnship option
Regards, Vinay Patil On Tue, Nov 22, 2016 at 4:44 PM, Maximilian Michels [via Apache Flink User Mailing List archive.] <[email protected]> wrote: > Hi Vinay, > > I was referring to setting up an deploying a YARN cluster directly > through a Java program, instead of using the command-line interface. > When you do that, you typically construct a YarnClusterDescriptor and > parameterize it, then you call deploy() to deploy the cluster and > create a YarnClusterClient for interaction. > > -Max > > > On Fri, Nov 18, 2016 at 12:20 PM, vinay patil <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=10277&i=0>> wrote: > > > Hi Max, > > > > When I provided the config file using --yarnship option , it worked. > Thank > > you. > > > > However I did not get this : " it's better to add it to the ship files > in > > the > > cluster descriptor" Can you please tell me how to do this ? > > > > Regards, > > Vinay Patil > > > > On Fri, Nov 18, 2016 at 4:12 AM, Maximilian Michels [via Apache Flink > User > > Mailing List archive.] <[hidden email]> wrote: > >> > >> The JVM only accepts Jar files in the classpath. You will have to load > >> your custom files from the working directory of the node where the lib > >> directory is shipped. > >> > >> By the way, the /lib directory is meant for Jar files. If you want to > >> ship a custom file, it's better to add it to the ship files in the > >> cluster descriptor or use the --yarnship option from the command-line. > >> > >> -Max > >> > >> > >> On Thu, Nov 17, 2016 at 5:37 AM, vinay patil <[hidden email]> wrote: > >> > >> > Hi Robert, > >> > > >> > Sorry for the late response. > >> > > >> > I checked the TM and JM logs but am not able to see that file added > in > >> > the > >> > classpath. > >> > > >> > I have deployed Flink on EMR, and placed the yaml file in > >> > /usr/lib/flink/lib > >> > folder. > >> > > >> > What am I doing wrong here ? > >> > > >> > Regards, > >> > Vinay Patil > >> > > >> > On Wed, Oct 26, 2016 at 1:25 PM, rmetzger0 [via Apache Flink User > >> > Mailing > >> > List archive.] <[hidden email]> wrote: > >> > >> >> > >> >> Hi Vinay, > >> >> > >> >> the JobManager and TaskManager logs contain the classpath used when > >> >> starting a container on YARN. Can you check if the yaml file is in > the > >> >> classpath? > >> >> > >> >> On Tue, Oct 25, 2016 at 8:28 AM, vinay patil <[hidden email]> wrote: > >> >>> > >> >>> Hi Max, > >> >>> > >> >>> As discussed here , I have put my yaml file in the flink lib > >> >>> directory, > >> >>> but > >> >>> still I am not able to get this file from classpath. > >> >>> > >> >>> I am using Flink 1.1.1 and cfg4j to load the file from classpath. > >> >>> > >> >>> Running the job on YARN in EMR using the below command: > >> >>> > >> >>> ./bin/flink run <jar_name> > >> >>> > >> >>> Can you please let me know what I am doing wrong here. > >> >>> > >> >>> Regards, > >> >>> Vinay Patil > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> View this message in context: > >> >>> > >> >>> http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p9702.html > >> >>> Sent from the Apache Flink User Mailing List archive. mailing list > >> >>> archive at Nabble.com. > >> >> > >> >> > >> >> > >> >> > >> >> ________________________________ > >> >> If you reply to this email, your message will be added to the > >> >> discussion > >> >> below: > >> >> > >> >> > >> >> http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p9719.html > >> >> To start a new topic under Apache Flink User Mailing List archive., > >> >> email > >> >> [hidden email] > >> >> To unsubscribe from Apache Flink User Mailing List archive., click > >> >> here. > >> >> NAML > >> > > >> > > >> > > >> > ________________________________ > >> > View this message in context: Re: add FLINK_LIB_DIR to classpath on > yarn > >> > -> > >> > add properties file to class path on yarn > >> > > >> > Sent from the Apache Flink User Mailing List archive. mailing list > >> > archive > >> > at Nabble.com. > >> > >> > >> ________________________________ > >> If you reply to this email, your message will be added to the > discussion > >> below: > >> > >> http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p10187.html > >> To start a new topic under Apache Flink User Mailing List archive., > email > >> [hidden email] > >> To unsubscribe from Apache Flink User Mailing List archive., click > here. > >> NAML > > > > > > > > ________________________________ > > View this message in context: Re: add FLINK_LIB_DIR to classpath on yarn > -> > > add properties file to class path on yarn > > Sent from the Apache Flink User Mailing List archive. mailing list > archive > > at Nabble.com. > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p10277.html > To start a new topic under Apache Flink User Mailing List archive., email > [email protected] > To unsubscribe from Apache Flink User Mailing List archive., click here > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx> > . > NAML > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p10279.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
