Re: Providing external files to flink classpath
Hi Vishwas,
took me some time to find out as well. If you have your properties file
under lib following will work:
val kafkaPropertiesInputStream =
getClass.getClassLoader.getResourceAsStream("lib/config/kafka.properties")
Hope this helps,
Maxim.
On Wed, Jul 17, 2019 at 7:23 PM Vishwas Siravara
wrote:
> Does the -yt option work for standalone cluster without dedicated resource
> manager ? So this property file is read by one of the dependencies inside
> my application as a file, so I can't really use Parameter tool to parse the
> config file.
>
> Thanks,
> Vishwas
>
> On Fri, Jun 28, 2019 at 11:08 PM Yun Tang wrote:
>
>> Hi Vishwas
>>
>>
>>1. You could use '-yt' to ship specified files to the class path,
>>please refer to [1] for more details.
>>2. If the properties are only loaded on client side before executing
>>the application, you could let your application to just read from local
>>property data. Flink support to load properties within the
>>ParameterTool [2].
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/cli.html#usage
>> [2]
>> https://github.com/apache/flink/blob/f1721293b0701d584d42bd68671181e332d2ad04/flink-java/src/main/java/org/apache/flink/api/java/utils/ParameterTool.java#L120
>>
>> Best
>> Yun Tang
>>
>> --
>> *From:* Vishwas Siravara
>> *Sent:* Saturday, June 29, 2019 0:43
>> *To:* user
>> *Subject:* Providing external files to flink classpath
>>
>> Hi ,
>> I am trying to add external property files to the flink classpath for
>> my application. These files are not a part of the fat jar. I put them
>> under the lib folder but flink cant find them? How can I manage
>> external property files that needs to be read by flink ?
>>
>> Thanks,
>> Vishwas
>>
>
Re: Providing external files to flink classpath
Does the -yt option work for standalone cluster without dedicated resource manager ? So this property file is read by one of the dependencies inside my application as a file, so I can't really use Parameter tool to parse the config file. Thanks, Vishwas On Fri, Jun 28, 2019 at 11:08 PM Yun Tang wrote: > Hi Vishwas > > >1. You could use '-yt' to ship specified files to the class path, >please refer to [1] for more details. >2. If the properties are only loaded on client side before executing >the application, you could let your application to just read from local >property data. Flink support to load properties within the >ParameterTool [2]. > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/cli.html#usage > [2] > https://github.com/apache/flink/blob/f1721293b0701d584d42bd68671181e332d2ad04/flink-java/src/main/java/org/apache/flink/api/java/utils/ParameterTool.java#L120 > > Best > Yun Tang > > -- > *From:* Vishwas Siravara > *Sent:* Saturday, June 29, 2019 0:43 > *To:* user > *Subject:* Providing external files to flink classpath > > Hi , > I am trying to add external property files to the flink classpath for > my application. These files are not a part of the fat jar. I put them > under the lib folder but flink cant find them? How can I manage > external property files that needs to be read by flink ? > > Thanks, > Vishwas >
Re: Providing external files to flink classpath
Hi Vishwas 1. You could use '-yt' to ship specified files to the class path, please refer to [1] for more details. 2. If the properties are only loaded on client side before executing the application, you could let your application to just read from local property data. Flink support to load properties within the ParameterTool [2]. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/cli.html#usage [2] https://github.com/apache/flink/blob/f1721293b0701d584d42bd68671181e332d2ad04/flink-java/src/main/java/org/apache/flink/api/java/utils/ParameterTool.java#L120 Best Yun Tang From: Vishwas Siravara Sent: Saturday, June 29, 2019 0:43 To: user Subject: Providing external files to flink classpath Hi , I am trying to add external property files to the flink classpath for my application. These files are not a part of the fat jar. I put them under the lib folder but flink cant find them? How can I manage external property files that needs to be read by flink ? Thanks, Vishwas
