Re: Dynamic file name prefix - StreamingFileSink

2020-10-14 Thread Piotr Nowojski
Great! Please let us know if it solves the issue or not. Best, Piotrek śr., 14 paź 2020 o 17:46 Vijayendra Yadav napisał(a): > Hi Piotrek, > > That is correct I was still in 1.10, I am upgrading to 1.11. > > Regards, > Vijay > > On Wed, Oct 14, 2020 at 6:12 AM Piotr Nowojski > wrote: > >> Hi Y

Re: Dynamic file name prefix - StreamingFileSink

2020-10-14 Thread Vijayendra Yadav
Hi Piotrek, That is correct I was still in 1.10, I am upgrading to 1.11. Regards, Vijay On Wed, Oct 14, 2020 at 6:12 AM Piotr Nowojski wrote: > Hi Yadav, > > What Flink version are you using? `getPartPrefix` and `getPartSufix` > methods were not public before 1.10.1/1.11.0, which might be caus

Re: Dynamic file name prefix - StreamingFileSink

2020-10-14 Thread Piotr Nowojski
Hi Yadav, What Flink version are you using? `getPartPrefix` and `getPartSufix` methods were not public before 1.10.1/1.11.0, which might be causing this problem for you. Other than that, if you are already using Flink 1.10.1 (or newer), maybe please double check what class are you extending? The e

Re: Dynamic file name prefix - StreamingFileSink

2020-10-13 Thread Vijayendra Yadav
*Thanks Ravi. I got following Error:* [ERROR] DynamicOutputFileConfig.scala:21: error: method getPartPrefix overrides nothing [ERROR] override def getPartPrefix: String = if(partPrefixFunction == null) partPrefix else partPrefixFunction.apply() [ERROR]^ [ERROR] DynamicOutputFileCo

Re: Dynamic file name prefix - StreamingFileSink

2020-10-13 Thread Ravi Bhushan Ratnakar
Hi Vijayendra, OutputFileConfig provides a builder method to create immutable objects with given 'prefix' and 'suffix'. The parameter which you are passing to '*withPartPrefix*' will only be evaluated at the time of calling this method '*withPartPrefix*'. So if you want to achieve a dynamic 'prefi

Dynamic file name prefix - StreamingFileSink

2020-10-12 Thread Vijayendra Yadav
Hi Team, I have tried to assign a dynamic prefix for file name, which contains datetime components. *The Problem is Job always takes initial datetime when job first starts and never refreshes later. * *How can I get dynamic current datetime in filename at sink time ?* *.withPartPrefix (ZonedDateT