Re: Dynamic file name prefix - StreamingFileSink

2020-10-14 Thread Piotr Nowojski
Great! Please let us know if it solves the issue or not. Best, Piotrek śr., 14 paź 2020 o 17:46 Vijayendra Yadav napisał(a): > Hi Piotrek, > > That is correct I was still in 1.10, I am upgrading to 1.11. > > Regards, > Vijay > > On Wed, Oct 14, 2020 at 6:12 AM Piotr Nowojski > wrote: > >> Hi

Re: Dynamic file name prefix - StreamingFileSink

2020-10-14 Thread Vijayendra Yadav
Hi Piotrek, That is correct I was still in 1.10, I am upgrading to 1.11. Regards, Vijay On Wed, Oct 14, 2020 at 6:12 AM Piotr Nowojski wrote: > Hi Yadav, > > What Flink version are you using? `getPartPrefix` and `getPartSufix` > methods were not public before 1.10.1/1.11.0, which might be

Re: Dynamic file name prefix - StreamingFileSink

2020-10-14 Thread Piotr Nowojski
Hi Yadav, What Flink version are you using? `getPartPrefix` and `getPartSufix` methods were not public before 1.10.1/1.11.0, which might be causing this problem for you. Other than that, if you are already using Flink 1.10.1 (or newer), maybe please double check what class are you extending? The

Re: Dynamic file name prefix - StreamingFileSink

2020-10-13 Thread Vijayendra Yadav
*Thanks Ravi. I got following Error:* [ERROR] DynamicOutputFileConfig.scala:21: error: method getPartPrefix overrides nothing [ERROR] override def getPartPrefix: String = if(partPrefixFunction == null) partPrefix else partPrefixFunction.apply() [ERROR]^ [ERROR]

Re: Dynamic file name prefix - StreamingFileSink

2020-10-13 Thread Ravi Bhushan Ratnakar
Hi Vijayendra, OutputFileConfig provides a builder method to create immutable objects with given 'prefix' and 'suffix'. The parameter which you are passing to '*withPartPrefix*' will only be evaluated at the time of calling this method '*withPartPrefix*'. So if you want to achieve a dynamic

Dynamic file name prefix - StreamingFileSink

2020-10-12 Thread Vijayendra Yadav
Hi Team, I have tried to assign a dynamic prefix for file name, which contains datetime components. *The Problem is Job always takes initial datetime when job first starts and never refreshes later. * *How can I get dynamic current datetime in filename at sink time ?* *.withPartPrefix