Hi, BucketingSink doesn’t support the feature that you are requesting, you can not specify a dynamically generated prefix/suffix.
Piotrek > On Aug 31, 2017, at 7:12 PM, Raja.Aravapalli <raja.aravapa...@target.com> > wrote: > > > Hi, > > I have a flink application that is streaming data into HDFS and I am using > Bucketing Sink for that. And, I want to know if is it possible to rename the > part files that is being created in the base hdfs directory. > > Right now I am using the below code for including the timestamp into > part-file name, but the problem I am facing is the timestamp is not changing > for the new part file that is being rolled! > > > BucketingSink<String> HdfsSink = new BucketingSink<String> (hdfsOutputPath); > > HdfsSink.setBucketer(new BasePathBucketer<String>()); > HdfsSink.setBatchSize(1024 * 1024 * hdfsOutputBatchSizeInMB); // this means > 'hdfsOutputBatchSizeInMB' MB > HdfsSink.setPartPrefix("PART-FILE-" + > Long.toString(System.currentTimeMillis())); > > > Can someone please suggest me, what code changes I can try so that I get a > new timestamp for every part file that is being rolled new? > > > Thanks a lot. > > Regards, > Raja.