subject:"Bucketing\/Rolling Sink\: How to overwrite method \"openNewPartFile\" \- to append a new timestamp to part file path every time a new part file is being created"

Re: Bucketing/Rolling Sink: How to overwrite method "openNewPartFile" - to append a new timestamp to part file path every time a new part file is being created

2017-10-09 Thread Kostas Kloudas

Hi Raja, To know about the method, I suppose you have looked at the source code of the sink. I think that including the timestamp of the element in the path file is not as easy as overriding the openNewPartFile. The reason is that the filenames serve as identities for the associated state of

Bucketing/Rolling Sink: How to overwrite method "openNewPartFile" - to append a new timestamp to part file path every time a new part file is being created

2017-10-06 Thread Raja . Aravapalli

Hi, I want to overwrite the method “openNewPartFile” in the BucketingSink Class such that it creates part file name with inclusion of timestamp whenever it rolls a new part file. Can someone share some thoughts on how I can do this. Thanks a ton, in advance. Regards, Raja.