Hi Just shooting away my thoughts. Based on your what you had described so far, I think your objective is to have some unique way to identify/filter the output based on the organization. If that's the case, you can implement a BucketAssigner with the logic to create a bucket key based on the organization data.
Cheers, Sivaprasanna On Thu, May 14, 2020 at 12:13 PM Jingsong Li <jingsongl...@gmail.com> wrote: > Hi, Dhurandar, > > Can you describe your needs? Why do you need to modify file names > flexibly? What kind of name do you want? > > Best, > Jingsong Lee > > On Thu, May 14, 2020 at 2:05 AM dhurandar S <dhurandarg...@gmail.com> > wrote: > >> Yes we looked at it , >> The problem is the file name gets generated in a dynamic fashion, based >> on which organization data we are getting we generate the file name from >> the coming data. >> >> Is there any way we can achieve this ?? >> >> On Tue, May 12, 2020 at 8:38 PM Yun Gao <yungao...@aliyun.com> wrote: >> >>> Hi Dhurandar: >>> >>> Currently StreamingFileSink should be able to change the prefix and >>> suffix of the filename[1], it could be changed to something like >>> <prefix>-0-0<suffix>. Could this solve your problem ? >>> >>> >>> Best, >>> Yun >>> >>> >>> >>> >>> [1] >>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/streamfile_sink.html#part-file-configuration >>> >>> >>> >>> ------------------------------------------------------------------ >>> 发件人：dhurandar S<dhurandarg...@gmail.com> >>> 日 期：2020年05月13日 05:13:04 >>> 收件人：user<user@flink.apache.org>; <d...@flink.apache.org> >>> 主 题：changing the output files names in Streamfilesink from part-00 to >>> something else >>> >>> We want to change the name of the file being generated as the output of >>> our StreamFileSink. >>> , when files are generated they are named part-00*, is there a way that >>> we can change the name. >>> >>> In Hadoop, we can change RecordWriters and MultipleOutputs. May I please >>> some help in this regard. This is causing blockers for us and will force us >>> t move to MR job >>> >>> -- >>> Thank you and regards, >>> Dhurandar >>> >>> >>> >> >> -- >> Thank you and regards, >> Dhurandar >> >> > > -- > Best, Jingsong Lee >
