So you want to be able to set a "global" suffix that should be appended to all different kinds of files that the sink writes, including intermediate files?
Aljoscha > On 29. Mar 2018, at 16:59, l...@lyft.com wrote: > > Sorry, I meant "I don't see a way of doing this apart from setting a part > file *suffix* with the required file extension. " > > > On 2018/03/29 14:55:43, l...@lyft.com <l...@lyft.com> wrote: >> Currently the BucketingSink allows addition of part prefix, pending >> prefix/suffix and in-progress prefix/suffix via setter methods. Can we also >> support setting part suffixes? >> An instance where this maybe useful: I am currently writing GZIP compressed >> output to S3 using the BucketingSink and I would want the uploaded files to >> have a ".gz" or ".zip" extensions (if the files does not have such an >> extensionelse they are written as garbled bytes and don't get rendered >> correctly for reading). I don't see a way of doing this apart from setting a >> part file prefix with the required file extension. >> >> Thanks >> Lakshmi >>
