Re: Compress DataSink Output

2016-08-19 Thread Wesley Kerr
hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.html > > Let me know if you need more information. > > Regards, > Robert > > [1]: > https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/hadoop_compatibility.html &g

Compress DataSink Output

2016-08-17 Thread Wesley Kerr
Hello - Forgive me if this has been asked before, but I'm trying to determine the best way to add compression to DataSink Outputs (starting with TextOutputFormat). Realistically I would like each partition file (based on parallelism) to be compressed independently with gzip, but am open to other