I'm writing a Flink processor that will read a bunch of JSON records from Kafka and then write them to S3 in parquet format using the FileSink. I've got most things in place, the only thing I haven't been able to figure out is how to change the compression codec used by the writer. Is there any recommended way to do this? Currently I'm using the AvroParquetWriters. forReflectRecord(PlayerEvent.class) to transform my POJOs to Avro and then write them as Parquet files. I've looked into the AvroParquetWriters class but couldn't figure out how to configure the compression codec (or even what codec was used). Is there a way to configure this or do I have to write my own implementation of the Parquet writer and if so, how would one do that?
Thankful for any help, Tim
