Hi Team, Hope you're all doing well. This is a query regarding the Parquet Encoding used by spark.
We are interested in reducing the parquet file size to as small as possible. Looking at the nature of our data, DELTA_BINARY_PACKED seems to be a good option. However, with dictionary disabled, the DefaultV1ValuesWriter class defaults to the PlainValuesWriter. Is there a way to create a custom parquet writer which can be used by Spark? Appreciate your help on this. Thanks, Ridha
