Hi Stephen, Thanks for raising the issue! You are right that the version is always 1 written by parquet-mr. This is something we need to fix. However, IMHO, the community does not have a clear answer on the definition of parquet format v2. Which feature are you referring to specifically in the version 2.6? It seems that you don't have to bother with the version and just set the config to enable it.
Best, Gang On Fri, Mar 15, 2024 at 6:02 PM Stephen Colebourne <[email protected]> wrote: > Hi all, > I'm trying to use the parquet-mr library to set format_version=2.6 (or > higher). > > When I review a file that is produced by the library, it appears that > the version is set to 1.0. Looking at the code in > org.apache.parquet.hadoop.ParquetFileWriter CURRENT_VERSION is hard > coded to 1.0. > > Is it a bug to hard code the version there? Am I missing something > obvious to select the format_version? > > thanks > Stephen >
