Hello Vinoo/Team, As per pyarrow Team , They don't see any concern , please check below. Please let us know *where it says Parquet V2 is not official *
"> *As per Apache Parquet Community Parquet V2 is not final yet so it is not > official . They are advising not to use Parquet V2 for writing (though code > is available ) .* This would be news to me. Parquet releases are listed (by the parquet community) at [1] The vote to release parquet 2.10 is here: [2] *Neither of these links mention anything about this being an experimental,unofficial, or non-finalized release.* I understand your concern. I believe your quotes are coming from your discussion on the parquet mailing list here [3]. This communication is unfortunate and confusing to me as well. [1] https://parquet.apache.org/blog/ [2] https://lists.apache.org/thread/fdf1zz0f3xzz5zpvo6c811xjswhm1zy6 [3] https://lists.apache.org/thread/4nzroc68czwxnp0ndqz15kp1vhcd7vg3"; On Mon, Apr 22, 2024 at 4:56 PM Prem Sahoo <prem.re...@gmail.com> wrote: > Hello Vinoo/Team,. > I was going through pyarrow and they have started using V2 as default . > isn't it they should avoid it as it is not official. > > > https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_table.html#pyarrow.parquet.write_table > > version{“1.0”, “2.4”, “2.6”}, default “2.6” > > Determine which Parquet logical types are available for use, whether the > reduced set from the Parquet 1.x.x format or the expanded logical types > added in later format versions. Files written with version=’2.4’ or ‘2.6’ > may not be readable in all Parquet implementations, so version=’1.0’ is > likely the choice that maximizes file compatibility. UINT32 and some > logical types are only available with version ‘2.4’. Nanosecond timestamps > are only available with version ‘2.6’. Other features such as compression > algorithms or the new serialized data page format must be enabled > separately (see ‘compression’ and ‘data_page_version’). >
