Sorry you have a point there. It was released in version 3.00. What version of spark are you using?
Technologist | Solutions Architect | Data Engineer | Generative AI London United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* The information provided is correct to the best of my knowledge but of course cannot be guaranteed . It is essential to note that, as with any advice, quote "one test result is worth one-thousand expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". On Mon, 15 Apr 2024 at 21:33, Prem Sahoo <prem.re...@gmail.com> wrote: > Thank you so much for the info! But do we have any release notes where it > says spark2.4.0 onwards supports parquet version 2. I was under the > impression Spark3.0 onwards it started supporting . > > > > > On Mon, Apr 15, 2024 at 4:28 PM Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > >> Well if I am correct, Parquet version 2 support was introduced in Spark >> version 2.4.0. Therefore, any version of Spark starting from 2.4.0 supports >> Parquet version 2. Assuming that you are using Spark version 2.4.0 or >> later, you should be able to take advantage of Parquet version 2 features. >> >> HTH >> >> Mich Talebzadeh, >> Technologist | Solutions Architect | Data Engineer | Generative AI >> London >> United Kingdom >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> https://en.everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* The information provided is correct to the best of my >> knowledge but of course cannot be guaranteed . It is essential to note >> that, as with any advice, quote "one test result is worth one-thousand >> expert opinions (Werner >> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >> >> >> On Mon, 15 Apr 2024 at 20:53, Prem Sahoo <prem.re...@gmail.com> wrote: >> >>> Thank you for the information! >>> I can use any version of parquet-mr to produce parquet file. >>> >>> regarding 2nd question . >>> Which version of spark is supporting parquet version 2? >>> May I get the release notes where parquet versions are mentioned ? >>> >>> >>> On Mon, Apr 15, 2024 at 2:34 PM Mich Talebzadeh < >>> mich.talebza...@gmail.com> wrote: >>> >>>> Parquet-mr is a Java library that provides functionality for working >>>> with Parquet files with hadoop. It is therefore more geared towards >>>> working with Parquet files within the Hadoop ecosystem, particularly using >>>> MapReduce jobs. There is no definitive way to check exact compatible >>>> versions within the library itself. However, you can have a look at this >>>> >>>> https://github.com/apache/parquet-mr/blob/master/CHANGES.md >>>> >>>> HTH >>>> >>>> Mich Talebzadeh, >>>> Technologist | Solutions Architect | Data Engineer | Generative AI >>>> London >>>> United Kingdom >>>> >>>> >>>> view my Linkedin profile >>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>> >>>> >>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>> >>>> >>>> >>>> *Disclaimer:* The information provided is correct to the best of my >>>> knowledge but of course cannot be guaranteed . It is essential to note >>>> that, as with any advice, quote "one test result is worth one-thousand >>>> expert opinions (Werner >>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>>> >>>> >>>> On Mon, 15 Apr 2024 at 18:59, Prem Sahoo <prem.re...@gmail.com> wrote: >>>> >>>>> Hello Team, >>>>> May I know how to check which version of parquet is supported by >>>>> parquet-mr 1.2.1 ? >>>>> >>>>> Which version of parquet-mr is supporting parquet version 2 (V2) ? >>>>> >>>>> Which version of spark is supporting parquet version 2? >>>>> May I get the release notes where parquet versions are mentioned ? >>>>> >>>>
