I filed a ticket [1] to discuss integration testing / feature compatibility testing
[1]: https://github.com/apache/parquet-format/issues/441 On Wed, Jun 26, 2024 at 10:03 AM Andrew Lamb <andrewlam...@gmail.com> wrote: > I will file a ticket for discussion shortly > > On Wed, Jun 26, 2024, 06:55 Julien Le Dem <jul...@apache.org> wrote: > >> I agree with Antoine for the separate thread/ticket for integration >> testing >> >> For the « implementation status » PR, I agree we can merge and iterate. I >> added a couple of comments that can easily be addressed afterwards. >> Julien >> >> On Wed, Jun 26, 2024 at 10:43 Antoine Pitrou <anto...@python.org> wrote: >> >> > >> > IMHO, we should either start a dedicated discussion thread for >> > integration testing, or open a GH issue and discuss it there. >> > >> > Regards >> > >> > Antoine. >> > >> > >> > >> > On Wed, 26 Jun 2024 09:21:33 +0200 >> > Alkis Evlogimenos >> > <alkis.evlogime...@databricks.com.INVALID> >> > wrote: >> > > It would be nice if the integration suite specifies how a driver can >> be >> > > executed. Then each implementation can provide a driver and the suite >> > will >> > > use that for validation. >> > > >> > > By specifying both reads and writes for the driver we get a lot more >> > power. >> > > Given an input we can roundtrip all combination of readers/writers and >> > > verify they can roundtrip. >> > > >> > > On Tue, Jun 25, 2024 at 6:42 PM Andrew Lamb < >> > andrewlamb11-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: >> > > >> > > > FWIW I started hacking up a prototype[1] of what a parquet-testing >> > > > integration suite might look like if anyone is interested >> > > > >> > > > >> > > > >> > > > [1]: https://github.com/apache/arrow-rs/pull/5956 >> > > > >> > > > On Tue, Jun 18, 2024 at 10:39 AM Alkis Evlogimenos >> > > > <alkis.evlogime...@databricks.com.invalid> wrote: >> > > > >> > > > > +1. >> > > > > >> > > > > I would suggest you address the comments first? I went through the >> > open >> > > > > ones and most of them make sense to me (and left few additional >> > > > comments). >> > > > > >> > > > > On Tue, Jun 18, 2024 at 12:42 PM Andrew Lamb < >> andrewlam...@gmail.com >> > > >> > > > > wrote: >> > > > > >> > > > > > Thank you >> > > > > > >> > > > > > On Mon, Jun 17, 2024 at 11:40 PM Micah Kornfield < >> > > > emkornfi...@gmail.com> >> > > > > > wrote: >> > > > > > >> > > > > > > Hi Andrew, >> > > > > > > I agree with this sentiment, I asked on the PR if there would >> > be >> > > > > another >> > > > > > > pass and then I can merge it. >> > > > > > > >> > > > > > > Cheers, >> > > > > > > Micah >> > > > > > > >> > > > > > > On Fri, Jun 14, 2024 at 3:20 AM Andrew Lamb < >> > andrewlam...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Hello Parquet Devs, >> > > > > > > > >> > > > > > > > I propose we merge the first (admittedly bare bones) >> > > > "Implementation >> > > > > > > > Status" page PR [1] to the website soon. I think this page >> is >> > vital >> > > > > to >> > > > > > > the >> > > > > > > > Parquet community (and to any attempt to extend the format) >> so >> > the >> > > > > > sooner >> > > > > > > > the better. >> > > > > > > > >> > > > > > > > The reason to merge the PR now is to have a base from which >> > to >> > > > build. >> > > > > > > That >> > > > > > > > PR is already over a year old and has so many comments it is >> > hard >> > > > to >> > > > > > > follow >> > > > > > > > or know what the path forward is. If we insist on sorting >> all >> > the >> > > > > > details >> > > > > > > > out before we merge it I fear it will never merge. >> > > > > > > > >> > > > > > > > Once we have a page, I think the next steps are to add a >> > preamble >> > > > > > > > explaining what it is for and to start trying to fill out >> the >> > chart >> > > > > for >> > > > > > > an >> > > > > > > > implementation (I am happy to try for parquet-rs). I suspect >> > during >> > > > > > that >> > > > > > > > process we will have to adjust some of the charts more. >> > > > > > > > >> > > > > > > > Thank you for your consideration (and thank you for all the >> > > > comments >> > > > > so >> > > > > > > > far) >> > > > > > > > >> > > > > > > > Andrew >> > > > > > > > >> > > > > > > > [1]: https://github.com/apache/parquet-site/pull/34 >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >> > >> > >> >
