IMHO, we should either start a dedicated discussion thread for integration testing, or open a GH issue and discuss it there.
Regards Antoine. On Wed, 26 Jun 2024 09:21:33 +0200 Alkis Evlogimenos <alkis.evlogime...@databricks.com.INVALID> wrote: > It would be nice if the integration suite specifies how a driver can be > executed. Then each implementation can provide a driver and the suite will > use that for validation. > > By specifying both reads and writes for the driver we get a lot more power. > Given an input we can roundtrip all combination of readers/writers and > verify they can roundtrip. > > On Tue, Jun 25, 2024 at 6:42 PM Andrew Lamb > <andrewlamb11-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > FWIW I started hacking up a prototype[1] of what a parquet-testing > > integration suite might look like if anyone is interested > > > > > > > > [1]: https://github.com/apache/arrow-rs/pull/5956 > > > > On Tue, Jun 18, 2024 at 10:39 AM Alkis Evlogimenos > > <alkis.evlogime...@databricks.com.invalid> wrote: > > > > > +1. > > > > > > I would suggest you address the comments first? I went through the open > > > ones and most of them make sense to me (and left few additional > > comments). > > > > > > On Tue, Jun 18, 2024 at 12:42 PM Andrew Lamb <andrewlam...@gmail.com> > > > wrote: > > > > > > > Thank you > > > > > > > > On Mon, Jun 17, 2024 at 11:40 PM Micah Kornfield < > > emkornfi...@gmail.com> > > > > wrote: > > > > > > > > > Hi Andrew, > > > > > I agree with this sentiment, I asked on the PR if there would be > > > another > > > > > pass and then I can merge it. > > > > > > > > > > Cheers, > > > > > Micah > > > > > > > > > > On Fri, Jun 14, 2024 at 3:20 AM Andrew Lamb <andrewlam...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hello Parquet Devs, > > > > > > > > > > > > I propose we merge the first (admittedly bare bones) > > "Implementation > > > > > > Status" page PR [1] to the website soon. I think this page is vital > > > > > > > > > to > > > > > the > > > > > > Parquet community (and to any attempt to extend the format) so the > > > > sooner > > > > > > the better. > > > > > > > > > > > > The reason to merge the PR now is to have a base from which to > > build. > > > > > That > > > > > > PR is already over a year old and has so many comments it is hard > > to > > > > > follow > > > > > > or know what the path forward is. If we insist on sorting all the > > > > details > > > > > > out before we merge it I fear it will never merge. > > > > > > > > > > > > Once we have a page, I think the next steps are to add a preamble > > > > > > explaining what it is for and to start trying to fill out the chart > > > > > > > > > for > > > > > an > > > > > > implementation (I am happy to try for parquet-rs). I suspect during > > > > > > > > > > that > > > > > > process we will have to adjust some of the charts more. > > > > > > > > > > > > Thank you for your consideration (and thank you for all the > > comments > > > so > > > > > > far) > > > > > > > > > > > > Andrew > > > > > > > > > > > > [1]: https://github.com/apache/parquet-site/pull/34 > > > > > > > > > > > > > > > > > > > > >
