I wanted to highlight that there is a PR[1] to fill out this matrix for cuda . Super exciting to see this starting
[1] https://github.com/apache/parquet-site/pull/99# On Thu, Jul 4, 2024 at 2:57 PM Andrew Lamb <[email protected]> wrote: > Thank you! A small step perhaps but a major step forward > > On Thu, Jul 4, 2024 at 1:03 PM Micah Kornfield <[email protected]> > wrote: > >> I just merged it. >> >> Cheers, >> Micah >> >> On Thu, Jul 4, 2024 at 2:32 AM Andrew Lamb <[email protected]> >> wrote: >> >> > The initial feature matrix PR[1], has all outstanding comments addressed >> > and is ready to merge from my opinion. >> > >> > Could one of the committers consider merging it so we can start filling >> the >> > matrix out? >> > >> > Thanks, >> > Andrew >> > >> > [1]: https://github.com/apache/parquet-site/pull/34 >> > >> > >> > On Wed, Jun 26, 2024 at 11:22 AM Andrew Lamb <[email protected]> >> > wrote: >> > >> > > I filed a ticket [1] to discuss integration testing / feature >> > > compatibility testing >> > > >> > > [1]: https://github.com/apache/parquet-format/issues/441 >> > > >> > > On Wed, Jun 26, 2024 at 10:03 AM Andrew Lamb <[email protected]> >> > > wrote: >> > > >> > >> I will file a ticket for discussion shortly >> > >> >> > >> On Wed, Jun 26, 2024, 06:55 Julien Le Dem <[email protected]> wrote: >> > >> >> > >>> I agree with Antoine for the separate thread/ticket for integration >> > >>> testing >> > >>> >> > >>> For the « implementation status » PR, I agree we can merge and >> > iterate. I >> > >>> added a couple of comments that can easily be addressed afterwards. >> > >>> Julien >> > >>> >> > >>> On Wed, Jun 26, 2024 at 10:43 Antoine Pitrou <[email protected]> >> > wrote: >> > >>> >> > >>> > >> > >>> > IMHO, we should either start a dedicated discussion thread for >> > >>> > integration testing, or open a GH issue and discuss it there. >> > >>> > >> > >>> > Regards >> > >>> > >> > >>> > Antoine. >> > >>> > >> > >>> > >> > >>> > >> > >>> > On Wed, 26 Jun 2024 09:21:33 +0200 >> > >>> > Alkis Evlogimenos >> > >>> > <[email protected]> >> > >>> > wrote: >> > >>> > > It would be nice if the integration suite specifies how a driver >> > can >> > >>> be >> > >>> > > executed. Then each implementation can provide a driver and the >> > suite >> > >>> > will >> > >>> > > use that for validation. >> > >>> > > >> > >>> > > By specifying both reads and writes for the driver we get a lot >> > more >> > >>> > power. >> > >>> > > Given an input we can roundtrip all combination of >> readers/writers >> > >>> and >> > >>> > > verify they can roundtrip. >> > >>> > > >> > >>> > > On Tue, Jun 25, 2024 at 6:42 PM Andrew Lamb < >> > >>> > [email protected]> wrote: >> > >>> > > >> > >>> > > > FWIW I started hacking up a prototype[1] of what a >> > parquet-testing >> > >>> > > > integration suite might look like if anyone is interested >> > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > [1]: https://github.com/apache/arrow-rs/pull/5956 >> > >>> > > > >> > >>> > > > On Tue, Jun 18, 2024 at 10:39 AM Alkis Evlogimenos >> > >>> > > > <[email protected]> wrote: >> > >>> > > > >> > >>> > > > > +1. >> > >>> > > > > >> > >>> > > > > I would suggest you address the comments first? I went >> through >> > >>> the >> > >>> > open >> > >>> > > > > ones and most of them make sense to me (and left few >> additional >> > >>> > > > comments). >> > >>> > > > > >> > >>> > > > > On Tue, Jun 18, 2024 at 12:42 PM Andrew Lamb < >> > >>> [email protected] >> > >>> > > >> > >>> > > > > wrote: >> > >>> > > > > >> > >>> > > > > > Thank you >> > >>> > > > > > >> > >>> > > > > > On Mon, Jun 17, 2024 at 11:40 PM Micah Kornfield < >> > >>> > > > [email protected]> >> > >>> > > > > > wrote: >> > >>> > > > > > >> > >>> > > > > > > Hi Andrew, >> > >>> > > > > > > I agree with this sentiment, I asked on the PR if there >> > would >> > >>> > be >> > >>> > > > > another >> > >>> > > > > > > pass and then I can merge it. >> > >>> > > > > > > >> > >>> > > > > > > Cheers, >> > >>> > > > > > > Micah >> > >>> > > > > > > >> > >>> > > > > > > On Fri, Jun 14, 2024 at 3:20 AM Andrew Lamb < >> > >>> > [email protected]> >> > >>> > > > > > > wrote: >> > >>> > > > > > > >> > >>> > > > > > > > Hello Parquet Devs, >> > >>> > > > > > > > >> > >>> > > > > > > > I propose we merge the first (admittedly bare bones) >> > >>> > > > "Implementation >> > >>> > > > > > > > Status" page PR [1] to the website soon. I think this >> > page >> > >>> is >> > >>> > vital >> > >>> > > > > to >> > >>> > > > > > > the >> > >>> > > > > > > > Parquet community (and to any attempt to extend the >> > >>> format) so >> > >>> > the >> > >>> > > > > > sooner >> > >>> > > > > > > > the better. >> > >>> > > > > > > > >> > >>> > > > > > > > The reason to merge the PR now is to have a base from >> > which >> > >>> > to >> > >>> > > > build. >> > >>> > > > > > > That >> > >>> > > > > > > > PR is already over a year old and has so many >> comments it >> > >>> is >> > >>> > hard >> > >>> > > > to >> > >>> > > > > > > follow >> > >>> > > > > > > > or know what the path forward is. If we insist on >> sorting >> > >>> all >> > >>> > the >> > >>> > > > > > details >> > >>> > > > > > > > out before we merge it I fear it will never merge. >> > >>> > > > > > > > >> > >>> > > > > > > > Once we have a page, I think the next steps are to >> add a >> > >>> > preamble >> > >>> > > > > > > > explaining what it is for and to start trying to fill >> out >> > >>> the >> > >>> > chart >> > >>> > > > > for >> > >>> > > > > > > an >> > >>> > > > > > > > implementation (I am happy to try for parquet-rs). I >> > >>> suspect >> > >>> > during >> > >>> > > > > > that >> > >>> > > > > > > > process we will have to adjust some of the charts >> more. >> > >>> > > > > > > > >> > >>> > > > > > > > Thank you for your consideration (and thank you for >> all >> > the >> > >>> > > > comments >> > >>> > > > > so >> > >>> > > > > > > > far) >> > >>> > > > > > > > >> > >>> > > > > > > > Andrew >> > >>> > > > > > > > >> > >>> > > > > > > > [1]: https://github.com/apache/parquet-site/pull/34 >> > >>> > > > > > > > >> > >>> > > > > > > >> > >>> > > > > > >> > >>> > > > > >> > >>> > > > >> > >>> > > >> > >>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> >> > >> >> > >> >
