+1 (binding) on agreeing on principle to add Variant to Parquet. Now on the specific plan,
> For repositories to host the Variant specification and library: > - apache/parquet-format will add documentation for the specification > - apache/parquet-java will add a new module for the Java implementation Gene has posted his doc with a plan "[DISCUSS] Moving Variant to Parquet Details" to collect feedback. Once he's done integrating the feedback and it's finalized, that will be the plan on how to do it. The doc itself is a better reference on how it's going to happen. Maybe that's a better artifact to vote on. My personal opinion is that we've been jumping the gun on voting a bit. I do appreciate the enthusiasm though. :) Voting is more of a procedural mechanism to formally record that we've achieved consensus. On Wed, Sep 11, 2024 at 11:45 AM Micah Kornfield <[email protected]> wrote: > +1 (binding) in principle on adding it. I think there are still a number > of issues to be worked out and we should try to come to a consensus in > Gene's doc [1] + discussion thread on the nitty gritty of what this > proposal actually means. > > [1] > > https://docs.google.com/document/d/1guEzBQjzOEEZvvibeZjNraKmZHWtxQR95O_DvtZU0xw/edit#heading=h.5ad5xy8ox6bp > > On Wed, Sep 11, 2024 at 9:21 AM Nong Li <[email protected]> wrote: > > > +1 > > > > On Wed, Sep 11, 2024 at 8:53 AM Gang Wu <[email protected]> wrote: > > > > > Let's just vote for the adoption in this thread and discuss the > location > > in > > > https://lists.apache.org/thread/xwd3mqjr9bdpg3jcnlprbyb4x09c9ymj > > > > > > Cast my own vote: +1 for adding the variant spec to parquet > > > > > > Thanks! > > > > > > Gang > > > > > > On Wed, Sep 11, 2024 at 5:27 PM Eduard Tudenhöfner < > > > [email protected]> > > > wrote: > > > > > > > +1 (non-binding) for adding the variant spec to parquet > > > > > > > > On Tue, Sep 10, 2024 at 11:08 PM Daniel Weeks <[email protected]> > > wrote: > > > > > > > > > +1 on adding the variant spec to Parquet > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 12:23 PM Russell Spitzer < > > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > +1 (Non-binding) This will be great for universal adoption of the > > > > variant > > > > > > type > > > > > > > > > > > > On Tue, Sep 10, 2024 at 2:14 PM [email protected] < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > +1 for adding the variant spec to Parquet. I'm looking forward > to > > > > > working > > > > > > > on the addition of shredding. > > > > > > > > > > > > > > As for the details, I think I also prefer a separate > repository, > > > > > > > `parquet-variant`, but I don't think we necessarily need to > > > determine > > > > > > that > > > > > > > question up front. > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 9:05 AM Gang Wu <[email protected]> > > wrote: > > > > > > > > > > > > > > > Hi Antoine, > > > > > > > > > > > > > > > > A separate project was my 1st proposal in the original > > discussion > > > > > > > > on the dev@iceberg ML :). > > > > > > > > > > > > > > > > TBH, I'm open to putting them either in existing repos or a > > > > dedicated > > > > > > > > parquet-variant repo. The intention of this thread is to try > to > > > > push > > > > > > the > > > > > > > > discussion to reach a consensus. > > > > > > > > > > > > > > > > Best, > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:54 PM Antoine Pitrou < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Gang, > > > > > > > > > > > > > > > > > > Assuming we do want to adopt this in Parquet, I would very > > much > > > > > > > > > recommend separate repositories for this. Putting the spec > > > inside > > > > > > > > > `parquet-format` breeds confusion, IMHO, and may discourage > > > third > > > > > > > > > parties from considering this standalone, non-Parquet, data > > > > format. > > > > > > > > > > > > > > > > > > (but for the same reason, I would recommand a separate > > project > > > as > > > > > > well > > > > > > > > > :-)) > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 10 Sep 2024 09:48:03 +0800 > > > > > > > > > Gang Wu <[email protected]> wrote: > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > I’d like to start a vote for adopting the Variant > > > specification > > > > > and > > > > > > > > > library > > > > > > > > > > from > > > > > > > > > > the Spark project. This allows the Variant binary format > > and > > > > > > > shredding > > > > > > > > > > format > > > > > > > > > > to be more broadly used by other interested projects and > > > > systems. > > > > > > > > > > > > > > > > > > > > For repositories to host the Variant specification and > > > library: > > > > > > > > > > - apache/parquet-format will add documentation for the > > > > > > specification > > > > > > > > > > - apache/parquet-java will add a new module for the Java > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > Please refer to the discussion thread: > > > > > > > > > > > > > > https://lists.apache.org/thread/6h58hj39lhqtcyd2hlsyvqm4lzdh4b9z > > > > > > > > > > > > > > > > > > > > This vote will be open for at least 72 hours. > > > > > > > > > > > > > > > > > > > > [ ] +1: Accept the proposal > > > > > > > > > > [ ] +0 > > > > > > > > > > [ ] -1: I don’t think this is a good idea because … > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
