So, I agree with Jacques that:
1) there should be two separate votes *at least*: one for adopting the spec itself, one for each donated implementation 2) there should be more explanation of what this implies for people not really familiar with Spark Regards Antoine. On Fri, 13 Sep 2024 15:22:43 -0700 "[email protected]" <[email protected]> wrote: > Yes, I think this is a vote for maintaining the variant spec here in the > Parquet community and accepting the code that goes along with it. > > On Fri, Sep 13, 2024 at 12:18 PM Jacques Nadeau > <[email protected]> wrote: > > > So this vote is actually a vote for: > > "accept donation of variant code from spark project" > > > > That's very different (to me) from "adopt variant from spark". > > > > I'm +1 (non-binding) for accepting donation of code from Spark. I'm -1 > > (non-binding) for adopting variant because I don't really understand what > > that means. > > > > > > > > On Wed, Sep 11, 2024 at 10:34 PM Antoine Pitrou <[email protected]> > > wrote: > > > > > > > > +0 on accepting Variant into the Parquet *project*, but that's not an > > > approval for sharing repos with the current Parquet format and > > > implementations. > > > > > > Also, I have the same impression of this vote being a bit prematurate. > > > Is the Variant type as proposed performant enough? Is it flexible > > > enough to enable interoperability with other systems than Spark? > > > > > > Regards > > > > > > Antoine. > > > > > > > > > On Thu, 12 Sep 2024 12:15:13 +0800 > > > Gang Wu <[email protected]> wrote: > > > > Sorry for the confusion. The intention of this vote is to formally > > accept > > > > the adoption from Spark and is a formal answer to the corresponding > > vote > > > > on the Spark side: > > > > https://lists.apache.org/thread/gqy02x1r5dj73woj4l8r0xxkrztd5qos > > > > Both parties should officially agree on the move before discussing the > > > > details. > > > > > > > > On Thu, Sep 12, 2024 at 7:21 AM Jacques Nadeau < > > > jacques-1odqgaof3lkdnm+yrofe0a-xmd5yjdbdmrexy1tmh2...@public.gmane.org> > > > wrote: > > > > > > > > > > My personal opinion is that we've been jumping the gun on voting a > > > bit. I > > > > > do appreciate the enthusiasm though. :) > > > > > > > > > > Yeah, felt the same to me (non-binding). It's hard to fully > > understand > > > the > > > > > specific actions that are outcomes of this vote. > > > > > > > > > > On Wed, Sep 11, 2024 at 1:11 PM Julien Le Dem < > > > [email protected]> wrote: > > > > > > > > > > > +1 (binding) on agreeing on principle to add Variant to Parquet. > > > > > > Now on the specific plan, > > > > > > > > > > > > > For repositories to host the Variant specification and library: > > > > > > > - apache/parquet-format will add documentation for the > > > specification > > > > > > > - apache/parquet-java will add a new module for the Java > > > implementation > > > > > > > > > > > > Gene has posted his doc with a plan "[DISCUSS] Moving Variant to > > > Parquet > > > > > > Details" to collect feedback. > > > > > > Once he's done integrating the feedback and it's finalized, that > > > will be > > > > > > the plan on how to do it. > > > > > > The doc itself is a better reference on how it's going to happen. > > > Maybe > > > > > > that's a better artifact to vote on. > > > > > > > > > > > > My personal opinion is that we've been jumping the gun on voting a > > > bit. I > > > > > > do appreciate the enthusiasm though. :) > > > > > > Voting is more of a procedural mechanism to formally record that > > > we've > > > > > > achieved consensus. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 11:45 AM Micah Kornfield < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > +1 (binding) in principle on adding it. I think there are still > > > a > > > > > number > > > > > > > of issues to be worked out and we should try to come to a > > > consensus in > > > > > > > Gene's doc [1] + discussion thread on the nitty gritty of what > > this > > > > > > > proposal actually means. > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1guEzBQjzOEEZvvibeZjNraKmZHWtxQR95O_DvtZU0xw/edit#heading=h.5ad5xy8ox6bp > > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 9:21 AM Nong Li < > > > [email protected]> wrote: > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 8:53 AM Gang Wu < > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > Let's just vote for the adoption in this thread and discuss > > > the > > > > > > > location > > > > > > > > in > > > > > > > > > > > > https://lists.apache.org/thread/xwd3mqjr9bdpg3jcnlprbyb4x09c9ymj > > > > > > > > > > > > > > > > > > Cast my own vote: +1 for adding the variant spec to parquet > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 5:27 PM Eduard Tudenhöfner < > > > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 (non-binding) for adding the variant spec to parquet > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:08 PM Daniel Weeks < > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 on adding the variant spec to Parquet > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 12:23 PM Russell Spitzer < > > > > > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1 (Non-binding) This will be great for universal > > > adoption of > > > > > > the > > > > > > > > > > variant > > > > > > > > > > > > type > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 2:14 PM > > > [email protected] < > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > +1 for adding the variant spec to Parquet. I'm > > > looking > > > > > > forward > > > > > > > to > > > > > > > > > > > working > > > > > > > > > > > > > on the addition of shredding. > > > > > > > > > > > > > > > > > > > > > > > > > > As for the details, I think I also prefer a separate > > > > > > > repository, > > > > > > > > > > > > > `parquet-variant`, but I don't think we necessarily > > > need to > > > > > > > > > determine > > > > > > > > > > > > that > > > > > > > > > > > > > question up front. > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 9:05 AM Gang Wu < > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Antoine, > > > > > > > > > > > > > > > > > > > > > > > > > > > > A separate project was my 1st proposal in the > > > original > > > > > > > > discussion > > > > > > > > > > > > > > on the dev@iceberg ML :). > > > > > > > > > > > > > > > > > > > > > > > > > > > > TBH, I'm open to putting them either in existing > > > repos > > > > > or a > > > > > > > > > > dedicated > > > > > > > > > > > > > > parquet-variant repo. The intention of this thread > > > is to > > > > > > try > > > > > > > to > > > > > > > > > > push > > > > > > > > > > > > the > > > > > > > > > > > > > > discussion to reach a consensus. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:54 PM Antoine Pitrou < > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Gang, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Assuming we do want to adopt this in Parquet, I > > > would > > > > > > very > > > > > > > > much > > > > > > > > > > > > > > > recommend separate repositories for this. Putting > > > > > > > > > > > > > > > > > > the > > > > > > spec > > > > > > > > > inside > > > > > > > > > > > > > > > `parquet-format` breeds confusion, IMHO, and may > > > > > > discourage > > > > > > > > > third > > > > > > > > > > > > > > > parties from considering this standalone, > > > non-Parquet, > > > > > > data > > > > > > > > > > format. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > (but for the same reason, I would recommand a > > > separate > > > > > > > > project > > > > > > > > > as > > > > > > > > > > > > well > > > > > > > > > > > > > > > :-)) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 10 Sep 2024 09:48:03 +0800 > > > > > > > > > > > > > > > Gang Wu < > > > [email protected]> wrote: > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I’d like to start a vote for adopting the > > > Variant > > > > > > > > > specification > > > > > > > > > > > and > > > > > > > > > > > > > > > library > > > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > the Spark project. This allows the Variant > > > binary > > > > > > format > > > > > > > > and > > > > > > > > > > > > > shredding > > > > > > > > > > > > > > > > format > > > > > > > > > > > > > > > > to be more broadly used by other interested > > > projects > > > > > > and > > > > > > > > > > systems. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > For repositories to host the Variant > > > specification > > > > > and > > > > > > > > > library: > > > > > > > > > > > > > > > > - apache/parquet-format will add documentation > > > for > > > > > the > > > > > > > > > > > > specification > > > > > > > > > > > > > > > > - apache/parquet-java will add a new module for > > > > > > > > > > > > > > > > > > > the > > > > > > Java > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please refer to the discussion thread: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread/6h58hj39lhqtcyd2hlsyvqm4lzdh4b9z > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This vote will be open for at least 72 hours. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1: Accept the proposal > > > > > > > > > > > > > > > > [ ] +0 > > > > > > > > > > > > > > > > [ ] -1: I don’t think this is a good idea > > > because … > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
