g.
> > > >
> > > > Lastly, I do agree with Andrew that it would be good to only maintain
> > > > a single arrow crate in crates.io in the long run. As he mentioned,
> > > > when the current arrow2 code base becomes stable, we could still
>
as
> > > long as we stick to the convention that breaking change will result in
> > > a major version bump.
> > >
> > > Thanks,
> > > QP
> > >
> > >
> > >
> > > On Tue, Aug 3, 2021 at 5:31 PM paddy horan
> >
addy horan
> wrote:
> > >
> > > Hi Jorge,
> > >
> > > I see value in consolidating development in a single repo and releasing
> > under the existing arrow crate. Regarding versioning, I think once we
> > follow semantic versioning we are fine. I don't
ding versioning, I think once we
> follow semantic versioning we are fine. I don't think it's worth migrating
> to a different repo and crate to comply with the de-facto standard you
> mention.
> >
> > Just one person's opinion though,
> > Paddy
> >
> >
> >
ill still contribute to Arrow
> > instead of Arrow2 just by virtue of it being the "official" implementation.
> > However, if both are in one repo people will want to contribute to the
> > "future", i.e. Arrow2.
> > - the experimental flag will be a clear
standard you mention.
Just one person's opinion though,
Paddy
-Original Message-
From: Jorge Cardoso Leitão
Sent: Tuesday, August 3, 2021 5:23 PM
To: dev@arrow.apache.org
Subject: Re: [Discuss] [Rust] Arrow2/parquet2 going foward
Hi Paddy,
> What do you think about moving Arr
community that Arrow2 is the future but that it is <1.0
> - existing users will be well supported in this transition
> - In general, I think the longer that development proceeds in separate
> repos the harder it will be to eventually merge the two in a way that
> supports existing
ge the two in a way that
> > supports existing users.
> >
> > Do you think would work?
> >
> > Paddy
> >
> > -Original Message-
> > From: Jorge Cardoso Leitão
> > Sent: Monday, August 2, 2021 1:59 PM
> > To: dev@arrow.apache.org
>
t; repos the harder it will be to eventually merge the two in a way that
> supports existing users.
>
> Do you think would work?
>
> Paddy
>
> -Original Message-
> From: Jorge Cardoso Leitão
> Sent: Monday, August 2, 2021 1:59 PM
> To: dev@arrow.apache.org
sting users.
Do you think would work?
Paddy
-Original Message-
From: Jorge Cardoso Leitão
Sent: Monday, August 2, 2021 1:59 PM
To: dev@arrow.apache.org
Subject: Re: [Discuss] [Rust] Arrow2/parquet2 going foward
Hi,
Sorry for the delay.
If there is a path towards an official rel
Hi,
Sorry for the delay.
If there is a path towards an official release under a <1.0.0 versioning
schema aligned with the rest of the Rust ecosystem and in line with the
stability of the API, then IMO we should move all development to within
Apache experimental asap (I can handle this and the
I think it would also be fine to push “beta” arrow2 crates out of a repo
under apache/ so long as they are not marked on crates.io as being
Apache-official releases. There’s a possible slippery slope there, but as
long as we are on a path to formalizing the releases I think it is okay.
On Fri,
Jorge -- do you feel like we have a resolution on what to do with arrow2 in
the near term?
The current state of affairs seems to me that arrow2 is released from
https://github.com/jorgecarleitao/arrow2 to crates.io (which is fine). Are
you happy with keeping development in the jorgecarleitao
I would be happy with this approach. Thank you for the suggestion
This hybrid approach of both arrow and arrow2 in the same repo seems better
to me than separate repos.
What I really care about is ensuring we don't have two crates/APIs
indefinitely -- as long as we are continually making
Apologies for being late to this discussion.
There is a hybrid option to consider here where we add the arrow2 code into
the arrow crate as a separate module, so we release one crate containing
the "old" API (which we can mark as deprecated) as well as the new API.
Java did a similar thing a long
What I meant is that when you decide arrow2 is suitable for release to
existing arrow users, I stand ready to help you incorporate it into arrow.
All the feedback I have heard so far from the rest of the community is that
we are ready. One might even say we are anxious to do so :)
Andrew
Hi,
I meant to stop releasing "arrow" in crates.io and start releasing it as
"arrow2" under a different versioning schema; like "psycopg" -> "psycopg2"
in pypi and others that suffered from large architectural changes that
required a different versioning that better represents the state of the
> If we do indeed have the expectation of stability over its whole public
surface,
I certainly do not have this expectation between major releases. Who does?
I believe it is a disservice to the overall community to release two API
incompatible Rust implementations of Arrow to crates.io. It will
Hi,
Whatever its technical faults may be, projects that rely on arrow (such as
> anything based on DataFusion, like my own) need to be supported as they
> have made the bet on Rust Arrow.
>
1.X versioning in Apache Arrow was never meant to represent stability of
their individual libraries, but
Hi Simon,
There are several Arrow implementations in parallel:
https://arrow.apache.org/docs/status.html
The Python and R versions are based on Arrow C++, others are
completely separate projects.
Arrow-rs and Arrow2 are referring to the Rust implementation, Arrow C++ is
not going to be replaced.
I'm interested to hear what the relation between arrow2, arrow-rs and the
main github apache/arrow is. Is the intention to replace the C++ codebase
with a rust implementation?
The reason I'm asking is that I'm adding complex number support in the C++
codebase. It may instead be a better idea to
What if we released "beta" [1] versions of arrow on cargo at whatever pace
was necessary? That way dependent crates could opt in to bleeding edge
functionality / APIs.
There is tension between full technical freedom to change APIs and the
needs of downstream projects for a more stable API.
5.0 is being released right now, which means from timing perspective this
is the worst moment for arrow2, indeed. You'd need to wait the full 3
months. On the other hand does releasing a 6.0 beta based on arrow2 on Aug
1st, rc on Sept 1st and releasing the stable on Oct 1st sound like a bad
plan?
Hi,
Arrow2 and parquet2 have passed the IP clearance vote and are ready to be
merged to apache/* repos.
My plan is to merge them and PR to both of them to the latest updates on my
own repo, so that I can temporarily (and hopefully permanently) archive the
versions of my account and move
24 matches
Mail list logo
