re inspired by the C++ implementation and have gradually been
> > > > migrated to "idiomatic" Rust, as "idiomatic" is becoming more well
> > > defined
> > > > in Rust.
> > > > > Arrow2 is inspired by the current crate and the pains of using it
ow namespace in crates.io with a major version
> > > bump. The absolute value in the major version doesn't really matter as
> > > long as we stick to the convention that breaking change will result in
> > > a major version bump.
> > >
> > > Thank
; Thanks,
> > QP
> >
> >
> >
> > On Tue, Aug 3, 2021 at 5:31 PM paddy horan
> wrote:
> > >
> > > Hi Jorge,
> > >
> > > I see value in consolidating development in a single repo and releasing
> > under the existing arrow crate. Re
in a single repo and releasing
> under the existing arrow crate. Regarding versioning, I think once we
> follow semantic versioning we are fine. I don't think it's worth migrating
> to a different repo and crate to comply with the de-facto standard you
> mention.
> >
>
e next main release with Arrow2
> > being the default and move the existing implementation behind a "legacy"
> > feature flag.
> >
> > Here is why I think this might work well:
> > - People contributing to the Arrow project will naturally contribute
> > to Arrow2
e de-facto standard you mention.
Just one person's opinion though,
Paddy
-Original Message-
From: Jorge Cardoso Leitão
Sent: Tuesday, August 3, 2021 5:23 PM
To: dev@arrow.apache.org
Subject: Re: [Discuss] [Rust] Arrow2/parquet2 going foward
Hi Paddy,
> What do you think about movi
ting Arrow
> community that Arrow2 is the future but that it is <1.0
> - existing users will be well supported in this transition
> - In general, I think the longer that development proceeds in separate
> repos the harder it will be to eventually merge the two in a way that
>
tually merge the two in a way that
> > supports existing users.
> >
> > Do you think would work?
> >
> > Paddy
> >
> > -Original Message-
> > From: Jorge Cardoso Leitão
> > Sent: Monday, August 2, 2021 1:59 PM
> > To: dev@arrow.apache
eparate
> repos the harder it will be to eventually merge the two in a way that
> supports existing users.
>
> Do you think would work?
>
> Paddy
>
> -----Original Message-----
> From: Jorge Cardoso Leitão
> Sent: Monday, August 2, 2021 1:59 PM
> To: dev@arrow.a
sting users.
Do you think would work?
Paddy
-Original Message-
From: Jorge Cardoso Leitão
Sent: Monday, August 2, 2021 1:59 PM
To: dev@arrow.apache.org
Subject: Re: [Discuss] [Rust] Arrow2/parquet2 going foward
Hi,
Sorry for the delay.
If there is a path towards an official release
Hi,
Sorry for the delay.
If there is a path towards an official release under a <1.0.0 versioning
schema aligned with the rest of the Rust ecosystem and in line with the
stability of the API, then IMO we should move all development to within
Apache experimental asap (I can handle this and the lik
I think it would also be fine to push “beta” arrow2 crates out of a repo
under apache/ so long as they are not marked on crates.io as being
Apache-official releases. There’s a possible slippery slope there, but as
long as we are on a path to formalizing the releases I think it is okay.
On Fri, Jul
Jorge -- do you feel like we have a resolution on what to do with arrow2 in
the near term?
The current state of affairs seems to me that arrow2 is released from
https://github.com/jorgecarleitao/arrow2 to crates.io (which is fine). Are
you happy with keeping development in the jorgecarleitao repo
I would be happy with this approach. Thank you for the suggestion
This hybrid approach of both arrow and arrow2 in the same repo seems better
to me than separate repos.
What I really care about is ensuring we don't have two crates/APIs
indefinitely -- as long as we are continually making progress
Apologies for being late to this discussion.
There is a hybrid option to consider here where we add the arrow2 code into
the arrow crate as a separate module, so we release one crate containing
the "old" API (which we can mark as deprecated) as well as the new API.
Java did a similar thing a long
What I meant is that when you decide arrow2 is suitable for release to
existing arrow users, I stand ready to help you incorporate it into arrow.
All the feedback I have heard so far from the rest of the community is that
we are ready. One might even say we are anxious to do so :)
Andrew
Hi,
I meant to stop releasing "arrow" in crates.io and start releasing it as
"arrow2" under a different versioning schema; like "psycopg" -> "psycopg2"
in pypi and others that suffered from large architectural changes that
required a different versioning that better represents the state of the new
> If we do indeed have the expectation of stability over its whole public
surface,
I certainly do not have this expectation between major releases. Who does?
I believe it is a disservice to the overall community to release two API
incompatible Rust implementations of Arrow to crates.io. It will
1
Hi,
Whatever its technical faults may be, projects that rely on arrow (such as
> anything based on DataFusion, like my own) need to be supported as they
> have made the bet on Rust Arrow.
>
1.X versioning in Apache Arrow was never meant to represent stability of
their individual libraries, but on
Hi Simon,
There are several Arrow implementations in parallel:
https://arrow.apache.org/docs/status.html
The Python and R versions are based on Arrow C++, others are
completely separate projects.
Arrow-rs and Arrow2 are referring to the Rust implementation, Arrow C++ is
not going to be replaced.
I'm interested to hear what the relation between arrow2, arrow-rs and the
main github apache/arrow is. Is the intention to replace the C++ codebase
with a rust implementation?
The reason I'm asking is that I'm adding complex number support in the C++
codebase. It may instead be a better idea to do
What if we released "beta" [1] versions of arrow on cargo at whatever pace
was necessary? That way dependent crates could opt in to bleeding edge
functionality / APIs.
There is tension between full technical freedom to change APIs and the
needs of downstream projects for a more stable API.
Whatev
5.0 is being released right now, which means from timing perspective this
is the worst moment for arrow2, indeed. You'd need to wait the full 3
months. On the other hand does releasing a 6.0 beta based on arrow2 on Aug
1st, rc on Sept 1st and releasing the stable on Oct 1st sound like a bad
plan?
23 matches
Mail list logo
