On Tue, Dec 7, 2021 at 5:43 PM Antoine Pitrou <anto...@python.org> wrote:
> > Le 07/12/2021 à 23:35, Niranda Perera a écrit : > > Hi all, > > > > I'd like to discuss a packaging change for arrow. > > > > AFAIU, there are two broad categories of frameworks that use Arrow. > > 1. Projects that only use Arrow core (ex: cudf, ray) - where they follow > > the Arrow format, but internally they are using their own Arrow impl. So, > > they mostly need to read/ write from the Arrow core public APIs to > convert > > to/from their internal impl. > > 2. Projects that use arrow intimately (ex: cylon) - where they use Arrow > > sub-components intimately (ex: compute, flight, etc). These may also > > depend/ support Type1 projects as well (ex: GCylon for with cudf) > > > > Now, as a member of the latter category, a major challenge we face is > > managing dependencies. We currently depend on Arrow v5 and cudf 21.10 but > > can not upgrade to v6 because Cudf is yet to upgrade the Arrow > > dependencies. But when we look at the version upgrade PR [1], there's > > hardly any API changes. > > Why don't cudf simply relax the version requirements if they know their > code runs with both Arrow 5.0 and 6.0? > > It's discussed here. https://github.com/rapidsai/cudf/pull/9686#issuecomment-969079069 > Regards > > Antoine. > -- Niranda Perera https://niranda.dev/ @n1r44 <https://twitter.com/N1R44>
