Le 07/12/2021 à 23:35, Niranda Perera a écrit :
Hi all, I'd like to discuss a packaging change for arrow. AFAIU, there are two broad categories of frameworks that use Arrow. 1. Projects that only use Arrow core (ex: cudf, ray) - where they follow the Arrow format, but internally they are using their own Arrow impl. So, they mostly need to read/ write from the Arrow core public APIs to convert to/from their internal impl. 2. Projects that use arrow intimately (ex: cylon) - where they use Arrow sub-components intimately (ex: compute, flight, etc). These may also depend/ support Type1 projects as well (ex: GCylon for with cudf) Now, as a member of the latter category, a major challenge we face is managing dependencies. We currently depend on Arrow v5 and cudf 21.10 but can not upgrade to v6 because Cudf is yet to upgrade the Arrow dependencies. But when we look at the version upgrade PR [1], there's hardly any API changes.
Why don't cudf simply relax the version requirements if they know their code runs with both Arrow 5.0 and 6.0?
Regards Antoine.
