As we just found in https://issues.apache.org/jira/browse/ARROW-6015, our 0.14.1 wheels have more problems (this time on Windows), so more evidence that we don't have the bandwidth to properly support these packages.
On Tue, Jul 16, 2019 at 3:08 PM Jacques Nadeau <jacq...@apache.org> wrote: > > I think what you suggest is highly dependent on who does the work. > > The first question is who is willing to do the work. Given that they are > volunteers, they'd probably need to propose something like this (but with > there own flavors/choices) and then we'd have to figure out how this > communicated to users (especially in the context that the same package > would potentially have different capabilities if used pip vs conda). > > On Mon, Jul 15, 2019 at 8:52 PM Suvayu Ali <fatkasuvayu+li...@gmail.com> > wrote: > > > Hi Wes, others, > > > > A few thoughts from a user. Firstly, I completely understand your > > frustration. I myself have delved into a bit of packaging for many > > scientific computing packages, like ROOT from CERN, although not at the > > scale of users that you face here. > > > > AIU, wheels are a Python-first spec, whereas Arrow is a C++ first library, > > with python bindings. I feel this is what causes the friction in the build > > chain for wheels. That said, I would like to propose the following. > > > > On Mon, Jul 15, 2019 at 10:06:41PM -0500, Wes McKinney wrote: > > > > > > * Our wheel become much more complex due to Flight (requiring gRPC, > > > OpenSSL, and other dependencies) and Gandiva (requiring LLVM and more) > > > > Disable the more advanced features and release reduced feature set wheels, > > say, only with: > > 1. core data structures, Table, etc, > > 2. various serialisation support (parquet, orc, etc), and > > 3. plasma. > > > > My justification being, it covers a significant proportion of the > > relatively non-expert usecases. (1) covers the interaction with other > > Python libraries, particularly pandas, (2) covers most I/O requirements, > > and plasma along with providing a way to manage Arrow objects in-memory for > > more advanced architectures, it also serves as a relatively simple bridge > > to other languages. Any users requiring Gandiva or Flight on Python could > > easily "upgrade" to the conda-forge releases. > > > > What do you think? > > > > Cheers, > > > > -- > > Suvayu > > > > Open source is the future. It sets us free. > >
