Re: [DISCUSS] Python Wheel Size

2023-01-01 Thread Weston Steimel
Apologies for being very late to this discussion, but if anyone is still interested in this work, I did quite long ago attempt something like this at https://github.com/westonsteimel/pyarrow-parquet. Eventually I gave up on that approach (due to the time taken for builds etc) and instead moved to

Re: [DISCUSS] Python Wheel Size

2022-10-10 Thread Wes McKinney
We've discussed this in the past, I think. In addition to having many optional components enabled, the pyarrow wheel also includes the unit tests directory which is of growing size. I think if we made a pyarrow-slim wheel with support only for core Arrow (IPC, etc.) and Parquet file reading, it mig

Re: [DISCUSS] Python Wheel Size

2022-10-03 Thread Antoine Pitrou
Hi Rusty, Le 02/10/2022 à 22:51, Rusty Conover a écrit : Hi Arrow Team, I'm using Apache Arrow with AWS Lambda Functions. The primary motivation is AWS Athena's user-defined functions[1]. Those functions process and return Arrow IPC segments. * The published Python wheels for Apache Arrow

[DISCUSS] Python Wheel Size

2022-10-03 Thread Rusty Conover
Hi Arrow Team, I'm using Apache Arrow with AWS Lambda Functions. The primary motivation is AWS Athena's user-defined functions[1]. Those functions process and return Arrow IPC segments. * The published Python wheels for Apache Arrow include almost every feature of Arrow. (Gandiva, Plasma, Fligh