Sounds good, I'll review the PR.
On Fri, 3 Nov 2023 at 14:08, Abdeali Kothari
wrote:
> Seeing more support for arrow based functions would be great.
> Gives more control to application developers. And so pandas just becomes 1
> of the available options.
>
> On Fri, 3 Nov 2023, 21:23 Luca
Seeing more support for arrow based functions would be great.
Gives more control to application developers. And so pandas just becomes 1
of the available options.
On Fri, 3 Nov 2023, 21:23 Luca Canali, wrote:
> Hi Enrico,
>
>
>
> +1 on supporting Arrow on par with Pandas. Besides the frameworks
Hi Enrico,
+1 on supporting Arrow on par with Pandas. Besides the frameworks and libraries
that you mentioned I add awkward array, a library used in High Energy Physics
(for those interested more details on how we tested awkward array with Spark
from back when mapInArrow was introduced can be
I'm definitely +1 to include this.
- It seems like an odd feature parity gap to have a map function but no
group apply function.
- There's currently no way to use large arrow types with applyInPandas,
which can lead to errors hitting the 2 GiB max string/binary array size. I
have a PR to Arrow
