ursday, July 7, 2022 5:15 AM
To: dev@arrow.apache.org
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
Hi Yaron,
Le 07/07/2022 à 10:48, Yaron Gvili a écrit :
> It looks like the main decision to make is whether accessing Substrait
> protobuf Python classes from
Hi Yaron,
Le 07/07/2022 à 10:48, Yaron Gvili a écrit :
It looks like the main decision to make is whether accessing Substrait protobuf
Python classes from PyArrow is what we want here, because it would require some
effort to implement as discussed.
I'm not the best person to comme
It looks like the main decision to make is whether accessing Substrait protobuf
Python classes from PyArrow is what we want here, because it would require some
effort to implement as discussed.
> These are two reasons to need the protobuf in python. However, I still don't
> fully
er in this thread, it has an LGPL v3+
> license. Is this license acceptable for a build dependency?
>
>
> Yaron.
> ________
> From: Yaron Gvili
> Sent: Wednesday, July 6, 2022 7:26 AM
> To: dev@arrow.apache.org
> Subject: Re: accessing Substrait
about UDFs.
Yaron.
From: Weston Pace
Sent: Tuesday, July 5, 2022 7:59 PM
To: dev@arrow.apache.org
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
If this is to be used within pyarrow then I think it makes sense as
something to do. Can you ex
Fs.
Yaron.
From: Weston Pace
Sent: Tuesday, July 5, 2022 7:59 PM
To: dev@arrow.apache.org
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
If this is to be used within pyarrow then I think it makes sense as
something to do. Can you expand a bit more on what you are
using Cython or C++.
>
>
> Yaron.
>
>
> From: Li Jin
> Sent: Tuesday, July 5, 2022, 4:22 PM
> To: dev@arrow.apache.org
> Subject: Re: accessing Substrait protobuf Python classes from PyArrow
>
> Yaron, do we need to parse the subtrait p
.
From: Li Jin
Sent: Tuesday, July 5, 2022, 4:22 PM
To: dev@arrow.apache.org
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
Yaron, do we need to parse the subtrait protobuf in Python so that we can
get the UDFs and register them with Pyarrow?
On
_
> From: Jeroen van Straten
> Sent: Monday, July 4, 2022 12:38 PM
> To: dev@arrow.apache.org
> Subject: Re: accessing Substrait protobuf Python classes from PyArrow
>
> Ah, I see. I guess that complicates things a little.
>
> About the exposure part in C++, the idea is
-of-refactorings
Yaron.
From: Jeroen van Straten
Sent: Monday, July 4, 2022 12:38 PM
To: dev@arrow.apache.org
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
Ah, I see. I guess that complicates things a little.
About the exposure part in C
or external project)
>
>
> Yaron.
> ____
> From: Antoine Pitrou
> Sent: Sunday, July 3, 2022 3:20 PM
> To: dev@arrow.apache.org
> Subject: Re: accessing Substrait protobuf Python classes from PyArrow
>
>
> I agree that giving direct acces
Sunday, July 3, 2022 3:20 PM
To: dev@arrow.apache.org
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
I agree that giving direct access to protobuf classes is not Arrow's
job. You can probably take the upstream (i.e. Substrait's) protobuf
definitions and com
22 8:55 AM
To: dev@arrow.apache.org ; Phillip Cloud <
cpcl...@gmail.com>
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
I'm somewhat confused by this answer because I think resolving the issue I
raised does not require any change outside PyArrow. I'll tr
__
> From: Yaron Gvili
> Sent: Saturday, July 2, 2022 8:55 AM
> To: dev@arrow.apache.org ; Phillip Cloud <
> cpcl...@gmail.com>
> Subject: Re: accessing Substrait protobuf Python classes from PyArrow
>
> I'm somewhat confused by this an
und like a reasonable approach?
Yaron.
From: Yaron Gvili
Sent: Saturday, July 2, 2022 8:55 AM
To: dev@arrow.apache.org ; Phillip Cloud
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
I'm somewhat confused by this answer because I think resolv
uly 1, 2022 12:42 PM
To: dev@arrow.apache.org ; Phillip Cloud
Subject: Re: accessing Substrait protobuf Python classes from PyArrow
Given that Acero does not do any planner / optimizer type tasks I'm
not sure you will find anything like this in arrow-cpp or pyarrow.
What you are describing I someti
Given that Acero does not do any planner / optimizer type tasks I'm
not sure you will find anything like this in arrow-cpp or pyarrow.
What you are describing I sometimes refer to as "plan slicing and
dicing". I have wondered if we will someday need this in Acero but I
fear it is a slippery slope
Hi,
Is there support for accessing Substrait protobuf Python classes (such as Plan)
from PyArrow? If not, how should such support be added? For example, should the
PyArrow build system pull in the Substrait repo as an external project and
build its protobuf Python classes, in a manner similar t
18 matches
Mail list logo
