Re: [VOTE] SPIP: Add FunctionCatalog

[Russell Spitzer]

+1 (for what it's worth)

Thanks for making such a robust proposal, i'm excited to see the new work 
coming from this

> On Mar 8, 2021, at 11:44 PM, Dongjoon Hyun  wrote:
> 
> +1 (binding)
> 
> Thank you, Ryan.
> 
> Bests,
> Dongjoon.
> 
> 
> On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  > wrote:
> +1 (non-binding)
> 
> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  > wrote:
> +1 (non-binding)
> 
> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  > wrote:
> +1 (binding)
> 
> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  > wrote:
> Hi everyone, I’d like to start a vote for the FunctionCatalog design proposal 
> (SPIP).
> 
> The proposal is to add a FunctionCatalog interface that can be used to load 
> and list functions for Spark to call. There are interfaces for scalar and 
> aggregate functions.
> 
> In the discussion we’ve come to consensus and I’ve updated the design doc to 
> match how functions will be called:
> 
> In addition to produceResult(InternalRow), which is optional, functions can 
> define produceResult methods with arguments that are Spark’s internal data 
> types, like UTF8String. Spark will prefer these methods when calling the UDF 
> using codgen.
> 
> I’ve also updated the AggregateFunction interface and merged it with the 
> partial aggregate interface because Spark doesn’t support non-partial 
> aggregates.
> 
> The full SPIP doc is here: 
> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>  
> 
> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll do a 
> final update of the PR and we can merge the API.
> 
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
> 
> -- 
> Ryan Blue
> -- 
> Twitter: https://twitter.com/holdenkarau 
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
>  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau 
> 
> 
> -- 
> John Zhuge

Re: [VOTE] SPIP: Add FunctionCatalog

[Dongjoon Hyun]

+1 (binding)

Thank you, Ryan.

Bests,
Dongjoon.


On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  wrote:

> +1 (non-binding)
>
> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:
>
>> +1 (non-binding)
>>
>> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:
>>
>>> +1 (binding)
>>>
>>> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>>>
 Hi everyone, I’d like to start a vote for the FunctionCatalog design
 proposal (SPIP).

 The proposal is to add a FunctionCatalog interface that can be used to
 load and list functions for Spark to call. There are interfaces for scalar
 and aggregate functions.

 In the discussion we’ve come to consensus and I’ve updated the design
 doc to match how functions will be called:

 In addition to produceResult(InternalRow), which is optional,
 functions can define produceResult methods with arguments that are
 Spark’s internal data types, like UTF8String. Spark will prefer these
 methods when calling the UDF using codgen.

 I’ve also updated the AggregateFunction interface and merged it with
 the partial aggregate interface because Spark doesn’t support non-partial
 aggregates.

 The full SPIP doc is here:
 https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl

 Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
 do a final update of the PR and we can merge the API.

 [ ] +1: Accept the proposal as an official SPIP
 [ ] +0
 [ ] -1: I don’t think this is a good idea because …
 --
 Ryan Blue

>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9  
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
>>
>> --
>> John Zhuge
>>
>

Re: [VOTE] SPIP: Add FunctionCatalog

[Chao Sun]

+1 (non-binding)

On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:

> +1 (non-binding)
>
> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:
>
>> +1 (binding)
>>
>> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>>
>>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>>> proposal (SPIP).
>>>
>>> The proposal is to add a FunctionCatalog interface that can be used to
>>> load and list functions for Spark to call. There are interfaces for scalar
>>> and aggregate functions.
>>>
>>> In the discussion we’ve come to consensus and I’ve updated the design
>>> doc to match how functions will be called:
>>>
>>> In addition to produceResult(InternalRow), which is optional, functions
>>> can define produceResult methods with arguments that are Spark’s
>>> internal data types, like UTF8String. Spark will prefer these methods
>>> when calling the UDF using codgen.
>>>
>>> I’ve also updated the AggregateFunction interface and merged it with
>>> the partial aggregate interface because Spark doesn’t support non-partial
>>> aggregates.
>>>
>>> The full SPIP doc is here:
>>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>>
>>> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
>>> do a final update of the PR and we can merge the API.
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don’t think this is a good idea because …
>>> --
>>> Ryan Blue
>>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9  
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
>
> --
> John Zhuge
>

Re: [VOTE] SPIP: Add FunctionCatalog

[John Zhuge]

+1 (non-binding)

On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:

> +1 (binding)
>
> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>
>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>> proposal (SPIP).
>>
>> The proposal is to add a FunctionCatalog interface that can be used to
>> load and list functions for Spark to call. There are interfaces for scalar
>> and aggregate functions.
>>
>> In the discussion we’ve come to consensus and I’ve updated the design doc
>> to match how functions will be called:
>>
>> In addition to produceResult(InternalRow), which is optional, functions
>> can define produceResult methods with arguments that are Spark’s
>> internal data types, like UTF8String. Spark will prefer these methods
>> when calling the UDF using codgen.
>>
>> I’ve also updated the AggregateFunction interface and merged it with the
>> partial aggregate interface because Spark doesn’t support non-partial
>> aggregates.
>>
>> The full SPIP doc is here:
>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>
>> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
>> do a final update of the PR and we can merge the API.
>>
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don’t think this is a good idea because …
>> --
>> Ryan Blue
>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>


-- 
John Zhuge

Re: [VOTE] SPIP: Add FunctionCatalog

[Holden Karau]

+1 (binding)

On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:

> Hi everyone, I’d like to start a vote for the FunctionCatalog design
> proposal (SPIP).
>
> The proposal is to add a FunctionCatalog interface that can be used to
> load and list functions for Spark to call. There are interfaces for scalar
> and aggregate functions.
>
> In the discussion we’ve come to consensus and I’ve updated the design doc
> to match how functions will be called:
>
> In addition to produceResult(InternalRow), which is optional, functions
> can define produceResult methods with arguments that are Spark’s internal
> data types, like UTF8String. Spark will prefer these methods when calling
> the UDF using codgen.
>
> I’ve also updated the AggregateFunction interface and merged it with the
> partial aggregate interface because Spark doesn’t support non-partial
> aggregates.
>
> The full SPIP doc is here:
> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>
> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll do
> a final update of the PR and we can merge the API.
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
> --
> Ryan Blue
>
-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

[VOTE] SPIP: Add FunctionCatalog

[Ryan Blue]

Hi everyone, I’d like to start a vote for the FunctionCatalog design
proposal (SPIP).

The proposal is to add a FunctionCatalog interface that can be used to load
and list functions for Spark to call. There are interfaces for scalar and
aggregate functions.

In the discussion we’ve come to consensus and I’ve updated the design doc
to match how functions will be called:

In addition to produceResult(InternalRow), which is optional, functions can
define produceResult methods with arguments that are Spark’s internal data
types, like UTF8String. Spark will prefer these methods when calling the
UDF using codgen.

I’ve also updated the AggregateFunction interface and merged it with the
partial aggregate interface because Spark doesn’t support non-partial
aggregates.

The full SPIP doc is here:
https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl

Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll do
a final update of the PR and we can merge the API.

[ ] +1: Accept the proposal as an official SPIP
[ ] +0
[ ] -1: I don’t think this is a good idea because …
-- 
Ryan Blue

Re: Apache Spark 2.4.8 (and EOL of 2.4)

[Liang-Chi Hsieh]

Thank you Dongjoon.

I'm going to cut the branch now. Hopefully I can make it soon (need to get
familiar with the process as first time :) )

Liang-Chi


Dongjoon Hyun-2 wrote
> Thank you, Liang-Chi! Next Monday sounds good.
> 
> To All. Please ping Liang-Chi if you have a missed backport.
> 
> Bests,
> Dongjoon.





--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [K8S] ExecutorPodsWatchSnapshotSource with no spark-exec-inactive label in 3.1?

[attilapiros]

Hi,

I do not think this could cause any problem.

The PODs polling snapshots would contain those PODs which were inactivated
before. Every time when the polling is triggered, so here it make sense to
handle them like deleted PODs and skip those.

On the other hand PODs watcher only informed about state changes
(https://kubernetes.io/docs/reference/kubernetes-api/common-parameters/common-parameters/#watch)
and here we are talking about inactivated PODs which I do not expect to have
many state changes.

Best regards,
Attila




--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org