Re: Improving Dynamic Allocation Logic for Spark 4+

2023-08-08 Thread Thomas Graves
> > - Advisory user input (e.g. a way to say after X is done I know I need Y > > where Y might be a bunch of GPU machines) You are thinking of something more advanced than the Stage Level Scheduling? Or perhaps configured differently or prestarting things you know you will need? Tom On Mon,

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Thomas graves
+1 Tom On Mon, Jun 19, 2023 at 9:41 PM Dongjoon Hyun wrote: > Please vote on releasing the following candidate as Apache Spark version > 3.4.1. > > The vote is open until June 23rd 1AM (PST) and passes if a majority +1 PMC > votes are cast, with a minimum of 3 +1 votes. > > [ ] +1 Release this

Re: [VOTE] Release Spark 3.3.1 (RC4)

2022-10-18 Thread Thomas graves
+1. Ran internal test suite. Tom On Sun, Oct 16, 2022 at 9:14 PM Yuming Wang wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.3.1. > > The vote is open until 11:59pm Pacific time October 21th and passes if a > majority +1 PMC votes are cast, with a

Re: [VOTE] Release Spark 3.3.1 (RC2)

2022-10-03 Thread Thomas Graves
+1. ran out internal tests and everything looks good. Tom Graves On Wed, Sep 28, 2022 at 12:20 AM Yuming Wang wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.3.1. > > The vote is open until 11:59pm Pacific time October 3th and passes if a > majority +1

Re: [VOTE][SPIP] Spark Connect

2022-06-16 Thread Thomas Graves
+1 for the concept. Correct me if I'm wrong, but at a high level this is proposing adding a new user API (which is language agnostic) and the proposal is to start with something like the Logical Plan, with the addition of being able to remotely call this API. +0 on architecture/design as it's not

Re: [VOTE] Release Spark 3.3.0 (RC5)

2022-06-07 Thread Thomas Graves
+1 Tom Graves On Sat, Jun 4, 2022 at 9:50 AM Maxim Gekk wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.3.0. > > The vote is open until 11:59pm Pacific time June 8th and passes if a majority > +1 PMC votes are cast, with a minimum of 3 +1 votes. > > [ ]

Re: [VOTE] Release Spark 3.3.0 (RC3)

2022-05-27 Thread Thomas graves
+1. Ran through internal tests. Tom Graves On Tue, May 24, 2022 at 12:14 PM Maxim Gekk wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.3.0. > > The vote is open until 11:59pm Pacific time May 27th and passes if a majority > +1 PMC votes are cast, with a

Re: [VOTE] Release Spark 3.3.0 (RC1)

2022-05-10 Thread Thomas graves
Is there going to be an rc2? I thought there were a couple of issue mentioned in the thread. On Tue, May 10, 2022 at 11:53 AM Maxim Gekk wrote: > > Hi All, > > Today is the last day for voting. Please, test the RC1 and vote. > > Maxim Gekk > > Software Engineer > > Databricks, Inc. > > > > On

Re: [VOTE] Spark 3.1.3 RC4

2022-02-16 Thread Thomas graves
+1 Tom On Mon, Feb 14, 2022 at 2:55 PM Holden Karau wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.1.3. > > The vote is open until Feb. 18th at 1 PM pacific (9 PM GMT) and passes if a > majority > +1 PMC votes are cast, with a minimum of 3 + 1 votes. >

Re: [VOTE] Spark 3.1.3 RC3

2022-02-02 Thread Thomas Graves
It was discussed doing all the maintenance lines back at beginning of December (Dec 6) when we were talking about release 3.2.1. Tom On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan wrote: > > Hi Holden, > > Not that I am against releasing 3.1.3 (given the fixes that have already > gone

Re: [VOTE][SPIP] Support Customized Kubernetes Schedulers Proposal

2022-01-11 Thread Thomas Graves
+1 (binding). One minor note since I haven't had time to look at the implementation details is please make sure resource aware scheduling and the stage level scheduling still work or any caveats are documented. Feel free to ping me if questions in these areas. Tom On Wed, Jan 5, 2022 at 7:07 PM

Re: [VOTE] Release Spark 3.2.1 (RC1)

2022-01-11 Thread Thomas Graves
+1, ran our internal tests and everything looks good. Tom On Mon, Jan 10, 2022 at 12:10 PM huaxin gao wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.2.1. > > The vote is open until Jan. 13th at 12 PM PST (8 PM UTC) and passes if a > majority > +1 PMC

Re: [VOTE] Release Spark 3.2.0 (RC7)

2021-10-11 Thread Thomas graves
+1 Tom Graves On Wed, Oct 6, 2021 at 11:49 AM Gengliang Wang wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.2.0. > > The vote is open until 11:59pm Pacific time October 11 and passes if a > majority +1 PMC votes are cast, with a minimum of 3 +1 votes.

Re: Removing references to Master

2021-07-09 Thread Thomas Graves
Hey everyone, Looking at this again since we cut spark 3.2 branch thinking this might be something to target for Spark 3.3. Based on the feedback, I'd like to propose using "Leader" to replace "Master". If there are objections to this please let me know in the next few days. Thanks, Tom On

Re: [VOTE] Release Spark 3.1.2 (RC1)

2021-05-26 Thread Thomas Graves
+1 Tom Graves On Mon, May 24, 2021 at 1:14 AM Dongjoon Hyun wrote: > > Please vote on releasing the following candidate as Apache Spark version > 3.1.2. > > The vote is open until May 27th 1AM (PST) and passes if a majority +1 PMC > votes are cast, with a minimum of 3 +1 votes. > > [ ] +1

GitHub action permissions

2020-02-28 Thread Thomas graves
Does anyone know how the GitHub action permissions are setup? I see a lot of random failures and want to be able to rerun them, but I don't seem to have a "rerun" button like some folks do. Thanks, Tom - To unsubscribe e-mail:

Committing while Jenkins down?

2019-10-10 Thread Thomas graves
This is directed towards committers/PMC members. It looks like Jenkins will be down for a while, what is everyone's thoughts on committing PRs while its down? Do we want to wait for Jenkins to come back up, manually run things ourselves and commit? Tom

Re: [RESULT][VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration and scheduling

2019-09-14 Thread Thomas graves
Resending fixing my typo. Hi all, > > The vote passed with 6 +1's (4 binding) and no -1's. > > +1s (* = binding) : > Bobby Evans* > Thomas Graves* > Dongjoon Hyun* > Felix Cheung* > Bryan Cutler > Ryan Blue > > > Thanks, > Tom Graves >

[RESULT][VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration and scheduling

2019-09-13 Thread Thomas graves
Hi all, The vote passed with 6 +1's (4 binding) and no -1's. +1s (* = binding) : Bobby Evans* Thomas Graves* Dongjoon Hyuni* Felix Cheung* Bryan Cutler Ryan Blue Thanks, Tom Graves - To unsubscribe e-mail: dev-unsubscr

Re: [VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration and scheduling

2019-09-13 Thread Thomas Graves
>> This is going to be really useful. Thanks for working on it! >> >> On Wed, Sep 11, 2019 at 9:38 AM Felix Cheung >> wrote: >>> >>> +1 >>> >>> >>> From: Thomas graves >>> Sent: Wedn

Re: Thoughts on Spark 3 release, or a preview release

2019-09-13 Thread Thomas Graves
+1, I think having preview release would be great. Tom On Fri, Sep 13, 2019 at 4:55 AM Stavros Kontopoulos < stavros.kontopou...@lightbend.com> wrote: > +1 as a contributor and as a user. Given the amount of testing required > for all the new cool stuff like java 11 support, major >

[VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration and scheduling

2019-09-04 Thread Thomas graves
Hey everyone, I'd like to call for a vote on SPARK-27495 SPIP: Support Stage level resource configuration and scheduling This is for supporting stage level resource configuration and scheduling. The basic idea is to allow the user to specify executor and task resource requirements for each

[RESULT][VOTE] SPIP: Public APIs for extended Columnar Processing Support

2019-05-29 Thread Thomas graves
Hi all, The vote passed with 9 +1's (4 binding) and 1 +0 and no -1's. +1s (* = binding) : Bobby Evans* Thomas Graves* DB Tsai* Felix Cheung* Bryan Cutler Kazuaki Ishizaki Tyson Condie Dongjoon Hyun Jason Lowe +0s: Xiangrui Meng Thanks, Tom Graves

Re: [VOTE][SPARK-27396] SPIP: Public APIs for extended Columnar Processing Support

2019-05-14 Thread Thomas graves
Thanks for replying, I'll extend the vote til May 26th to allow your and other people feedback who haven't had time to look at it. Tom On Mon, May 13, 2019 at 4:43 PM Holden Karau wrote: > > I’d like to ask this vote period to be extended, I’m interested but I don’t > have the cycles to review

Re: [VOTE][SPARK-27396] SPIP: Public APIs for extended Columnar Processing Support

2019-05-13 Thread Thomas graves
It would be nice to get feedback from people who responded on the other vote thread - Reynold, Matei, Xiangrui, does the new version look good? Thanks, Tom On Mon, May 13, 2019 at 8:22 AM Jason Lowe wrote: > > +1 (non-binding) > > Jason > > On Tue, May 7, 2019 at 1:37 PM Th

[VOTE][SPARK-27396] SPIP: Public APIs for extended Columnar Processing Support

2019-05-07 Thread Thomas graves
Hi everyone, I'd like to call for another vote on SPARK-27396 - SPIP: Public APIs for extended Columnar Processing Support. The proposal is to extend the support to allow for more columnar processing. We had previous vote and discussion threads and have updated the SPIP based on the comments to

Re: [VOTE] SPIP: Executor Plugin (SPARK-24918)

2018-08-28 Thread Thomas Graves
+1 Tom On Tue, Aug 28, 2018 at 8:50 AM Imran Rashid wrote: > There has been discussion on jira & the PR, all generally positive, so I'd > like to call a vote for this spip. > > I'll start with own +1. > > On Fri, Aug 3, 2018 at 11:59 AM Imran Rashid wrote: > >> I'd like to propose adding a