Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications

Wed, 16 Nov 2022 21:22:33 -0800 

+1 




Ruifeng Zheng
ruife...@foxmail.com



 




------------------ Original ------------------
From:                                                                           
                                             "Wenchen Fan"                      
                                                              
<cloud0...@gmail.com&gt;;
Date:&nbsp;Thu, Nov 17, 2022 10:27 AM
To:&nbsp;"Ye Zhou"<zhouye...@gmail.com&gt;;
Cc:&nbsp;"Jungtaek Lim"<kabhwan.opensou...@gmail.com&gt;;"Chris 
Nauroth"<cnaur...@apache.org&gt;;"Maciej"<mszymkiew...@gmail.com&gt;;"dev"<dev@spark.apache.org&gt;;
Subject:&nbsp;Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability 
for large applications



+1, I'm looking forward to it!


On Thu, Nov 17, 2022 at 9:44 AM Ye Zhou <zhouye...@gmail.com&gt; wrote:

+1 (non-binding)Thanks for proposing this improvement to SHS, it resolves the 
main performance&nbsp;issue within SHS.


On Wed, Nov 16, 2022 at 1:15 PM Jungtaek Lim <kabhwan.opensou...@gmail.com&gt; 
wrote:

+1

Nice to see the chance for driver to reduce resource usage and increase 
stability, especially the fact that the driver is SPOF. It's even promising to 
have a future plan to pre-bake the kvstore for SHS from the driver.


Thanks for driving the effort, Gengliang!


On Thu, Nov 17, 2022 at 5:32 AM Chris Nauroth <cnaur...@apache.org&gt; wrote:

+1 (non-binding)

Gengliang, thank you for the SPIP.

Chris Nauroth








On Wed, Nov 16, 2022 at 4:27 AM Maciej <mszymkiew...@gmail.com&gt; wrote:

+1
 
 On 11/16/22 13:19, Yuming Wang wrote:
 &gt; +1, non-binding
 &gt; 
 &gt; On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangji...@baidu.com 
 &gt; <mailto:yangji...@baidu.com&gt;&gt; wrote:
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;+1, non-binding____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;Yang Jie____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;*??????**: *Mridul Muralidharan <mri...@gmail.com
 &gt;&nbsp; &nbsp; &nbsp;<mailto:mri...@gmail.com&gt;&gt;
 &gt;&nbsp; &nbsp; &nbsp;*????**: *2022??11??16????????17:35
 &gt;&nbsp; &nbsp; &nbsp;*??????**: *Kent Yao <y...@apache.org 
<mailto:y...@apache.org&gt;&gt;
 &gt;&nbsp; &nbsp; &nbsp;*????**: *Gengliang Wang <ltn...@gmail.com
 &gt;&nbsp; &nbsp; &nbsp;<mailto:ltn...@gmail.com&gt;&gt;, dev 
<dev@spark.apache.org
 &gt;&nbsp; &nbsp; &nbsp;<mailto:dev@spark.apache.org&gt;&gt;
 &gt;&nbsp; &nbsp; &nbsp;*????**: *Re: [VOTE][SPIP] Better Spark UI scalability 
and Driver
 &gt;&nbsp; &nbsp; &nbsp;stability for large applications____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;+1____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;Would be great to see history server performance 
improvements and
 &gt;&nbsp; &nbsp; &nbsp;lower resource utilization at driver !____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;Regards,____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;Mridul ____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;__ __
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp;On Wed, Nov 16, 2022 at 2:38 AM Kent Yao 
<y...@apache.org
 &gt;&nbsp; &nbsp; &nbsp;<mailto:y...@apache.org&gt;&gt; wrote:____
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+1, non-binding
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Gengliang Wang <ltn...@gmail.com 
<mailto:ltn...@gmail.com&gt;&gt; ??
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;2022??11??16??????16:36??????
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Hi all,
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; I??d like to start a vote for SPIP: 
"Better Spark UI scalability and Driver stability for large applications"
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; The goal of the SPIP is to improve 
the Driver's stability by supporting storing Spark's UI data on RocksDB. 
Furthermore, to fasten the read and write operations on RocksDB, it introduces 
a new Protobuf serializer.
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Please also refer to the following:
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Previous discussion in the dev 
mailing list: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability 
for large applications
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Design Doc: Better Spark UI 
scalability and Driver stability for large applications
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; JIRA: SPARK-41053
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Please vote on the SPIP for the 
next 72 hours:
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; [ ] +1: Accept the proposal as an 
official SPIP
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; [ ] +0
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; [ ] -1: I don??t think this is a 
good idea because ??
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt;
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Kind Regards,
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; Gengliang
 &gt; 
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; 
&nbsp;---------------------------------------------------------------------
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;To unsubscribe e-mail: 
dev-unsubscr...@spark.apache.org
 &gt;&nbsp; &nbsp; &nbsp; &nbsp; 
&nbsp;<mailto:dev-unsubscr...@spark.apache.org&gt;____
 &gt; 
 
 -- 
 Best regards,
 Maciej Szymkiewicz
 
 Web: https://zero323.net
 PGP: A30CEF0C31A501EC
 
 
 
 



-- 

Zhou, Ye&nbsp; ????

Reply via email to