RE: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Kazuaki Ishizaki
+1 if we add them to Alternative config. Kazuaki Ishizaki From: Takeshi Yamamuro To: Wenchen Fan Cc: Spark dev list Date: 2020/02/13 16:02 Subject:[EXTERNAL] Re: [DISCUSS] naming policy of Spark configs +1; the idea sounds reasonable. Bests, Takeshi On Thu, Feb 13

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Takeshi Yamamuro
+1; the idea sounds reasonable. Bests, Takeshi On Thu, Feb 13, 2020 at 12:39 PM Wenchen Fan wrote: > Hi Dongjoon, > > It's too much work to revisit all the configs that added in 3.0, but I'll > revisit the recent commits that update config names and see if they follow > the new policy. > > > Hi

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Wenchen Fan
Hi Dongjoon, It's too much work to revisit all the configs that added in 3.0, but I'll revisit the recent commits that update config names and see if they follow the new policy. Hi Reynold, There are a few interval configs: spark.sql.streaming.fileSink.log.compactInterval spark.sql.streaming.co

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Jungtaek Lim
+1 Thanks for the proposal. Looks very reasonable to me. On Thu, Feb 13, 2020 at 10:53 AM Hyukjin Kwon wrote: > +1. > > 2020년 2월 13일 (목) 오전 9:30, Gengliang Wang 님이 > 작성: > >> +1, this is really helpful. We should make the SQL configurations >> consistent and more readable. >> >> On Wed, Feb 12,

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Hyukjin Kwon
+1. 2020년 2월 13일 (목) 오전 9:30, Gengliang Wang 님이 작성: > +1, this is really helpful. We should make the SQL configurations > consistent and more readable. > > On Wed, Feb 12, 2020 at 3:33 PM Rubén Berenguel > wrote: > >> I love it, it will make configs easier to read and write. Thanks Wenchen. >> >

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Gengliang Wang
+1, this is really helpful. We should make the SQL configurations consistent and more readable. On Wed, Feb 12, 2020 at 3:33 PM Rubén Berenguel wrote: > I love it, it will make configs easier to read and write. Thanks Wenchen. > > R > > On 13 Feb 2020, at 00:15, Dongjoon Hyun wrote: > >  > Tha

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Rubén Berenguel
I love it, it will make configs easier to read and write. Thanks Wenchen. R > On 13 Feb 2020, at 00:15, Dongjoon Hyun wrote: > >  > Thank you, Wenchen. > > The new policy looks clear to me. +1 for the explicit policy. > > So, are we going to revise the existing conf names before 3.0.0 relea

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Reynold Xin
This is really cool. We should also be more opinionated about how we specify time and intervals. On Wed, Feb 12, 2020 at 3:15 PM, Dongjoon Hyun < dongjoon.h...@gmail.com > wrote: > > Thank you, Wenchen. > > > The new policy looks clear to me. +1 for the explicit policy. > > > So, are we go

Re: [DISCUSS] naming policy of Spark configs

2020-02-12 Thread Dongjoon Hyun
Thank you, Wenchen. The new policy looks clear to me. +1 for the explicit policy. So, are we going to revise the existing conf names before 3.0.0 release? Or, is it applied to new up-coming configurations from now? Bests, Dongjoon. On Wed, Feb 12, 2020 at 7:43 AM Wenchen Fan wrote: > Hi all,