Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

Another correctness issue: https://issues.apache.org/jira/browse/SPARK-32659 On Tue, Aug 25, 2020 at 11:25 PM Sean Owen wrote: > That isn't a blocker (see comments - not a regression). > That said I think we have a fix ready to merge now, if there are no > objections. > > On Tue, Aug 25, 2020

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

That isn't a blocker (see comments - not a regression). That said I think we have a fix ready to merge now, if there are no objections. On Tue, Aug 25, 2020 at 10:24 AM Dongjoon Hyun wrote: > > For the correctness blocker, we have the following, Tom. > > -

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

t; > On Thu, Aug 13, 2020 at 8:19 PM Jason Moore > > wrote: > > Thank you so much! Any update on getting the RC1 up for vote? > > Jason. > > > -- > *From:* 郑瑞峰 > *Sent:* Wednesday, 5 August 2020 12:54 PM > *To:* Jason Moore > ; Sp

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

Release   Hi all, I am going to prepare the realease of 3.0.1 RC1, with the help of Wenchen. -- 原始邮件 -- 发件人: "Jason Moore" ; 发送时间: 2020年7月30日(星期四) 上午10:35 收件人:  "dev"; 主题: Re: [DISCUSS] Apache Spark 3.0.1 Release Hi all,   Dis

Re: [DISCUSS] Apache Spark 3.0.1 Release

enchen. > > > -- 原始邮件 -- > *发件人:* "Jason Moore" ; > *发送时间:* 2020年7月30日(星期四) 上午10:35 > *收件人:* "dev"; > *主题:* Re: [DISCUSS] Apache Spark 3.0.1 Release > > Hi all, > > > > Discussion around 3.0.1 seems to have trickled away. Wha

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

Jason. *From:* 郑瑞峰 mailto:ruife...@foxmail.com>> *Sent:* Wednesday, 5 August 2020 12:54 PM *To:* Jason Moore ; Spark dev list mailto:dev@spark.apache.org>> *Subject:* 回复： [DISCUSS] Apache Spark 3.0.1 Release Hi all, I am go

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

PM Jason Moore > wrote: > >> Thank you so much! Any update on getting the RC1 up for vote? >> >> Jason. >> >> >> -- >> *From:* 郑瑞峰 >> *Sent:* Wednesday, 5 August 2020 12:54 PM >> *To:* Jason Moore ; Spark

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

gt; *Sent:* Wednesday, 5 August 2020 12:54 PM > *To:* Jason Moore ; Spark dev list < > dev@spark.apache.org> > *Subject:* 回复： [DISCUSS] Apache Spark 3.0.1 Release > > Hi all, > I am going to prepare the realease of 3.0.1 RC1, with the help of Wenchen. > > > ---

Re: 回复： [DISCUSS] Apache Spark 3.0.1 Release

Thank you so much! Any update on getting the RC1 up for vote? Jason. From: 郑瑞峰 Sent: Wednesday, 5 August 2020 12:54 PM To: Jason Moore ; Spark dev list Subject: 回复： [DISCUSS] Apache Spark 3.0.1 Release Hi all, I am going to prepare the realease of 3.0.1 RC1

回复： [DISCUSS] Apache Spark 3.0.1 Release

Hi all, I am going to prepare the realease of 3.0.1 RC1, with the help of Wenchen. -- 原始邮件 -- 发件人: "Jason Moore"

Re: [DISCUSS] Apache Spark 3.0.1 Release

>> > I can take care of 2.4.7 unless someone else wants to do it. >> >>>>>>> > >> >>>>>>> > On Tue, Jun 30, 2020 at 8:29 PM Jason Moore < >> jason.mo...@quantium.com.au> wrote: >> >>>>>>> >&

Re: [DISCUSS] Apache Spark 3.0.1 Release

Yamamuro Date: Wednesday, 15 July 2020 at 9:00 am To: Shivaram Venkataraman Cc: "dev@spark.apache.org" Subject: Re: [DISCUSS] Apache Spark 3.0.1 Release > Just wanted to check if there are any blockers that we are still waiting for > to start the new release process. I don't see any

Re: [DISCUSS] Apache Spark 3.0.1 Release

ore < > jason.mo...@quantium.com.au> wrote: > >>>>>>> >> > >>>>>>> >> Hi all, > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>&

Re: [DISCUSS] Apache Spark 3.0.1 Release

;> >>>>>>> >> Hi all, >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> Could I get some input on the severity of this one that I found >>>>>>> &g

Re: [DISCUSS] Apache Spark 3.0.1 Release

gt; >> >>>>>> >> Could I get some input on the severity of this one that I found >>>>>> yesterday? If that’s a correctness issue, should it block this patch? >>>>>> Let >>>>>> me know under the ticket if there’s m

Re: [DISCUSS] Apache Spark 3.0.1 Release

one that I found >>>>> yesterday? If that’s a correctness issue, should it block this patch? >>>>> Let >>>>> me know under the ticket if there’s more info that I can provide to help. >>>>> >> >>>>> >>

Re: [DISCUSS] Apache Spark 3.0.1 Release

Thanks Dongjoon, you're right. I referenced to a wrong caused-by jira link in https://issues.apache.org/jira/browse/SPARK-32307. Bests, Yi On Wed, Jul 15, 2020 at 3:03 AM Dongjoon Hyun wrote: > Hi, Yi. > > Could you explain why you think that is a blocker? For the given example > from the JIRA

Re: [DISCUSS] Apache Spark 3.0.1 Release

> Just wanted to check if there are any blockers that we are still waiting for to start the new release process. I don't see any on-going blocker in my area. Thanks for the notification. Bests, Tkaeshi On Wed, Jul 15, 2020 at 4:03 AM Dongjoon Hyun wrote: > Hi, Yi. > > Could you explain why you

Re: [DISCUSS] Apache Spark 3.0.1 Release

Hi, Yi. Could you explain why you think that is a blocker? For the given example from the JIRA description, spark.udf.register("key", udf((m: Map[String, String]) => m.keys.head.toInt)) Seq(Map("1" -> "one", "2" -> "two")).toDF("a").createOrReplaceTempView("t") checkAnswer(sql("SELECT key(a)

Re: [DISCUSS] Apache Spark 3.0.1 Release

This probably be a blocker: https://issues.apache.org/jira/browse/SPARK-32307 On Tue, Jul 14, 2020 at 11:13 PM Sean Owen wrote: > https://issues.apache.org/jira/browse/SPARK-32234 ? > > On Tue, Jul 14, 2020 at 9:57 AM Shivaram Venkataraman > wrote: > > > > Hi all > > > > Just wanted to check

Re: [DISCUSS] Apache Spark 3.0.1 Release

https://issues.apache.org/jira/browse/SPARK-32234 ? On Tue, Jul 14, 2020 at 9:57 AM Shivaram Venkataraman wrote: > > Hi all > > Just wanted to check if there are any blockers that we are still waiting for > to start the new release process. > > Thanks > Shivaram >

Re: [DISCUSS] Apache Spark 3.0.1 Release

Hi all Just wanted to check if there are any blockers that we are still waiting for to start the new release process. Thanks Shivaram On Sun, Jul 5, 2020, 06:51 wuyi wrote: > Ok, after having another look, I think it only affects local cluster deploy > mode, which is for testing only. > > >

Re: [DISCUSS] Apache Spark 3.0.1 Release

Ok, after having another look, I think it only affects local cluster deploy mode, which is for testing only. wuyi wrote > Please also includes https://issues.apache.org/jira/browse/SPARK-32120 in > Spark 3.0.1. It's a regression compares to Spark 3.0.0-preview2. > > Thanks, > Yi Wu > > >

Re: [DISCUSS] Apache Spark 3.0.1 Release

Please also includes https://issues.apache.org/jira/browse/SPARK-32120 in Spark 3.0.1. It's a regression compares to Spark 3.0.0-preview2. Thanks, Yi Wu Yuanjian Li wrote > Hi dev-list, > > I’m writing this to raise the discussion about Spark 3.0.1 feasibility > since 4 blocker issues were

Re: [DISCUSS] Apache Spark 3.0.1 Release

>>>> >> >>>> >> https://issues.apache.org/jira/browse/SPARK-32136 >>>> >> >>>> >> >>>> >> >>>> >> Thanks, >>>> >> >>>> >> Jason. >>>&g

Re: [DISCUSS] Apache Spark 3.0.1 Release

;> >> >>> >> >>> >> Thanks, >>> >> >>> >> Jason. >>> >> >>> >> >>> >> >>> >> From: Jungtaek Lim >>> >> Date: Wednesday, 1 July 2020 at 10:20 am >>>

Re: [DISCUSS] Apache Spark 3.0.1 Release

gt;> >> >> >> >> >> >> >> Thanks, >> >> >> >> Jason. >> >> >> >> >> >> >> >> From: Jungtaek Lim >> >> Date: Wednesday, 1 July 2020 at 10:20 am >> >> To: Shiva

Re: [DISCUSS] Apache Spark 3.0.1 Release

gt; Date: Wednesday, 1 July 2020 at 10:20 am > >> To: Shivaram Venkataraman > >> Cc: Prashant Sharma , 郑瑞峰 , > Gengliang Wang , gurwls223 < > gurwls...@gmail.com>, Dongjoon Hyun , Jules > Damji , Holden Karau , Reynold > Xin , Yuanjian Li , " > d

Re: [DISCUSS] Apache Spark 3.0.1 Release

ma , 郑瑞峰 , >> Gengliang Wang , gurwls223 >> , Dongjoon Hyun , Jules Damji >> , Holden Karau , Reynold Xin >> , Yuanjian Li , >> "dev@spark.apache.org" , Takeshi Yamamuro >> >> Subject: Re: [DISCUSS] Apache Spark 3.0.1 Release >> >

Re: [DISCUSS] Apache Spark 3.0.1 Release

*Shivaram Venkataraman >> *Cc: *Prashant Sharma , 郑瑞峰 , >> Gengliang Wang , gurwls223 < >> gurwls...@gmail.com>, Dongjoon Hyun , Jules >> Damji , Holden Karau , >> Reynold Xin , Yuanjian Li , >> "dev@spark.apache.org" , Takeshi Yamamuro < &g

Re: [DISCUSS] Apache Spark 3.0.1 Release

ivaram Venkataraman > *Cc: *Prashant Sharma , 郑瑞峰 , > Gengliang Wang , gurwls223 < > gurwls...@gmail.com>, Dongjoon Hyun , Jules > Damji , Holden Karau , Reynold > Xin , Yuanjian Li , " > dev@spark.apache.org" , Takeshi Yamamuro < > linguin@gmail.com> &g

Re: [DISCUSS] Apache Spark 3.0.1 Release

gt;>;"Reynold Xin"mailto:r...@databricks.com>>;"Shivaram Venkataraman"mailto:shiva...@eecs.berkeley.edu>>;"Yuanjian Li"mailto:xyliyuanj...@gmail.com>>;"Spark dev list"mailto:dev@spark.apache.org>>;"Takeshi Yamamuro&quo

Re: [DISCUSS] Apache Spark 3.0.1 Release

t;>> *发送时间:* 2020年6月24日(星期三) 下午4:15 >>> *收件人:* "Hyukjin Kwon"; >>> *抄送:* "Dongjoon Hyun";"Jungtaek Lim"< >>> kabhwan.opensou...@gmail.com>;"Jules Damji";"Holden >>> Karau";"Reynold Xin";"Shiv

Re: [DISCUSS] Apache Spark 3.0.1 Release

quot;;"Jungtaek Lim"< >> kabhwan.opensou...@gmail.com>;"Jules Damji";"Holden >> Karau";"Reynold Xin";"Shivaram >> Venkataraman";"Yuanjian Li"< >> xyliyuanj...@gmail.com>;"Spark dev list";&

Re: [DISCUSS] Apache Spark 3.0.1 Release

2020年6月24日(星期三) 下午4:15 > *收件人:* "Hyukjin Kwon"; > *抄送:* "Dongjoon Hyun";"Jungtaek Lim"< > kabhwan.opensou...@gmail.com>;"Jules Damji";"Holden > Karau";"Reynold Xin";"Shivaram > Venkataraman";"Yuanjian L

回复： [DISCUSS] Apache Spark 3.0.1 Release

I volunteer to be a release manager of 3.0.1, if nobody is working on this. --原始邮件-- 发件人:"Gengliang Wang"https://issues.apache.org/jira/browse/SPARK-32051and includethe fix into 3.0.1 if possible. Looks like APIs designed to work with Scala 2.11 Java bring

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1, the issues mentioned are really serious. On Tue, Jun 23, 2020 at 7:56 PM Hyukjin Kwon wrote: > +1. > > Just as a note, > - SPARK-31918 is > fixed now, and there's no blocker. - When we build SparkR, we should use > the latest R version at

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1. Just as a note, - SPARK-31918 is fixed now, and there's no blocker. - When we build SparkR, we should use the latest R version at least 4.0.0+. 2020년 6월 24일 (수) 오전 11:20, Dongjoon Hyun 님이 작성: > +1 > > Bests, > Dongjoon. > > On Tue, Jun 23,

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1 Bests, Dongjoon. On Tue, Jun 23, 2020 at 1:19 PM Jungtaek Lim wrote: > +1 on a 3.0.1 soon. > > Probably it would be nice if some Scala experts can take a look at > https://issues.apache.org/jira/browse/SPARK-32051 and include the fix > into 3.0.1 if possible. > Looks like APIs designed to

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1 on a 3.0.1 soon. Probably it would be nice if some Scala experts can take a look at https://issues.apache.org/jira/browse/SPARK-32051 and include the fix into 3.0.1 if possible. Looks like APIs designed to work with Scala 2.11 & Java bring ambiguity in Scala 2.12 & Java. On Wed, Jun 24, 2020

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1 (non-binding) Sent from my iPhone Pardon the dumb thumb typos :) > On Jun 23, 2020, at 11:36 AM, Holden Karau wrote: > >  > +1 on a patch release soon > >> On Tue, Jun 23, 2020 at 10:47 AM Reynold Xin wrote: >> +1 on doing a new patch release soon. I saw some of these issues when >>

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1 on a patch release soon On Tue, Jun 23, 2020 at 10:47 AM Reynold Xin wrote: > +1 on doing a new patch release soon. I saw some of these issues when > preparing the 3.0 release, and some of them are very serious. > > > On Tue, Jun 23, 2020 at 8:06 AM, Shivaram Venkataraman < >

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1 on doing a new patch release soon. I saw some of these issues when preparing the 3.0 release, and some of them are very serious. On Tue, Jun 23, 2020 at 8:06 AM, Shivaram Venkataraman < shiva...@eecs.berkeley.edu > wrote: > > > > +1 Thanks Yuanjian -- I think it'll be great to have a

Re: [DISCUSS] Apache Spark 3.0.1 Release

+1 Thanks Yuanjian -- I think it'll be great to have a 3.0.1 release soon. Shivaram On Tue, Jun 23, 2020 at 3:43 AM Takeshi Yamamuro wrote: > > Thanks for the heads-up, Yuanjian! > > > I also noticed branch-3.0 already has 39 commits after Spark 3.0.0. > wow, the updates are so quick. Anyway,

Re: [DISCUSS] Apache Spark 3.0.1 Release

Thanks for the heads-up, Yuanjian! > I also noticed branch-3.0 already has 39 commits after Spark 3.0.0. wow, the updates are so quick. Anyway, +1 for the release. Bests, Takeshi On Tue, Jun 23, 2020 at 4:59 PM Yuanjian Li wrote: > Hi dev-list, > > I’m writing this to raise the discussion

[DISCUSS] Apache Spark 3.0.1 Release

Hi dev-list, I’m writing this to raise the discussion about Spark 3.0.1 feasibility since 4 blocker issues were found after Spark 3.0.0: 1. [SPARK-31990] The state store compatibility broken will cause a correctness issue when