Re: Release Apache Spark 2.4.4

Thank you, DB, Takeshi, Hyukjin, Sean, Kazuaki, Holden, Wenchen! I'll create tag for 2.4.4-rc1 next Monday. For SPARK-27234, it looks like that to me, too. Thanks, Dongjoon. On Wed, Aug 14, 2019 at 9:13 AM Holden Karau wrote: > That looks like more of a feature than a bug fix unless I’m missi

Re: Release Apache Spark 2.4.4

That looks like more of a feature than a bug fix unless I’m missing something? On Tue, Aug 13, 2019 at 11:58 PM Hyukjin Kwon wrote: > Adding Shixiong > > WDYT? > > 2019년 8월 14일 (수) 오후 2:30, Terry Kim 님이 작성: > >> Can the following be included? >> >> [SPARK-27234][SS][PYTHON] Use InheritableThread

Re: Release Apache Spark 2.4.4

Adding Shixiong WDYT? 2019년 8월 14일 (수) 오후 2:30, Terry Kim 님이 작성: > Can the following be included? > > [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in > EpochTracker (to support Python UDFs) > > > Thanks, > Terry > > On Tue, A

Re: Release Apache Spark 2.4.4

Can the following be included? [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) Thanks, Terry On Tue, Aug 13, 2019 at 10:24 PM Wenchen Fan wrote: > +1 > > On Wed, Aug 14, 2019 at 12:52 P

Re: Release Apache Spark 2.4.4

+1 On Wed, Aug 14, 2019 at 12:52 PM Holden Karau wrote: > +1 > Does anyone have any critical fixes they’d like to see in 2.4.4? > > On Tue, Aug 13, 2019 at 5:22 PM Sean Owen wrote: > >> Seems fine to me if there are enough valuable fixes to justify another >> release. If there are any other imp

Re: Release Apache Spark 2.4.4

+1 Does anyone have any critical fixes they’d like to see in 2.4.4? On Tue, Aug 13, 2019 at 5:22 PM Sean Owen wrote: > Seems fine to me if there are enough valuable fixes to justify another > release. If there are any other important fixes imminent, it's fine to > wait for those. > > > On Tue, A

RE: Release Apache Spark 2.4.4

Thanks, Dongjoon! +1 Kazuaki Ishizaki, From: Hyukjin Kwon To: Takeshi Yamamuro Cc: Dongjoon Hyun , dev , User Date: 2019/08/14 09:21 Subject:[EXTERNAL] Re: Release Apache Spark 2.4.4 +1 2019년 8월 14일 (수) 오전 9:13, Takeshi Yamamuro 님 이 작성: Hi, Thanks for your

Re: Release Apache Spark 2.4.4

Seems fine to me if there are enough valuable fixes to justify another release. If there are any other important fixes imminent, it's fine to wait for those. On Tue, Aug 13, 2019 at 6:16 PM Dongjoon Hyun wrote: > > Hi, All. > > Spark 2.4.3 was released three months ago (8th May). > As of today (

Re: Release Apache Spark 2.4.4

+1 2019년 8월 14일 (수) 오전 9:13, Takeshi Yamamuro 님이 작성: > Hi, > > Thanks for your notification, Dongjoon! > I put some links for the other committers/PMCs to access the info easily: > > A commit list in github from the last release: > https://github.com/apache/spark/compare/5ac2014e6c118fbeb1fe8e5c8

Re: Release Apache Spark 2.4.4

Hi, Thanks for your notification, Dongjoon! I put some links for the other committers/PMCs to access the info easily: A commit list in github from the last release: https://github.com/apache/spark/compare/5ac2014e6c118fbeb1fe8e5c8064c4a8ee9d182a...branch-2.4 A issue list in jira: https://issues.a

Re: Release Apache Spark 2.4.4

+1 On Tue, Aug 13, 2019 at 4:16 PM Dongjoon Hyun wrote: > > Hi, All. > > Spark 2.4.3 was released three months ago (8th May). > As of today (13th August), there are 112 commits (75 JIRAs) in `branch-24` > since 2.4.3. > > It would be great if we can have Spark 2.4.4. > Shall we start `2.4.4 RC1`

Release Apache Spark 2.4.4

Hi, All. Spark 2.4.3 was released three months ago (8th May). As of today (13th August), there are 112 commits (75 JIRAs) in `branch-24` since 2.4.3. It would be great if we can have Spark 2.4.4. Shall we start `2.4.4 RC1` next Monday (19th August)? Last time, there was a request for K8s issue a

Re: Re: Release Apache Spark 2.4.4 before 3.0.0

r@spark.apache.org>, Apache Spark PMC > Date: 2019/07/13 07:18 > Subject:[EXTERNAL] Re: Release Apache Spark 2.4.4 before 3.0.0 > -- > > > > Thank you, Jacek. > > BTW, I added `@private` since we need PMC's help to make

Re: Re: Release Apache Spark 2.4.4 before 3.0.0

TERNAL] Re: Release Apache Spark 2.4.4 before 3.0.0 Thank you, Jacek. BTW, I added `@private` since we need PMC's help to make an Apache Spark release. Can I get more feedbacks from the other PMC members? Please me know if you have any concerns (e.g. Release date or Release manager?)

Re: Release Apache Spark 2.4.4 before 3.0.0

Hi Dongjoon, Should we also consider fixing https://issues.apache.org/jira/browse/SPARK-27812 before the cut? Best, Stavros On Mon, Jul 15, 2019 at 7:04 PM Dongjoon Hyun wrote: > Hi, Apache Spark PMC members. > > Can we cut Apache Spark 2.4.4 next Monday (22nd July)? > > Bests, > Dongjoon. > >

Re: Release Apache Spark 2.4.4 before 3.0.0

Hi, Apache Spark PMC members. Can we cut Apache Spark 2.4.4 next Monday (22nd July)? Bests, Dongjoon. On Fri, Jul 12, 2019 at 3:18 PM Dongjoon Hyun wrote: > Thank you, Jacek. > > BTW, I added `@private` since we need PMC's help to make an Apache Spark > release. > > Can I get more feedbacks f

Re: Release Apache Spark 2.4.4 before 3.0.0

Thank you, Jacek. BTW, I added `@private` since we need PMC's help to make an Apache Spark release. Can I get more feedbacks from the other PMC members? Please me know if you have any concerns (e.g. Release date or Release manager?) As one of the community members, I assumed the followings (if

Re: Release Apache Spark 2.4.4 before 3.0.0

Hi, Thanks Dongjoon Hyun for stepping up as a release manager! Much appreciated. If there's a volunteer to cut a release, I'm always to support it. In addition, the more frequent releases the better for end users so they have a choice to upgrade and have all the latest fixes or wait. It's their

Re: Release Apache Spark 2.4.4 before 3.0.0

Additionally, one more correctness patch landed yesterday. - SPARK-28015 Check stringToDate() consumes entire input for the and -[m]m formats Bests, Dongjoon. On Tue, Jul 9, 2019 at 10:11 AM Dongjoon Hyun wrote: > Thank you for the reply, Sean. Sure. 2.4.x should be a LTS version

Re: Release Apache Spark 2.4.4 before 3.0.0

Thank you for the reply, Sean. Sure. 2.4.x should be a LTS version. The main reason of 2.4.4 release (before 3.0.0) is to have a better basis for comparison to 3.0.0. For example, SPARK-27798 had an old bug, but its correctness issue is only exposed at Spark 2.4.3. It would be great if we can have

Re: Release Apache Spark 2.4.4 before 3.0.0

We will certainly want a 2.4.4 release eventually. In fact I'd expect 2.4.x gets maintained for longer than the usual 18 months, as it's the last 2.x branch. It doesn't need to happen before 3.0, but could. Usually maintenance releases happen 3-4 months apart and the last one was 2 months ago. If t

Release Apache Spark 2.4.4 before 3.0.0

Hi, All. Spark 2.4.3 was released two months ago (8th May). As of today (9th July), there exist 45 fixes in `branch-2.4` including the following correctness or blocker issues. - SPARK-26038 Decimal toScalaBigInt/toJavaBigInteger not work for decimals not fitting in long - SPARK-26045 Err