Additionally, one more correctness patch landed yesterday. - SPARK-28015 Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats
Bests, Dongjoon. On Tue, Jul 9, 2019 at 10:11 AM Dongjoon Hyun <dongjoon.h...@gmail.com> wrote: > Thank you for the reply, Sean. Sure. 2.4.x should be a LTS version. > > The main reason of 2.4.4 release (before 3.0.0) is to have a better basis > for comparison to 3.0.0. > For example, SPARK-27798 had an old bug, but its correctness issue is only > exposed at Spark 2.4.3. > It would be great if we can have a better basis. > > Bests, > Dongjoon. > > > On Tue, Jul 9, 2019 at 9:52 AM Sean Owen <sro...@gmail.com> wrote: > >> We will certainly want a 2.4.4 release eventually. In fact I'd expect >> 2.4.x gets maintained for longer than the usual 18 months, as it's the >> last 2.x branch. >> It doesn't need to happen before 3.0, but could. Usually maintenance >> releases happen 3-4 months apart and the last one was 2 months ago. If >> these are significant issues, sure. It'll probably be August before >> it's out anyway. >> >> On Tue, Jul 9, 2019 at 11:15 AM Dongjoon Hyun <dongjoon.h...@gmail.com> >> wrote: >> > >> > Hi, All. >> > >> > Spark 2.4.3 was released two months ago (8th May). >> > >> > As of today (9th July), there exist 45 fixes in `branch-2.4` including >> the following correctness or blocker issues. >> > >> > - SPARK-26038 Decimal toScalaBigInt/toJavaBigInteger not work for >> decimals not fitting in long >> > - SPARK-26045 Error in the spark 2.4 release package with the >> spark-avro_2.11 dependency >> > - SPARK-27798 from_avro can modify variables in other rows in local >> mode >> > - SPARK-27907 HiveUDAF should return NULL in case of 0 rows >> > - SPARK-28157 Make SHS clear KVStore LogInfo for the blacklist >> entries >> > - SPARK-28308 CalendarInterval sub-second part should be padded >> before parsing >> > >> > It would be great if we can have Spark 2.4.4 before we are going to get >> busier for 3.0.0. >> > If it's okay, I'd like to volunteer for an 2.4.4 release manager to >> roll it next Monday. (15th July). >> > How do you think about this? >> > >> > Bests, >> > Dongjoon. >> >