Additionally, one more correctness patch landed yesterday.
- SPARK-28015 Check stringToDate() consumes entire input for the yyyy
and yyyy-[m]m formats
Bests,
Dongjoon.
On Tue, Jul 9, 2019 at 10:11 AM Dongjoon Hyun <dongjoon.h...@gmail.com>
wrote:
> Thank you for the reply, Sean. Sure. 2.4.x should be a LTS version.
>
> The main reason of 2.4.4 release (before 3.0.0) is to have a better basis
> for comparison to 3.0.0.
> For example, SPARK-27798 had an old bug, but its correctness issue is only
> exposed at Spark 2.4.3.
> It would be great if we can have a better basis.
>
> Bests,
> Dongjoon.
>
>
> On Tue, Jul 9, 2019 at 9:52 AM Sean Owen <sro...@gmail.com> wrote:
>
>> We will certainly want a 2.4.4 release eventually. In fact I'd expect
>> 2.4.x gets maintained for longer than the usual 18 months, as it's the
>> last 2.x branch.
>> It doesn't need to happen before 3.0, but could. Usually maintenance
>> releases happen 3-4 months apart and the last one was 2 months ago. If
>> these are significant issues, sure. It'll probably be August before
>> it's out anyway.
>>
>> On Tue, Jul 9, 2019 at 11:15 AM Dongjoon Hyun <dongjoon.h...@gmail.com>
>> wrote:
>> >
>> > Hi, All.
>> >
>> > Spark 2.4.3 was released two months ago (8th May).
>> >
>> > As of today (9th July), there exist 45 fixes in `branch-2.4` including
>> the following correctness or blocker issues.
>> >
>> > - SPARK-26038 Decimal toScalaBigInt/toJavaBigInteger not work for
>> decimals not fitting in long
>> > - SPARK-26045 Error in the spark 2.4 release package with the
>> spark-avro_2.11 dependency
>> > - SPARK-27798 from_avro can modify variables in other rows in local
>> mode
>> > - SPARK-27907 HiveUDAF should return NULL in case of 0 rows
>> > - SPARK-28157 Make SHS clear KVStore LogInfo for the blacklist
>> entries
>> > - SPARK-28308 CalendarInterval sub-second part should be padded
>> before parsing
>> >
>> > It would be great if we can have Spark 2.4.4 before we are going to get
>> busier for 3.0.0.
>> > If it's okay, I'd like to volunteer for an 2.4.4 release manager to
>> roll it next Monday. (15th July).
>> > How do you think about this?
>> >
>> > Bests,
>> > Dongjoon.
>>
>
