So I one of the things which we’re planning on backporting internally is
DSv2, which I think being available in a community release in a 2 branch
would be more broadly useful. Anything else on top of that would be on a
case by case basis for if they make an easier upgrade path to 3.

If we’re worried about people using 2.5 as a long term home we could always
mark it with “-transitional” or something similar?

On Fri, Jun 12, 2020 at 4:33 PM Sean Owen <sro...@gmail.com> wrote:

> What is the functionality that would go into a 2.5.0 release, that can't
> be in a 2.4.7 release? I think that's the key question. 2.4.x is the 2.x
> maintenance branch, and I personally could imagine being open to more
> freely backporting a few new features for 2.x users, whereas usually it's
> only bug fixes. Making 2.5.0 implies that 2.5.x is the 2.x maintenance
> branch but there's something too big for a 'normal' maintenance release,
> and I think the whole question turns on what that is.
>
> If it's things like JDK 11 support, I think that is unfortunately fairly
> 'breaking' because of dependency updates. But maybe that's not it.
>
>
> On Fri, Jun 12, 2020 at 4:38 PM Holden Karau <hol...@pigscanfly.ca> wrote:
>
>> Hi Folks,
>>
>> As we're getting closer to Spark 3 I'd like to revisit a Spark 2.5
>> release. Spark 3 brings a number of important changes, and by its nature is
>> not backward compatible. I think we'd all like to have as smooth an upgrade
>> experience to Spark 3 as possible, and I believe that having a Spark 2
>> release some of the new functionality while continuing to support the older
>> APIs and current Scala version would make the upgrade path smoother.
>>
>> This pattern is not uncommon in other Hadoop ecosystem projects, like
>> Hadoop itself and HBase.
>>
>> I know that Ryan Blue has indicated he is already going to be maintaining
>> something like that internally at Netflix, and we'll be doing the same
>> thing at Apple. It seems like having a transitional release could benefit
>> the community with easy migrations and help avoid duplicated work.
>>
>> I want to be clear I'm volunteering to do the work of managing a 2.5
>> release, so hopefully, this wouldn't create any substantial burdens on the
>> community.
>>
>> Cheers,
>>
>> Holden
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Reply via email to