Re: Moving Scala 2.12 forward one step

[Matei Zaharia]

That would be awesome. I’m not sure whether we want 3.0 to be right after 2.3 
(I guess this Scala issue is one reason to start discussing that), but even if 
we do, I imagine that wouldn’t be out for at least 4-6 more months after 2.3, 
and that’s a long time to go without Scala 2.12 support. If we decide to do 2.4 
next instead, that’s even longer.

Matei

> On Sep 1, 2017, at 1:52 AM, Sean Owen  wrote:
> 
> OK, what I'll do is focus on some changes that can be merged to master 
> without impacting the 2.11 build (e.g. putting kafka-0.8 behind a profile, 
> maybe, or adding the 2.12 REPL). Anything that is breaking, we can work on in 
> a series of open PRs, or maybe a branch, yea. It's unusual but might be 
> worthwhile.
> 
> On Fri, Sep 1, 2017 at 9:44 AM Matei Zaharia  wrote:
> If the changes aren’t that hard, I think we should also consider building a 
> Scala 2.12 version of Spark 2.3 in a separate branch. I’ve definitely seen 
> concerns from some large Scala users that Spark isn’t supporting 2.12 soon 
> enough. I thought SPARK-14220 was blocked mainly because the changes are 
> hard, but if not, maybe we can release such a branch sooner.
> 
> Matei
> 
> > On Aug 31, 2017, at 3:59 AM, Sean Owen  wrote:
> >
> > I don't think there's a target. The changes aren't all that hard (see the 
> > SPARK-14220 umbrella) but there are some changes that are hard or 
> > impossible without changing key APIs, as far as we can see. That would 
> > suggest 3.0.
> >
> > One motivation I have here for getting it as far as possible otherwise is 
> > so people could, if they wanted, create a 2.12 build themselves without 
> > much work even if it were not supported upstream. This particular change is 
> > a lot of the miscellaneous stuff you'd have to fix to get to that point.
> >
> 


-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Moving Scala 2.12 forward one step

[Sean Owen]

OK, what I'll do is focus on some changes that can be merged to master
without impacting the 2.11 build (e.g. putting kafka-0.8 behind a profile,
maybe, or adding the 2.12 REPL). Anything that is breaking, we can work on
in a series of open PRs, or maybe a branch, yea. It's unusual but might be
worthwhile.

On Fri, Sep 1, 2017 at 9:44 AM Matei Zaharia 
wrote:

> If the changes aren’t that hard, I think we should also consider building
> a Scala 2.12 version of Spark 2.3 in a separate branch. I’ve definitely
> seen concerns from some large Scala users that Spark isn’t supporting 2.12
> soon enough. I thought SPARK-14220 was blocked mainly because the changes
> are hard, but if not, maybe we can release such a branch sooner.
>
> Matei
>
> > On Aug 31, 2017, at 3:59 AM, Sean Owen  wrote:
> >
> > I don't think there's a target. The changes aren't all that hard (see
> the SPARK-14220 umbrella) but there are some changes that are hard or
> impossible without changing key APIs, as far as we can see. That would
> suggest 3.0.
> >
> > One motivation I have here for getting it as far as possible otherwise
> is so people could, if they wanted, create a 2.12 build themselves without
> much work even if it were not supported upstream. This particular change is
> a lot of the miscellaneous stuff you'd have to fix to get to that point.
> >
>
>

Re: Moving Scala 2.12 forward one step

[Matei Zaharia]

If the changes aren’t that hard, I think we should also consider building a 
Scala 2.12 version of Spark 2.3 in a separate branch. I’ve definitely seen 
concerns from some large Scala users that Spark isn’t supporting 2.12 soon 
enough. I thought SPARK-14220 was blocked mainly because the changes are hard, 
but if not, maybe we can release such a branch sooner.

Matei

> On Aug 31, 2017, at 3:59 AM, Sean Owen  wrote:
> 
> I don't think there's a target. The changes aren't all that hard (see the 
> SPARK-14220 umbrella) but there are some changes that are hard or impossible 
> without changing key APIs, as far as we can see. That would suggest 3.0.
> 
> One motivation I have here for getting it as far as possible otherwise is so 
> people could, if they wanted, create a 2.12 build themselves without much 
> work even if it were not supported upstream. This particular change is a lot 
> of the miscellaneous stuff you'd have to fix to get to that point.
> 
> On Thu, Aug 31, 2017 at 11:04 AM Saisai Shao  wrote:
> Hi Sean,
> 
> Do we have a planned target version for Scala 2.12 support? Several other 
> projects like Zeppelin, Livy which rely on Spark repl also require changes to 
> support this Scala 2.12.
> 
> Thanks
> Jerry
> 
> On Thu, Aug 31, 2017 at 5:55 PM, Sean Owen  wrote:
> No, this doesn't let Spark build and run on 2.12. It makes changes that will 
> be required though, the ones that are really no loss to the current 2.11 
> build.
> 
> 
> On Thu, Aug 31, 2017, 10:48 Denis Bolshakov  wrote:
> Hello,
> 
> Sounds amazing. Is there any improvements in benchmarks?
> 
> 
> On 31 August 2017 at 12:25, Sean Owen  wrote:
> Calling attention to the question of Scala 2.12 again for moment. I'd like to 
> make a modest step towards support. Have a look again, if you would, at 
> SPARK-14280:
> 
> https://github.com/apache/spark/pull/18645
> 
> This is a lot of the change for 2.12 that doesn't break 2.11, and really 
> doesn't add any complexity. It's mostly dependency updates and clarifying 
> some code. Other items like dealing with Kafka 0.8 support, the 2.12 REPL, 
> etc, are not  here.
> 
> So, this still doesn't result in a working 2.12 build but it's most of the 
> miscellany that will be required.
> 
> I'd like to merge it but wanted to flag it for feedback as it's not trivial.
> 
> 
> 
> -- 
> //with Best Regards
> --Denis Bolshakov
> e-mail: bolshakov.de...@gmail.com
> 


-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Moving Scala 2.12 forward one step

[Sean Owen]

I don't think there's a target. The changes aren't all that hard (see the
SPARK-14220 umbrella) but there are some changes that are hard or
impossible without changing key APIs, as far as we can see. That would
suggest 3.0.

One motivation I have here for getting it as far as possible otherwise is
so people could, if they wanted, create a 2.12 build themselves without
much work even if it were not supported upstream. This particular change is
a lot of the miscellaneous stuff you'd have to fix to get to that point.

On Thu, Aug 31, 2017 at 11:04 AM Saisai Shao  wrote:

> Hi Sean,
>
> Do we have a planned target version for Scala 2.12 support? Several other
> projects like Zeppelin, Livy which rely on Spark repl also require changes
> to support this Scala 2.12.
>
> Thanks
> Jerry
>
> On Thu, Aug 31, 2017 at 5:55 PM, Sean Owen  wrote:
>
>> No, this doesn't let Spark build and run on 2.12. It makes changes that
>> will be required though, the ones that are really no loss to the current
>> 2.11 build.
>>
>> On Thu, Aug 31, 2017, 10:48 Denis Bolshakov 
>> wrote:
>>
>>> Hello,
>>>
>>> Sounds amazing. Is there any improvements in benchmarks?
>>>
>>>
>>> On 31 August 2017 at 12:25, Sean Owen  wrote:
>>>
 Calling attention to the question of Scala 2.12 again for moment. I'd
 like to make a modest step towards support. Have a look again, if you
 would, at SPARK-14280:

 https://github.com/apache/spark/pull/18645

 This is a lot of the change for 2.12 that doesn't break 2.11, and
 really doesn't add any complexity. It's mostly dependency updates and
 clarifying some code. Other items like dealing with Kafka 0.8 support, the
 2.12 REPL, etc, are not  here.

 So, this still doesn't result in a working 2.12 build but it's most of
 the miscellany that will be required.

 I'd like to merge it but wanted to flag it for feedback as it's not
 trivial.

>>>
>>>
>>>
>>> --
>>> //with Best Regards
>>> --Denis Bolshakov
>>> e-mail: bolshakov.de...@gmail.com
>>>
>>
>

Re: Moving Scala 2.12 forward one step

[Saisai Shao]

Hi Sean,

Do we have a planned target version for Scala 2.12 support? Several other
projects like Zeppelin, Livy which rely on Spark repl also require changes
to support this Scala 2.12.

Thanks
Jerry

On Thu, Aug 31, 2017 at 5:55 PM, Sean Owen  wrote:

> No, this doesn't let Spark build and run on 2.12. It makes changes that
> will be required though, the ones that are really no loss to the current
> 2.11 build.
>
> On Thu, Aug 31, 2017, 10:48 Denis Bolshakov 
> wrote:
>
>> Hello,
>>
>> Sounds amazing. Is there any improvements in benchmarks?
>>
>>
>> On 31 August 2017 at 12:25, Sean Owen  wrote:
>>
>>> Calling attention to the question of Scala 2.12 again for moment. I'd
>>> like to make a modest step towards support. Have a look again, if you
>>> would, at SPARK-14280:
>>>
>>> https://github.com/apache/spark/pull/18645
>>>
>>> This is a lot of the change for 2.12 that doesn't break 2.11, and really
>>> doesn't add any complexity. It's mostly dependency updates and clarifying
>>> some code. Other items like dealing with Kafka 0.8 support, the 2.12 REPL,
>>> etc, are not  here.
>>>
>>> So, this still doesn't result in a working 2.12 build but it's most of
>>> the miscellany that will be required.
>>>
>>> I'd like to merge it but wanted to flag it for feedback as it's not
>>> trivial.
>>>
>>
>>
>>
>> --
>> //with Best Regards
>> --Denis Bolshakov
>> e-mail: bolshakov.de...@gmail.com
>>
>

Re: Moving Scala 2.12 forward one step

[Sean Owen]

No, this doesn't let Spark build and run on 2.12. It makes changes that
will be required though, the ones that are really no loss to the current
2.11 build.

On Thu, Aug 31, 2017, 10:48 Denis Bolshakov 
wrote:

> Hello,
>
> Sounds amazing. Is there any improvements in benchmarks?
>
>
> On 31 August 2017 at 12:25, Sean Owen  wrote:
>
>> Calling attention to the question of Scala 2.12 again for moment. I'd
>> like to make a modest step towards support. Have a look again, if you
>> would, at SPARK-14280:
>>
>> https://github.com/apache/spark/pull/18645
>>
>> This is a lot of the change for 2.12 that doesn't break 2.11, and really
>> doesn't add any complexity. It's mostly dependency updates and clarifying
>> some code. Other items like dealing with Kafka 0.8 support, the 2.12 REPL,
>> etc, are not  here.
>>
>> So, this still doesn't result in a working 2.12 build but it's most of
>> the miscellany that will be required.
>>
>> I'd like to merge it but wanted to flag it for feedback as it's not
>> trivial.
>>
>
>
>
> --
> //with Best Regards
> --Denis Bolshakov
> e-mail: bolshakov.de...@gmail.com
>

Re: Moving Scala 2.12 forward one step

[Denis Bolshakov]

Hello,

Sounds amazing. Is there any improvements in benchmarks?


On 31 August 2017 at 12:25, Sean Owen  wrote:

> Calling attention to the question of Scala 2.12 again for moment. I'd like
> to make a modest step towards support. Have a look again, if you would, at
> SPARK-14280:
>
> https://github.com/apache/spark/pull/18645
>
> This is a lot of the change for 2.12 that doesn't break 2.11, and really
> doesn't add any complexity. It's mostly dependency updates and clarifying
> some code. Other items like dealing with Kafka 0.8 support, the 2.12 REPL,
> etc, are not  here.
>
> So, this still doesn't result in a working 2.12 build but it's most of the
> miscellany that will be required.
>
> I'd like to merge it but wanted to flag it for feedback as it's not
> trivial.
>



-- 
//with Best Regards
--Denis Bolshakov
e-mail: bolshakov.de...@gmail.com