[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944879#comment-16944879
]
Sean R. Owen commented on SPARK-27681:
--
I looked at this one more time today after clearing out some earlier 2.13
related issues.
I'm pretty sure this is what we should do, all in all, which is much more in
line with [~vanzin]'s take:
- Generally, don't change all the Spark {{Seq}} usages in methods and return
values. Just too much change.
- Definitely fix all the compile errors within Spark that result in 2.13, by
adding {{.toSeq}} or {{.toMap}} where applicable to get immutable versions from
mutable Seqs, and Maps from MapViews (similar but different change in 2.13).
This is what SPARK-29292 covers.
- ... and that may be it. Maybe we find a few corner cases where a public API
method really does need to fix its Seq type to make sense, but I hadn't found
it yet after fixing most of core
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean R. Owen
>Assignee: Sean R. Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839674#comment-16839674
]
Marcelo Vanzin commented on SPARK-27681:
bq. 2043 in method parameters, 1488 in method return types
Just a small fraction of those are public APIs, I'll bet. So probably not that
huge of a problem.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839341#comment-16839341
]
Lukas Rytz commented on SPARK-27681:
Uh, I just did "find usages" in IntelliJ for the scala.Seq type alias: 2043 in
method parameters, 1488 in method return types, 4525 total. So this is not
going to be very easy.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839280#comment-16839280
]
Lukas Rytz commented on SPARK-27681:
I think the best solution for 2.13, if we forget about 2.12, would be to look
at uses of {{scala.Seq}} individually.
Methods with a Seq parameter can continue to accept {{collection.Seq}} if that
makes sense, i.e., if the implementation can work fine with mutable Seqs. If
that's not the case and {{collection.Seq}} was used accidentally, the method
should probably be deprecated.
When used as result type, it would be better to replace it by {{immutable.Seq}}
if the result is actually immutable. A more precise type is good in general,
and it makes sure that the result is still compatible with {{scala.Seq}}, which
a user might expect, as noted by [~vanzin] above.
Unfortunately, changing the result type is *not* binary compatible, so it
cannot be done in a 2.12 minor release of Spark. If there will be separate
source files for 2.12 and 2.13, you could use a type alias that points to
{{collection.Seq}} in 2.12, and to {{immutable.Seq}} in 2.13, and use it in
these cases. Otherwise, the only solution is to use {{collection.Seq}}
everywhere.
Or maybe the Spark version that will actually cross-build with 2.13 does need
to be binary compatible on 2.12? In this case, using a more precise return type
would be good for 2.12 as well.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838799#comment-16838799
]
Marcelo Vanzin commented on SPARK-27681:
bq. The case to consider is, roughly, where Seq is an argument not a return
type.
If this change is restricted to that it's probably fine. But it's still a
tricky change. You'll probably have to explicitly use that class in the problem
spots (i.e. no importing at the top level or it may change things you don't
want to change). Then people will have to remember to do that (or reviewers
remember to catch it during review). It just feels a little too brittle.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838781#comment-16838781
]
Sean Owen commented on SPARK-27681:
---
You're right, the change is kinda breaking! we can just accept that, or
explicitly patch over the breaking change where we think it's important. I
agree, using the convenient Seq alias is just going to keep happening, but
doesn't mean we may not be explicit in cases where it matters enough, in a
public API here or there.
(We won't be able to avoid fixing Spark methods that need to return a
non-immutable Seq and return scala.Seq now, so there's at least that.)
Yep your example is just what we don't want to do. It just breaks things. The
case to consider is, roughly, where Seq is an argument not a return type.
Yes, mutable and immutable Seqs have always been separate subtypes of
scala.collection.Seq, and neither is or will be assignable to the other. Your
example works in 2.12 but not in 2.13. But scala.collection.Seq is the parent
of both mutable and immutable Seqs.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838766#comment-16838766
]
Marcelo Vanzin commented on SPARK-27681:
bq. But, user code that passes a non-immutable Seq to a Spark API that accepts
scala.Seq now will no longer compile in 2.13.
That's true, but isn't that what the Scala developers intend anyway? That will
be true for any Scala code, not just Spark. That's assuming they didn't add any
implicit conversion from a mutable Seq to an immutable one, which would solve
that problem.
My problem with your suggestion is that now Spark developers will have to
always remember to import that different {{Seq}} type. And if they don't
remember, probably nothing will break until it's too late to notice. It's a
counter-intuitive change for developers, and I in particular am not seeing a
lot of benefits from it.
Here's an example of your proposed change that would break user code (just ran
it on Scala 2.13-RC1):
{code}
scala> def foo(): scala.collection.Seq[String] = Nil
foo: ()scala.collection.Seq[String]
scala> val s: Seq[String] = foo()
^
error: type mismatch;
found : Seq[String] (in scala.collection)
required: Seq[String] (in scala.collection.immutable)
{code}
So, aside from Spark developers having to remember to use the different {{Seq}}
type, user code might also have to change so that their internal APIs also use
the different type, or things like the above may occur.
BTW I also checked and there's no automatic promotion from mutable to immutable
seqs:
{code}
scala> val s: Seq[String] = scala.collection.mutable.ArrayBuffer[String]()
^
error: type mismatch;
found : scala.collection.mutable.ArrayBuffer[String]
required: Seq[String]
{code}
So I sort of understand your desire to keep things more similar, but I'm not
really seeing the advantages you see.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. In many cases, this will be
> fine, as Spark users using Scala 2.13 will also have this changed alias. In
> some cases it may be undesirable, as it will cause some code to compile in
> 2.12 but not in 2.13. In some cases, making the type {{scala.collection.Seq}}
> explicit so that it doesn't vary can help avoid this, so that Spark apps
> might cross-compile for 2.12 and 2.13 with the same source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838744#comment-16838744
]
Sean Owen commented on SPARK-27681:
---
OK, take it down a notch; we are largely in agreement, but there's a little
more to it.
Right, so if all Spark's {{scala.Seq}} in APIs switch their implementation,
then so do callers' {{scala.Seq}}, so who cares? The problem is that Spark, and
user code, _do_ sometimes use Seqs that aren't known to be immutable. Your
point is specifically that {{def foo(): scala.Seq}} and {{val a: scala.Seq =
foo()}} work fine in 2.12 and 2.13, but changing to {{def foo():
scala.collection.Seq}} works in 2.12 but not 2.13, unless you use the type more
explicitly. Yeah, let's avoid that.
There's a mirror-image issue. A Spark method that returns a non-immutable Seq
as {{scala.Seq}} works now but won't compile in 2.13. OK, but that's Spark's
problem; we either change the Spark API or change its implementation, though I
doubt it'll be possible in all cases.
But, user code that passes a non-immutable Seq to a Spark API that accepts
{{scala.Seq}} now will no longer compile in 2.13. That is if {{def foo(a:
scala.Seq)}}, it works with any Seq in 2.12 but fails to compile for
{{scala.collection.Seq}} arguments in 2.13. Whereas {{def foo(a:
scala.collection.Seq)}} continues to work in both.
Fixing the type to {{scala.collection.Seq}} isn't so much about making sure
it's the same type in 2.12 and 2.13, but fixing this at a type that can accept
any Seq as it does already, in 2.13. The cases that may have to change are the
covariant or contravariant cases, I forget which is which. Definitely not all
of them.
Which ones? I'd punt that to later, but I think it'll be method arguments that
are Seqs and Spark methods whose return type don't compile otherwise in 2.13.
Oh, and the description definitely has to be updated, I'll do that.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838715#comment-16838715
]
Marcelo Vanzin commented on SPARK-27681:
bq. varargs methods in Scala will change their signature as they are
implemented with Seq
Again, why is that a problem? That will only happen in the 2.13 version of
Spark.There are absolutely no backwards binary compatibility guarantees in that
case.
The 2.12 build will keep working just like it does today.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838693#comment-16838693
]
Marcelo Vanzin commented on SPARK-27681:
bq. scala.Seq is just a type def for scala.collection.Seq
In 2.12. If you force all the API in 2.13 to explicitly use
{{scala.collection.Seq}}, all the existing user code that does not explicitly
import that class will now be using a different type, and may run into issues.
Again, what I'm asking is why it is important for a method {{def foo(s:
Seq[Any])}} to be exposed as {{def foo(s: scala.collection.Seq[Any])}} in both
Scala 2.12 and 2.13, instead of using whatever is the default {{Seq}} alias for
the respective Scala version.
I really don't know how to word my question any more clearly.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837907#comment-16837907
]
Sean Owen commented on SPARK-27681:
---
One last thought for now ... when {{Seq}} becomes {{immutable}} we may find
some code doesn't compile as it is only returning {{collection.Seq}}. We may at
least have to update a lot of methods to explicitly specify the more generic
{{Seq}} anyway. But can keep it to just that much change.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837854#comment-16837854
]
Sean Owen commented on SPARK-27681:
---
I don't think it would change source compatibility; {{scala.Seq}} is just a
type def for {{scala.collection.Seq}}, so it's a no-op w.r.t. the source and
binary. Or am I missing something i really should know about how type def work?
certainly wouldn't be worth breaking source compatibility; the point if
anything would be to keep a little binary compatibility. If we're not concerned
about Scala binary compatibility, I'm imagining maybe the corner case of a Java
lib that refers to {{Seq}} in its API? that's a weak reason. At best I would
consider this later.
Counter-argument: we should be changing the API to explicitly return immutable
Seqs anyway; that's the motivation for the Scala 2.13 change to begin with.
There's another wrinkle here which I don't think we can help anyway. varargs
methods in Scala will change their signature as they are implemented with Seq.
That we can't control.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837753#comment-16837753
]
Marcelo Vanzin commented on SPARK-27681:
bq. we don't need to do this until we support Scala 2.13
I guess my question is why do we need to do it? Your explanation above
indicates that the {{Seq}} alias in 2.13 will just be a different type. That's
fine since 2.12 and 2.13 builds are not binary compatible anyway.
I'd understand this change if the {{Seq}} alias was being removed altogether
and you'd need an explicit import, but that doesn't seem to be the case.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837752#comment-16837752
]
Marcelo Vanzin commented on SPARK-27681:
Also the change you're proposing likely would break source compatibility, which
would make upgrading to Spark 3 harder than just a recompile...
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837751#comment-16837751
]
Sean Owen commented on SPARK-27681:
---
To my understanding, we don't need to do this until we support Scala 2.13. We
_can_ do it now, as it just makes explicit the type of {{Seq}} that is already
used by using the type {{scala.Seq}}. We don't actually want or need to change
to use {{scala.collection.immutable.Seq}} over {{scala.collection.Seq}} in
theory or practice. But indeed, that change wouldn't be so bad anyway as 2.13
builds aren't expected to be binary compatible with 2.12. I'd only argue for it
as the API already doesn't promise immutable or mutable {{Seq}} and this change
would maintain that, but yeah it's not even strictly required.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias
[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837749#comment-16837749
]
Marcelo Vanzin commented on SPARK-27681:
OOC, what happens if we do nothing?
As far as I can see, Scala 2.12 builds will have scala.Seq in the API, Scala
2.13 builds will have scala.collection.immutable.Seq, and that should be fine,
right, since those builds are not binary compatible with each other anyway?
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
>Affects Versions: 3.0.0
>Reporter: Sean Owen
>Assignee: Sean Owen
>Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
