While that's not happening soon (2.13 isn't out), note that some of the changes to collections will be fairly breaking changes.
https://issues.apache.org/jira/browse/SPARK-25075 https://docs.scala-lang.org/overviews/core/collections-migration-213.html Some of this may impact a public API, so may need to start proactively fixing stuff for 2.13 before 3.0 comes out where possible. Here's an example: Traversable goes away. We have a method SparkConf.setAll(Traversable). We can't support 2.13 while that still exists. Of course, we can decide to deprecate it with replacement (use Iterable) and remove it in the version that supports 2.13. But that would mean a little breaking change, and we either have to accept that for a future 3.x release, or it waits until 4.x. I wanted to put that on the radar now to gather opinions about whether this will probably be acceptable, or whether we really need to get methods like that changed before 3.0. Also: there's plenty of straightforward but medium-sized changes we can make now in anticipation of 2.13 support, like, make the type of Seq we use everywhere explicit (will be good for a like 1000 file change I'm sure). Or see if we can swap out Traversable everywhere. Remove MutableList, etc. I was going to start fiddling with that unless it just sounds too disruptive. --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
