[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501675#comment-17501675 ] Ismael Juma commented on SPARK-25075: - Yeah, I was asking since Kafka plans to drop support for Scala 2.12 next year https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=181308218#content/view/181308218 > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Assignee: Sean R. Owen >Priority: Major > Fix For: 3.2.0 > > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498640#comment-17498640 ] Sean R. Owen commented on SPARK-25075: -- Unknown, though I'd guess not this year. What depends on that though, dropping 2.12 support? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Assignee: Sean R. Owen >Priority: Major > Fix For: 3.2.0 > > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498639#comment-17498639 ] Ismael Juma commented on SPARK-25075: - Is there a very rough timeline for 4.0 or it completely unknown at this stage? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Assignee: Sean R. Owen >Priority: Major > Fix For: 3.2.0 > > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498622#comment-17498622 ] Sean R. Owen commented on SPARK-25075: -- I don't think that's the plan. Certainly not to remove 2.12 before 4.0, but probably not to change defaults soon either. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Assignee: Sean R. Owen >Priority: Major > Fix For: 3.2.0 > > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498615#comment-17498615 ] Yang Jie commented on SPARK-25075: -- Do we plan to switch Scala 2.13 to the default Scala version in Spark 3.3? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Assignee: Sean R. Owen >Priority: Major > Fix For: 3.2.0 > > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403446#comment-17403446
]
Dongjoon Hyun commented on SPARK-25075:
---
FYI, here is the pre-built Apache Spark 3.2.0 RC1 spark-shell (Scala 2.13.5)
result.
{code}
$ bin/spark-shell
...
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 3.2.0
/_/
Using Scala version 2.13.5 (OpenJDK 64-Bit Server VM, Java 1.8.0_302)
Type in expressions to have them evaluated.
Type :help for more information.
Spark context Web UI available at http://localhost:4040
Spark context available as 'sc' (master = local[*], app id =
local-1629769598806).
Spark session available as 'spark'.
{code}
> Build and test Spark against Scala 2.13
> ---
>
> Key: SPARK-25075
> URL: https://issues.apache.org/jira/browse/SPARK-25075
> Project: Spark
> Issue Type: Umbrella
> Components: Build, MLlib, Project Infra, Spark Core, SQL
>Affects Versions: 3.0.0
>Reporter: Guillaume Massé
>Priority: Major
>
> This umbrella JIRA tracks the requirements for building and testing Spark
> against the current Scala 2.13 milestone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403445#comment-17403445
]
Dongjoon Hyun commented on SPARK-25075:
---
[~ekrich]. Yes, SPARK-34218 completed it already for Apache Spark 3.2.0. You
can see the pre-built binaries at Apache Spark 3.2.0 RC1 vote artifacts.
- https://dist.apache.org/repos/dist/dev/spark/v3.2.0-rc1-bin/
{code}
spark-3.2.0-bin-hadoop3.2-scala2.13.tgz
spark-3.2.0-bin-hadoop3.2-scala2.13.tgz.asc
spark-3.2.0-bin-hadoop3.2-scala2.13.tgz.sha512
{code}
The RC1 vote failed and we will proceed to RC2. Please download the Scala 2.13
pre-build artifacts and provide your feedbacks, [~ekrich].
> Build and test Spark against Scala 2.13
> ---
>
> Key: SPARK-25075
> URL: https://issues.apache.org/jira/browse/SPARK-25075
> Project: Spark
> Issue Type: Umbrella
> Components: Build, MLlib, Project Infra, Spark Core, SQL
>Affects Versions: 3.0.0
>Reporter: Guillaume Massé
>Priority: Major
>
> This umbrella JIRA tracks the requirements for building and testing Spark
> against the current Scala 2.13 milestone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403305#comment-17403305 ] Eric Richardson commented on SPARK-25075: - Is it possible to have both the 2.12 and 2.13 versions pre-built for download? I think this could accelerate adoption to 2.13 and thus Scala 3. It is fine if the 2.12 is considered production and 2.13 is experimental. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403154#comment-17403154 ] Yang Jie commented on SPARK-25075: -- [~dongjoon] Thank you for your reply > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402964#comment-17402964 ] Dongjoon Hyun commented on SPARK-25075: --- [~LuciferYang]. Not yet although I believe Apache Spark community can switch the default in Year 2022. We will see the adoption of Scala 2.13 after Apache Spark 3.2.0 release and stabilize our release first at 3.2.1 and 3.2.2. We can start to build a consensus early next year and target it for Apache Spark 3.3.0. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402927#comment-17402927 ] Yang Jie commented on SPARK-25075: -- Do we plan to use Scala 2.13 as the default version in Spark 3.3.0? I found that Scala 2.13 makes it easier to pass Spark UTs with JDK 17-ea > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384934#comment-17384934 ] Eric Richardson commented on SPARK-25075: - Great news that you will have 2.12 and 2.13 artifacts. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384660#comment-17384660 ] Gengliang Wang commented on SPARK-25075: [~dongjoon]Thanks for the ping. Yes, we should have both Scala 2.12/2.13 artifacts in Spark 3.2. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384469#comment-17384469 ] Dongjoon Hyun commented on SPARK-25075: --- > It looks like scala 2.12 will still be the default, correct? Yes, [~tgraves]. Our MiMa check is only verifying 2.12. After we have the official 2.13 bits, we are able to switch the default Scala version. > Are we planning on releasing the Spark tgz artifacts for 2.13 and 2.12 or > only 2.12? Both. SPARK-34218 added Scala 2.13 tgz artifacts and Maven publishing. Already, we have both Scala 2.12/2.13 Snapshot Maven artifacts. cc [~Gengliang.Wang] since he is the Apache Spark 3.2.0 release manager. The above is the plan and if something goes out of order, we should fix it. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383529#comment-17383529 ] Thomas Graves commented on SPARK-25075: --- Just wanted to check the plans for scala 2.13 in 3.2. It looks like scala 2.12 will still be the default, correct? Are we planning on releasing the Spark tgz artifacts for 2.13 and 2.12 or only 2.12? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350084#comment-17350084 ] Ismael Juma commented on SPARK-25075: - Yes, makes sense. When I mentioned Kafka 3.0 and Spark 3.2, I meant from the point of view of users that want to use both together. I understand that Spark itself will go with Kafka 2.8 for Spark 3.2 and evaluate the upgrade to Kafka 3.0 once it's available (expected sometime in July/August). > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350073#comment-17350073 ] Dongjoon Hyun commented on SPARK-25075: --- It sounds like two issues. - For the deprecation policy of Apache Kafka 3.0, it sounds reasonable to me (as a user). - For the relation with Apache Spark 3.2 and Apache Kafka 3.0, let's see after Apache Kafka 3.0 is released. You know, I'm a big fan of Kafka and I usually catch up the new release of Apache Kafka to bring the latest client-side bug fixes to Apache Spark. However, the followings are the current AS-IS status which Apache Spark community are trying to stabilize. - The latest Apache Spark release (3.1.1) was shipped Apache Kafka 2.6.0. Apache Spark 3.1.2 RC1 vote starts in a few days. - Apache Spark master branch has been using Apache Kafka 2.8. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350068#comment-17350068 ] Ismael Juma commented on SPARK-25075: - Thanks Dongjoon, I totally agree that stable Scala 2.13 support needs to come first. I was asking because we are also trying to figure out the strategy for Kafka when it comes to Scala versions support. We have supported Scala 2.13 for a while, but Scala 2.12 is still widely used due to Spark. I think I will propose to deprecate Scala 2.12 support in Apache Kafka 3.0, which would roughly align with Spark 3.2. And removal would happen around 1 year after that. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350063#comment-17350063 ] Dongjoon Hyun commented on SPARK-25075: --- [~ijuma]. We cannot consider dropping Scala 2.12 until we have a *stable* Scala 2.13. At least, one Scala version should be stable, shouldn't it? For now, Apache Spark 3.2 is going to be the first release with Scala 2.13 support and `Feature Freeze` for Spark 3.2 is July 1st. - https://spark.apache.org/versioning-policy.html After releasing Spark 3.2 with Scala 2.13, we will evaluate the adoption of Scala 2.13 and the stability in the end-user's perspective (apps). Some users might try Scala 3 on Apache Spark 3.2 because Scala community claims Scala 3 is binary compatible with Scala 2.13. We are expecting some references from them or bug reports. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349932#comment-17349932 ] Ismael Juma commented on SPARK-25075: - [~dongjoon] When do you intend to drop support for Scala 2.12? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346242#comment-17346242 ] Dongjoon Hyun commented on SPARK-25075: --- Yes, we are still focusing on supporting Scala 2.13 first because it's not officially released yet. In addition, we are intentionally avoiding Scala 3 because Apache Spark community supports at most 2 Scala versions historically. BTW, the binary compatibility between Scala 2.13 and Scala 3 is one of the goals of Scala 3. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346063#comment-17346063 ] Guillaume Martres commented on SPARK-25075: --- Note that the Scala 2.13 snapshots of Spark are usable from Scala 3 just like other Scala 2.13 libraries ([https://scalacenter.github.io/scala-3-migration-guide/docs/compatibility/classpath.html]), I don't think there's a Jira on building Spark on Scala 3 but there was a thread on the mailing-list: [http://apache-spark-developers-list.1001551.n3.nabble.com/Scala-3-support-approach-td30316.html] > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346060#comment-17346060 ] Ismaël Mejía commented on SPARK-25075: -- Don't want to be a party pooper but just for awareness [Scala 3 is out|https://www.scala-lang.org/blog/2021/05/14/scala3-is-here.html]. I suppose this will be a future target. Is there already a Jira for this? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289104#comment-17289104 ] Guillaume Martres commented on SPARK-25075: --- I've opened https://issues.apache.org/jira/browse/SPARK-34507. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288935#comment-17288935 ] Dongjoon Hyun commented on SPARK-25075: --- Thank you always, [~tisue]! > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288814#comment-17288814 ] Seth Tisue commented on SPARK-25075: Scala 2.13.5 is now available. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287437#comment-17287437 ] Dongjoon Hyun commented on SPARK-25075: --- [~MasseGuillaume]. Feel free to create a new independent Jira issue if you think that's a problem. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287109#comment-17287109 ] Guillaume Martres commented on SPARK-25075: --- [~dongjoon] I think something is wrong with the published snapshots, it seems to depend on both Scala 2.12 and Scala 2.13 artifacts, leading to crashes at runtime, and indeed if I look at [https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/3.2.0-SNAPSHOT/spark-parent_2.13-3.2.0-20210219.011324-25.pom] I see: 2.12.10 So I assume a config file wasn't updated somewhere. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284343#comment-17284343 ] Dongjoon Hyun commented on SPARK-25075: --- Thank you for sharing the progress, [~tisue]. We will try Scala 2.13.5 definitely. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284288#comment-17284288 ] Seth Tisue commented on SPARK-25075: We'll be releasing Scala 2.13.5 soon (thread [https://contributors.scala-lang.org/t/scala-2-13-5-planning/4825/10),] hopefully within a week or two, with a fix [https://github.com/scala/bug/issues/12038] > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283182#comment-17283182
]
Dongjoon Hyun commented on SPARK-25075:
---
Thanks, [~smarter] . Yes, we are moving forward. The latest blocker is Scala
2.13 bug which was reported when we upgrade the following.
{code:java}
[SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0
{code}
- [https://github.com/apache/spark/pull/31223]
The above PR was and reverted due to Scala 2.12.13's bug which is also open for
Scala 2.13, too. During reverting Scala 2.12.13, we didn't downgrade Scala
2.13's version because it's still testing. However, it will be an obvious
blocker for Scala 2.13 release in Apache Spark.
> Build and test Spark against Scala 2.13
> ---
>
> Key: SPARK-25075
> URL: https://issues.apache.org/jira/browse/SPARK-25075
> Project: Spark
> Issue Type: Umbrella
> Components: Build, MLlib, Project Infra, Spark Core, SQL
>Affects Versions: 3.0.0
>Reporter: Guillaume Massé
>Priority: Major
>
> This umbrella JIRA tracks the requirements for building and testing Spark
> against the current Scala 2.13 milestone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283059#comment-17283059 ] Guillaume Martres commented on SPARK-25075: --- Looks like 2.13 snapshot builds are now available: http://apache-spark-developers-list.1001551.n3.nabble.com/FYI-Scala-2-13-Maven-Artifacts-td30616.html, thanks [~dongjoon] ! > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263909#comment-17263909
]
Dongjoon Hyun commented on SPARK-25075:
---
[~stephenk]. Apache Spark release script doesn't have that logic yet. We need
to add that logic into our release script during 3.2.0 timeframe.
{quote}can you add context about points 1 and 2 please (why can't/aren't they
published)?
{quote}
Yes, currently, we are focusing on 3.1.0 first, but we can start publishing
3.2.0_2.13 too concurrently. `master` branch is not blocked by 3.1.0 release
process. You can make a PR to `master` branch.
{quote}Does a 3.2.0_2.13 snapshot require 3.1.0 being released?
{quote}
Here is the code part where you are interested in.
-
https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh#L418
> Build and test Spark against Scala 2.13
> ---
>
> Key: SPARK-25075
> URL: https://issues.apache.org/jira/browse/SPARK-25075
> Project: Spark
> Issue Type: Umbrella
> Components: Build, MLlib, Project Infra, Spark Core, SQL
>Affects Versions: 3.0.0
>Reporter: Guillaume Massé
>Priority: Major
>
> This umbrella JIRA tracks the requirements for building and testing Spark
> against the current Scala 2.13 milestone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263837#comment-17263837 ] Stephen Kestle commented on SPARK-25075: [~dongjoon] can you add context about points 1 and 2 please (why can't/aren't they published)? I'm guessing that the current focus is on 3.1.0 RCs (which is not scala 2.13). Does a 3.2.0_2.13 snapshot require 3.1.0 being released? When might it start? (And would this support be in an early milestone)? A month ago, I told myself that this was likely to be 6-12 months away, but on recent inspection, perhaps I should expect _something_ a bit sooner (would help start to integrate my code bases) > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258667#comment-17258667 ] Dongjoon Hyun commented on SPARK-25075: --- [~smarter]. Sorry, but unfortunately, from my assesement, the current status is a little different. 1. Apache Spark community is not able to publish Scala 2.13-based Maven artifacts yet. 2. Apache Spark community is not able to provide Scala 2.13-based binary distribution yet. 3. As you see at this JIRA, the target version of this Jira is 3.2.0, not 3.1.0. 4. For Apache Spark 3.1.0, we already created RC1 without SPARK-33894 and SPARK-33894 is marked as Spark 3.1.1. * [https://github.com/apache/spark/releases/tag/v3.1.0-rc1] Due to (1)~(4), Apache Spark 3.1.0 RC1 will have only Scala 2.12 libraries and binaries during vote period. Of course, I guess we will roll more RCs with more improvements; at least SPARK-33894 will be a part of 3.1.0. However, I don't think we can say Scala 2.13 is supported without the official Scala 2.13 binaries and Scala 2.13 Maven artifacts. I guess you also agree that those are mandatory. cc [~hyukjin.kwon] and [~srowen] > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258218#comment-17258218 ] Guillaume Martres commented on SPARK-25075: --- Now that 2.13 support is basically complete, would it be possible to publish a preview release of spark 3.1 built against scala 2.13 on maven for testing purposes? Thanks! > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157497#comment-17157497 ] Apache Spark commented on SPARK-25075: -- User 'karolchmist' has created a pull request for this issue: https://github.com/apache/spark/pull/29105 > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157495#comment-17157495 ] Apache Spark commented on SPARK-25075: -- User 'karolchmist' has created a pull request for this issue: https://github.com/apache/spark/pull/29105 > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147144#comment-17147144 ] Dongjoon Hyun commented on SPARK-25075: --- It's great! Thanks, [~smarter]. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146941#comment-17146941 ] Guillaume Martres commented on SPARK-25075: --- 2.13.3 has just been released with a fix for the miscompilation issue I mentioned above. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108310#comment-17108310 ] Guillaume Martres commented on SPARK-25075: --- I've found what seems to be a miscompilation issue in Scala 2.13 with my branch of Spark, see https://github.com/scala/bug/issues/12002 > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096870#comment-17096870 ] Guillaume Martres commented on SPARK-25075: --- Based on [~srowen]'s branch from SPARK-29292, I was able to get spark-core and its dependencies to compile with Scala 2.13.2 after making a few changes. The result is at https://github.com/smarter/spark/tree/scala-2.13, I've published it at https://bintray.com/smarter/maven for my own needs but I don't intend to update it or work on it further. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090996#comment-17090996 ] Dongjoon Hyun commented on SPARK-25075: --- Thank you for updates, [~smarter]. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090582#comment-17090582 ] Guillaume Martres commented on SPARK-25075: --- > SPARK-30132 which is blocked on Scala 2.13.2 2.13.2 is out now. > SPARK-27683 and SPARK-30090 sound like non-negligible effort as well The first one isn't actually a blocker, as noted in that issue, TraversableOnce is still there (as an alias) in 2.13 > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042491#comment-17042491 ] H. Vetinari commented on SPARK-25075: - For those trying to find an update on this (like me) - from the currently open subtasks there's SPARK-30132 which is blocked on Scala 2.13.2 (hopefully released soon). SPARK-27683 and SPARK-30090 sound like non-negligible effort as well, and both haven't seen much activity recently. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, MLlib, Project Infra, Spark Core, SQL >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981173#comment-16981173 ] Sean R. Owen commented on SPARK-25075: -- Yeah this change will be tricky but not terrible. One challenge is that until one can get the whole project to compile, it's hard to know all the issues. I've fixed a few obvious ones, and will file a few more classes of issue I've found. The major change is indeed the Seq / Map change. That's solved with a lot of .toSeq and .toMap across the project. I'm going to wait until branch-3.0 is cut to deal with that in one big change. In the meantime, I've been making that change steadily throughout the project and have uncovered some more issues I'll file as subtasks. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932666#comment-16932666 ] Jacob Niebloom commented on SPARK-25075: I am a possible new contributor the Spark. Is there a way I can help on this? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841287#comment-16841287
]
Lukas Rytz commented on SPARK-25075:
In our own interest of testing Scala the 2.13 RCs, I took a stab at compiling
spark core on 2.13.
My goal was *not* to get something running, just compiling and seeing what kind
of breaking changes there are. So I chose a slightly adventurous methodology: I
updated the {{scalaVersion}} to 2.13.0-pre-06392a5-SNAPSHOT (a very recent
local build), but forced the {{scalaBinaryVersion}} to 2.12, so that the 2.12
dependencies end up on the classpath. That way I didn't have to worry about
missing / incompatible dependencies.
The first step was to avoid using {{scala.Seq}}, I used scalafix to rewrite all
references of the type {{scala.Seq}} to {{scala.collection.Seq}}. As discussed
on https://issues.apache.org/jira/browse/SPARK-27681, this is not necessarily
the best solution, but the easiest.
Here's a list of other breaking changes:
* {{foo(someMutableOrGenericCollection: _*)}} no longer works, because varargs
de-sugars to {{scala.Seq}}, so an immutable collection is now required. Calling
{{.toSeq}} works, but is inefficient. Better build an immutable collection from
the beginning. For arrays, {{immutable.ArraySeq.unsafeWrapArray}} can be used.
Maybe the standard library should provide an unsafe {{immutable.SeqWrapper}}
that wraps a {{collection.Seq}} for the cases when the users are certain it's
safe.
* Views are quite different in 2.13, for example {{seq.view}} is no longer a
{{Seq}}, views are a separate hierarchy. This needs some adjustments, not too
difficult.
* Parallel collections are a separate module now
([https://github.com/scala/scala-parallel-collections]), no longer in the
standard library. However, you might want to use StreamConverters instead to do
parallel processing via a Java stream
([https://github.com/scala/scala/blob/2.13.x/src/library/scala/jdk/StreamConverters.scala]).
* Subclasses of collections need some adjustments: {{BoundedPriorityQueue}},
{{TimeStampedHashMap}}. For example, {{++=}} cannot be overridden anymore, as
it's a final alias for {{addAll}} now.
The other changes are relatively minor. The brannch is here - for reference,
too hacky to be actually useful:
[https://github.com/lrytz/spark/commits/2.13-experiment]. {{sbt core/compile}}
passes (except for scalastyle).
Overall, this is more or less what we expected in terms of breaking changes. We
definietly want to use the time between now and 2.13.0 final to improve
migration documentation
([https://docs.scala-lang.org/overviews/core/collections-migration-213.html])
and scalafix rules ([https://github.com/scala/scala-rewrites] is in its early
days).
> Build and test Spark against Scala 2.13
> ---
>
> Key: SPARK-25075
> URL: https://issues.apache.org/jira/browse/SPARK-25075
> Project: Spark
> Issue Type: Umbrella
> Components: Build, Project Infra
>Affects Versions: 3.0.0
>Reporter: Guillaume Massé
>Priority: Major
>
> This umbrella JIRA tracks the requirements for building and testing Spark
> against the current Scala 2.13 milestone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837579#comment-16837579 ] Stavros Kontopoulos commented on SPARK-25075: - Scala team added some more info to explain this in detail: https://github.com/scala/docs.scala-lang/pull/1326/files > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837516#comment-16837516 ] Sean Owen commented on SPARK-25075: --- OK, so we can possibly defer this by waiting until 2.13 support to go explicitly import scala.collection.Seq everywhere rather than use the alias in order to keep the type the same. I mean, we could get going on it now too; that's a lot of imports! Traversable: one usage is in a public API. We'll have to deprecate that and replace with Iterable, then remove when 2.13 support is added. Likewise there are some usages of removed collection classes, though so far looks all internal. Right now I'm mostly interested in identifying changes we can't solve by deprecating something in 3.x and then removing in 3.(x+1) to allow for 2.13 support. Probably nothing that will truly only be solvable in 4.x that I see so far, but, maybe best to start on these changes before 3.0. Anyone's welcome to; I may start pushing some minor proactive changes for 2.13. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837417#comment-16837417 ] Stefan Zeiger commented on SPARK-25075: --- Returning a Seq is not a problem but if you want it to be a scala.Seq it needs to be immutable now. Nothing changes if you always use scala.collection.Seq or scala.collection.immutable.Seq explicitly. This is recommended for cross-building. The main problem with Seq is the interaction with varargs. They are of type scala.Seq so you cannot pass an existing mutable Seq to a varargs method with `_*` anymore. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837403#comment-16837403 ] Stavros Kontopoulos commented on SPARK-25075: - Yes will take some time one of the blockers here: [https://github.com/twitter/chill/issues/316.] Thanks [~MasseGuillaume] havent checked the Spark side yet. `scala.Seq` is an alias right? so it should work out of the box no? > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837299#comment-16837299 ] Sean Owen commented on SPARK-25075: --- Given that 2.13 isn't out yet and most dependencies won't cross publish for a while, I doubt we'd try it soon. However any breaking changes we need to make to support it later we should try now. Uh oh does the collections API change mean returning Seq is a problem? Hoo boy. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837260#comment-16837260 ] Guillaume Massé commented on SPARK-25075: - Supporting both Scala 2.12 and Scala 2.13 will be an interesting chalenge. You can take a look at our work in Akka: [https://github.com/akka/akka/pull/26043.] To make the transition easier, there is a library and a bunch of scalafix migration rules. Take a look at: [https://github.com/scala/scala-collection-compat.] See also the migration guide at: [https://docs.scala-lang.org/overviews/core/collections-migration-213.html] I took a quick look, and there is a few Traversable instances in the public API. There is also a bunch of scala.Seq. This will makes things harder to migrate. Based on the version policy (https://spark.apache.org/versioning-policy.html), this would require a bump to Spark 4.X > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837234#comment-16837234 ] Stavros Kontopoulos commented on SPARK-25075: - [~srowen] since Scala 2.11 has been removed from master ([https://github.com/apache/spark/pull/23098)] should we try add 2.13 support as experimental in Spark 3.0.0 or it should be like Spark 3.1.0? Scala 2.13 is at RC1 at the moment, but will be out soon. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 3.0.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718988#comment-16718988 ] Sean Owen commented on SPARK-25075: --- You can just get the whole list of dependencies with `mvn dependency:tree` BTW -- in case that wasn't what you were using. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 2.4.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718987#comment-16718987 ] Sean Owen commented on SPARK-25075: --- It's fine to track that explicitly and try it out, but we won't add Scala 2.13 support unless and until we drop 2.11. > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 2.4.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716302#comment-16716302 ] Darcy Shen commented on SPARK-25075: I maintained a list of scala libraries which spark uses. https://github.com/scala/scala-dev/issues/563#issuecomment-425363609 > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 2.1.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574839#comment-16574839 ] Guillaume Massé commented on SPARK-25075: - Scala 2.13 is currently in the milestone phases (We are at 2.13.0-M4 at the moment of writing this). Since Sparks build with 2.12, we can start the migration to 2.13.0-M4 to find any incompatibilities. It's a good timing to add anything to 2.13.X before we finalize the collection API. To ease the process of the migration to 2.13 the Scala Center and the Scala team created automatic migration rules and compatibility library available at [https://github.com/scala/scala-collection-compat.] > Build and test Spark against Scala 2.13 > --- > > Key: SPARK-25075 > URL: https://issues.apache.org/jira/browse/SPARK-25075 > Project: Spark > Issue Type: Umbrella > Components: Build, Project Infra >Affects Versions: 2.1.0 >Reporter: Guillaume Massé >Priority: Major > > This umbrella JIRA tracks the requirements for building and testing Spark > against the current Scala 2.13 milestone. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
