[
https://issues.apache.org/jira/browse/SPARK-50411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell updated SPARK-50411:
--------------------------------------
Summary: Spark Connect as the default API in Spark 4 (was: Spark Connect
the default API in Spark 4)
> Spark Connect as the default API in Spark 4
> -------------------------------------------
>
> Key: SPARK-50411
> URL: https://issues.apache.org/jira/browse/SPARK-50411
> Project: Spark
> Issue Type: Epic
> Components: Connect, SQL
> Affects Versions: 4.0.0
> Reporter: Herman van Hövell
> Assignee: Herman van Hövell
> Priority: Major
>
> We propose to enable Spark Connect as the default interface for Scala and
> Python applications in Spark 4. With this change, we will ensure that by
> default, all Scala and Python applications will run using the Spark Connect
> API so that developers can 1) benefit from stability, upgradability, and ease
> of use and 2) use Spark from non-JVM languages.
> As a lot of cases are using the full Spark API, we want to make it simple to
> switch back for compatibility reasons. We propose to add a new Spark
> configuration spark.api.mode with the following options:
> * connect: Spark uses the Spark Connect as the default API (default)
> * classic: Spark uses classic mode. The classic mode is today’s default.
> With this configuration, we open up Spark to a new generation of developers
> while ensuring all existing workloads continue to run without any impact!
> SPIP:
> https://docs.google.com/document/d/1C0kuQEliG78HujVwdnSk0wjNwHEDdwo2o8aVq7kbhTo/edit?tab=t.0#heading=h.r2c3xrbiklu3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
