Herman van Hövell created SPARK-50411:
-----------------------------------------
Summary: We to make Spark Connect the default API in Spark 4
Key: SPARK-50411
URL: https://issues.apache.org/jira/browse/SPARK-50411
Project: Spark
Issue Type: Epic
Components: Connect, SQL
Affects Versions: 4.0.0
Reporter: Herman van Hövell
Assignee: Herman van Hövell
We propose to enable Spark Connect as the default interface for Scala and
Python applications in Spark 4. With this change, we will ensure that by
default, all Scala and Python applications will run using the Spark Connect API
so that developers can 1) benefit from stability, upgradability, and ease of
use and 2) use Spark from non-JVM languages.
As a lot of cases are using the full Spark API, we want to make it simple to
switch back for compatibility reasons. We propose to add a new Spark
configuration spark.api.mode with the following options:
* connect: Spark uses the Spark Connect as the default API (default)
* classic: Spark uses classic mode. The classic mode is today’s default.
With this configuration, we open up Spark to a new generation of developers
while ensuring all existing workloads continue to run without any impact!
SPIP:
https://docs.google.com/document/d/1C0kuQEliG78HujVwdnSk0wjNwHEDdwo2o8aVq7kbhTo/edit?tab=t.0#heading=h.r2c3xrbiklu3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
