Just wondering if you have noticed SPARK-54002 (https://github.com/apache/spark/pull/52706), which makes BeeLine support the Spark Connect JDBC driver and provides an SQL CLI to connect to the Connect Server, providing a similar experience to spark-sql.
Thanks, Cheng Pan > On Jul 21, 2026, at 00:50, Dhruv Pratap via dev <[email protected]> wrote: > > Hi all, > > The Spark Connect entry-point umbrella (SPARK-49194) has landed Connect > support for spark-shell (SPARK-48936), spark-submit (SPARK-48960), and > pyspark. The one interactive entry point still missing it is the spark-sql > CLI: there is currently no way to run spark-sql against a Connect server > (spark-sql --remote sc://...). > > Before writing any code, two questions for the Connect maintainers: > > 1. Tracking: should this be a sub-task under SPARK-49194, or a new ticket? > The umbrella lists pyspark/spark-shell/spark-submit and its sub-tasks are all > resolved, so I want to confirm the right home. > 2. Process: does this warrant a SPIP, or is it an incremental improvement > completing the already-accepted Connect pattern? It is user-facing but adds > no new public API (details below). > > Proposed approach (kept minimal): > - A new SparkConnectSQLCLIDriver in sql/connect/client/jvm, selected on > --remote / spark.api.mode=connect, mirroring how ConnectRepl backs > spark-shell --remote (SPARK-48936). > - Builds a remote session via SparkSession.builder().client(...), runs > statements via spark.sql(...), and formats results client-side. > - No change to the existing SparkSQLCLIDriver / sql/hive-thriftserver path; > the legacy driver stays the default. No existing public API changes. > - MVP first (-e/-f + interactive loop + basic output); follow-ups for full > Hive-style output parity. > > I have a short design write-up I can share/attach to the JIRA. > > This is complementary to the draft "Add Unified Spark CLI" PR (#57313, cc > @nchammas): that unifies the bin/ scripts and its sql subcommand currently > wraps the existing spark-sql shell, so a Connect-native spark-sql would slot > under it cleanly. Happy to coordinate to avoid duplicate CLI churn. > > Thanks, > Dhruv Pratap (Netflix) > Apache JIRA: dhruvpratap > [email protected] <mailto:[email protected]> ยท [email protected] > <mailto:[email protected]>
