This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 66f42b5b292 [SPARK-45145][EXAMPLE] Add JavaSparkSQLCli example 66f42b5b292 is described below commit 66f42b5b292cb4afff14b42f0d31835067ebdf5e Author: Dongjoon Hyun <dh...@apple.com> AuthorDate: Wed Sep 13 00:20:18 2023 -0700 [SPARK-45145][EXAMPLE] Add JavaSparkSQLCli example ### What changes were proposed in this pull request? This PR aims to add a simple Java example, `JavaSparkSQLCli.java`. Like SparkPi example, we can take advantage of this minimal example with `bin/run-example` and we can run a simple SQL query as a canary test during RC testing and voting phase. - https://spark.apache.org/docs/3.5.0/ ``` ./bin/run-example SparkPi 10 ``` After this PR, ``` $ bin/run-example sql.JavaSparkSQLCli "SELECT 'Spark SQL' col" +---------+ |col | +---------+ |Spark SQL| +---------+ ``` ### Why are the changes needed? Although there exists `org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver` and `bin/spark-sql` shell environment, it requires `-Phive -Phive-thriftserver` explicitly. ``` $ bin/spark-shell --version Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 3.5.0 /_/ ... $ bin/run-example org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver ... Error: Failed to load class org.apache.spark.examples.org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver. Failed to load main class org.apache.spark.examples.org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver. You need to build Spark with -Phive and -Phive-thriftserver. 23/09/12 23:06:38 INFO ShutdownHookManager: Shutdown hook called 23/09/12 23:06:38 INFO ShutdownHookManager: Deleting directory /private/var/folders/d4/dr6zxyvd4cl38877bj3fxs_m0000gn/T/spark-2a9ee4a3-64c3-492c-a268-a71b9aa44a2a ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually. ``` $ build/sbt test:package $ bin/run-example sql.JavaSparkSQLCli "SELECT 'Spark SQL' col" ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #42900 from dongjoon-hyun/SPARK-45145. Authored-by: Dongjoon Hyun <dh...@apple.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .../apache/spark/examples/sql/JavaSparkSQLCli.java | 41 ++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQLCli.java b/examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQLCli.java new file mode 100644 index 00000000000..d5b033c46bf --- /dev/null +++ b/examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQLCli.java @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.spark.examples.sql; + +import org.apache.spark.sql.SparkSession; + +/** + * Example Usage: + * <pre> + * bin/run-example sql.JavaSparkSQLCli "SELECT 'Spark SQL' col" + * </pre> + */ +public class JavaSparkSQLCli { + + public static void main(String[] args) { + SparkSession spark = SparkSession + .builder() + .appName("Java Spark SQL Cli") + .getOrCreate(); + + for (String a: args) { + spark.sql(a).show(false); + } + + spark.stop(); + } +} --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org