talatuyarer commented on code in PR #36305: URL: https://github.com/apache/beam/pull/36305#discussion_r2389830165
########## website/www/site/content/en/documentation/dsls/sql/shell.md: ########## @@ -112,7 +178,31 @@ When you're satisfied with the logic of your SQL statements, you can submit the ## Specifying the Runner -By default, Beam uses the `DirectRunner` to run the pipeline on the machine where you're executing the commands. If you want to run the pipeline with a different runner, you must perform two steps: +By default, Beam uses the `DirectRunner` to run the pipeline on the machine where you're executing the commands. If you want to run the pipeline with a different runner, you have two options: + +### Option 1: Using beam-sql.sh Script (Recommended) + +When using the `beam-sql.sh` script, you can specify the runner directly via command-line options: + +```bash +# Use Dataflow runner +./beam-sql.sh --runner dataflow + +# Use Dataflow runner with specific IO connectors +./beam-sql.sh --runner dataflow --io kafka --io iceberg +``` + +Then, configure the runner using the `SET` command ([reference page](/documentation/dsls/sql/set/)): + +``` +0: BeamSQL> SET runner='DataflowRunner'; +0: BeamSQL> SET projectId='your-gcp-project'; +0: BeamSQL> SET tempLocation='gs://your-bucket/temp'; +``` + +### Option 2: Building from Source Review Comment: Removed this ection on the document ########## website/www/site/content/en/documentation/dsls/sql/shell.md: ########## @@ -26,23 +26,89 @@ This page describes how to work with the shell, but does not focus on specific f ## Quickstart -To use Beam SQL shell, you must first clone the [Beam SDK repository](https://github.com/apache/beam). Then, from the root of the repository clone, execute the following commands to run the shell: +There are two ways to get started with the Beam SQL shell: + +### Option 1: Using beam-sql.sh Script (Recommended) + +The easiest way to get started is using the `beam-sql.sh` script, which automatically downloads and sets up the Beam SQL shell with all dependencies. + +#### Installation + +1. **Download the script:** + ```bash + curl -O https://raw.githubusercontent.com/apache/beam/master/scripts/beam-sql.sh + chmod +x beam-sql.sh + ``` + +2. **Run the shell:** + ```bash + ./beam-sql.sh + ``` + +The script will automatically: +- Download the latest stable Beam version (2.67.0 by default) +- Build a self-contained JAR with all dependencies +- Cache the JAR for future use (stored in `~/.beamshell/cache/`) +- Launch the Beam SQL shell + +#### Prerequisites + +- **Java**: Java 8 or higher must be installed and available in your PATH +- **Maven** (optional): If Maven is not installed, the script will automatically download and set up a Maven wrapper +- **curl** (optional): Required only if Maven is not installed (for downloading the Maven wrapper) + +#### Command-line Options + +The `beam-sql.sh` script supports several options: + +```bash +./beam-sql.sh [--version <beam_version>] [--runner <runner_name>] [--io <io_connector>] [--list-versions] [-h|--help] +``` + +**Options:** +- `--version <beam_version>`: Specify the Apache Beam version (default: 2.67.0) +- `--runner <runner_name>`: Specify the Beam runner to use (default: direct). Supported: direct, dataflow +- `--io <io_connector>`: Specify an IO connector to include (e.g., iceberg, kafka). Can be used multiple times +- `--list-versions`: List all available Beam versions from Maven Central and exit +- `-h, --help`: Show help message + +**Examples:** + +```bash +# Use a specific Beam version +./beam-sql.sh --version 2.66.0 + +# Include Kafka IO connector +./beam-sql.sh --io kafka + +# Use Dataflow runner with multiple IO connectors +./beam-sql.sh --runner dataflow --io kafka --io iceberg + +# List available versions +./beam-sql.sh --list-versions +``` + +### Option 2: Building from Source Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
