Gargi-jais11 commented on code in PR #455: URL: https://github.com/apache/ozone-site/pull/455#discussion_r3385730075
########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. -- Which interfaces it supports (ofs, o3, Java). -- Information about configuration files it reads (core-site.xml, client log4j). -- Environment variables and common JVM flags (`-D`) that can be used at the CLI. +## Installation -**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section. +### Binary distribution + +Download an official release tarball from the [Apache Ozone download page](https://dlcdn.apache.org/ozone) and extract it on every machine that will run client commands or applications: + +```bash +curl -L -O https://dlcdn.apache.org/ozone/<version>/ozone-<version>.tar.gz +tar -xzf ozone-<version>.tar.gz +cd ozone-<version> +export OZONE_HOME=$(pwd) +export PATH=$OZONE_HOME/bin:$PATH +``` + +The distribution layout relevant to clients: + +| Path | Purpose | +|------|---------| +| `bin/ozone` | Main entry point for CLI and client subcommands | +| `etc/hadoop/` | Configuration directory (`OZONE_CONF_DIR`) | +| `share/ozone/lib/` | Client and filesystem JARs | +| `share/ozone/lib/ozone-filesystem-hadoop3-*.jar` | Hadoop-compatible filesystem driver | + +### Docker + +For development and testing, the official `apache/ozone` image includes a preconfigured client. From inside a running container: + +```bash +docker compose exec om ozone version +docker compose exec om ozone sh volume list / +``` + +See the [Docker installation guide](../../quick-start/installation/docker) for a full quick-start. + +### Build from source + +To build the distribution locally: + +```bash +mvn clean package -DskipTests=true -Pdist +``` + +The tarball is produced under `hadoop-ozone/dist/target/`. + +## Configuration + +Before using the client, point it at a running Ozone cluster by placing configuration files in `OZONE_CONF_DIR` (defaults to `$OZONE_HOME/etc/hadoop`). + +### Generate a starting configuration + +Use `ozone genconf` to create a template `ozone-site.xml`: + +```bash +ozone genconf /path/to/output +cp /path/to/output/ozone-site.xml $OZONE_HOME/etc/hadoop/ +``` + +Edit the generated file and set at minimum the client-relevant properties: + +| Property | Description | Default port | +|----------|-------------|--------------| +| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 | +| `ozone.scm.client.address` | SCM client service address | 9860 | Review Comment: ```suggestion | Property | Description | Default | |----------|-------------|--------------| | `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 | | `ozone.scm.client.address` | SCM client service address | No default address; set this to your SCM host (for example, scm1:9860) | | `ozone.scm.client.port` | SCM client service port when the address omits a port | 9860 | ``` ########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. -- Which interfaces it supports (ofs, o3, Java). -- Information about configuration files it reads (core-site.xml, client log4j). -- Environment variables and common JVM flags (`-D`) that can be used at the CLI. +## Installation -**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section. +### Binary distribution + +Download an official release tarball from the [Apache Ozone download page](https://dlcdn.apache.org/ozone) and extract it on every machine that will run client commands or applications: + +```bash +curl -L -O https://dlcdn.apache.org/ozone/<version>/ozone-<version>.tar.gz +tar -xzf ozone-<version>.tar.gz +cd ozone-<version> +export OZONE_HOME=$(pwd) +export PATH=$OZONE_HOME/bin:$PATH +``` + +The distribution layout relevant to clients: + +| Path | Purpose | +|------|---------| +| `bin/ozone` | Main entry point for CLI and client subcommands | +| `etc/hadoop/` | Configuration directory (`OZONE_CONF_DIR`) | +| `share/ozone/lib/` | Client and filesystem JARs | +| `share/ozone/lib/ozone-filesystem-hadoop3-*.jar` | Hadoop-compatible filesystem driver | + +### Docker + +For development and testing, the official `apache/ozone` image includes a preconfigured client. From inside a running container: + +```bash +docker compose exec om ozone version +docker compose exec om ozone sh volume list / +``` + +See the [Docker installation guide](../../quick-start/installation/docker) for a full quick-start. + +### Build from source + +To build the distribution locally: + +```bash +mvn clean package -DskipTests=true -Pdist +``` + +The tarball is produced under `hadoop-ozone/dist/target/`. + +## Configuration + +Before using the client, point it at a running Ozone cluster by placing configuration files in `OZONE_CONF_DIR` (defaults to `$OZONE_HOME/etc/hadoop`). + +### Generate a starting configuration + +Use `ozone genconf` to create a template `ozone-site.xml`: + +```bash +ozone genconf /path/to/output +cp /path/to/output/ozone-site.xml $OZONE_HOME/etc/hadoop/ +``` + +Edit the generated file and set at minimum the client-relevant properties: + +| Property | Description | Default port | +|----------|-------------|--------------| +| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 | +| `ozone.scm.client.address` | SCM client service address | 9860 | + +### Verify configuration + +```bash +# Show resolved environment +ozone envvars + +# Read a specific config value +ozone getconf -confKey ozone.om.address +``` + +Use `--config /path/to/conf` on any `ozone` command to override `OZONE_CONF_DIR` for a single invocation: + +```bash +ozone --config /etc/ozone sh volume list / +``` + +## Running the client + +All client operations are invoked through the `ozone` script. Common client subcommands: + +| Command | Description | +|---------|-------------| +| `ozone sh` | Object-store shell (volumes, buckets, keys) via the native RPC protocol | +| `ozone fs` | Hadoop-compatible filesystem shell (equivalent to `hadoop fs`) | +| `ozone s3` | S3-related CLI operations | +| `ozone getconf` | Print configuration values | +| `ozone version` | Print the Ozone version | + +Example workflow after configuration: + +```bash +# Admin: create volumne and bucket +ozone sh volume create /vol1 +ozone sh bucket create /vol1/bucket1 +``` + +By default, CLI tools suppress logging output. Pass `--loglevel DEBUG` (or any Log4j level) to enable verbose client logging: + +```bash +ozone --loglevel DEBUG sh volume info /vol1 +``` + +### Client log4j configuration + +Client-side logging is controlled by `log4j.properties` in `$OZONE_CONF_DIR`. The distribution ships a template at `etc/hadoop/log4j.properties`. + +Key settings: + +| Setting | Default | Description | +|---------|---------|-------------| +| `hadoop.root.logger` | `INFO,console` | Root Log4j logger and appenders | +| `hadoop.log.dir` | Set via JVM property | Directory for log files | +| `hadoop.log.file` | `hadoop.log` | Log file name | + +The `ozone` script injects these as JVM system properties at runtime: + +``` +-Dhadoop.log.dir=${OZONE_LOG_DIR} +-Dhadoop.log.file=${OZONE_LOGFILE} +-Dhadoop.root.logger=${OZONE_ROOT_LOGGER} +``` + +For CLI subcommands (`ozone sh`, `ozone fs`, etc.), logging defaults to **OFF** unless you pass `--loglevel` or set `OZONE_ROOT_LOGGER` / `OZONE_LOGLEVEL` explicitly. This keeps routine command output clean. Review Comment: I believe info also we should add and write it as a note. ```suggestion **Note** - For CLI subcommands (`ozone sh`, `ozone fs`, etc.), logging defaults to **OFF** unless you pass `--loglevel` or set `OZONE_ROOT_LOGGER` / `OZONE_LOGLEVEL` explicitly. This keeps routine command output clean. **Precedence:** The JVM system properties injected by the ozone script (`-Dhadoop.root.logger, -Dhadoop.log.dir, -Dhadoop.log.file)` take precedence over values in **log4j.properties.** Editing **log4j.properties** alone does not enable CLI logging; use `--loglevel`, `OZONE_ROOT_LOGGER`, or `OZONE_SH_OPTS / OZONE_FS_OPTS` as shown below. ``` ########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. -- Which interfaces it supports (ofs, o3, Java). -- Information about configuration files it reads (core-site.xml, client log4j). -- Environment variables and common JVM flags (`-D`) that can be used at the CLI. +## Installation -**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section. +### Binary distribution + +Download an official release tarball from the [Apache Ozone download page](https://dlcdn.apache.org/ozone) and extract it on every machine that will run client commands or applications: + +```bash +curl -L -O https://dlcdn.apache.org/ozone/<version>/ozone-<version>.tar.gz +tar -xzf ozone-<version>.tar.gz +cd ozone-<version> +export OZONE_HOME=$(pwd) +export PATH=$OZONE_HOME/bin:$PATH +``` + +The distribution layout relevant to clients: + +| Path | Purpose | +|------|---------| +| `bin/ozone` | Main entry point for CLI and client subcommands | +| `etc/hadoop/` | Configuration directory (`OZONE_CONF_DIR`) | +| `share/ozone/lib/` | Client and filesystem JARs | +| `share/ozone/lib/ozone-filesystem-hadoop3-*.jar` | Hadoop-compatible filesystem driver | + +### Docker + +For development and testing, the official `apache/ozone` image includes a preconfigured client. From inside a running container: + +```bash +docker compose exec om ozone version +docker compose exec om ozone sh volume list / +``` + +See the [Docker installation guide](../../quick-start/installation/docker) for a full quick-start. + +### Build from source + +To build the distribution locally: + +```bash +mvn clean package -DskipTests=true -Pdist +``` + +The tarball is produced under `hadoop-ozone/dist/target/`. + +## Configuration + +Before using the client, point it at a running Ozone cluster by placing configuration files in `OZONE_CONF_DIR` (defaults to `$OZONE_HOME/etc/hadoop`). + +### Generate a starting configuration + +Use `ozone genconf` to create a template `ozone-site.xml`: + +```bash +ozone genconf /path/to/output +cp /path/to/output/ozone-site.xml $OZONE_HOME/etc/hadoop/ +``` + +Edit the generated file and set at minimum the client-relevant properties: + +| Property | Description | Default port | +|----------|-------------|--------------| +| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 | +| `ozone.scm.client.address` | SCM client service address | 9860 | + +### Verify configuration + +```bash +# Show resolved environment +ozone envvars + +# Read a specific config value +ozone getconf -confKey ozone.om.address +``` + +Use `--config /path/to/conf` on any `ozone` command to override `OZONE_CONF_DIR` for a single invocation: + +```bash +ozone --config /etc/ozone sh volume list / +``` + +## Running the client + +All client operations are invoked through the `ozone` script. Common client subcommands: + +| Command | Description | +|---------|-------------| +| `ozone sh` | Object-store shell (volumes, buckets, keys) via the native RPC protocol | +| `ozone fs` | Hadoop-compatible filesystem shell (equivalent to `hadoop fs`) | +| `ozone s3` | S3-related CLI operations | +| `ozone getconf` | Print configuration values | +| `ozone version` | Print the Ozone version | Review Comment: The subcommand table is very incomplete The table lists five commands. The ozone script registers many more that are relevant : ``` `ozone admin:`Cluster administration (pipeline, container, replication ops) `ozone debug:` Diagnostic tools (replaces deprecated auditparser, checknative) `ozone tenant:` Multi-tenancy management `ozone insight `: Metrics and log insight `ozone freon :` Load generator / performance testing `ozone dtutil :` Delegation token utilities `ozone classpath :` Print effective classpath (useful for embedding the client) ``` If not specify all then atleast we should add `ozone admin` and `ozone debug` as well to the table. ########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. -- Which interfaces it supports (ofs, o3, Java). Review Comment: There is no section for `Hadoop filesystem integration (O3FS and OFS)` ########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. -- Which interfaces it supports (ofs, o3, Java). -- Information about configuration files it reads (core-site.xml, client log4j). -- Environment variables and common JVM flags (`-D`) that can be used at the CLI. +## Installation -**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section. +### Binary distribution + +Download an official release tarball from the [Apache Ozone download page](https://dlcdn.apache.org/ozone) and extract it on every machine that will run client commands or applications: + +```bash +curl -L -O https://dlcdn.apache.org/ozone/<version>/ozone-<version>.tar.gz +tar -xzf ozone-<version>.tar.gz +cd ozone-<version> +export OZONE_HOME=$(pwd) +export PATH=$OZONE_HOME/bin:$PATH +``` + +The distribution layout relevant to clients: + +| Path | Purpose | +|------|---------| +| `bin/ozone` | Main entry point for CLI and client subcommands | +| `etc/hadoop/` | Configuration directory (`OZONE_CONF_DIR`) | +| `share/ozone/lib/` | Client and filesystem JARs | +| `share/ozone/lib/ozone-filesystem-hadoop3-*.jar` | Hadoop-compatible filesystem driver | + +### Docker + +For development and testing, the official `apache/ozone` image includes a preconfigured client. From inside a running container: + +```bash +docker compose exec om ozone version +docker compose exec om ozone sh volume list / +``` + +See the [Docker installation guide](../../quick-start/installation/docker) for a full quick-start. + +### Build from source + +To build the distribution locally: + +```bash +mvn clean package -DskipTests=true -Pdist +``` + +The tarball is produced under `hadoop-ozone/dist/target/`. + +## Configuration + +Before using the client, point it at a running Ozone cluster by placing configuration files in `OZONE_CONF_DIR` (defaults to `$OZONE_HOME/etc/hadoop`). + +### Generate a starting configuration + +Use `ozone genconf` to create a template `ozone-site.xml`: Review Comment: ```suggestion Use `ozone genconf` to generate a template `ozone-site.xml `containing the minimally required Ozone properties and their default values: ``` ########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. Review Comment: ```suggestion This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other: **the native RPC protocol**, **the Hadoop-compatible filesystem**, and **the S3-compatible REST AP**I. ``` ########## versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md: ########## @@ -2,12 +2,176 @@ sidebar_label: Ozone --- -# The Ozone Client +The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library. -Document how to install, configure and run Ozone's client. This should also include +This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other. -- Which interfaces it supports (ofs, o3, Java). -- Information about configuration files it reads (core-site.xml, client log4j). -- Environment variables and common JVM flags (`-D`) that can be used at the CLI. +## Installation -**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section. +### Binary distribution + +Download an official release tarball from the [Apache Ozone download page](https://dlcdn.apache.org/ozone) and extract it on every machine that will run client commands or applications: + +```bash +curl -L -O https://dlcdn.apache.org/ozone/<version>/ozone-<version>.tar.gz +tar -xzf ozone-<version>.tar.gz +cd ozone-<version> +export OZONE_HOME=$(pwd) +export PATH=$OZONE_HOME/bin:$PATH +``` + +The distribution layout relevant to clients: + +| Path | Purpose | +|------|---------| +| `bin/ozone` | Main entry point for CLI and client subcommands | +| `etc/hadoop/` | Configuration directory (`OZONE_CONF_DIR`) | +| `share/ozone/lib/` | Client and filesystem JARs | +| `share/ozone/lib/ozone-filesystem-hadoop3-*.jar` | Hadoop-compatible filesystem driver | + +### Docker + +For development and testing, the official `apache/ozone` image includes a preconfigured client. From inside a running container: + +```bash +docker compose exec om ozone version +docker compose exec om ozone sh volume list / +``` + +See the [Docker installation guide](../../quick-start/installation/docker) for a full quick-start. + +### Build from source + +To build the distribution locally: + +```bash +mvn clean package -DskipTests=true -Pdist +``` + +The tarball is produced under `hadoop-ozone/dist/target/`. + +## Configuration + +Before using the client, point it at a running Ozone cluster by placing configuration files in `OZONE_CONF_DIR` (defaults to `$OZONE_HOME/etc/hadoop`). + +### Generate a starting configuration + +Use `ozone genconf` to create a template `ozone-site.xml`: + +```bash +ozone genconf /path/to/output +cp /path/to/output/ozone-site.xml $OZONE_HOME/etc/hadoop/ +``` + +Edit the generated file and set at minimum the client-relevant properties: + +| Property | Description | Default port | +|----------|-------------|--------------| +| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 | +| `ozone.scm.client.address` | SCM client service address | 9860 | + +### Verify configuration + +```bash +# Show resolved environment +ozone envvars + +# Read a specific config value +ozone getconf -confKey ozone.om.address +``` + +Use `--config /path/to/conf` on any `ozone` command to override `OZONE_CONF_DIR` for a single invocation: + +```bash +ozone --config /etc/ozone sh volume list / +``` + +## Running the client + +All client operations are invoked through the `ozone` script. Common client subcommands: + +| Command | Description | +|---------|-------------| +| `ozone sh` | Object-store shell (volumes, buckets, keys) via the native RPC protocol | +| `ozone fs` | Hadoop-compatible filesystem shell (equivalent to `hadoop fs`) | +| `ozone s3` | S3-related CLI operations | +| `ozone getconf` | Print configuration values | +| `ozone version` | Print the Ozone version | + +Example workflow after configuration: + +```bash +# Admin: create volumne and bucket Review Comment: Typo : ```suggestion # Admin: create volume and bucket ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
