ptlrs commented on code in PR #101:
URL: https://github.com/apache/ozone-site/pull/101#discussion_r1839604640
##########
docs/06-troubleshooting/15-docker.md:
##########
@@ -0,0 +1,7 @@
+---
+sidebar_label: Docker
+---
+
+# Troubleshooting Docker
Review Comment:
Fixed
##########
package.json:
##########
@@ -1,5 +1,5 @@
{
- "packageManager": "[email protected]",
+ "packageManager": "[email protected]",
Review Comment:
With `[email protected]`, running docker-compose fails with the following error:
```bash
0.280 ! Corepack is about to download
https://registry.npmjs.org/pnpm/-/pnpm-8.7.1.tgz
1.109 ERR_PNPM_LOCKFILE_BREAKING_CHANGE Lockfile
/ozone-site/pnpm-lock.yaml not compatible with current pnpm
1.109
1.109 Run with the --force parameter to recreate the lockfile.
------
failed to solve: process "/bin/sh -c pnpm install --prod --frozen-lockfile"
did not complete successfully: exit code: 1
`docker-compose` process finished with exit code 17
```
I added the docusaurus/theme-mermaid package via `pnpm add
@docusaurus/[email protected]` on my mac. This updated the `pnpm-lock.yaml`
and the `package.json` files.
##########
cspell.yaml:
##########
@@ -48,6 +48,7 @@ words:
- ratis
- OM
- SCM
+- DN
Review Comment:
Fixed
##########
docs/08-developer-guide/02-run/02-docker-compose.md:
##########
@@ -10,3 +10,143 @@ sidebar_label: Docker Compose
- Define the Ozone runner image at `ozone-docker-runner` and its purpose to
wrap Ozone binaries.
- How to define which image gets used in the cluster (runner or other
pre-built image).
- Changing configurations in Docker Compose (`docker-config` file,
transformation.py, etc).
+
+This guide explains how to run Apache Ozone using Docker Compose, either with
locally built sources or pre-built images.
+
+## Prerequisites
+
+- Docker Engine 20.10.0 or higher
+- Docker Compose V2
Review Comment:
Fixed
##########
docs/08-developer-guide/02-run/02-docker-compose.md:
##########
@@ -10,3 +10,143 @@ sidebar_label: Docker Compose
- Define the Ozone runner image at `ozone-docker-runner` and its purpose to
wrap Ozone binaries.
- How to define which image gets used in the cluster (runner or other
pre-built image).
- Changing configurations in Docker Compose (`docker-config` file,
transformation.py, etc).
+
+This guide explains how to run Apache Ozone using Docker Compose, either with
locally built sources or pre-built images.
+
+## Prerequisites
+
+- Docker Engine 20.10.0 or higher
+- Docker Compose V2
+- Built Ozone distribution (if running from local build)
+
+## Running Ozone
+
+You can run Ozone either using your locally built version or using official
pre-built Docker images.
+
+### Option 1: Running from Local Build
+
+If you've built Ozone from source, follow these steps:
+
+1. Navigate to the compose directory in your build output:
+
+ ```bash
+ cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/ozone
+ ```
+
+2. Start the cluster:
+
+ ```bash
+ docker-compose up -d
+ ```
+
+The local build uses the `ozone-docker-runner` image, which is automatically
created during the build process to wrap your compiled Ozone binaries.
+
+### Option 2: Running from Pre-built Images
+
+If you haven't built Ozone locally, you can quickly start a cluster using
official pre-built images from Docker Hub:
+
+1. Create a directory for your Ozone deployment:
+
+ ```bash
+ mkdir ozone && cd ozone
+ ```
+
+2. Download the example compose file:
+
+ ```bash
+ curl \
+
https://raw.githubusercontent.com/apache/ozone/master/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
\
+ -o docker-compose.yaml
+ ```
+
+3. Start the cluster:
+
+ ```bash
+ docker-compose up -d
+ ```
+
+This will pull the official Apache Ozone images from Docker Hub.
+
+## Architecture Diagram
+
+This image shows the containers that will be created when running the
`docker-compose up -d` command.
+
+```mermaid
+graph TB
+ subgraph "Apache Ozone Architecture"
+ scm["SCM<br/>Storage Container Manager<br/>(Port: 9876, 9860)"]
+ om["OM<br/>Ozone Manager<br/>(Port: 9874, 9862)"]
+ s3g["S3G<br/>S3 Gateway<br/>(Port: 9878)"]
+ recon["Recon<br/>Monitoring Service<br/>(Port: 9888)"]
+ httpfs["HttpFS<br/>HTTP FileSystem<br/>(Port: 14000)"]
+
+ subgraph "Data Nodes"
+ dn1["DataNode 1<br/>(Port: 19864, 9882)"]
+ dnN["DataNode N<br/>(Port: 19864, 9882)"]
+ end
+
+ %% Connections
+ scm --> dn1
+ scm --> dnN
+ om --> dn1
+ om --> dnN
+ s3g --> om
+ s3g --> scm
+ httpfs --> om
+ httpfs --> scm
+ recon --> scm
+ recon --> om
+ end
+
+ classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px;
+ classDef datanode fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
+ classDef manager fill:#e8f5e9,stroke:#388e3c,stroke-width:2px;
+
+ class dn1,dnN datanode;
+ class scm,om manager;
+```
+
+## Cluster Configuration
+
+### Default Services
+
+The default Docker Compose configuration includes:
+
+- Storage Container Manager (SCM)
+- Ozone Manager (OM)
+- S3 Gateway
+- Recon (Monitoring Service)
+- Datanodes
+- HttpFS
+
+### Port Mappings
+
+The default setup exposes the following ports:
+
+- SCM: 9876 (RPC), 9860 (Client)
+- OM: 9874 (RPC), 9862 (Client)
+- S3 Gateway: 9878
+- Recon: 9888
+- HttpFS: 14000
+- Datanodes: 19864 (Container), 9882 (Client)
+
+## Cluster Management
+
+Common Docker Compose commands:
+
+```bash
+# Start the cluster
+docker-compose up -d
Review Comment:
Fixed
##########
docs/08-developer-guide/02-run/02-docker-compose.md:
##########
@@ -10,3 +10,143 @@ sidebar_label: Docker Compose
- Define the Ozone runner image at `ozone-docker-runner` and its purpose to
wrap Ozone binaries.
- How to define which image gets used in the cluster (runner or other
pre-built image).
- Changing configurations in Docker Compose (`docker-config` file,
transformation.py, etc).
+
+This guide explains how to run Apache Ozone using Docker Compose, either with
locally built sources or pre-built images.
+
+## Prerequisites
+
+- Docker Engine 20.10.0 or higher
+- Docker Compose V2
+- Built Ozone distribution (if running from local build)
+
+## Running Ozone
+
+You can run Ozone either using your locally built version or using official
pre-built Docker images.
+
+### Option 1: Running from Local Build
+
+If you've built Ozone from source, follow these steps:
+
+1. Navigate to the compose directory in your build output:
+
+ ```bash
+ cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/ozone
+ ```
+
+2. Start the cluster:
+
+ ```bash
+ docker-compose up -d
+ ```
+
+The local build uses the `ozone-docker-runner` image, which is automatically
created during the build process to wrap your compiled Ozone binaries.
+
+### Option 2: Running from Pre-built Images
+
+If you haven't built Ozone locally, you can quickly start a cluster using
official pre-built images from Docker Hub:
+
+1. Create a directory for your Ozone deployment:
+
+ ```bash
+ mkdir ozone && cd ozone
+ ```
+
+2. Download the example compose file:
+
+ ```bash
+ curl \
+
https://raw.githubusercontent.com/apache/ozone/master/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
\
+ -o docker-compose.yaml
+ ```
+
+3. Start the cluster:
+
+ ```bash
+ docker-compose up -d
+ ```
+
+This will pull the official Apache Ozone images from Docker Hub.
+
+## Architecture Diagram
Review Comment:
As a new user, it is not immediately apparent how the Ozone services will be
divvied across containers. It could be the case that all Ozone services run
within one container. There are several possible configurations.
I added the mermaid diagram to visualize how many containers will be
available and how each Ozone service maps to those containers.
It just so happens that each Ozone service is a separate container and the
container diagram looks similar to what could easily have been an Ozone
architecture diagram. Perhaps the title and wording add to the confusion.
If it is still a cause of confusion, I can remove the diagram. Not a
problem.
##########
docs/08-developer-guide/02-run/02-docker-compose.md:
##########
@@ -10,3 +10,143 @@ sidebar_label: Docker Compose
- Define the Ozone runner image at `ozone-docker-runner` and its purpose to
wrap Ozone binaries.
- How to define which image gets used in the cluster (runner or other
pre-built image).
- Changing configurations in Docker Compose (`docker-config` file,
transformation.py, etc).
+
+This guide explains how to run Apache Ozone using Docker Compose, either with
locally built sources or pre-built images.
+
+## Prerequisites
+
+- Docker Engine 20.10.0 or higher
+- Docker Compose V2
+- Built Ozone distribution (if running from local build)
+
+## Running Ozone
+
+You can run Ozone either using your locally built version or using official
pre-built Docker images.
+
+### Option 1: Running from Local Build
+
+If you've built Ozone from source, follow these steps:
+
+1. Navigate to the compose directory in your build output:
+
+ ```bash
+ cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/ozone
+ ```
+
+2. Start the cluster:
+
+ ```bash
+ docker-compose up -d
+ ```
+
+The local build uses the `ozone-docker-runner` image, which is automatically
created during the build process to wrap your compiled Ozone binaries.
+
+### Option 2: Running from Pre-built Images
+
+If you haven't built Ozone locally, you can quickly start a cluster using
official pre-built images from Docker Hub:
+
+1. Create a directory for your Ozone deployment:
+
+ ```bash
+ mkdir ozone && cd ozone
+ ```
+
+2. Download the example compose file:
+
+ ```bash
+ curl \
+
https://raw.githubusercontent.com/apache/ozone/master/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
\
+ -o docker-compose.yaml
+ ```
+
+3. Start the cluster:
+
+ ```bash
+ docker-compose up -d
+ ```
+
+This will pull the official Apache Ozone images from Docker Hub.
+
+## Architecture Diagram
+
+This image shows the containers that will be created when running the
`docker-compose up -d` command.
+
+```mermaid
+graph TB
+ subgraph "Apache Ozone Architecture"
+ scm["SCM<br/>Storage Container Manager<br/>(Port: 9876, 9860)"]
+ om["OM<br/>Ozone Manager<br/>(Port: 9874, 9862)"]
+ s3g["S3G<br/>S3 Gateway<br/>(Port: 9878)"]
+ recon["Recon<br/>Monitoring Service<br/>(Port: 9888)"]
+ httpfs["HttpFS<br/>HTTP FileSystem<br/>(Port: 14000)"]
+
+ subgraph "Data Nodes"
+ dn1["DataNode 1<br/>(Port: 19864, 9882)"]
+ dnN["DataNode N<br/>(Port: 19864, 9882)"]
+ end
+
+ %% Connections
+ scm --> dn1
+ scm --> dnN
+ om --> dn1
+ om --> dnN
+ s3g --> om
+ s3g --> scm
+ httpfs --> om
+ httpfs --> scm
+ recon --> scm
+ recon --> om
+ end
+
+ classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px;
+ classDef datanode fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
+ classDef manager fill:#e8f5e9,stroke:#388e3c,stroke-width:2px;
+
+ class dn1,dnN datanode;
+ class scm,om manager;
+```
+
+## Cluster Configuration
+
+### Default Services
+
+The default Docker Compose configuration includes:
+
+- Storage Container Manager (SCM)
+- Ozone Manager (OM)
+- S3 Gateway
+- Recon (Monitoring Service)
+- Datanodes
+- HttpFS
+
+### Port Mappings
+
+The default setup exposes the following ports:
Review Comment:
>Default ports will be covered in
docs/administrator-guide/configuration/basic/network/
Removed the port mappings section here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
