ptlrs commented on code in PR #101: URL: https://github.com/apache/ozone-site/pull/101#discussion_r1842716649
########## docs/08-developer-guide/02-run/02-docker-compose.md: ########## @@ -10,3 +10,143 @@ sidebar_label: Docker Compose - Define the Ozone runner image at `ozone-docker-runner` and its purpose to wrap Ozone binaries. - How to define which image gets used in the cluster (runner or other pre-built image). - Changing configurations in Docker Compose (`docker-config` file, transformation.py, etc). + +This guide explains how to run Apache Ozone using Docker Compose, either with locally built sources or pre-built images. + +## Prerequisites + +- Docker Engine 20.10.0 or higher +- Docker Compose V2 +- Built Ozone distribution (if running from local build) + +## Running Ozone + +You can run Ozone either using your locally built version or using official pre-built Docker images. + +### Option 1: Running from Local Build + +If you've built Ozone from source, follow these steps: + +1. Navigate to the compose directory in your build output: + + ```bash + cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/ozone + ``` + +2. Start the cluster: + + ```bash + docker-compose up -d + ``` + +The local build uses the `ozone-docker-runner` image, which is automatically created during the build process to wrap your compiled Ozone binaries. + +### Option 2: Running from Pre-built Images + +If you haven't built Ozone locally, you can quickly start a cluster using official pre-built images from Docker Hub: + +1. Create a directory for your Ozone deployment: + + ```bash + mkdir ozone && cd ozone + ``` + +2. Download the example compose file: + + ```bash + curl \ + https://raw.githubusercontent.com/apache/ozone/master/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml \ + -o docker-compose.yaml + ``` + +3. Start the cluster: + + ```bash + docker-compose up -d + ``` + +This will pull the official Apache Ozone images from Docker Hub. + +## Architecture Diagram + +This image shows the containers that will be created when running the `docker-compose up -d` command. + +```mermaid +graph TB + subgraph "Apache Ozone Architecture" + scm["SCM<br/>Storage Container Manager<br/>(Port: 9876, 9860)"] + om["OM<br/>Ozone Manager<br/>(Port: 9874, 9862)"] + s3g["S3G<br/>S3 Gateway<br/>(Port: 9878)"] + recon["Recon<br/>Monitoring Service<br/>(Port: 9888)"] + httpfs["HttpFS<br/>HTTP FileSystem<br/>(Port: 14000)"] + + subgraph "Data Nodes" + dn1["DataNode 1<br/>(Port: 19864, 9882)"] + dnN["DataNode N<br/>(Port: 19864, 9882)"] + end + + %% Connections + scm --> dn1 + scm --> dnN + om --> dn1 + om --> dnN + s3g --> om + s3g --> scm + httpfs --> om + httpfs --> scm + recon --> scm + recon --> om + end + + classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px; + classDef datanode fill:#e1f5fe,stroke:#0288d1,stroke-width:2px; + classDef manager fill:#e8f5e9,stroke:#388e3c,stroke-width:2px; + + class dn1,dnN datanode; + class scm,om manager; +``` + +## Cluster Configuration + +### Default Services + +The default Docker Compose configuration includes: + +- Storage Container Manager (SCM) +- Ozone Manager (OM) +- S3 Gateway +- Recon (Monitoring Service) +- Datanodes +- HttpFS + +### Port Mappings + +The default setup exposes the following ports: + +- SCM: 9876 (RPC), 9860 (Client) +- OM: 9874 (RPC), 9862 (Client) +- S3 Gateway: 9878 +- Recon: 9888 +- HttpFS: 14000 +- Datanodes: 19864 (Container), 9882 (Client) + +## Cluster Management + +Common Docker Compose commands: + +```bash +# Start the cluster +docker-compose up -d + +# Stop the cluster +docker-compose down + +# View service logs +docker-compose logs -f [service_name] + +# Scale data nodes +docker-compose up -d --scale datanode=3 Review Comment: Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
