snazy commented on code in PR #3022: URL: https://github.com/apache/polaris/pull/3022#discussion_r2513267475
########## getting-started/ceph/README.md: ########## @@ -0,0 +1,138 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Getting Started with Apache Polaris and Ceph + +## Overview + +This guide describes how to spin up a **single-node Ceph cluster** with **RADOS Gateway (RGW)** for S3-compatible storage and configure it for use by **Polaris**. + +This example cluster is configured for basic access key authentication only. +It does not include STS (Security Token Service) or temporary credentials. +All access to the Ceph RGW (RADOS Gateway) and Polaris integration uses static S3-style credentials (as configured via radosgw-admin user create). + +Spark is used as a query engine. This example assumes a local Spark installation. +See the [Spark Notebooks Example](../spark/README.md) for a more advanced Spark setup. + +## Starting the Example + +The services are started **in sequence**: +1. Monitor + Manager +2. OSD +3. RGW +4. Polaris + +Note: this example pulls the `apache/polaris:latest` image, but assumes the image is `1.2.0-incubating` or later. + + +### 1. Start monitor and manager +```shell +docker-compose up -d mon1 mgr +``` + +### 2. Start OSD +```shell +docker-compose up -d osd1 +``` + +### 3. Start RGW +```shell +docker-compose up -d rgw1 +``` +#### Check status +```shell +docker exec -it cephpolaris-mon1-1 ceph -s Review Comment: ```suggestion docker exec --interactive --tty cephpolaris-mon1-1 ceph -s ``` People may not intuitively know what a single-char option means. I find it more descriptive to use the long-arg variants (here and in the other places). ########## getting-started/ceph/docker-compose.yml: ########## @@ -0,0 +1,227 @@ +networks: Review Comment: This file and all other files must have a license header. ########## getting-started/ceph/docker-compose.yml: ########## @@ -0,0 +1,227 @@ +networks: + cluster-net: + driver: bridge + +services: + + mon1: + image: ${CEPH_CONTAINER_IMAGE} + entrypoint: "/bin/sh" + command: + - "-c" + - >- + set -ex; + mkdir -p /var/lib/ceph/osd/ceph-0; + ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'; + ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring \ + --gen-key -n client.admin \ + --cap mon 'allow *' --cap osd 'allow *' --cap mgr 'allow *' --cap mds 'allow *'; + ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring \ + --gen-key -n client.bootstrap-osd \ + --cap mon 'profile bootstrap-osd' --cap mgr 'allow r'; + ceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring; + ceph-authtool /tmp/ceph.mon.keyring --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring; + chown ceph:ceph /tmp/ceph.mon.keyring; + monmaptool --create --add mon1 ${MON_IP} --fsid ${FSID} /tmp/monmap --clobber; + sudo -u ceph ceph-mon --mkfs -i mon1 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring; + ceph-mon -i mon1 -f -d; + environment: + MON_IP: ${MON_IP} + CEPH_PUBLIC_NETWORK: ${MON1_CEPH_PUBLIC_NETWORK} + FSID: ${FSID} + volumes: + - ./ceph-conf:/etc/ceph + - ./bootstrap-osd:/var/lib/ceph/bootstrap-osd + - ./osd1:/var/lib/ceph/osd/ceph-0/ + networks: + - cluster-net + + mgr: + image: ${CEPH_CONTAINER_IMAGE} + entrypoint: "/bin/sh" + command: + - "-c" + - >- + set -ex; + mkdir -p /var/lib/ceph/mgr/ceph-mgr; + ceph auth get-or-create mgr.mgr mon 'allow profile mgr' osd 'allow *' mds 'allow *' > /var/lib/ceph/mgr/ceph-mgr/keyring; + ceph-mgr -f -i mgr; + volumes: + - ./ceph-conf:/etc/ceph + depends_on: + - mon1 + networks: + - cluster-net + ports: + - ${DASHBOARD_PORT}:${INTERNAL_DASHBOARD_PORT} + + osd1: + pid: host + privileged: true + image: ${CEPH_CONTAINER_IMAGE} + environment: + OSD_UUID_1: ${OSD_UUID_1} + entrypoint: "/bin/sh" + command: + - "-c" + - >- + set -ex; + mkdir -p /var/lib/ceph/osd/ceph-0; + chown -R ceph:ceph /var/lib/ceph/osd/ceph-0; + ceph-authtool --create-keyring /var/lib/ceph/osd/ceph-0/keyring \ + --gen-key -n osd.0 \ + --cap osd 'allow *' \ + --cap mon 'allow profile osd'; + ceph auth del osd.0 || true; + ceph auth add osd.0 -i /var/lib/ceph/osd/ceph-0/keyring; + ceph osd new ${OSD_UUID_1} -n client.bootstrap-osd -k /var/lib/ceph/bootstrap-osd/ceph.keyring; + ceph-osd -i 0 --mkfs --osd-data /var/lib/ceph/osd/ceph-0 --osd-uuid ${OSD_UUID_1} \ + --keyring /var/lib/ceph/osd/ceph-0/keyring; + ceph-osd -f -i 0; + volumes: + - ./ceph-conf:/etc/ceph + - ./bootstrap-osd:/var/lib/ceph/bootstrap-osd + depends_on: + - mon1 + networks: + - cluster-net + + mds1: + image: ${CEPH_CONTAINER_IMAGE} + entrypoint: "/bin/sh" + command: + - "-c" + - >- + set -ex; + mkdir -p /var/lib/ceph/mds/ceph-admin; + ceph-authtool --create-keyring /var/lib/ceph/mds/ceph-admin/keyring --gen-key -n mds. --cap mds 'allow *'; + ceph-mds -f -i admin; + hostname: "ceph-mds1-host" + environment: + CEPHFS_CREATE: 1 + volumes: + - ./ceph-conf:/etc/ceph + depends_on: + - osd1 + networks: + - cluster-net + rgw1: + image: ${CEPH_CONTAINER_IMAGE} + container_name: rgw1 + environment: + MON_IP: ${MON_IP} + CEPH_PUBLIC_NETWORK: ${MON1_CEPH_PUBLIC_NETWORK} + RGW_ACCESS_KEY: ${RGW_ACCESS_KEY} + RGW_SECRET_KEY: ${RGW_SECRET_KEY} + entrypoint: "/bin/sh" + command: + - "-c" + - >- + set -ex; + mkdir -p /var/lib/ceph/radosgw/ceph-rgw1; + ceph auth get-or-create client.rgw1 mon 'allow rw' osd 'allow rwx'; + ceph auth caps client.rgw1 mon 'allow rw' osd 'allow rwx'; + ceph-authtool --create-keyring /var/lib/ceph/radosgw/ceph-rgw1/keyring --gen-key -n client.rgw1 --cap osd 'allow *' --cap mon 'allow *'; + ceph auth del client.rgw1 || true; + ceph auth add client.rgw1 -i /var/lib/ceph/radosgw/ceph-rgw1/keyring; + radosgw-admin user create --uid="polaris-user" \ + --display-name="Polaris User" \ + --access-key="${RGW_ACCESS_KEY}" \ + --secret-key="${RGW_SECRET_KEY}" || true; + echo ">>> RGW user created (access=${RGW_ACCESS_KEY}, secret=${RGW_SECRET_KEY})"; + radosgw -n client.rgw1 --rgw-frontends="beast port=7480" --foreground; + ports: + - "7480:7480" # RGW HTTP endpoint (S3) + - "7481:7481" + volumes: + - ./ceph-conf:/etc/ceph + depends_on: + - osd1 + networks: + - cluster-net + + setup_bucket: + image: peakcom/s5cmd:latest + depends_on: + - rgw1 + environment: + AWS_ACCESS_KEY_ID: ${RGW_ACCESS_KEY} + AWS_SECRET_ACCESS_KEY: ${RGW_SECRET_KEY} + S3_ENDPOINT_URL: ${S3_ENDPOINT_URL} + S3_REGION: ${S3_REGION} + S3_POLARIS_BUCKET: ${S3_POLARIS_BUCKET} + entrypoint: "/bin/sh" + command: + - "-c" + - >- + set -ex; + echo ">>> Waiting for RGW to become ready..."; + sleep 5; + echo ">>> Create bucket if not exist..."; + /s5cmd --endpoint-url ${S3_ENDPOINT_URL} mb s3://${S3_POLARIS_BUCKET} || true; + tail -f /dev/null; + networks: + - cluster-net + + polaris: + image: apache/polaris:latest + ports: + # API port + - "8181:8181" + # Optional, allows attaching a debugger to the Polaris JVM + - "5005:5005" + depends_on: + - rgw1 + environment: + JAVA_DEBUG: true + JAVA_DEBUG_PORT: "*:5005" + AWS_REGION: us-west-2 Review Comment: Hm, this region is different from the one in the dot-env file, which is confusing. ########## getting-started/ceph/.env.example: ########## @@ -0,0 +1,15 @@ +LANG=en_US.utf8 +TZ=UTC +CEPH_CONTAINER_IMAGE=quay.io/ceph/ceph:v19.2.3 +DASHBOARD_PORT=8443 +INTERNAL_DASHBOARD_PORT=8443 +RGW_PORT=8080 Review Comment: Does it make sense to group the variables together add add some brief comments about their meanings for Ceph newbies? ########## getting-started/ceph/README.md: ########## @@ -0,0 +1,138 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Getting Started with Apache Polaris and Ceph + +## Overview + +This guide describes how to spin up a **single-node Ceph cluster** with **RADOS Gateway (RGW)** for S3-compatible storage and configure it for use by **Polaris**. + +This example cluster is configured for basic access key authentication only. +It does not include STS (Security Token Service) or temporary credentials. Review Comment: Would you mind adding a getting-started with IAM/STS as a follow-up of this PR? ########## getting-started/ceph/.env.example: ########## @@ -0,0 +1,15 @@ +LANG=en_US.utf8 +TZ=UTC +CEPH_CONTAINER_IMAGE=quay.io/ceph/ceph:v19.2.3 Review Comment: IIRC you can use the `env_file:` compose attribute to "import" a dot-env file. ########## getting-started/ceph/.env.example: ########## @@ -0,0 +1,15 @@ +LANG=en_US.utf8 +TZ=UTC +CEPH_CONTAINER_IMAGE=quay.io/ceph/ceph:v19.2.3 Review Comment: Can you move the image reference directly into the `docker-compose.yml`? Renovate will create version-bump PR for the image automatically, but it can't do it here. ########## getting-started/ceph/README.md: ########## @@ -0,0 +1,138 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Getting Started with Apache Polaris and Ceph + +## Overview + +This guide describes how to spin up a **single-node Ceph cluster** with **RADOS Gateway (RGW)** for S3-compatible storage and configure it for use by **Polaris**. + +This example cluster is configured for basic access key authentication only. +It does not include STS (Security Token Service) or temporary credentials. +All access to the Ceph RGW (RADOS Gateway) and Polaris integration uses static S3-style credentials (as configured via radosgw-admin user create). + +Spark is used as a query engine. This example assumes a local Spark installation. +See the [Spark Notebooks Example](../spark/README.md) for a more advanced Spark setup. + +## Starting the Example + +The services are started **in sequence**: +1. Monitor + Manager +2. OSD +3. RGW +4. Polaris + +Note: this example pulls the `apache/polaris:latest` image, but assumes the image is `1.2.0-incubating` or later. + + +### 1. Start monitor and manager +```shell +docker-compose up -d mon1 mgr Review Comment: ```suggestion docker compose up -d mon1 mgr ``` Better replace `docker-compose` with `docker compose` as that should work with Podman + Docker CLI compatibility as well. ########## getting-started/ceph/.env.example: ########## @@ -0,0 +1,15 @@ +LANG=en_US.utf8 +TZ=UTC +CEPH_CONTAINER_IMAGE=quay.io/ceph/ceph:v19.2.3 Review Comment: I think it's easier for users to just execute the `docker compose up ...` commands without a prerequisite step. Just followed the readme's first instruction and it complained about a lot of unset variables. ########## getting-started/ceph/docker-compose.yml: ########## @@ -0,0 +1,227 @@ +networks: + cluster-net: + driver: bridge + +services: + + mon1: + image: ${CEPH_CONTAINER_IMAGE} Review Comment: Better hard-code the image name here, so that Renovate can update the image version via PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
