This is an automated email from the ASF dual-hosted git repository. yuqi1129 pushed a commit to branch feat/mcp-governance-task3-6 in repository https://gitbox.apache.org/repos/asf/gravitino.git
commit 61d3bdf085a12e7e31602dc852b67e601ec3f77b Author: yuqi <[email protected]> AuthorDate: Thu Jun 11 16:37:39 2026 +0800 Add localtest example and docs. --- mcp-server/dev/INSPECTOR_DEMO.md | 208 ++++++++++++++++++++++++++ mcp-server/dev/start_inspector_demo.sh | 260 +++++++++++++++++++++++++++++++++ mcp-server/dev/stop_inspector_demo.sh | 77 ++++++++++ 3 files changed, 545 insertions(+) diff --git a/mcp-server/dev/INSPECTOR_DEMO.md b/mcp-server/dev/INSPECTOR_DEMO.md new file mode 100644 index 0000000000..d3a86f9b46 --- /dev/null +++ b/mcp-server/dev/INSPECTOR_DEMO.md @@ -0,0 +1,208 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Hands-on MCP Authorization Demo (with MCP Inspector) + +Drive the Gravitino MCP server interactively through the +[MCP Inspector](https://github.com/modelcontextprotocol/inspector) and watch +per-user authorization and audit work end to end, as two principals +(`admin` and `bob`). + +This exercises the three governance moments: + +1. **Scoped discovery** — `admin` and `bob` run the same call and get different, + authorization-scoped results. +2. **Write denied** — `bob` (read-only) attempts a write and is denied by + Gravitino authorization, surfaced as an explicit error through MCP. +3. **Audit trail** — every call produces an audit record attributed to the + correct principal with an allow/deny outcome. + +--- + +## 1. Prerequisites + +- A built Gravitino distribution. If you don't have one: + ```bash + ./gradlew compileDistribution -x test -PskipWeb=true + # produces distribution/package/ + ``` +- `uv` available in the `mcp-server/` directory (the project already uses it). +- Node.js (the start script launches the Inspector via `npx @modelcontextprotocol/inspector`). +- If you run behind an HTTP proxy, the scripts already export + `NO_PROXY=localhost,127.0.0.1`; make sure your shell doesn't force the proxy + for loopback in some other way. + +--- + +## 2. Start the demo environment + +From the `mcp-server/` directory: + +```bash +./dev/start_inspector_demo.sh +``` + +This script (idempotent — safe to re-run): + +1. Enables `simple` auth + authorization in `distribution/package/conf/gravitino.conf` + (backing up the original). +2. Starts Gravitino (or reuses a running one). +3. Provisions demo data into metalake **`mcp_authz_it`**: + - `cat_allowed` and `cat_denied` (two model catalogs) + - user **`bob`** + - a reader role granting `bob` `USE_CATALOG` on `cat_allowed` **only** +4. Starts the MCP server in HTTP mode at `http://127.0.0.1:8000/mcp`. +5. Starts the **MCP Inspector** at `http://localhost:6274/` (with the session-token + requirement disabled, so the plain URL works directly). + +On success it prints the connection details and the two principals' tokens. You +should see the provisioning summary confirming the different slices: + +``` +[demo] admin sees catalogs: ['cat_allowed', 'cat_denied'] +[demo] bob sees catalogs: ['cat_allowed'] +``` + +> If a distribution isn't found, set `GRAVITINO_HOME=/path/to/distribution`. + +--- + +## 3. Open the Inspector + +The start script already launched it. Just open: + +``` +http://localhost:6274/ +``` + +(No session token needed — the script starts it with `DANGEROUSLY_OMIT_AUTH=true` +for local convenience.) + +### Connect + +| Field | Value | +|----------------|------------------------------------------| +| Transport Type | `Streamable HTTP` | +| URL | `http://127.0.0.1:8000/mcp` | +| Header Name | `Authorization` | +| Header Value | `Basic YWRtaW46ZHVtbXk=` (this is `admin`) | + +The two principal tokens (these are Gravitino simple-auth headers, i.e. +`Basic base64("<user>:dummy")`): + +| Principal | Authorization header value | +|-----------|--------------------------------| +| `admin` | `Basic YWRtaW46ZHVtbXk=` | +| `bob` | `Basic Ym9iOmR1bW15` | + +Click **Connect**, then **List Tools** — you should see the full read + write +tool surface (catalogs, schemas, tables, filesets, topics, models, tags, …). + +> **Identity is the header.** The Inspector sends your `Authorization` header on +> every request; the MCP server forwards it verbatim to Gravitino, which +> authorizes against that principal. To switch principals, change the header +> value and reconnect. + +--- + +## 4. The three scenarios + +Keep a terminal tailing the audit log while you click: + +```bash +tail -f gravitino-mcp-audit.log +``` + +### Scenario 1 — Scoped discovery + +1. Connected as **admin**, run tool **`get_list_of_catalogs`**. + → Returns **both** `cat_allowed` and `cat_denied`. +2. Reconnect with the **bob** header (`Basic Ym9iOmR1bW15`), run + **`get_list_of_catalogs`** again. + → Returns **only** `cat_allowed`. + +Same call, different results — sourced entirely from Gravitino's list filtering, +not from any logic in the MCP server. + +### Scenario 2 — Write denied by authorization + +Still connected as **bob**, run **`create_tag`** with arguments: + +```json +{ "name": "test_tag", "comment": "x", "properties": {} } +``` + +→ You get an explicit error, e.g. +`User 'bob' is not authorized to perform operation 'createTag' on metadata 'mcp_authz_it'`. + +It's a real authorization denial, not a hidden tool or a silent no-op. Reconnect +as **admin** and run the same `create_tag` — it succeeds (admin owns the metalake). + +### Scenario 3 — Audit trail + +Look at the audit log you've been tailing. You should see discrete, correctly +attributed records, for example: + +```json +{"timestamp": "...", "principal": "admin", "tool": "get_list_of_catalogs", "outcome": "allow"} +{"timestamp": "...", "principal": "bob", "tool": "get_list_of_catalogs", "outcome": "allow"} +{"timestamp": "...", "principal": "bob", "tool": "create_tag", "outcome": "deny", "error_type": "McpError"} +``` + +`admin`'s reads attributed to `admin`, `bob`'s reads to `bob`, and `bob`'s denied +write to `bob` with a `deny` outcome. + +--- + +## 5. Tear down + +```bash +./dev/stop_inspector_demo.sh +``` + +Stops the Inspector, the MCP server, and Gravitino, and restores the original +`gravitino.conf`. + +--- + +## 6. Troubleshooting + +**The audit log looks empty / stuck at 0 bytes.** +The MCP server opens the audit file once at startup and holds the handle. If you +`rm` the file while the server is running, the process keeps writing to the now +unlinked inode and a new empty file appears at the path — so you see nothing. +Don't delete it mid-run; truncate instead (`: > gravitino-mcp-audit.log`), or +restart the server. The start script truncates rather than deletes for exactly +this reason. + +**`HTTP 000` / `502` when curling localhost.** +An HTTP proxy is intercepting loopback traffic. Export +`NO_PROXY=localhost,127.0.0.1` (the scripts already do) or pass `curl --noproxy '*'`. + +**MCP server fails to bind (`can't assign requested address`).** +`localhost` resolved to a non-loopback address. The scripts bind to the +`127.0.0.1` literal to avoid this. + +**`ModuleNotFoundError: No module named 'mcp_server'`.** +Launch with `uv run python -m mcp_server ...` (the scripts do); `uv run mcp_server` +needs an editable install. + +**Re-running the start script.** +It's idempotent: it reuses a running Gravitino/MCP server and drops+recreates the +demo metalake, so you can re-run it freely. diff --git a/mcp-server/dev/start_inspector_demo.sh b/mcp-server/dev/start_inspector_demo.sh new file mode 100755 index 0000000000..65ac519f72 --- /dev/null +++ b/mcp-server/dev/start_inspector_demo.sh @@ -0,0 +1,260 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# Bring up a persistent local demo environment for hands-on MCP authorization +# testing with the MCP Inspector: +# +# 1. Gravitino server with simple auth + authorization enabled. +# 2. Demo data: metalake + two catalogs, user "bob", a role granting bob +# access to only one catalog (so admin and bob see different slices). +# 3. MCP server in HTTP transport mode. +# +# Unlike run_authz_integration_test.sh, this script LEAVES everything running so +# you can drive it from the Inspector. Run stop_inspector_demo.sh to tear down. +# +# Usage: +# ./dev/start_inspector_demo.sh +# GRAVITINO_HOME=/path/to/distribution ./dev/start_inspector_demo.sh + +set -euo pipefail + +# Everything is local; never route through an HTTP proxy. +export NO_PROXY="localhost,127.0.0.1" +export no_proxy="localhost,127.0.0.1" + +MCP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +REPO_ROOT="$(cd "${MCP_DIR}/.." && pwd)" + +GRAVITINO_PORT="${GRAVITINO_PORT:-8090}" +GRAVITINO_URI="http://127.0.0.1:${GRAVITINO_PORT}"; +MCP_PORT="${MCP_PORT:-8000}" +MCP_URL="http://127.0.0.1:${MCP_PORT}/mcp"; +MCP_METALAKE="${MCP_METALAKE:-mcp_authz_it}" +MCP_PID_FILE="${MCP_DIR}/.inspector-demo-mcp.pid" +ADMIN_AUTH="Basic $(printf '%s' 'admin:dummy' | base64)" + +# MCP Inspector ports (defaults baked into @modelcontextprotocol/inspector). +INSPECTOR_UI_PORT="${INSPECTOR_UI_PORT:-6274}" +INSPECTOR_PROXY_PORT="${INSPECTOR_PROXY_PORT:-6277}" +INSPECTOR_PID_FILE="${MCP_DIR}/.inspector-demo-inspector.pid" + +log() { echo "[demo] $*"; } + +# --------------------------------------------------------------------------- +# 1. Resolve the Gravitino distribution +# --------------------------------------------------------------------------- +if [[ -z "${GRAVITINO_HOME:-}" ]]; then + GRAVITINO_HOME="${REPO_ROOT}/distribution/package" +fi +if [[ ! -x "${GRAVITINO_HOME}/bin/gravitino.sh" ]]; then + log "Distribution not found at ${GRAVITINO_HOME}." + log "Build it first: ./gradlew compileDistribution -x test -PskipWeb=true" + log "Or set GRAVITINO_HOME to an existing distribution." + exit 1 +fi +log "Using GRAVITINO_HOME=${GRAVITINO_HOME}" + +# --------------------------------------------------------------------------- +# 2. Enable simple auth + authorization (idempotent) +# --------------------------------------------------------------------------- +CONF="${GRAVITINO_HOME}/conf/gravitino.conf" +if ! grep -q "gravitino.authorization.enable = true" "${CONF}"; then + cp "${CONF}" "${CONF}.inspector-demo.bak" + sed -i.tmp \ + -e '/^gravitino.authenticators/d' \ + -e '/^gravitino.authorization.enable/d' \ + -e '/^gravitino.authorization.serviceAdmins/d' \ + "${CONF}" + rm -f "${CONF}.tmp" + cat >> "${CONF}" <<EOF + +# --- injected by start_inspector_demo.sh (restored on stop) --- +gravitino.authenticators = simple +gravitino.authorization.enable = true +gravitino.authorization.serviceAdmins = admin +EOF + log "Configured simple auth + authorization (serviceAdmins=admin)" +else + log "Authorization already enabled in config" +fi + +# --------------------------------------------------------------------------- +# 3. Start Gravitino (if not already up) +# --------------------------------------------------------------------------- +if curl -sf --noproxy '*' -H "Authorization: ${ADMIN_AUTH}" \ + "${GRAVITINO_URI}/api/version" >/dev/null 2>&1; then + log "Gravitino already running on ${GRAVITINO_PORT}" +else + log "Starting Gravitino server..." + "${GRAVITINO_HOME}/bin/gravitino.sh" start + log "Waiting for Gravitino to become healthy..." + for i in $(seq 1 60); do + if curl -sf --noproxy '*' -H "Authorization: ${ADMIN_AUTH}" \ + "${GRAVITINO_URI}/api/version" >/dev/null 2>&1; then + log "Gravitino is up." + break + fi + if [[ "${i}" == "60" ]]; then + log "ERROR: Gravitino did not become healthy in time" + exit 1 + fi + sleep 2 + done +fi + +# --------------------------------------------------------------------------- +# 4. Provision demo data (idempotent: drop + recreate the metalake) +# --------------------------------------------------------------------------- +log "Provisioning demo data into metalake '${MCP_METALAKE}'..." +( + cd "${MCP_DIR}" + GRAVITINO_URI="${GRAVITINO_URI}" MCP_METALAKE="${MCP_METALAKE}" \ + uv run python <<'PY' +import base64 +import os + +import httpx + +from tests.integration.gravitino_setup import GravitinoFixture + +uri = os.environ["GRAVITINO_URI"] +ml = os.environ["MCP_METALAKE"] + + +def hdr(user): + return "Basic " + base64.b64encode(f"{user}:dummy".encode()).decode() + + +# Drop an existing metalake so the script can be re-run cleanly. +client = httpx.Client( + base_url=uri, headers={"Authorization": hdr("admin")}, timeout=30 +) +if client.get(f"/api/metalakes/{ml}").status_code == 200: + client.put( + f"/api/metalakes/{ml}", + json={"updates": [{"@type": "setProperty", "property": "in-use", "value": "false"}]}, + ) + client.delete(f"/api/metalakes/{ml}?force=true") + print(f"[demo] removed existing metalake '{ml}'") +client.close() + +GravitinoFixture(uri, ml).provision() +print(f"[demo] provisioned metalake '{ml}': cat_allowed, cat_denied, user bob, reader role") + +# Show the resulting authorization slice. +for user in ("admin", "bob"): + r = httpx.get( + f"{uri}/api/metalakes/{ml}/catalogs?details=true", + headers={"Authorization": hdr(user)}, + ) + names = [c["name"] for c in r.json().get("catalogs", [])] + print(f"[demo] {user} sees catalogs: {names}") +PY +) + +# --------------------------------------------------------------------------- +# 5. Start the MCP server in HTTP mode (if not already up) +# --------------------------------------------------------------------------- +if nc -z 127.0.0.1 "${MCP_PORT}" 2>/dev/null; then + log "An MCP server is already listening on ${MCP_PORT}; leaving it as-is." +else + log "Starting MCP server (HTTP) on ${MCP_URL}..." + ( + cd "${MCP_DIR}" + # Truncate (not delete) the audit log so the running process keeps its handle. + : > gravitino-mcp-audit.log + nohup uv run python -m mcp_server \ + --metalake "${MCP_METALAKE}" \ + --gravitino-uri "${GRAVITINO_URI}" \ + --transport http \ + --mcp-url "${MCP_URL}" > "${MCP_DIR}/.inspector-demo-mcp.out" 2>&1 & + echo $! > "${MCP_PID_FILE}" + ) + for i in $(seq 1 30); do + nc -z 127.0.0.1 "${MCP_PORT}" 2>/dev/null && break + sleep 1 + done + if nc -z 127.0.0.1 "${MCP_PORT}" 2>/dev/null; then + log "MCP server is up (pid $(cat "${MCP_PID_FILE}"))." + else + log "ERROR: MCP server did not start; see ${MCP_DIR}/.inspector-demo-mcp.out" + exit 1 + fi +fi + +# --------------------------------------------------------------------------- +# 6. Start the MCP Inspector (UI on :6274, proxy on :6277) +# --------------------------------------------------------------------------- +# DANGEROUSLY_OMIT_AUTH disables the session-token requirement so the plain +# http://localhost:6274/ URL works without a token (fine for a local demo). +# MCP_AUTO_OPEN_ENABLED=false keeps it from popping a browser when backgrounded. +# The Inspector binds to "localhost" (may be IPv6 ::1), so probe via localhost. +if nc -z localhost "${INSPECTOR_UI_PORT}" 2>/dev/null; then + log "An Inspector is already listening on ${INSPECTOR_UI_PORT}; leaving it as-is." +else + log "Starting MCP Inspector on http://localhost:${INSPECTOR_UI_PORT} ..." + ( + cd "${MCP_DIR}" + DANGEROUSLY_OMIT_AUTH=true \ + MCP_AUTO_OPEN_ENABLED=false \ + nohup npx @modelcontextprotocol/inspector \ + > "${MCP_DIR}/.inspector-demo-inspector.out" 2>&1 & + echo $! > "${INSPECTOR_PID_FILE}" + ) + for i in $(seq 1 60); do + nc -z localhost "${INSPECTOR_UI_PORT}" 2>/dev/null && break + sleep 1 + done + if nc -z localhost "${INSPECTOR_UI_PORT}" 2>/dev/null; then + log "Inspector is up (pid $(cat "${INSPECTOR_PID_FILE}"))." + else + log "WARN: Inspector did not come up; see ${MCP_DIR}/.inspector-demo-inspector.out" + log " (You can still start it manually: npx @modelcontextprotocol/inspector)" + fi +fi + +# --------------------------------------------------------------------------- +# 7. Print connection details +# --------------------------------------------------------------------------- +cat <<EOF + +============================================================ + Demo environment is ready. +============================================================ + Gravitino : ${GRAVITINO_URI} (simple auth + authorization) + MCP server: ${MCP_URL} (Streamable HTTP) + Metalake : ${MCP_METALAKE} + Audit log : ${MCP_DIR}/gravitino-mcp-audit.log + + >>> Open the Inspector: http://localhost:${INSPECTOR_UI_PORT}/ + + In the Inspector, connect with: + Transport Type : Streamable HTTP + URL : ${MCP_URL} + Header Name : Authorization + Header Value : ${ADMIN_AUTH} <- admin + Basic $(printf '%s' 'bob:dummy' | base64) <- bob + + Watch audit records live: + tail -f ${MCP_DIR}/gravitino-mcp-audit.log + + Tear everything down (includes the Inspector): + ./dev/stop_inspector_demo.sh +============================================================ +EOF diff --git a/mcp-server/dev/stop_inspector_demo.sh b/mcp-server/dev/stop_inspector_demo.sh new file mode 100755 index 0000000000..0335dac0f2 --- /dev/null +++ b/mcp-server/dev/stop_inspector_demo.sh @@ -0,0 +1,77 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# Tear down the demo environment started by start_inspector_demo.sh: +# stops the MCP server, stops Gravitino, and restores the original config. + +set -uo pipefail + +MCP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +REPO_ROOT="$(cd "${MCP_DIR}/.." && pwd)" + +MCP_PORT="${MCP_PORT:-8000}" +MCP_PID_FILE="${MCP_DIR}/.inspector-demo-mcp.pid" +INSPECTOR_UI_PORT="${INSPECTOR_UI_PORT:-6274}" +INSPECTOR_PROXY_PORT="${INSPECTOR_PROXY_PORT:-6277}" +INSPECTOR_PID_FILE="${MCP_DIR}/.inspector-demo-inspector.pid" + +log() { echo "[demo] $*"; } + +if [[ -z "${GRAVITINO_HOME:-}" ]]; then + GRAVITINO_HOME="${REPO_ROOT}/distribution/package" +fi + +# 1. Stop the MCP Inspector (npx spawns child processes; free its ports too). +if [[ -f "${INSPECTOR_PID_FILE}" ]]; then + INSPECTOR_PID="$(cat "${INSPECTOR_PID_FILE}")" + kill "${INSPECTOR_PID}" 2>/dev/null && \ + log "Stopped Inspector (pid ${INSPECTOR_PID})" || true + rm -f "${INSPECTOR_PID_FILE}" +fi +for port in "${INSPECTOR_UI_PORT}" "${INSPECTOR_PROXY_PORT}"; do + lsof -ti :"${port}" 2>/dev/null | xargs kill -9 2>/dev/null && \ + log "Freed Inspector port ${port}" || true +done + +# 2. Stop the MCP server. +if [[ -f "${MCP_PID_FILE}" ]]; then + MCP_PID="$(cat "${MCP_PID_FILE}")" + if kill "${MCP_PID}" 2>/dev/null; then + log "Stopped MCP server (pid ${MCP_PID})" + fi + rm -f "${MCP_PID_FILE}" +fi +# Belt and suspenders: free the port if anything is still bound. +lsof -ti :"${MCP_PORT}" 2>/dev/null | xargs kill -9 2>/dev/null && \ + log "Freed port ${MCP_PORT}" || true + +# 3. Stop Gravitino. +if [[ -x "${GRAVITINO_HOME}/bin/gravitino.sh" ]]; then + "${GRAVITINO_HOME}/bin/gravitino.sh" stop || true + log "Stopped Gravitino" +fi + +# 4. Restore the original config. +CONF="${GRAVITINO_HOME}/conf/gravitino.conf" +if [[ -f "${CONF}.inspector-demo.bak" ]]; then + mv "${CONF}.inspector-demo.bak" "${CONF}" + log "Restored original gravitino.conf" +fi + +log "Teardown complete."
