This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new bcd0ec4f74 fix(deploy): load example datasets against the account's
real email (#8723)
bcd0ec4f74 is described below
commit bcd0ec4f742264503114aa4388d82df8d35e9647
Author: Meng Wang <[email protected]>
AuthorDate: Sun Sep 27 23:45:05 2026 +0000
fix(deploy): load example datasets against the account's real email (#8723)
### What changes were proposed in this PR?
`bin/single-node/examples/load-examples.sh` derived the owner email that
file-service matches dataset lookups against by concatenating
`@example.com`
onto the username. On a single-node deployment that never matches: the
admin
seeded from `USER_SYS_ADMIN_USERNAME` has the username itself as its
email
(`AuthResource.createAdminUser`), so the loader logged in as that admin,
skipped
its register branch, and every example file upload failed with
`400 Dataset not found`. Both example datasets were created but left
with no
files and no version, so the two example workflows pointed at CSV paths
that do
not exist.
The owner email now comes from the access token's `email` claim, which
is
whatever the server actually stored — correct for the seeded admin and
for an
account the script had to register itself. The register call keeps an
`@`-shaped address, which the server's format check requires.
Second, the loader now exits non-zero once it has logged an error. Each
failure
is still non-fatal so one bad file does not abandon the rest, but they
used to
leave no trace: the container printed `=== Example data loading complete
===`
and exited 0. That is why the broken lookup went unnoticed for two
months.
### Any related issues, documentation, discussions?
Closes #8721.
Introduced by #6402, which had to give the register call a regex-valid
address
and changed the shared variable rather than splitting its two roles.
`release/v1.3` carries it too.
### How was this PR tested?
New `bin/single-node/tests/test_load_examples_sh.sh` (9 assertions,
picked up
automatically by build.yml's shell-test discovery) drives the loader
against a
stub `curl` — no docker, no network, no sleeping. It pins the owner
email to the
token's claim, asserts it is never `<username>@example.com`, and asserts
that a
failed upload or version create exits non-zero. Both fixes were reverted
individually to confirm the relevant assertions go red and the suite
exits 1.
End to end on a fresh single-node deployment (`down --volumes` then
`up --with-examples`): the loader now reports `Owner email resolved as
'texera'`
and uploads both files, `dataset_version` goes from 0 rows to 2 (both
`v1`), and
presigned download of the two paths the example workflows scan returns
HTTP 200.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-5)
---
bin/single-node/examples/load-examples.sh | 56 ++++++-
bin/single-node/tests/test_load_examples_sh.sh | 199 +++++++++++++++++++++++++
2 files changed, 251 insertions(+), 4 deletions(-)
diff --git a/bin/single-node/examples/load-examples.sh
b/bin/single-node/examples/load-examples.sh
index e6452d1152..93ad591ece 100755
--- a/bin/single-node/examples/load-examples.sh
+++ b/bin/single-node/examples/load-examples.sh
@@ -27,7 +27,15 @@
TEXERA_DASHBOARD_SERVICE_URL=${TEXERA_DASHBOARD_SERVICE_URL:-"http://dashboard-s
TEXERA_FILE_SERVICE_URL=${TEXERA_FILE_SERVICE_URL:-"http://file-service:9092/api"}
USERNAME=${TEXERA_EXAMPLE_USERNAME:-"texera"}
PASSWORD=${TEXERA_EXAMPLE_PASSWORD:-"texera"}
-OWNER_EMAIL="[email protected]"
+# Email this script registers the account with, on the path where it has to
+# create one. The server requires an `@`-shaped address here.
+REGISTER_EMAIL="[email protected]"
+# The account's real email, resolved from the access token in authenticate().
+# It is not derivable from $USERNAME: a freshly registered account carries
+# $REGISTER_EMAIL, while an admin seeded from USER_SYS_ADMIN_USERNAME carries
+# the username itself (AuthResource.createAdminUser). file-service looks
+# datasets up by owner email, so guessing it fails every upload (#8721).
+OWNER_EMAIL=""
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
DATASET_DIR="$SCRIPT_DIR/datasets"
@@ -36,6 +44,12 @@ WORKFLOW_DIR="$SCRIPT_DIR/workflows"
MAX_RETRIES=60
RETRY_INTERVAL=5
+# Counts print_error calls so main() can exit non-zero. Each failure below is
+# deliberately non-fatal so one bad file does not abandon the rest, but they
+# used to leave no trace at all: the loader reported success even when nothing
+# was uploaded (#8721).
+FAILURES=0
+
# Color codes for output
GREEN='\033[0;32m'
RED='\033[0;31m'
@@ -47,6 +61,7 @@ print_status() {
}
print_error() {
+ FAILURES=$((FAILURES + 1))
echo -e "${RED}[ERROR]${NC} $1"
}
@@ -54,6 +69,33 @@ print_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
+# Read one claim out of a server-issued JWT's payload. Only the payload is
+# decoded: the token came back from our own authenticated call, so there is
+# nothing to verify here -- this just avoids guessing a value the server has
+# already told us. jq does the base64 work because the `base64` binary's decode
+# flag differs between GNU and BSD, and this script's tests run on both.
+jwt_claim() {
+ local payload="$1" claim="$2"
+ payload=$(printf '%s' "$payload" | cut -d. -f2)
+ while [ $(( ${#payload} % 4 )) -ne 0 ]; do
+ payload="${payload}="
+ done
+ printf '%s' "$payload" \
+ | jq -Rr --arg c "$claim" '(gsub("-";"+") | gsub("_";"/") | @base64d |
fromjson)[$c] // empty'
+}
+
+# Resolve the authenticated account's email. file-service identifies a dataset
+# by (ownerEmail, datasetName), so this has to be the address the server
+# actually stored for this account.
+resolve_owner_email() {
+ OWNER_EMAIL=$(jwt_claim "$TOKEN" email 2>/dev/null || true)
+ if [ -z "$OWNER_EMAIL" ]; then
+ print_error "Could not read the 'email' claim from the access token"
+ return 1
+ fi
+ print_status "Owner email resolved as '$OWNER_EMAIL'"
+}
+
# Wait for a service to become healthy
wait_for_service() {
local service_name="$1"
@@ -86,17 +128,19 @@ authenticate() {
if echo "$LOGIN_RESPONSE" | grep -q '"accessToken"'; then
TOKEN=$(echo "$LOGIN_RESPONSE" | grep -o '"accessToken":"[^"]*' | cut
-d'"' -f4)
print_status "Login successful"
+ resolve_owner_email || return 1
return 0
fi
print_status "User doesn't exist, attempting to register..."
REGISTER_RESPONSE=$(curl -s -X POST
"$TEXERA_DASHBOARD_SERVICE_URL/auth/register" \
-H "Content-Type: application/json" \
- -d "{\"username\": \"$USERNAME\", \"email\": \"$OWNER_EMAIL\",
\"password\": \"$PASSWORD\"}")
+ -d "{\"username\": \"$USERNAME\", \"email\": \"$REGISTER_EMAIL\",
\"password\": \"$PASSWORD\"}")
if echo "$REGISTER_RESPONSE" | grep -q '"accessToken"'; then
TOKEN=$(echo "$REGISTER_RESPONSE" | grep -o '"accessToken":"[^"]*' |
cut -d'"' -f4)
print_status "Registration successful"
+ resolve_owner_email || return 1
return 0
fi
@@ -261,8 +305,7 @@ load_workflows() {
wid=$(echo "$response" | grep -o '"wid":[0-9]*' | cut -d':' -f2)
print_status "Workflow '$workflow_name' created with ID $wid"
else
- print_error "Failed to create workflow '$workflow_name'"
- print_error "Response: $response"
+ print_error "Failed to create workflow '$workflow_name': $response"
fi
done
@@ -281,6 +324,11 @@ main() {
load_datasets
load_workflows
+ if [ "$FAILURES" -ne 0 ]; then
+ echo -e "${RED}[ERROR]${NC} === Example data loading finished with
$FAILURES error(s) ==="
+ return 1
+ fi
+
print_status "=== Example data loading complete ==="
}
diff --git a/bin/single-node/tests/test_load_examples_sh.sh
b/bin/single-node/tests/test_load_examples_sh.sh
new file mode 100755
index 0000000000..a79b6f2aa0
--- /dev/null
+++ b/bin/single-node/tests/test_load_examples_sh.sh
@@ -0,0 +1,199 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Tests for bin/single-node/examples/load-examples.sh. Run from the repo root:
+# bash bin/single-node/tests/test_load_examples_sh.sh
+# Exits 0 if every check passes, 1 otherwise.
+#
+# The loader only talks to Texera over HTTP, so a stub `curl` placed ahead of
+# the real one on PATH is enough to drive every branch deterministically -- no
+# docker, no network, no sleeping. The two things guarded here both regressed
+# silently once (#8721):
+#
+# * the ownerEmail sent to file-service must be the email the server issued
+# in the token, not one concatenated from the username; and
+# * a failed upload must make the loader exit non-zero instead of printing
+# "complete".
+
+set -u
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
+LOADER="$REPO_ROOT/bin/single-node/examples/load-examples.sh"
+
+PASS=0
+FAIL=0
+
+_pass() { printf " \e[32m✓\e[0m %s\n" "$1"; PASS=$((PASS+1)); }
+_fail() {
+ printf " \e[31m✗\e[0m %s\n" "$1"
+ [[ $# -ge 2 ]] && printf " %s\n" "$2"
+ FAIL=$((FAIL+1))
+}
+
+# The loader itself needs jq, so a runner without it cannot exercise anything.
+if ! command -v jq >/dev/null 2>&1; then
+ _pass "skip: jq not installed — load-examples.sh cannot be exercised"
+ printf "\n%d passed, %d failed\n" "$PASS" "$FAIL"
+ exit 0
+fi
+
+# base64url, the JWT flavour: standard base64 with a different alphabet and no
+# padding. Built with jq so no `base64` binary is needed (GNU takes -d, BSD
-D).
+_b64url() { jq -Rr '@base64' | tr '+/' '-_' | tr -d '='; }
+
+# A syntactically real JWT whose payload is $1. Never verified by anything —
+# the loader only reads claims out of a token its own login call returned.
+_mk_jwt() {
+ printf '%s.%s.%s' \
+ "$(printf '%s' '{"alg":"HS256","typ":"JWT"}' | _b64url)" \
+ "$(printf '%s' "$1" | _b64url)" \
+ "c2lnbmF0dXJl"
+}
+
+# One isolated run of the loader: a scratch tree with one dataset and one
+# workflow, a stub curl on PATH, and the loader copied in so that its
+# SCRIPT_DIR-derived datasets/ and workflows/ point at the scratch tree.
+# Behaviour knobs are read from the environment by the stub (STUB_*).
+# Called directly (never in a command substitution, which would lose the
+# variables): sets $RUN_RC, $RUN_OUT and $RUN_LOG for the caller. The login
+# token is $1; any further KEY=VAL arguments become stub knobs.
+_run_loader() {
+ local token="$1"; shift
+ local tmp
+ tmp=$(mktemp -d)
+ mkdir -p "$tmp/bin" "$tmp/examples/datasets/iris-species"
"$tmp/examples/workflows"
+ printf 'sepal,species\n1.0,setosa\n' >
"$tmp/examples/datasets/iris-species/Iris.csv"
+ printf 'an example\n' >
"$tmp/examples/datasets/iris-species/description.txt"
+ printf '{"operators":[],"links":[]}\n' > "$tmp/examples/workflows/Example
Workflow.json"
+ cp "$LOADER" "$tmp/examples/load-examples.sh"
+
+ RUN_LOG="$tmp/requests.log"
+ : > "$RUN_LOG"
+
+ cat > "$tmp/bin/curl" <<STUB
+#!/usr/bin/env bash
+# Stub curl: log the requested URL, then answer with the shape the loader
+# greps for. Only the cases the loader actually calls are handled.
+url=""
+for a in "\$@"; do case "\$a" in http*) url="\$a"; break;; esac; done
+printf '%s\n' "\$url" >> "$RUN_LOG"
+
+case "\$url" in
+ *healthcheck*) printf '200' ;;
+ */auth/login)
+ if [ -n "\${STUB_NO_LOGIN:-}" ]; then printf '{"error":"no such user"}'
+ else printf '{"accessToken":"%s"}' "\${STUB_LOGIN_TOKEN}"; fi ;;
+ */auth/register) printf '{"accessToken":"%s"}'
"\${STUB_REGISTER_TOKEN:-\${STUB_LOGIN_TOKEN}}" ;;
+ */dataset/list|*/workflow/list) printf '[]' ;;
+ */dataset/create) printf '{"did":1}' ;;
+ *multipart-upload/part*) printf '200' ;;
+ *type=init*)
+ if [ -n "\${STUB_FAIL_INIT:-}" ]; then printf
'{"code":400,"message":"Dataset not found"}'
+ else printf '{"missingParts":[1],"completedPartsCount":0}'; fi ;;
+ *type=finish*) printf '{"message":"upload finished"}' ;;
+ *type=abort*) printf '{"message":"aborted"}' ;;
+ */version/create)
+ if [ -n "\${STUB_FAIL_VERSION:-}" ]; then printf '{"code":500}'
+ else printf '{"datasetVersion":{"dvid":1}}'; fi ;;
+ */workflow/create) printf '{"wid":1}' ;;
+ *) printf '{}' ;;
+esac
+STUB
+ chmod +x "$tmp/bin/curl"
+
+ RUN_RC=0
+ RUN_OUT=$(PATH="$tmp/bin:$PATH" STUB_LOGIN_TOKEN="$token" \
+ env "$@" bash "$tmp/examples/load-examples.sh" 2>&1) || RUN_RC=$?
+}
+
+ADMIN_TOKEN=$(_mk_jwt
'{"sub":"texera","userId":1,"email":"texera","role":"ADMIN"}')
+
+# 1) The owner email used for dataset lookups comes from the token, so an
+# account whose email is not "<username>@example.com" still gets its files.
+# This is the #8721 regression itself: every upload 400'd because the
+# loader guessed [email protected] for an admin stored as "texera".
+_run_loader "$ADMIN_TOKEN"
+if [[ "$RUN_RC" == 0 ]]; then
+ _pass "all-success run exits 0"
+else
+ _fail "all-success run should exit 0" "rc=$RUN_RC out=$(echo "$RUN_OUT" |
tail -3)"
+fi
+
+if grep -q 'ownerEmail=texera&' "$RUN_LOG"; then
+ _pass "ownerEmail is taken from the token's email claim"
+else
+ _fail "ownerEmail should be the token's email claim" \
+ "$(grep -o 'ownerEmail=[^&]*' "$RUN_LOG" | sort -u | tr '\n' ' ')"
+fi
+
+if grep -q '[email protected]' "$RUN_LOG"; then
+ _fail "ownerEmail must not be concatenated from the username" \
+ "$(grep -o 'ownerEmail=[^&]*' "$RUN_LOG" | sort -u | tr '\n' ' ')"
+else
+ _pass "ownerEmail is never guessed as <username>@example.com"
+fi
+
+# 2) A failed upload has to surface in the exit status. The loader keeps going
+# so one bad file does not abandon the rest, but it must not claim success.
+_run_loader "$ADMIN_TOKEN" STUB_FAIL_INIT=1
+if [[ "$RUN_RC" != 0 ]]; then
+ _pass "failed upload makes the loader exit non-zero"
+else
+ _fail "failed upload should exit non-zero" "rc=$RUN_RC"
+fi
+if [[ "$RUN_OUT" == *"error(s)"* && "$RUN_OUT" != *"loading complete"* ]]; then
+ _pass "failed run reports the error count, not 'complete'"
+else
+ _fail "failed run should not report completion" "$(echo "$RUN_OUT" | tail
-2)"
+fi
+
+# 3) Same for a failed version create — the other formerly-silent path.
+_run_loader "$ADMIN_TOKEN" STUB_FAIL_VERSION=1
+if [[ "$RUN_RC" != 0 ]]; then
+ _pass "failed version create makes the loader exit non-zero"
+else
+ _fail "failed version create should exit non-zero" "rc=$RUN_RC"
+fi
+
+# 4) The register fallback still sends an @-shaped address (the server rejects
+# anything else), and the owner email then comes from the register token.
+REG_TOKEN=$(_mk_jwt
'{"sub":"texera","userId":2,"email":"[email protected]","role":"REGULAR"}')
+_run_loader "$ADMIN_TOKEN" STUB_NO_LOGIN=1 STUB_REGISTER_TOKEN="$REG_TOKEN"
+if [[ "$RUN_RC" == 0 ]] && grep -q '/auth/register' "$RUN_LOG"; then
+ _pass "register fallback runs when login fails"
+else
+ _fail "register fallback should run and succeed" "rc=$RUN_RC out=$(echo
"$RUN_OUT" | tail -3)"
+fi
+if grep -q 'ownerEmail=texera%40example.com\|[email protected]'
"$RUN_LOG"; then
+ _pass "a registered account's own email is used for lookups"
+else
+ _fail "registered account should use its register email" \
+ "$(grep -o 'ownerEmail=[^&]*' "$RUN_LOG" | sort -u | tr '\n' ' ')"
+fi
+
+# 5) A token with no email claim must abort with a clear message rather than
+# sending an empty ownerEmail and getting an opaque 400 per file.
+_run_loader "$(_mk_jwt '{"sub":"texera","userId":1,"role":"ADMIN"}')"
+if [[ "$RUN_RC" != 0 ]] && [[ "$RUN_OUT" == *"email"* ]]; then
+ _pass "token without an email claim aborts with a clear message"
+else
+ _fail "missing email claim should abort clearly" "rc=$RUN_RC out=$(echo
"$RUN_OUT" | tail -2)"
+fi
+
+printf "\n%d passed, %d failed\n" "$PASS" "$FAIL"
+(( FAIL == 0 ))
