This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new b1e3115d71 TIKA-4913: tika-server UAT gains an inference pass with a
mock engine (#3205)
b1e3115d71 is described below
commit b1e3115d71032726bbb00c9bb5849a057c37e88e
Author: Tim Allison <[email protected]>
AuthorDate: Fri Sep 18 21:54:07 2026 -0400
TIKA-4913: tika-server UAT gains an inference pass with a mock engine
(#3205)
---
CHANGES.txt | 2 +
.../integration-testing/run-uat-script.adoc | 44 +++++
.../pages/maintainers/release-guides/docker.adoc | 4 +-
pom.xml | 2 +
release-tools/uat/MockInferenceServer.java | 184 +++++++++++++++++++++
release-tools/uat/run-uat.sh | 110 +++++++++++-
release-tools/uat/test-files/testVideo_3s.mp4 | Bin 0 -> 48013 bytes
release-tools/uat/uat-inference-config.json | 17 ++
.../apache/tika/server/e2e/RunUatSmokeTest.java | 138 +++++++++++++++-
tika-server/docker-build/docker-tool.sh | 146 ++++++++++++++--
tika-server/docker-build/full/Dockerfile.snapshot | 1 +
11 files changed, 623 insertions(+), 25 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bd6c9aed9..014218bc0c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,7 @@
Release 4.1.0 - unreleased
+ * The tika-server UAT gains an inference pass (TIKA-4913).
+
* Calls to a hosted inference engine retry a 429, 502, 503 or 504 answer
with a jittered backoff (TIKA-4912).
diff --git
a/docs/modules/ROOT/pages/advanced/integration-testing/run-uat-script.adoc
b/docs/modules/ROOT/pages/advanced/integration-testing/run-uat-script.adoc
index 6aaa3d7fa4..dc687b9b85 100644
--- a/docs/modules/ROOT/pages/advanced/integration-testing/run-uat-script.adoc
+++ b/docs/modules/ROOT/pages/advanced/integration-testing/run-uat-script.adoc
@@ -83,6 +83,50 @@ Coverage includes:
failure when `TIKA_UAT_REQUIRE_OCR=1` is set. `docker-tool.sh test-uat` sets
that env
var for the `-full` image, so OCR is asserted there and skipped for the
minimal image.
+=== The inference pass
+
+A second run, against a server started on
`release-tools/uat/uat-inference-config.json`,
+exercises what the default-mode run cannot: the engine wiring between
tika-server, its forked
+parse worker and an inference engine. Nothing real is called.
`MockInferenceServer.java` in
+the same directory stands in for a hosted OCR model and an embedding model; it
speaks the two
+OpenAI-shaped endpoints Tika's engines use plus the health check, has no
dependencies beyond
+the JDK, and runs from source (`java
release-tools/uat/MockInferenceServer.java 18080 --flaky`).
+With `--flaky` it refuses every other request with a 429 first, the way a
concurrency-capped
+engine answers, so the pass proves the client's retry as well.
+
+With `TIKA_UAT_INFERENCE=1` the script adds:
+
+* T40/T41: the mock OCR model is the configured text recognizer; its markdown
comes back as
+ text, and the HTML table it emits (document-OCR models write tables that
way) is parsed
+ into a `<table>`, not escaped.
+* T42/T43: the `IMAGES` binding puts a vector on an image, the `TEXT` binding
one on a
+ document's text, on `/rmeta`.
+* T44/T45: the per-request switches, `text-recognizers.enabled: false` and an
+ `inference.bindings` subset, through `/tika/config` and `/rmeta/config`
+ (`allowPerRequestConfig` is on in the inference config, so the T18 gating
checks are
+ skipped in this pass; the default-mode run covers them).
+* T46: the `MEDIA` bindings cut a 3 s clip into one segment per channel
sharing a
+ correlator. This needs ffmpeg on the server: a hard failure with
`TIKA_UAT_REQUIRE_MEDIA=1`
+ (the `-full` image ships it), a skip otherwise.
+* T47: the mock's `/stats` (reached at `TIKA_UAT_MOCK_URL`, default
`http://localhost:18080`)
+ shows refusals, so the checks above passed only because the client retried.
A mock started
+ without `--flaky` reports no refusals and T47 is a skip. `docker-tool.sh`
and the e2e test
+ start it flaky only when the server under test carries the retry
(TIKA-4912), detected
+ from its `tika-http-jdk` jar; `TIKA_UAT_MOCK_FLAKY=1` or `=0` overrides that
in the tool.
+
+The config names the engine key as `${env:TIKA_UAT_MOCK_KEY}`, so the pass
also proves
+environment interpolation reaches the forked workers. `docker-tool.sh
test-uat` runs this
+pass against both images after the default-mode run, with the mock on the host
and the
+container reaching it as `host.docker.internal`; the `RunUatSmokeTest` e2e
test runs it
+against a forked `java -jar` server.
+
+`docker-tool.sh test-uat-snapshot` builds the two images the docker-snapshot
workflow
+publishes, from `Dockerfile.snapshot` over this checkout's
`tika-server-standard` zip, and
+runs both passes against each. No download and no signature check, so it works
on any
+build, and what it tests is the image itself: that is how the full image's
ffmpeg is
+verified before a release, and it is the run that showed the snapshot
Dockerfile had
+been missed when ffmpeg was added to the release one.
+
The `allowPipes=false` "refuse to start" gating and the
stale-`enableUnsecureFeatures`
upgrade behavior are *start-time* properties, so they are not in this script
(which runs
against an already-running server) -- they are covered by the
diff --git a/docs/modules/ROOT/pages/maintainers/release-guides/docker.adoc
b/docs/modules/ROOT/pages/maintainers/release-guides/docker.adoc
index 5123be81dd..2b16d23a31 100644
--- a/docs/modules/ROOT/pages/maintainers/release-guides/docker.adoc
+++ b/docs/modules/ROOT/pages/maintainers/release-guides/docker.adoc
@@ -26,7 +26,9 @@ Starting with 4.0.0-alpha-1, the Dockerfiles and the GitHub
Actions workflow
that publishes them live in this repository:
* `tika-server/docker-build/{minimal,full}/Dockerfile` — `apache/tika`
(server) release builds
-* `tika-server/docker-build/{minimal,full}/Dockerfile.snapshot` — snapshot
builds
+* `tika-server/docker-build/{minimal,full}/Dockerfile.snapshot` — snapshot
builds; keep the
+ package lists in step with the release Dockerfiles (`docker-tool.sh
test-uat-snapshot` runs
+ the UAT against images built from these, ffmpeg asserted on full)
* `tika-grpc/docker-build/Dockerfile` — `apache/tika-grpc` release *and*
snapshot builds
* `.github/workflows/docker-release.yml` — the release publishing workflow
* `.github/workflows/docker-snapshot.yml` — the snapshot publishing workflow
(auto on push to `main`, skipped when the version is not a `-SNAPSHOT`)
diff --git a/pom.xml b/pom.xml
index 5f4088b2c8..bd83168dea 100644
--- a/pom.xml
+++ b/pom.xml
@@ -346,6 +346,8 @@ least three +1 Tika PMC votes are cast.
<!-- UAT test fixtures (binary/markup test documents carry no
license header) -->
<inputExclude>release-tools/uat/test-files/**</inputExclude>
+ <!-- UAT server config: JSON carries no license header -->
+
<inputExclude>release-tools/uat/uat-inference-config.json</inputExclude>
<!-- subprojects already checked, added for RAT 0.17, see also
RAT-97 -->
<inputExclude>tika-*/**</inputExclude>
diff --git a/release-tools/uat/MockInferenceServer.java
b/release-tools/uat/MockInferenceServer.java
new file mode 100644
index 0000000000..bed6e2ea14
--- /dev/null
+++ b/release-tools/uat/MockInferenceServer.java
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.InetSocketAddress;
+import java.nio.charset.StandardCharsets;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import com.sun.net.httpserver.HttpExchange;
+import com.sun.net.httpserver.HttpServer;
+
+/**
+ * A stand-in for a hosted inference engine, for the tika-server UAT. Speaks
the two
+ * OpenAI-shaped endpoints Tika's engines call, plus the health check, with no
dependencies
+ * beyond the JDK, so it runs from source:
+ *
+ * <pre>
+ * java release-tools/uat/MockInferenceServer.java [port] [--flaky]
+ * </pre>
+ *
+ * <ul>
+ * <li>{@code GET /v1/models}: the VLM health check.</li>
+ * <li>{@code POST /v1/embeddings}: one 4-dimension unit vector per input,
keyed by index,
+ * whatever the input's modality (text, image, audio or video); the
vector encodes the
+ * input's position so a client can tell vectors apart.</li>
+ * <li>{@code POST /v1/chat/completions}: fixed markdown with an HTML table,
as a document-OCR
+ * model returns it.</li>
+ * </ul>
+ * With {@code --flaky}, every other POST is answered 429 first, the way a
concurrency-capped
+ * engine refuses the excess, so a client's retry is what makes the run pass.
+ * {@code GET /stats} reports the counts, for a test to assert on.
+ */
+public class MockInferenceServer {
+
+ static final String OCR_MARKDOWN = "# Mock OCR\n\nThe quick brown fox.\n\n"
+ + "<table>\n<tr><th>Region</th><th>Q1</th></tr>\n"
+ + "<tr><td>North</td><td>120</td></tr>\n</table>\n";
+
+ static final AtomicInteger POSTS = new AtomicInteger();
+ static final AtomicInteger REFUSED = new AtomicInteger();
+ static final AtomicInteger EMBEDDED_INPUTS = new AtomicInteger();
+ static final AtomicInteger OCR_CALLS = new AtomicInteger();
+
+ public static void main(String[] args) throws IOException {
+ int port = 0;
+ boolean flaky = false;
+ for (String a : args) {
+ if ("--flaky".equals(a)) {
+ flaky = true;
+ } else {
+ port = Integer.parseInt(a);
+ }
+ }
+ boolean refuseEveryOther = flaky;
+ HttpServer server = HttpServer.create(new InetSocketAddress("0.0.0.0",
port), 0);
+ server.createContext("/v1/models", ex -> reply(ex, 200,
+ "{\"data\":[{\"id\":\"mock-embed\"},{\"id\":\"mock-ocr\"}]}"));
+ server.createContext("/stats", ex -> reply(ex, 200, "{\"posts\":" +
POSTS.get()
+ + ",\"refused\":" + REFUSED.get() + ",\"embeddedInputs\":" +
EMBEDDED_INPUTS.get()
+ + ",\"ocrCalls\":" + OCR_CALLS.get() + "}"));
+ server.createContext("/v1/embeddings", ex -> {
+ String body = read(ex);
+ if (refuse(ex, refuseEveryOther)) {
+ return;
+ }
+ int n = countInputs(body);
+ EMBEDDED_INPUTS.addAndGet(n);
+ StringBuilder sb = new StringBuilder("{\"data\":[");
+ for (int i = 0; i < n; i++) {
+ // unit vector, position-dependent so vectors differ by input
+ double a = (i + 1) * 0.5;
+ sb.append(i > 0 ? "," : "").append("{\"index\":").append(i)
+
.append(",\"embedding\":[").append(Math.cos(a)).append(',')
+ .append(Math.sin(a)).append(",0.0,0.0]}");
+ }
+ reply(ex, 200,
sb.append("],\"model\":\"mock-embed\"}").toString());
+ });
+ server.createContext("/v1/chat/completions", ex -> {
+ read(ex);
+ if (refuse(ex, refuseEveryOther)) {
+ return;
+ }
+ OCR_CALLS.incrementAndGet();
+ reply(ex, 200,
"{\"model\":\"mock-ocr\",\"choices\":[{\"index\":0,\"message\":{\"role\":"
+ + "\"assistant\",\"content\":" + json(OCR_MARKDOWN) +
"}}],"
+ +
"\"usage\":{\"prompt_tokens\":10,\"completion_tokens\":20}}");
+ });
+ server.start();
+ System.out.println("mock inference server on port " +
server.getAddress().getPort()
+ + (flaky ? " (flaky: every other POST is refused with 429
first)" : ""));
+ }
+
+ /** With refusal on, odd-numbered POSTs get a 429 the way a saturated
engine answers. */
+ private static boolean refuse(HttpExchange ex, boolean refuseEveryOther)
throws IOException {
+ int n = POSTS.incrementAndGet();
+ if (refuseEveryOther && n % 2 == 1) {
+ REFUSED.incrementAndGet();
+ reply(ex, 429, "{\"detail\":\"Concurrency limit exceeded: 2/2
concurrent requests\"}");
+ return true;
+ }
+ return false;
+ }
+
+ /** Counts the items of the top-level "input" array without a JSON parser:
strings or objects. */
+ static int countInputs(String body) {
+ int at = body.indexOf("\"input\"");
+ if (at < 0) {
+ return 0;
+ }
+ int start = body.indexOf('[', at);
+ if (start < 0) {
+ return 1;
+ }
+ int depth = 0;
+ int items = 0;
+ boolean inString = false;
+ boolean sawValue = false;
+ for (int i = start; i < body.length(); i++) {
+ char c = body.charAt(i);
+ if (inString) {
+ if (c == '\\') {
+ i++;
+ } else if (c == '"') {
+ inString = false;
+ }
+ continue;
+ }
+ if (c == '"') {
+ inString = true;
+ sawValue = true;
+ } else if (c == '[' || c == '{') {
+ depth++;
+ if (depth == 2) {
+ sawValue = true;
+ }
+ } else if (c == ']' || c == '}') {
+ depth--;
+ if (depth == 0) {
+ return items + (sawValue ? 1 : 0);
+ }
+ } else if (c == ',' && depth == 1) {
+ items++;
+ sawValue = false;
+ } else if (depth == 1 && !Character.isWhitespace(c)) {
+ sawValue = true;
+ }
+ }
+ return items;
+ }
+
+ private static String read(HttpExchange ex) throws IOException {
+ try (var in = ex.getRequestBody()) {
+ return new String(in.readAllBytes(), StandardCharsets.UTF_8);
+ }
+ }
+
+ private static void reply(HttpExchange ex, int status, String body) throws
IOException {
+ byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
+ ex.getResponseHeaders().set("Content-Type", "application/json");
+ ex.sendResponseHeaders(status, bytes.length);
+ try (OutputStream out = ex.getResponseBody()) {
+ out.write(bytes);
+ }
+ }
+
+ private static String json(String s) {
+ return '"' + s.replace("\\", "\\\\").replace("\"",
"\\\"").replace("\n", "\\n") + '"';
+ }
+}
diff --git a/release-tools/uat/run-uat.sh b/release-tools/uat/run-uat.sh
index 25f2627f37..0e924c7ab5 100755
--- a/release-tools/uat/run-uat.sh
+++ b/release-tools/uat/run-uat.sh
@@ -43,6 +43,17 @@ OCRPNG="$FILES/testOCR_spacing.png"
# the OCR check is skipped when OCR is unavailable (minimal image / no
tesseract).
REQUIRE_OCR="${TIKA_UAT_REQUIRE_OCR:-}"
+# Set TIKA_UAT_INFERENCE=1 when the server was started with
uat-inference-config.json
+# pointed at a running MockInferenceServer: the inference checks (T40+) then
run, and
+# the media check is a hard failure when TIKA_UAT_REQUIRE_MEDIA=1 (the -full
image ships
+# ffmpeg) and a skip otherwise. Left unset, every inference check is skipped.
+INFERENCE="${TIKA_UAT_INFERENCE:-}"
+REQUIRE_MEDIA="${TIKA_UAT_REQUIRE_MEDIA:-}"
+VIDEO="$FILES/testVideo_3s.mp4"
+# TIKA_UAT_MOCK_URL: where the mock is reachable from THIS script (its /stats
endpoint);
+# the server reaches it through the baseUrl in its config, which may differ
(docker).
+MOCK_URL="${TIKA_UAT_MOCK_URL:-http://localhost:18080}";
+
PASS=0
FAIL=0
SKIP=0
@@ -202,7 +213,13 @@ assert_contains "T17 POST /rmeta/form" "Content-Type"
"$(curl -s -X POST -F "upl
# Per-request /config endpoints must be 403 in default mode
(allowPerRequestConfig=false).
# Note: the unpack config-variant is /unpack/all/config; /unpack/config does
not exist
# (the {id} template requires a leading slash, so it 404s) -- the old docs
were wrong.
+# uat-inference-config.json turns allowPerRequestConfig on (T44/T45 need it),
so under
+# TIKA_UAT_INFERENCE the gate checks are skipped: they are covered by the
default-mode run.
+if [[ -n "$INFERENCE" ]]; then
+ skip "T18/T18b per-request config gating -- allowPerRequestConfig is on in
the inference config"
+fi
for ep in meta/config rmeta/config tika/config unpack/all/config; do
+ [[ -n "$INFERENCE" ]] && break
RESP=$(curl -s -w '\n%{http_code}' -X POST -F "file=@$PDF" "$BASE/$ep")
CODE=$(printf '%s' "$RESP" | tail -n1)
BODY=$(printf '%s' "$RESP" | sed '$d')
@@ -216,13 +233,15 @@ done
# --- SECURITY GATING (gate 2: content-based multipart 'config' part) ---
# A 'config' part on an endpoint that accepts one must be 403 even when the
path
# has no /config (TikaResource.setupMultipartConfig). Exercises the second
enforcement point.
-RESP=$(curl -s -w '\n%{http_code}' -X POST -F "file=@$PDF" -F
'config={"parsers":[{"pdf-parser":{}}]}' "$BASE/unpack")
-CODE=$(printf '%s' "$RESP" | tail -n1)
-BODY=$(printf '%s' "$RESP" | sed '$d')
-if [[ "$CODE" == "403" ]] && printf '%s' "$BODY" | grep -qiF "disabled"; then
- ok "T18b POST /unpack with config part blocked (403 + 'disabled')"
-else
- bad "T18b POST /unpack with config part" "403 + 'disabled'" "HTTP $CODE
$BODY"
+if [[ -z "$INFERENCE" ]]; then
+ RESP=$(curl -s -w '\n%{http_code}' -X POST -F "file=@$PDF" -F
'config={"parsers":[{"pdf-parser":{}}]}' "$BASE/unpack")
+ CODE=$(printf '%s' "$RESP" | tail -n1)
+ BODY=$(printf '%s' "$RESP" | sed '$d')
+ if [[ "$CODE" == "403" ]] && printf '%s' "$BODY" | grep -qiF "disabled"; then
+ ok "T18b POST /unpack with config part blocked (403 + 'disabled')"
+ else
+ bad "T18b POST /unpack with config part" "403 + 'disabled'" "HTTP $CODE
$BODY"
+ fi
fi
# --- SECURITY GATING: /status is NOT registered by default (must be 404, not
200) ---
@@ -253,6 +272,83 @@ else
skip "T30 OCR PUT /tika/text -- no OCR text (tesseract not available on
server)"
fi
+# --- INFERENCE (conditional: server started with uat-inference-config.json +
mock engine) ---
+# The mock engine (MockInferenceServer.java) stands in for a hosted OCR model
and an
+# embedding model. These checks prove the wiring between tika-server, its
forked worker
+# and an engine: config load, the text-recognizers list, the four binding
kinds, the
+# per-request switches, and ffmpeg where the image ships it.
+if [[ -n "$INFERENCE" ]]; then
+ # T40: the mock OCR model is the text recognizer for a standalone image; its
markdown
+ # comes back as text, and the HTML table it emits is parsed, not escaped.
+ OCR_OUT=$(curl -s -X PUT -T "$OCRPNG" "$BASE/tika/text")
+ assert_contains "T40 mock OCR PUT /tika/text (image -> mock text)" "The
quick brown fox" "$OCR_OUT"
+ OCR_XML=$(curl -s -X PUT -T "$OCRPNG" "$BASE/tika/xml")
+ assert_contains "T41 mock OCR HTML table parsed" "<table>" "$OCR_XML"
+ if printf '%s' "$OCR_XML" | grep -qF '<table'; then
+ bad "T41b mock OCR table not escaped" "no <table" "escaped tags in
output"
+ else
+ ok "T41b mock OCR table not escaped"
+ fi
+
+ # T42: the IMAGES binding puts a vector on the image; the TEXT binding one
on its text.
+ IMG_META=$(curl -s -X PUT -T "$OCRPNG" -H 'Accept: application/json'
"$BASE/rmeta")
+ assert_contains "T42 IMAGES binding: tk:chunks on /rmeta" '"tk:chunks"'
"$IMG_META"
+ # tk:chunks is a JSON string inside the metadata JSON, so its quotes are
escaped on the wire
+ assert_contains "T42b IMAGES binding: producer pictures"
'\"producer\":\"pictures\"' "$IMG_META"
+ TXT_META=$(curl -s -X PUT -T "$HTML" -H 'Accept: application/json'
"$BASE/rmeta")
+ assert_contains "T43 TEXT binding: producer text on an html doc"
'\"producer\":\"text\"' "$TXT_META"
+
+ # T44: per-request switches (allowPerRequestConfig is on in the UAT config):
the
+ # recognizer off removes the OCR text; a bindings subset runs only that
binding.
+ OFF=$(curl -s -X POST -F "file=@$OCRPNG" \
+ -F 'config={"parse-context":{"text-recognizers":{"enabled":false}}}'
"$BASE/tika/config/text")
+ if printf '%s' "$OFF" | grep -qF "The quick brown fox"; then
+ bad "T44 text-recognizers.enabled=false per request" "no OCR text" "OCR
text present"
+ else
+ ok "T44 text-recognizers.enabled=false per request"
+ fi
+ SUBSET=$(curl -s -X POST -F "file=@$OCRPNG" \
+ -F 'config={"parse-context":{"inference":{"bindings":["text"]}}}' \
+ -H 'Accept: application/json' "$BASE/rmeta/config")
+ if printf '%s' "$SUBSET" | grep -qF '\"producer\":\"pictures\"'; then
+ bad "T45 inference.bindings subset per request" "no pictures chunk"
"pictures binding ran"
+ else
+ ok "T45 inference.bindings subset per request"
+ fi
+
+ # T46: MEDIA bindings need ffmpeg on the server. A 3 s clip with sound gives
one
+ # segment per channel sharing a correlator. Hard failure only with
TIKA_UAT_REQUIRE_MEDIA.
+ if [[ -f "$VIDEO" ]]; then
+ VID_META=$(curl -s -X PUT -T "$VIDEO" -H 'Accept: application/json'
"$BASE/rmeta")
+ if printf '%s' "$VID_META" | grep -qF '\"producer\":\"video\"' \
+ && printf '%s' "$VID_META" | grep -qF '\"producer\":\"audio\"' \
+ && printf '%s' "$VID_META" | grep -qF '\"correlator\":\"t:0-3000\"';
then
+ ok "T46 MEDIA bindings: video + audio segment vectors (ffmpeg)"
+ elif [[ -n "$REQUIRE_MEDIA" ]]; then
+ bad "T46 MEDIA bindings" "video and audio chunks with correlator
t:0-3000 (ffmpeg required)" "$(printf '%s' "$VID_META" | head -c 400)"
+ else
+ skip "T46 MEDIA bindings -- no segment vectors (ffmpeg not available on
server)"
+ fi
+ else
+ skip "T46 MEDIA bindings -- $VIDEO missing"
+ fi
+
+ # T47: the mock started with --flaky refuses every other POST with 429
first, as a
+ # concurrency-capped engine does; the checks above passed only if the client
retried.
+ # The mock's /stats says whether refusals happened at all.
+ STATS=$(curl -s --max-time 5 "$MOCK_URL/stats" || true)
+ REFUSED=$(printf '%s' "$STATS" | sed -n 's/.*"refused":\([0-9]*\).*/\1/p')
+ if [[ -z "$REFUSED" ]]; then
+ skip "T47 engine 429 retried -- mock /stats unreachable at $MOCK_URL"
+ elif [[ "$REFUSED" == "0" ]]; then
+ skip "T47 engine 429 retried -- mock not started with --flaky (no
refusals)"
+ else
+ ok "T47 engine 429 retried ($REFUSED refusals absorbed by the client's
retries)"
+ fi
+else
+ skip "T40-T47 inference checks -- TIKA_UAT_INFERENCE not set"
+fi
+
echo "== UAT done: $PASS passed, $FAIL failed, $SKIP skipped =="
if [[ $FAIL -gt 0 ]]; then
printf 'FAILED: %s\n' "${FAILED_NAMES[*]}"
diff --git a/release-tools/uat/test-files/testVideo_3s.mp4
b/release-tools/uat/test-files/testVideo_3s.mp4
new file mode 100644
index 0000000000..843fe75465
Binary files /dev/null and b/release-tools/uat/test-files/testVideo_3s.mp4
differ
diff --git a/release-tools/uat/uat-inference-config.json
b/release-tools/uat/uat-inference-config.json
new file mode 100644
index 0000000000..a14db0866a
--- /dev/null
+++ b/release-tools/uat/uat-inference-config.json
@@ -0,0 +1,17 @@
+{
+ "server": { "allowPerRequestConfig": true },
+ "engines": {
+ "mock-ocr": { "openai-vlm-parser": {
+ "baseUrl": "MOCK_BASE_URL", "model": "mock-ocr", "apiKey":
"${env:TIKA_UAT_MOCK_KEY}", "maxTokens": 512 } },
+ "mock-embed": { "openai-embedding-engine": {
+ "baseUrl": "MOCK_BASE_URL", "model": "mock-embed", "apiKey":
"${env:TIKA_UAT_MOCK_KEY}",
+ "mediaInput": "object", "maxBatchSize": 8 } }
+ },
+ "text-recognizers": [ { "engine": "mock-ocr" } ],
+ "inference": [
+ { "id": "text", "engine": "mock-embed", "input": "TEXT", "tasks":
["embed"] },
+ { "id": "pictures", "engine": "mock-embed", "input": "IMAGES", "tasks":
["embed"] },
+ { "id": "video", "engine": "mock-embed", "input": "MEDIA", "modality":
"visual" },
+ { "id": "audio", "engine": "mock-embed", "input": "MEDIA", "modality":
"audio" }
+ ]
+}
diff --git
a/tika-e2e-tests/tika-server/src/test/java/org/apache/tika/server/e2e/RunUatSmokeTest.java
b/tika-e2e-tests/tika-server/src/test/java/org/apache/tika/server/e2e/RunUatSmokeTest.java
index e99291ebbe..dc405a5847 100644
---
a/tika-e2e-tests/tika-server/src/test/java/org/apache/tika/server/e2e/RunUatSmokeTest.java
+++
b/tika-e2e-tests/tika-server/src/test/java/org/apache/tika/server/e2e/RunUatSmokeTest.java
@@ -109,6 +109,131 @@ public class RunUatSmokeTest {
}
}
+ /**
+ * The UAT's inference pass: release-tools/uat/MockInferenceServer.java
stands in for a
+ * hosted OCR model and an embedding model (started flaky, so every other
request is
+ * refused first), the server runs on uat-inference-config.json with the
engine key taken
+ * from the environment, and run-uat.sh's T40-T47 checks exercise the
forked worker's
+ * engine wiring. Media needs ffmpeg on the PATH: asserted when it is
there, skipped by
+ * the script when it is not.
+ */
+ @Test
+ void testRestUatInferencePasses() throws Exception {
+ Assumptions.assumeFalse(IS_WINDOWS, "run-uat.sh requires bash;
skipping on Windows");
+ Path serverJar = locateServerJar();
+ Path uatScript = locateUatScript();
+ Assumptions.assumeTrue(serverJar != null, "tika-server-standard-*.jar
not found; build the dist first");
+ Assumptions.assumeTrue(uatScript != null,
"release-tools/uat/run-uat.sh not found");
+ Path uatDir = uatScript.getParent();
+ Path mockSource = uatDir.resolve("MockInferenceServer.java");
+ Path configTemplate = uatDir.resolve("uat-inference-config.json");
+ Assumptions.assumeTrue(Files.exists(mockSource) &&
Files.exists(configTemplate),
+ "mock engine or inference config missing next to run-uat.sh");
+
+ int mockPort = findFreePort();
+ // --flaky only once the shipped client retries a 429 (TIKA-4912);
before that the
+ // refusals would fail the run for a reason this test is not about
+ boolean clientRetries = clientRetries(serverJar);
+ ProcessBuilder mockPb = clientRetries
+ ? new ProcessBuilder("java", mockSource.toString(),
String.valueOf(mockPort), "--flaky")
+ : new ProcessBuilder("java", mockSource.toString(),
String.valueOf(mockPort));
+ mockPb.redirectErrorStream(true);
+ Process mock = mockPb.start();
+ drainAsync(mock);
+ Process server = null;
+ try {
+ awaitHttp("http://localhost:"; + mockPort + "/v1/models", mock,
"mock inference server");
+ int port = findFreePort();
+ String baseUrl = "http://localhost:"; + port;
+ String json = Files.readString(configTemplate, UTF_8)
+ .replace("MOCK_BASE_URL", "http://localhost:"; + mockPort);
+ Path config = writeConfig("uat-inference", json);
+ server = startServer(serverJar,
java.util.Map.of("TIKA_UAT_MOCK_KEY", "uat-secret"),
+ "-p", String.valueOf(port), "-h", "localhost", "-c",
config.toString());
+ awaitServerStartup(server, baseUrl);
+
+ ProcessBuilder pb = new ProcessBuilder("bash",
uatScript.toString(), baseUrl);
+ pb.environment().put("TIKA_UAT_INFERENCE", "1");
+ pb.environment().put("TIKA_UAT_MOCK_URL", "http://localhost:"; +
mockPort);
+ if (onPath("ffmpeg") && onPath("ffprobe")) {
+ pb.environment().put("TIKA_UAT_REQUIRE_MEDIA", "1");
+ }
+ pb.redirectErrorStream(true);
+ Process uat = pb.start();
+ String output = drain(uat);
+ boolean finished = uat.waitFor(180, TimeUnit.SECONDS);
+ if (!finished) {
+ uat.destroyForcibly();
+ fail("run-uat.sh (inference) did not finish within 180s");
+ }
+ log.info("run-uat.sh (inference) output:\n{}", output);
+ assertEquals(0, uat.exitValue(),
+ "run-uat.sh (inference) reported failures against " +
baseUrl + ":\n" + output);
+ if (clientRetries) {
+ assertTrue(output.contains("PASS T47"),
+ "the flaky mock's refusals must have been retried:\n"
+ output);
+ }
+ } finally {
+ if (server != null) {
+ stop(server);
+ }
+ mock.destroy();
+ mock.waitFor(10, TimeUnit.SECONDS);
+ }
+ }
+
+ /** Whether the server's HTTP client retries a 429:
TikaHttpClient.build(int, int) exists. */
+ private static boolean clientRetries(Path serverJar) {
+ try (DirectoryStream<Path> jars = Files.newDirectoryStream(
+ serverJar.getParent().resolve("lib"), "tika-http-jdk-*.jar")) {
+ for (Path jar : jars) {
+ try (java.util.jar.JarFile jf = new
java.util.jar.JarFile(jar.toFile())) {
+ java.util.jar.JarEntry e =
jf.getJarEntry("org/apache/tika/http/TikaHttpClient.class");
+ if (e == null) {
+ return false;
+ }
+ String bytes = new
String(jf.getInputStream(e).readAllBytes(), StandardCharsets.ISO_8859_1);
+ return bytes.contains("DEFAULT_MAX_RETRIES");
+ }
+ }
+ } catch (Exception e) {
+ // fall through
+ }
+ return false;
+ }
+
+ private static boolean onPath(String tool) {
+ try {
+ Process p = new ProcessBuilder(tool,
"-version").redirectErrorStream(true).start();
+ p.getInputStream().readAllBytes();
+ return p.waitFor(10, TimeUnit.SECONDS) && p.exitValue() == 0;
+ } catch (Exception e) {
+ return false;
+ }
+ }
+
+ private static void awaitHttp(String url, Process p, String what) throws
Exception {
+ HttpClient client = HttpClient.newHttpClient();
+ Instant deadline = Instant.now().plus(Duration.ofSeconds(30));
+ while (Instant.now().isBefore(deadline)) {
+ if (!p.isAlive()) {
+ fail(what + " exited with " + p.exitValue());
+ }
+ try {
+ HttpResponse<String> r =
client.send(HttpRequest.newBuilder(URI.create(url))
+ .timeout(Duration.ofSeconds(2)).GET().build(),
+ HttpResponse.BodyHandlers.ofString());
+ if (r.statusCode() == 200) {
+ return;
+ }
+ } catch (Exception e) {
+ // not up yet
+ }
+ Thread.sleep(250);
+ }
+ fail(what + " did not answer at " + url + " within 30s");
+ }
+
@Test
void testPipesEndpointWithoutAllowPipesRefusesToStart() throws Exception {
Path serverJar = locateServerJar();
@@ -144,6 +269,11 @@ public class RunUatSmokeTest {
// ----- helpers -----
private Process forkServer(Path serverJar, String... args) throws
Exception {
+ return forkServer(serverJar, java.util.Map.of(), args);
+ }
+
+ private Process forkServer(Path serverJar, java.util.Map<String, String>
env, String... args)
+ throws Exception {
String[] cmd = new String[args.length + 3];
cmd[0] = "java";
cmd[1] = "-jar";
@@ -151,12 +281,18 @@ public class RunUatSmokeTest {
System.arraycopy(args, 0, cmd, 3, args.length);
ProcessBuilder pb = new ProcessBuilder(cmd);
pb.directory(serverJar.getParent().toFile());
+ pb.environment().putAll(env);
pb.redirectErrorStream(true);
return pb.start();
}
private Process startServer(Path serverJar, String... args) throws
Exception {
- Process p = forkServer(serverJar, args);
+ return startServer(serverJar, java.util.Map.of(), args);
+ }
+
+ private Process startServer(Path serverJar, java.util.Map<String, String>
env, String... args)
+ throws Exception {
+ Process p = forkServer(serverJar, env, args);
drainAsync(p);
return p;
}
diff --git a/tika-server/docker-build/docker-tool.sh
b/tika-server/docker-build/docker-tool.sh
index 4df74f7406..ac5d03fef0 100755
--- a/tika-server/docker-build/docker-tool.sh
+++ b/tika-server/docker-build/docker-tool.sh
@@ -37,6 +37,10 @@ while getopts ":h" opt; do
echo " docker-tool.sh build <TIKA_DOCKER_VERSION> <TIKA_VERSION>
Builds <TIKA_DOCKER_VERSION> images for <TIKA_VERSION>."
echo " docker-tool.sh test <TIKA_DOCKER_VERSION> Tests images for
<TIKA_DOCKER_VERSION>."
echo " docker-tool.sh test-uat <TIKA_DOCKER_VERSION> Runs the
tika-server REST UAT against images for <TIKA_DOCKER_VERSION>."
+ echo " docker-tool.sh test-uat-snapshot Builds the minimal and full
images from the local tika-server-standard zip"
+ echo " with the Dockerfile.snapshot
files CI publishes from, and runs the REST UAT"
+ echo " (both passes) against them. No
download, no signature: the artifact under test"
+ echo " is what this checkout built."
echo " Requires
TIKA_MAIN env var or sibling tika-main checkout (../tika-main)."
echo " docker-tool.sh publish <TIKA_VERSION> <BUILD_NUMBER>
Builds multi-arch images and pushes three tags per image:"
echo " <TIKA_VERSION>
(mutable), <TIKA_VERSION>-<BUILD_NUMBER> (immutable),"
@@ -102,36 +106,116 @@ test_docker_image() {
stop_test_container "$container_name"
}
-test_docker_image_uat() {
- container_name=$1
- image=$image_name:$1
- uat_script=$2
- require_ocr=${3:-} # non-empty => OCR must work (the -full image ships
tesseract)
-
- docker run -d --name "$container_name" -p 127.0.0.1:9998:9998 "$image" \
- || die "couldn't start $image"
-
+wait_for_version() {
# Wait up to 30s for /version to respond.
for i in $(seq 1 30); do
if curl -fsS --max-time 2 http://localhost:9998/version >/dev/null 2>&1;
then
- break
+ return 0
fi
sleep 1
done
+ return 1
+}
- if TIKA_UAT_REQUIRE_OCR="$require_ocr" "$uat_script" http://localhost:9998;
then
- echo "$(tput setaf 2)Image: $image - UAT passed$(tput sgr0)"
- stop_test_container "$container_name"
+report_uat() {
+ # report_uat <image> <container> <exit code of the UAT>
+ if [[ "$3" == "0" ]]; then
+ echo "$(tput setaf 2)Image: $1 - UAT passed$(tput sgr0)"
+ stop_test_container "$2"
else
- echo "$(tput setaf 1)Image: $image - UAT failed$(tput sgr0)"
+ echo "$(tput setaf 1)Image: $1 - UAT failed$(tput sgr0)"
echo "--- last 40 lines of container log ---"
- docker logs --tail 40 "$container_name" || true
+ docker logs --tail 40 "$2" || true
echo "--- end log ---"
- stop_test_container "$container_name"
+ stop_test_container "$2"
exit 1
fi
}
+test_docker_image_uat() {
+ container_name=$1
+ image=$image_name:$1
+ uat_script=$2
+ require_ocr=${3:-} # non-empty => OCR must work (the -full image ships
tesseract)
+
+ docker run -d --name "$container_name" -p 127.0.0.1:9998:9998 "$image" \
+ || die "couldn't start $image"
+ wait_for_version
+ TIKA_UAT_REQUIRE_OCR="$require_ocr" "$uat_script" http://localhost:9998
+ report_uat "$image" "$container_name" $?
+}
+
+# Whether the image's HTTP client retries a 429 (TIKA-4912): the images carry
no unzip, so the
+# jar is copied out of a stopped container and inspected on the host.
+image_client_retries() {
+ local probe="$1-probe" tmp rc=1
+ tmp="$(mktemp -d /tmp/tika-uat-probe.XXXXXX)"
+ docker create --name "$probe" "$1" >/dev/null 2>&1 || { rm -rf "$tmp";
return 1; }
+ if docker cp "$probe:/opt/tika-server/lib/." "$tmp/" >/dev/null 2>&1; then
+ local jar
+ jar=$(ls "$tmp"/tika-http-jdk-*.jar 2>/dev/null | head -1)
+ if [[ -n "$jar" ]] && unzip -p "$jar"
org/apache/tika/http/TikaHttpClient.class 2>/dev/null \
+ | grep -q DEFAULT_MAX_RETRIES; then
+ rc=0
+ fi
+ fi
+ docker rm -f "$probe" >/dev/null 2>&1
+ rm -rf "$tmp"
+ return $rc
+}
+
+# Second pass with an inference config: a mock engine
(release-tools/uat/MockInferenceServer.java)
+# on the host stands in for a hosted OCR model and an embedding model, started
--flaky so every
+# other request is refused first; the container reaches it as
host.docker.internal. Proves the
+# forked worker's engine wiring, ${env:...} interpolation into the forks, the
client's retry, and
+# ffmpeg where the image ships it (a hard failure on -full, a skip on minimal).
+test_docker_image_uat_inference() {
+ container_name="$1-inference"
+ image=$image_name:$1
+ uat_script=$2
+ require_media=${3:-}
+ uat_dir="$(dirname "$uat_script")"
+ mock_port=18080
+
+ # The mock refuses every other request only when the image's HTTP client
retries a 429
+ # (TIKA-4912): detected from the class in the image's tika-http-jdk jar, so
an older image
+ # is tested without refusals rather than failing for a reason this pass is
not about.
+ # TIKA_UAT_MOCK_FLAKY=1 or =0 overrides the detection.
+ flaky=""
+ if [[ -n "${TIKA_UAT_MOCK_FLAKY:-}" ]]; then
+ [[ "$TIKA_UAT_MOCK_FLAKY" == "1" ]] && flaky="--flaky"
+ elif image_client_retries "$image"; then
+ flaky="--flaky"
+ fi
+ java "$uat_dir/MockInferenceServer.java" $mock_port $flaky >
/tmp/tika-uat-mock.log 2>&1 &
+ mock_pid=$!
+ for i in $(seq 1 20); do
+ curl -fsS --max-time 1 "http://localhost:$mock_port/v1/models"; >/dev/null
2>&1 && break
+ sleep 0.5
+ done
+ curl -fsS --max-time 2 "http://localhost:$mock_port/v1/models"; >/dev/null
2>&1 \
+ || { kill $mock_pid 2>/dev/null; die "mock inference server did not start
(see /tmp/tika-uat-mock.log)"; }
+
+ config="$(mktemp /tmp/tika-uat-inference-config.XXXXXX.json)"
+ sed "s|MOCK_BASE_URL|http://host.docker.internal:$mock_port|g"
"$uat_dir/uat-inference-config.json" > "$config"
+ # the images run as UID 35002; a mktemp file is 0600, which that user cannot
read
+ chmod 644 "$config"
+
+ docker run -d --name "$container_name" -p 127.0.0.1:9998:9998 \
+ --add-host=host.docker.internal:host-gateway \
+ -e TIKA_UAT_MOCK_KEY=uat-secret \
+ -v "$config:/tika-config.json:ro" \
+ "$image" -c /tika-config.json \
+ || { kill $mock_pid 2>/dev/null; die "couldn't start $image with the
inference config"; }
+ wait_for_version
+ TIKA_UAT_INFERENCE=1 TIKA_UAT_REQUIRE_MEDIA="$require_media" \
+ TIKA_UAT_MOCK_URL="http://localhost:$mock_port"; "$uat_script"
http://localhost:9998
+ rc=$?
+ kill $mock_pid 2>/dev/null
+ rm -f "$config"
+ report_uat "$image (inference config)" "$container_name" $rc
+}
+
shift $((OPTIND -1))
subcommand=$1; shift
tika_docker_version=$1; shift
@@ -162,6 +246,36 @@ case "$subcommand" in
fi
test_docker_image_uat ${tika_docker_version} "$uat_script"
test_docker_image_uat "${tika_docker_version}-full" "$uat_script" 1
+ test_docker_image_uat_inference ${tika_docker_version} "$uat_script"
+ test_docker_image_uat_inference "${tika_docker_version}-full"
"$uat_script" 1
+ ;;
+
+ test-uat-snapshot)
+ # The same images the docker-snapshot workflow publishes:
Dockerfile.snapshot over the
+ # unzipped local distribution. Run after `./mvnw install -pl
tika-server/tika-server-standard -am`.
+ repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+ uat_script="${repo_root}/release-tools/uat/run-uat.sh"
+ [[ -x "$uat_script" ]] || die "UAT script not found or not executable:
$uat_script"
+ zip=$(ls
"${repo_root}"/tika-server/tika-server-standard/target/tika-server-standard-*.zip
2>/dev/null | head -1)
+ [[ -f "$zip" ]] || die "no tika-server-standard zip under
tika-server/tika-server-standard/target; build it first"
+ snapshot_version=$(basename "$zip" .zip | sed 's/^tika-server-standard-//')
+ tika_docker_version="${snapshot_version}-snapshot"
+ build_dir="$(mktemp -d /tmp/tika-uat-snapshot.XXXXXX)"
+ for flavor in minimal full; do
+ ctx="$build_dir/$flavor"
+ mkdir -p "$ctx/tika-server"
+ unzip -q "$zip" -d "$ctx/tika-server"
+ cp "${repo_root}/tika-server/docker-build/$flavor/Dockerfile.snapshot"
"$ctx/Dockerfile"
+ tag="${tika_docker_version}"
+ [[ "$flavor" == "full" ]] && tag="${tika_docker_version}-full"
+ docker build -q -t "${image_name}:${tag}" --build-arg
"TIKA_VERSION=${snapshot_version}" "$ctx" \
+ || die "couldn't build the $flavor snapshot image"
+ done
+ rm -rf "$build_dir"
+ test_docker_image_uat "${tika_docker_version}" "$uat_script"
+ test_docker_image_uat "${tika_docker_version}-full" "$uat_script" 1
+ test_docker_image_uat_inference "${tika_docker_version}" "$uat_script"
+ test_docker_image_uat_inference "${tika_docker_version}-full"
"$uat_script" 1
;;
publish)
diff --git a/tika-server/docker-build/full/Dockerfile.snapshot
b/tika-server/docker-build/full/Dockerfile.snapshot
index a5a7285624..4318dee335 100644
--- a/tika-server/docker-build/full/Dockerfile.snapshot
+++ b/tika-server/docker-build/full/Dockerfile.snapshot
@@ -26,6 +26,7 @@ RUN set -eux \
&& DEBIAN_FRONTEND=noninteractive apt-get install --yes
--no-install-recommends $JRE \
gdal-bin \
imagemagick \
+ ffmpeg \
tesseract-ocr \
tesseract-ocr-eng \
tesseract-ocr-ita \
