This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new c18bc4dd2e TIKA-4884: cleanups for TIKA-4872 (#3143)
c18bc4dd2e is described below
commit c18bc4dd2ec53080690db8b8f90a32eae3fbfc2f
Author: Tim Allison <[email protected]>
AuthorDate: Tue Sep 8 16:23:56 2026 -0400
TIKA-4884: cleanups for TIKA-4872 (#3143)
---
CHANGES.txt | 11 +++
docs/modules/ROOT/pages/configuration/index.adoc | 6 +-
.../pages/configuration/parsers/vlm-parsers.adoc | 4 +
.../pages/migration-to-4x/chunk-strategies.adoc | 6 +-
.../inference-handler-requirements.adoc | 4 +
.../parser/enricher/CompositeContentEnricher.java | 21 +++++-
.../tika/parser/enricher/ContentEnrichersTest.java | 16 ++++
.../org/apache/tika/parser/mock}/MockEnricher.java | 4 +-
.../org/apache/tika/inference/ChunkSerializer.java | 2 +-
.../apache/tika/parser/vlm/AbstractVLMParser.java | 14 +++-
.../org/apache/tika/parser/vlm/VLMOCRConfig.java | 2 +-
.../resources/config-examples/pdf-parser-full.json | 2 +-
.../apache/tika/pipes/core/PipesClientTest.java | 1 +
.../tika/config/loader/ContentEnricherLoader.java | 20 +++--
.../config/loader/ContentEnricherLoaderTest.java | 48 ++++++++++++
.../tika/config/loader/TestLegacyOcrEnricher.java | 25 ++-----
.../org/apache/tika/server/core/CXFTestBase.java | 9 ++-
.../server/standard/ContentEnricherWireTest.java | 86 ++++++++++++++++++++++
.../configs/tika-config-content-enrichers.json | 11 +++
19 files changed, 255 insertions(+), 37 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 64bfc36c74..673a175fe8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,16 @@
Release 4.1.0 - unreleased
+ * "content-enrichers": _mime-exclude now matches the real media type, so
+ excluding image/tiff also drops a legacy image/ocr-tiff engine (it was
+ a no-op). The zero-media-types check at config load asks the engine
+ itself, so a _mime-include list no longer masks an unreachable one.
+ VLM parsers implement Closeable and release their HTTP client
+ (TIKA-4884).
+
+ * tika-inference and tika-vlm are experimental: classes, config keys and
+ metadata output may change in minor releases without deprecation or
+ compatibility shims (TIKA-4884).
+
* A missing OOXML relationship target no longer aborts the whole file:
the threaded-comment and person lookups in xlsx and the page lookups in
vsdx went straight to POI's getRelatedPart, whose unchecked
diff --git a/docs/modules/ROOT/pages/configuration/index.adoc
b/docs/modules/ROOT/pages/configuration/index.adoc
index dc90e9fee9..76a807b913 100644
--- a/docs/modules/ROOT/pages/configuration/index.adoc
+++ b/docs/modules/ROOT/pages/configuration/index.adoc
@@ -144,12 +144,14 @@ the engine explicitly instead of relying on which OCR
module happens to be on th
An enricher advertises its *real* media types (`image/png`, ...) and does not
compete with the
parser registered for those types: `image-parser` still parses the image and
calls the enricher.
(The bundled OCR engines still advertise legacy `image/ocr-*` types; those are
mapped to the
-real type, so naming them here just works.) *Every* enricher matching a media
type runs, in the
+real type, so naming them here just works, and an entry's
`_mime-include`/`_mime-exclude` name
+real types too: `image/tiff`, not `image/ocr-tiff`.) *Every* enricher matching
a media type runs, in the
order listed — so an OCR engine followed by a VLM that tags images is two
entries, both invoked
per image. The list is authoritative: a media type no configured enricher
matches gets no
enrichment — never a classpath engine you did not name — and a named engine
that reports no
media types at startup (missing native binary, unreachable inference server)
fails config load
-rather than going silently inert. Failures are best-effort: one enricher
failing does not stop
+rather than going silently inert; that check is made against the engine
itself, so a
+`_mime-include` list cannot stand in for an unreachable engine. Failures are
best-effort: one enricher failing does not stop
the others, and every failure is still reported through the parser's normal
exception handling
(timeouts abort the chain immediately). With no `content-enrichers`
configured, behavior is
unchanged — whichever OCR engine is on the classpath is used, exactly as
before, and a WARN is
diff --git a/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
b/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
index 65b55f7daf..a74d4bb70b 100644
--- a/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
+++ b/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
@@ -22,6 +22,10 @@ to remote Vision-Language Model (VLM) endpoints. These
parsers send images
(or PDFs) to an external API and convert the model's markdown response into
structured XHTML.
+NOTE: `tika-vlm` is experimental. Its classes, configuration keys and metadata
output may
+change in minor releases without a deprecation cycle, and no compatibility
shims are kept
+for earlier shapes.
+
Three implementations are provided out of the box. None is auto-loaded: each
must be
named explicitly in your configuration. (Changed in 4.1.0: `openai-vlm-parser`
previously
auto-registered via SPI.) To use a VLM as the OCR engine for embedded images
and rendered
diff --git a/docs/modules/ROOT/pages/migration-to-4x/chunk-strategies.adoc
b/docs/modules/ROOT/pages/migration-to-4x/chunk-strategies.adoc
index 18d9e4c76e..1acbc366db 100644
--- a/docs/modules/ROOT/pages/migration-to-4x/chunk-strategies.adoc
+++ b/docs/modules/ROOT/pages/migration-to-4x/chunk-strategies.adoc
@@ -22,6 +22,10 @@ Tika 4.x introduces a unified chunking and embedding
pipeline (`tika-inference`)
`tk:chunks` array on each document's metadata. This page describes how those
chunks reach
Elasticsearch and OpenSearch, and why the shipped strategy is the one it is.
+NOTE: `tika-inference` is experimental. Its classes, configuration keys and
the `tk:chunks`
+layout may change in minor releases without a deprecation cycle, and no
compatibility shims
+are kept for earlier shapes.
+
== The `tk:chunks` Field
All chunk data lands in one metadata field, `tk:chunks`, as a JSON array.
There is no
@@ -52,7 +56,7 @@ A single chunk looks like:
----
{
"text": "Revenue grew 15% year-over-year...",
- "vector": "base64-encoded-float32-le",
+ "vector": "base64-encoded-float32-be",
"locators": {
"text": [{"start_offset": 0, "end_offset": 120}],
"paginated": [{"page": 1}]
diff --git
a/docs/modules/ROOT/pages/migration-to-4x/inference-handler-requirements.adoc
b/docs/modules/ROOT/pages/migration-to-4x/inference-handler-requirements.adoc
index 5e56bc9931..6bd4ccc521 100644
---
a/docs/modules/ROOT/pages/migration-to-4x/inference-handler-requirements.adoc
+++
b/docs/modules/ROOT/pages/migration-to-4x/inference-handler-requirements.adoc
@@ -23,6 +23,10 @@ metadata field, splits it with the `MarkdownChunker`, and
sends the resulting ch
embeddings endpoint. The chunker splits on markdown structure, so the content
handler that
produced `tk:content` has to be `MARKDOWN` for it to do its job.
+NOTE: `tika-inference` is experimental. Its classes, configuration keys and
the `tk:chunks`
+layout may change in minor releases without a deprecation cycle, and no
compatibility shims
+are kept for earlier shapes.
+
Since 4.x that is the default, so the common case needs no configuration. This
page records
why, and what happens if you override it.
diff --git
a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
index 398b3fb225..e26587516b 100644
---
a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
+++
b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
@@ -20,6 +20,7 @@ import java.io.Serializable;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -27,6 +28,7 @@ import java.util.Set;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.ParserDecorator;
/**
* Media-type-keyed registry of content enrichers: ordinary {@link Parser}s
that a container
@@ -36,7 +38,8 @@ import org.apache.tika.parser.Parser;
* <p>
* Members advertise their <em>real</em> media types ({@code image/png});
legacy engines
* still advertising the {@code image/ocr-*} pseudo-types are keyed under the
real type, so
- * they are nameable here unmodified. An enricher does not compete with the
parser
+ * they are nameable here unmodified and a {@code _mime-exclude} on one
matches the real type
+ * too. An enricher does not compete with the parser
* registered for the same type: that parser still runs and calls the enricher.
*
* @since Apache Tika 4.1
@@ -51,9 +54,13 @@ public class CompositeContentEnricher implements
Serializable {
Map<MediaType, List<Parser>> tmp = new HashMap<>();
ParseContext empty = new ParseContext();
for (Parser enricher : enrichers) {
+ Set<MediaType> excluded = excludedRealTypes(enricher);
for (MediaType mediaType : enricher.getSupportedTypes(empty)) {
// legacy engines advertise image/ocr-*; key under the real
type
MediaType keyType =
stripLegacyOcrPrefix(mediaType.getBaseType());
+ if (excluded.contains(keyType)) {
+ continue;
+ }
List<Parser> forType = tmp.computeIfAbsent(keyType, k -> new
ArrayList<>());
if (!forType.contains(enricher)) {
forType.add(enricher);
@@ -64,6 +71,18 @@ public class CompositeContentEnricher implements
Serializable {
this.enricherMap = Collections.unmodifiableMap(tmp);
}
+ // decorator excludes are literal; "image/tiff" must also drop a legacy
"image/ocr-tiff"
+ private static Set<MediaType> excludedRealTypes(Parser enricher) {
+ if (!(enricher instanceof ParserDecorator.MimeFilteringDecorator
decorator)) {
+ return Collections.emptySet();
+ }
+ Set<MediaType> excluded = new HashSet<>();
+ for (MediaType excludeType : decorator.getExcludeTypes()) {
+ excluded.add(stripLegacyOcrPrefix(excludeType.getBaseType()));
+ }
+ return excluded;
+ }
+
private static MediaType stripLegacyOcrPrefix(MediaType mediaType) {
String subtype = mediaType.getSubtype();
if (subtype.startsWith(LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX)) {
diff --git
a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
index 416e08993e..53635f8e49 100644
---
a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
+++
b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
@@ -19,6 +19,7 @@ package org.apache.tika.parser.enricher;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
import java.util.Collections;
@@ -38,6 +39,7 @@ import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.ParserDecorator;
public class ContentEnrichersTest {
@@ -93,6 +95,20 @@ public class ContentEnrichersTest {
assertNull(explicit.overrideSeenDuringParse);
}
+ @Test
+ public void testExcludeAppliesToLegacyPseudoType() {
+ // either spelling of the excluded type must work
+ Set<MediaType> legacy = Set.of(OCR_PNG, MediaType.image("ocr-tiff"));
+ for (String spelling : new String[]{"image/tiff", "image/ocr-tiff"}) {
+ Parser excluded = ParserDecorator.withoutTypes(new
RecordingParser(legacy),
+ Collections.singleton(MediaType.parse(spelling)));
+ CompositeContentEnricher enrichers =
+ new CompositeContentEnricher(List.of(excluded));
+ assertEquals(Set.of(PNG), enrichers.getSupportedTypes(), spelling);
+
assertTrue(enrichers.getEnrichers(MediaType.image("tiff")).isEmpty(), spelling);
+ }
+ }
+
@Test
public void testLegacyFallbackMintsAndRestores() throws Exception {
RecordingParser composite = new
RecordingParser(Collections.singleton(OCR_PNG));
diff --git
a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
b/tika-core/src/test/java/org/apache/tika/parser/mock/MockEnricher.java
similarity index 93%
copy from
tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
copy to tika-core/src/test/java/org/apache/tika/parser/mock/MockEnricher.java
index 73b499b270..924b3ba132 100644
---
a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
+++ b/tika-core/src/test/java/org/apache/tika/parser/mock/MockEnricher.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.tika.pipes.core;
+package org.apache.tika.parser.mock;
import java.util.Collections;
import java.util.Set;
@@ -30,7 +30,7 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.sax.XHTMLContentHandler;
-/** Fixture: proves the invocation and config path with no OCR binary
installed. */
+/** Fixture for {@code "content-enrichers"}: nameable as {@code
mock-enricher}, no OCR binary. */
@TikaComponent(name = "mock-enricher", spi = false)
public class MockEnricher implements Parser {
diff --git
a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/ChunkSerializer.java
b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/ChunkSerializer.java
index 6a8f66bea0..2119532c65 100644
---
a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/ChunkSerializer.java
+++
b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/ChunkSerializer.java
@@ -34,7 +34,7 @@ import org.apache.tika.metadata.TikaCoreProperties;
/**
* Serializes and deserializes a list of {@link Chunk} objects to/from JSON.
- * Vectors are stored as base64-encoded little-endian float32 via
+ * Vectors are stored as base64-encoded big-endian float32 via
* {@link VectorSerializer}. Locators are nested under a {@code "locators"}
* object with optional {@code text}, {@code paginated}, {@code spatial},
* and {@code temporal} arrays.
diff --git
a/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/AbstractVLMParser.java
b/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/AbstractVLMParser.java
index 1c4bd89011..5c46a4748c 100644
---
a/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/AbstractVLMParser.java
+++
b/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/AbstractVLMParser.java
@@ -18,6 +18,7 @@ package org.apache.tika.parser.vlm;
import static org.apache.tika.sax.XHTMLContentHandler.XHTML;
+import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.util.Base64;
@@ -62,7 +63,7 @@ import org.apache.tika.sax.XHTMLContentHandler;
*
* @since Apache Tika 4.0
*/
-public abstract class AbstractVLMParser implements Parser, Initializable {
+public abstract class AbstractVLMParser implements Parser, Initializable,
Closeable {
private static final long serialVersionUID = 1L;
@@ -227,7 +228,9 @@ public abstract class AbstractVLMParser implements Parser,
Initializable {
@Override
public void initialize() throws TikaConfigException {
- this.httpClient = buildHttpClient();
+ if (httpClient == null) {
+ httpClient = buildHttpClient();
+ }
String healthUrl = getHealthCheckUrl(defaultConfig);
if (healthUrl == null) {
// No health check configured (e.g. Claude) — assume available
@@ -252,6 +255,13 @@ public abstract class AbstractVLMParser implements Parser,
Initializable {
}
}
+ @Override
+ public void close() throws IOException {
+ if (httpClient != null) {
+ httpClient.close();
+ }
+ }
+
// ---- shared helpers ---------------------------------------------------
protected VLMOCRConfig getConfig(ParseContext parseContext)
diff --git
a/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/VLMOCRConfig.java
b/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/VLMOCRConfig.java
index b3f3a3c80e..7a01f8d042 100644
---
a/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/VLMOCRConfig.java
+++
b/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/VLMOCRConfig.java
@@ -22,7 +22,7 @@ import org.apache.tika.exception.TikaConfigException;
import org.apache.tika.utils.StringUtils;
/**
- * Configuration for {@link VLMOCRParser}.
+ * Configuration for {@link AbstractVLMParser} and its subclasses.
* <p>
* The parser expects an OpenAI-compatible chat completions endpoint
* (e.g. from vLLM, Ollama, or a local FastAPI server). The image is
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/pdf-parser-full.json
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/pdf-parser-full.json
index 19aca7fd22..8754565dd7 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/pdf-parser-full.json
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/pdf-parser-full.json
@@ -62,7 +62,7 @@
// Options: PNG, TIFF, JPEG
"imageFormat": "PNG",
// Image quality (0.0-1.0) for lossy formats.
- "imageQuality": 1.0,
+ "imageQuality": 0.5,
// Rendered-image color model.
// Options: RGB, GRAY
"imageType": "GRAY",
diff --git
a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
index 8233f939eb..d4d55fe941 100644
---
a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
+++
b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
@@ -45,6 +45,7 @@ import org.apache.tika.metadata.HttpHeaders;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.mock.MockEnricher;
import org.apache.tika.pipes.api.FetchEmitTuple;
import org.apache.tika.pipes.api.ParseMode;
import org.apache.tika.pipes.api.PipesResult;
diff --git
a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
index 781b3feb93..5f5540a546 100644
---
a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
+++
b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
@@ -26,6 +26,7 @@ import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.tika.exception.TikaConfigException;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.ParserDecorator;
import org.apache.tika.parser.enricher.CompositeContentEnricher;
/**
@@ -54,16 +55,23 @@ class ContentEnricherLoader implements
ComponentLoader<CompositeContentEnricher>
throw new TikaConfigException(
"Failed to load content enricher: " + entry.getKey(),
e);
}
- // this type snapshot lasts the life of the process, so an engine
reporting
- // nothing (missing binary, dead server) must fail load, not go
silently inert
- if (enricher.getSupportedTypes(empty).isEmpty()) {
+ // lifetime snapshot: an empty engine must fail load, not go
inert; ask the
+ // engine itself, since a _mime-include answers for the decorator
+ if (unwrap(enricher).getSupportedTypes(empty).isEmpty()) {
throw new TikaConfigException("Content enricher \"" +
entry.getKey()
- + "\" advertises no media types. Is the engine
unavailable "
- + "(missing native binary, unreachable inference
server) or "
- + "configured to skip enrichment?");
+ + "\" advertises no media types (a _mime-include list
does not "
+ + "count). Is the engine unavailable (missing native
binary, "
+ + "unreachable inference server) or configured to skip
enrichment?");
}
enrichers.add(enricher);
}
return new CompositeContentEnricher(enrichers);
}
+
+ private static Parser unwrap(Parser parser) {
+ while (parser instanceof ParserDecorator decorator) {
+ parser = decorator.getWrappedParser();
+ }
+ return parser;
+ }
}
diff --git
a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
index 49acce1070..0f7490104c 100644
---
a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
+++
b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
@@ -26,6 +26,8 @@ import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.CompositeParser;
@@ -83,6 +85,52 @@ public class ContentEnricherLoaderTest {
"unexpected message: " + e.getMessage());
}
+ @ParameterizedTest
+ @ValueSource(strings = {"image/tiff", "image/ocr-tiff"})
+ public void testMimeExcludeReachesLegacyPseudoType(String excluded) throws
Exception {
+ TikaLoader loader = load("""
+ {
+ "content-enrichers": [
+ {"test-legacy-ocr-enricher": {"_mime-exclude":
["EXCLUDED"]}}
+ ]
+ }
+ """.replace("EXCLUDED", excluded));
+ CompositeContentEnricher enrichers =
loader.get(CompositeContentEnricher.class);
+ assertEquals(java.util.Set.of(MediaType.image("png")),
enrichers.getSupportedTypes());
+ assertTrue(enrichers.getEnrichers(MediaType.image("tiff")).isEmpty());
+ assertEquals(1, enrichers.getEnrichers(MediaType.image("png")).size());
+ }
+
+ @Test
+ public void testMimeIncludeDoesNotMaskUnavailableEngine() throws Exception
{
+ TikaLoader loader = load("""
+ {
+ "content-enrichers": [
+ {"test-unavailable-enricher": {"_mime-include":
["image/png"]}}
+ ]
+ }
+ """);
+ org.apache.tika.exception.TikaConfigException e =
+ org.junit.jupiter.api.Assertions.assertThrows(
+ org.apache.tika.exception.TikaConfigException.class,
+ () -> loader.get(CompositeContentEnricher.class));
+ assertTrue(e.getMessage().contains("advertises no media types"),
+ "unexpected message: " + e.getMessage());
+ }
+
+ @Test
+ public void testMimeIncludeNarrowsLegacyEngine() throws Exception {
+ TikaLoader loader = load("""
+ {
+ "content-enrichers": [
+ {"test-legacy-ocr-enricher": {"_mime-include":
["image/png"]}}
+ ]
+ }
+ """);
+ CompositeContentEnricher enrichers =
loader.get(CompositeContentEnricher.class);
+ assertEquals(java.util.Set.of(MediaType.image("png")),
enrichers.getSupportedTypes());
+ }
+
@Test
public void testNoContentEnrichersConfigured() throws Exception {
TikaLoader loader = load("""
diff --git
a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestLegacyOcrEnricher.java
similarity index 58%
rename from
tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
rename to
tika-serialization/src/test/java/org/apache/tika/config/loader/TestLegacyOcrEnricher.java
index 73b499b270..fa331cae43 100644
---
a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
+++
b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestLegacyOcrEnricher.java
@@ -14,13 +14,11 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.tika.pipes.core;
+package org.apache.tika.config.loader;
-import java.util.Collections;
import java.util.Set;
import org.xml.sax.ContentHandler;
-import org.xml.sax.SAXException;
import org.apache.tika.annotation.TikaComponent;
import org.apache.tika.io.TikaInputStream;
@@ -28,29 +26,20 @@ import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
-import org.apache.tika.sax.XHTMLContentHandler;
-/** Fixture: proves the invocation and config path with no OCR binary
installed. */
-@TikaComponent(name = "mock-enricher", spi = false)
-public class MockEnricher implements Parser {
+/** Fixture: an OCR engine still advertising {@code image/ocr-*} pseudo-types.
*/
+@TikaComponent(name = "test-legacy-ocr-enricher", spi = false)
+public class TestLegacyOcrEnricher implements Parser {
private static final long serialVersionUID = 1L;
- public static final String MARKER_KEY = "mock-enricher";
- public static final String MARKER_TEXT = "MOCK-ENRICHED-TEXT";
-
@Override
public Set<MediaType> getSupportedTypes(ParseContext context) {
- return Collections.singleton(MediaType.image("png"));
+ return Set.of(MediaType.image("ocr-png"), MediaType.image("ocr-tiff"));
}
@Override
- public void parse(TikaInputStream tis, ContentHandler handler, Metadata
metadata,
- ParseContext context) throws SAXException {
- metadata.set(MARKER_KEY, "ENRICHED");
- XHTMLContentHandler xhtml = new XHTMLContentHandler(handler, metadata);
- xhtml.startDocument();
- xhtml.characters(MARKER_TEXT);
- xhtml.endDocument();
+ public void parse(TikaInputStream stream, ContentHandler handler, Metadata
metadata,
+ ParseContext context) {
}
}
diff --git
a/tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/CXFTestBase.java
b/tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/CXFTestBase.java
index 155b7dcc87..32b57813d1 100644
---
a/tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/CXFTestBase.java
+++
b/tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/CXFTestBase.java
@@ -306,8 +306,8 @@ public abstract class CXFTestBase {
}
/**
- * Creates a default test config with pipes configuration.
- * If the tika config contains metadata-filters, they are merged into the
pipes config.
+ * Creates a default test config with pipes configuration, carrying
parse-context,
+ * metadata-filters, presets and content-enrichers over from the tika
config.
*
* @param tikaConfigPath path to the tika config (may contain
metadata-filters)
*
@@ -357,6 +357,11 @@ public abstract class CXFTestBase {
if (metadataFilters != null && !metadataFilters.isEmpty()) {
root.set("metadata-filters", metadataFilters);
}
+ // the worker loads enrichers from its own config
+ JsonNode contentEnrichers = tikaConfig.get("content-enrichers");
+ if (contentEnrichers != null && !contentEnrichers.isEmpty()) {
+ root.set("content-enrichers", contentEnrichers);
+ }
// The worker resolves preset names from its own config, so
presets must be
// visible there just like the parse-context defaults above.
JsonNode presets = tikaConfig.get("presets");
diff --git
a/tika-server/tika-server-standard/src/test/java/org/apache/tika/server/standard/ContentEnricherWireTest.java
b/tika-server/tika-server-standard/src/test/java/org/apache/tika/server/standard/ContentEnricherWireTest.java
new file mode 100644
index 0000000000..bc2fe4d868
--- /dev/null
+++
b/tika-server/tika-server-standard/src/test/java/org/apache/tika/server/standard/ContentEnricherWireTest.java
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.server.standard;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+import java.awt.image.BufferedImage;
+import java.io.ByteArrayOutputStream;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.util.ArrayList;
+import java.util.List;
+import javax.imageio.ImageIO;
+
+import jakarta.ws.rs.core.Response;
+import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
+import org.apache.cxf.jaxrs.client.WebClient;
+import org.apache.cxf.jaxrs.lifecycle.SingletonResourceProvider;
+import org.junit.jupiter.api.Test;
+
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.parser.mock.MockEnricher;
+import org.apache.tika.serialization.JsonMetadataList;
+import org.apache.tika.server.core.CXFTestBase;
+import org.apache.tika.server.core.resource.RecursiveMetadataResource;
+import org.apache.tika.server.core.writer.MetadataListMessageBodyWriter;
+
+/** A config-named enricher reaches the forked worker and its output comes
back (TIKA-4872). */
+public class ContentEnricherWireTest extends CXFTestBase {
+
+ private static final String META_PATH = "/rmeta";
+
+ @Override
+ protected void setUpResources(JAXRSServerFactoryBean sf) {
+ sf.setResourceClasses(RecursiveMetadataResource.class);
+ sf.setResourceProvider(RecursiveMetadataResource.class,
+ new SingletonResourceProvider(new
RecursiveMetadataResource(tikaResource)));
+ }
+
+ @Override
+ protected void setUpProviders(JAXRSServerFactoryBean sf) {
+ List<Object> providers = new ArrayList<>();
+ providers.add(new MetadataListMessageBodyWriter());
+ sf.setProviders(providers);
+ }
+
+ @Override
+ protected InputStream getTikaConfigInputStream() {
+ return
getClass().getResourceAsStream("/configs/tika-config-content-enrichers.json");
+ }
+
+ @Test
+ public void testEnricherOutputSurvivesFork() throws Exception {
+ ByteArrayOutputStream png = new ByteArrayOutputStream();
+ ImageIO.write(new BufferedImage(10, 10, BufferedImage.TYPE_INT_RGB),
"png", png);
+
+ Response response = WebClient
+ .create(endPoint + META_PATH)
+ .accept("application/json")
+ .put(png.toByteArray());
+
+ Reader reader = new InputStreamReader((InputStream)
response.getEntity(), UTF_8);
+ List<Metadata> metadataList = JsonMetadataList.fromJson(reader);
+ assertEquals(1, metadataList.size());
+ Metadata metadata = metadataList.get(0);
+ assertEquals("ENRICHED", metadata.get(MockEnricher.MARKER_KEY));
+ assertContains(MockEnricher.MARKER_TEXT,
metadata.get(TikaCoreProperties.TIKA_CONTENT));
+ }
+}
diff --git
a/tika-server/tika-server-standard/src/test/resources/configs/tika-config-content-enrichers.json
b/tika-server/tika-server-standard/src/test/resources/configs/tika-config-content-enrichers.json
new file mode 100644
index 0000000000..d2be254675
--- /dev/null
+++
b/tika-server/tika-server-standard/src/test/resources/configs/tika-config-content-enrichers.json
@@ -0,0 +1,11 @@
+{
+ "content-enrichers": [ { "mock-enricher": {} } ],
+ "auto-detect-parser": {
+ "throwOnZeroBytes": false
+ },
+ "parse-context": {
+ "timeout-limits": {
+ "progressTimeoutMillis": 30000
+ }
+ }
+}
