This is an automated email from the ASF dual-hosted git repository. tballison pushed a commit to branch add-skills-for-users in repository https://gitbox.apache.org/repos/asf/tika.git
commit ca0ede08c376a6dd557ef1ab067d0d4a17de015f Author: tallison <[email protected]> AuthorDate: Wed Aug 19 18:02:47 2026 -0400 add skills to help agents use tika --- .skills/file-forensics/SKILL.md | 400 ++++++++++++++++++++++ .skills/file-forensics/demo/README.md | 21 ++ .skills/file-forensics/demo/budget.xlsx | Bin 0 -> 8396 bytes .skills/file-forensics/demo/contract.pdf | Bin 0 -> 64872 bytes .skills/file-forensics/demo/memo.docx | Bin 0 -> 52399 bytes .skills/file-forensics/demo/quarterly-report.docm | Bin 0 -> 17322 bytes .skills/file-forensics/file-forensics-config.json | 46 +++ .skills/file-to-markdown-docker/SKILL.md | 223 ++++++++++++ .skills/file-to-markdown/SKILL.md | 316 +++++++++++++++++ AGENTS.md | 15 + README.md | 25 +- 11 files changed, 1044 insertions(+), 2 deletions(-) diff --git a/.skills/file-forensics/SKILL.md b/.skills/file-forensics/SKILL.md new file mode 100644 index 0000000000..6b9ac53f3d --- /dev/null +++ b/.skills/file-forensics/SKILL.md @@ -0,0 +1,400 @@ +--- +name: file-forensics +description: > + Examine what a file claims about itself and what it actually contains — + powered by Apache Tika. True content-based type detection (extensions lie), + provenance claims (authors, dates, creating application), revision and + tamper signals (PDF incremental updates, tracked changes, hidden slides, + zip integrity), hidden and embedded content (attachments, macros), + risk indicators (PDF JavaScript actions, encryption), and content digests. + Evidence gathering, not verdicts: Tika reports what the file asserts and + what parsing observed; it does not attribute authorship or validate + signatures. Use for triaging suspicious files, provenance questions, + e-discovery-style review, or "is this file what it claims to be." +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# File forensics with Apache Tika + +**What this can and cannot tell you.** Tika cannot tell you who wrote a +file. It can tell you — exhaustively — what the file *claims* about itself +and what it actually *contains*, and those claims are the evidence: a +`dc:creator` value is an assertion some software recorded, not an identity; +a template-default author, an inconsistency between claimed dates, or a +conspicuously *missing* field is as much a finding as a present one. Report +what the file says; let the human draw conclusions. + +**The one rule for reading Tika's output:** the key prefix tells you *who is +asserting each fact*. + +- **`tk:*`** — Tika's own parse-time observations, which the file cannot + forge into place: magic-byte-detected type, embedded-item structure, + encryption status, digests of the actual bytes. +- **Everything else** (`dc:`, `pdf:docinfo:`, `extended-properties:`, + `xmp:`, ...) — the file's claims about itself, recorded by whatever + software touched it, editable by anyone with a hex editor. + +Disagreement between the two classes is where findings live. + +**And one rule with no exceptions: no date in a file is an observation.** +Creation, modification, and print timestamps — including the per-revision +timestamps inside PDF incremental updates — are written by software under +the control of whoever produced the file, and can be set to anything. Tika +can observe that a revision layer *exists*, how large it is, and what it +*contains*; when it was written is only ever claimed. Report timestamps as +"the file records...", never "this happened at...". + +Setup (getting tika-app or a tika-server) is covered by the +`file-to-markdown` companion skill; this skill assumes one is available and +uses the same invocation shapes. **Parse suspect files in isolation — this +is not optional for forensics work.** tika-server and `tika-app -f` both +parse in crash-isolated forked processes; for hostile-file triage prefer the +Docker route with the input mounted read-only (see `file-to-markdown-docker`) +so the parse can neither modify the evidence nor touch anything else. + +The one-command forensics rig — the stock Tika image started with this +skill's config (which IS the unlock on the server surface: none of these +switches are on by default there): + +```bash +docker run -d --rm --name tika-forensics -p 127.0.0.1:9998:9998 \ + -v "$(pwd)/file-forensics-config.json:/file-forensics-config.json:ro" \ + apache/tika:latest-full -c /file-forensics-config.json +curl -T suspect.file http://localhost:9998/rmeta > suspect.rmeta.json +docker stop tika-forensics # when done +``` + +Guaranteed OCR, read-only config, process isolation, explicit named +configuration — in one command. + +## Workflow: capture once, then converse over the JSON + +This skill is **not** "run Tika and read the output into context." It's two +phases: + +**Phase 1 — capture** (once per file): parse to disk, with a digest, and +make a compact metadata-only view for the conversation: + +```bash +java -jar tika-app.jar --config=file-forensics-config.json -J suspect.file > suspect.rmeta.json +# (--digest=sha256 is only needed if you are NOT using the config; +# the config's built-in digester already covers it) +# or, against the docker rig from the isolation section above: +# curl -T suspect.file http://localhost:9998/rmeta > suspect.rmeta.json + +jq 'map(del(."tk:content"))' suspect.rmeta.json > suspect.meta.json +``` + +`suspect.rmeta.json` is the full evidence record — a JSON array where entry +0 is the file and entries 1+ are everything embedded in it, content +included. `suspect.meta.json` is the same array with the (possibly huge) +extracted text stripped out: small enough to inspect freely. + +**Phase 2 — investigate conversationally.** Each question the user asks +becomes a targeted query against the saved files, and only the answering +values enter the conversation: + +```bash +jq 'length' suspect.meta.json # how many embedded items? +jq '.[0] | keys' suspect.meta.json # what fields exist at all? +jq '.[].["tk:embedded-resource-type"]' suspect.meta.json +jq '.[0]."pdf:incremental-update-count"' suspect.meta.json +grep -o 'Normal.dotm' suspect.meta.json # quick claim checks +jq '.[2]."tk:content"' suspect.rmeta.json # content of ONE entry, + # only when asked +``` + +Never load the full rmeta JSON into context — a document with a large text +body or many attachments makes it enormous, and the conversation only ever +needs slices. The saved files also make the session auditable: the evidence +the answers came from is on disk, unchanged, re-queryable. + +## The forensics config: turn on what default parsing leaves off + +This skill ships `file-forensics-config.json` (in this skill's directory): +one config for every surface — its `"server": {}` element is required for +tika-server's `-c` and harmlessly ignored by tika-app; don't remove it. +the named, explicit parse configuration for investigation work. Use it on +every surface, every time — an examination should be able to state exactly +what configuration produced its output, and "whatever that tool defaults +to" is not that. It also makes results identical across surfaces, because +the defaults are NOT the same everywhere: + +**What it changes per surface:** + +- **tika-server, pipes mode, or the library:** the config is the difference + between seeing revision history/macros and not — none of these switches + are on there. +- **tika-app single-file mode:** the CLI quietly applies its own convenience + config (it prints a banner saying so) that already enables the marquee + items — `parseIncrementalUpdates`, `extractMacros`, `extractActions`, + `extractInlineImages`. On tika-app, `file-forensics-config.json` still adds + `extractFontNames`, `extractScripts`, `includeMissingRows`, + deleted/moved-content for legacy `.doc` (on `.docx` the convenience + config already includes it, so testing this switch on a docx shows no + diff — that's expected), and pins `accessCheckMode` to + `DONT_CHECK` (tika-app's convenience config sets a more restrictive + mode). A bare `tika-app -J` will already show PDF revisions and macros — + the config's job there is making that explicit and reproducible rather + than unlocking it. + +The full switch list: + +- `pdf-parser.parseIncrementalUpdates` — each prior revision of an + incrementally-updated PDF is parsed as its own embedded document, so you + can read what the file said *before* the last save(s) +- `pdf-parser.extractActions` — full detail of automatic actions/JavaScript +- `pdf-parser.extractFontNames` — font inventory (a provenance fingerprint: + fonts betray the producing toolchain) +- `pdf-parser.extractInlineImages` — every image drawn in the PDF becomes + an embedded entry: extractable to disk with `-Z`, OCR-able, and part of the + inventory. (Repeated images are deduplicated by default — + `extractUniqueInlineImagesOnly` — which is usually what you want.) +- Office (`office-parser` + `ooxml-parser` — set on both; + 4.x config is per-component): `extractMacros` (macro code as MACRO-typed + embedded entries), `includeDeletedContent` and `includeMoveFromContent` + (tracked-change deletions and moved-away text in the output), + `includeMissingRows` (spreadsheet row gaps) +- `jsoup-parser.extractScripts` — script bodies in HTML (including HTML + email bodies and HTML attachments) appear in output instead of being + silently dropped: JavaScript in a document is something a reviewer should + see + +Some defaults are already forensics-friendly and are deliberately NOT +changed: overlapping/duplicate text is kept +(`suppressDuplicateOverlappingText: false` — text hidden under other text +stays visible), annotation/AcroForm/bookmark text is extracted, and +`accessCheckMode: DONT_CHECK` means Tika extracts regardless of the file's +claimed copy/extract restrictions while still *recording* those claims +under `access-permission:*` — the restriction flags are themselves +evidence. + +```bash +java -jar tika-app.jar --config=file-forensics-config.json -J suspect.file > suspect.rmeta.json +``` + +The config also configures a SHA-256 digester, so `tk:digest:SHA-256` +appears on every entry on **every** surface — no per-command flag to forget. + +If a parser name doesn't bind on your build, `--list-parser-names` prints +the registered names. Expect bigger output and slower parses than default +config — that's the point. + +## Identity: is it what it claims to be? + +```bash +java -jar tika-app.jar -d suspect.file # content-based media type, extension ignored +``` + +**Anchor on `tk:content-type-magic-detected`** — what the leading bytes +said, computed by Tika from content alone. The top-level `Content-Type` is +the *routing* type, and on tika-server a caller-supplied `Content-Type` +header influences it by design — so on the HTTP surface that field can +reflect upstream input, accidental or adversarial, not just Tika's own +conclusion. An extension or routed type that disagrees with +`tk:content-type-magic-detected` is a classic finding (a "pdf" that is a +zip; an "xlsx" that is plain-text CSV). Caller-supplied types are tracked +under `tk:content-type-hint`, `tk:content-type-override`, and +`tk:content-type-parser-override` (which of these appears depends on how +the type was supplied) — all input, none evidence. + +## Provenance claims + +The file's own story about its origin — read it as testimony, not fact: + +- `dc:creator` and related Dublin Core fields — claimed author(s) +- `pdf:docinfo:creator-tool`, `pdf:docinfo:producer` — what claims to have + made the PDF (e.g. a phishing PDF "authored" by a word processor that + doesn't match its claimed corporate origin) +- `extended-properties:*` (Office) — creating application/version, template + (`Normal.dotm` = stock Word), total edit time, claimed created/modified + timestamps +- **`tk:orig-resource-name`** — for xlsx, the absolute path where the file + was last saved (Excel records it in the workbook): drive letters, network + shares, and `C:\Users\<name>\...` usernames leak here. (The `tk:` prefix + here means Tika surfaced it; the *value* is still the file's own record — + treat it as a claim like the rest of this section.) + +Findings look like: dates that precede the claimed creating tool's release; +edit time of 0 on a "carefully drafted" document; a template name from an +organization other than the claimed author's; identical creator strings +across supposedly independent documents. + +## Revision and tamper signals + +- **`pdf:incremental-update-count`** (emitted by default in 4.x) — a PDF + above 0 has been modified after its initial write; each increment is a + save layered on top of the previous file. Signed-then-modified PDFs show + here. `pdf:eof-offsets` lists the byte offset of each revision's end. +- **Reading the prior versions themselves:** with the forensics config's + `parseIncrementalUpdates` on, each earlier revision appears as its own + entry in the rmeta array, typed `tk:embedded-resource-type: VERSION`, with + `pdf:incremental-update-number` giving its place in history — **0-indexed + from the earliest**; the final state is entry 0 of the array and carries + no update number. Each VERSION entry is the complete file as it existed at + that save (the bytes up to that revision's EOF offset), so its `tk:content` + is what the document said *then* — diff a version's content against entry + 0's to see what changed between saves. The layer structure and content + differences are observations; any timestamps inside each revision are that + revision's own claims — order is established by the byte layering, timing + is not. (Note: the final entry has no + `pdf:incremental-update-number` key *at all* — `jq` prints `null` for a + missing key and a null value alike, so "null on entry 0" is expected, not + an error.) +- `tk:version-count` / `tk:version-number` — the format-general spelling of + the same idea (also 0-indexed earliest-first; the latest version carries + none). PDFs emit `tk:version-count` alongside the pdf-specific key; other + formats that retain prior versions may adopt it, but not all parsers emit + these yet. +- `msoffice:has-track-changes`, `msoffice:has-comments` — content the + author may not have meant to ship. Parse output includes tracked-change + and comment text; a "deleted" tracked change is still in the file. +- `extended-properties:HiddenSlides` — slides present but not shown. +- `zip:*` integrity family (zip-based formats): `zip:integrity-check-result`, + `zip:duplicate-entry-names`, `zip:local-header-only-entries`, + `zip:central-directory-only-entries` — structural anomalies associated + with crafted or tampered archives (a duplicate entry name can make two + tools see two different "same" files). + +## Hidden and embedded content + +The rmeta array *is* the embedded-content inventory: every entry past index +0 is something inside the file — attachments, embedded objects, images, +archive members, prior VERSIONs — each with its own metadata, +`tk:embedded-resource-type` (`INLINE`, `ATTACHMENT`, `MACRO`, `METADATA`, +`THUMBNAIL`, `VERSION`, ...), `tk:embedded-depth`, and path. + +**Show the human the literal files.** The metadata inventory is for the +agent; the extracted bytes are for the person. Extract everything embedded +to disk so they can open the images, hand an attachment to another tool, or +open a prior PDF revision side-by-side with the final: + +```bash +java -jar tika-app.jar --config=file-forensics-config.json -Z --extract-dir=evidence/suspect-embedded suspect.file +``` + +- With the forensics config, this includes **macro source** (MACRO entries) + and **each prior PDF revision as a standalone, openable PDF** (VERSION + entries). +- **Read code from the extracted files, not from `tk:content`.** The default + content handler is Markdown, which escapes underscores and other + characters — VBA/script source read via `jq '.[N]."tk:content"'` comes + back visually corrupted (`VB\_Name`). The `-z`/`-Z` extracted files are + pristine bytes. (Alternative: capture with `-J -t` for unescaped plain + text per entry.) +- Extracted files are **renumbered** (`00000001.jpg`, ...); original names + live in each rmeta entry's `tk:resource-name`, and `-z`/`-Z` writes a + sidecar `<name>.json` metadata dump for mapping numbers back to names. +- Digest what you extracted (`sha256sum evidence/suspect-embedded/*/*`) so + each artifact is pinned the same way the container file is. +- Mechanics and gotchas (Pipes-mode delay, chatty stderr) are in + `file-to-markdown`; the server-side equivalent is `/unpack`, which returns + the embedded files as a zip over HTTP. **`/unpack` names differ from + `-z`/`-Z`:** plain sequential names (`1.jpg`, `2.pdf`, ...) and **no + sidecar JSON** — map names back via each rmeta entry's `tk:resource-name` + yourself. + +**Macros:** Office macro code is surfaced as embedded entries typed `MACRO`, +but only when macro extraction is enabled — it is **off by default**; +`file-forensics-config.json` (above) turns it on. Read the extracted VBA as +text. Presence of macros is a data point, not a verdict — +plenty of legitimate spreadsheets have them. + +## Risk indicators + +- `pdf:action-triggers`, `pdf:action-types`, `pdf:js-name` — automatic + actions and JavaScript wired into a PDF (open actions are a common + malicious-document mechanism, and also used legitimately by forms) +- `pdf:has-xfa`, `pdf:has-acro-form-fields` — active form machinery +- `tk:encrypted` — the file (or an embedded item) is encrypted; content + Tika couldn't read is content nobody scanned +- `tk:exception:*` — parse failures, per embedded item; a file that + crashes parsers is itself a signal worth recording + +## Signatures — presence, not validity + +`pdf:has-signature-fields` and the `tk:signature:*` family (`name`, `date`, +`reason`, `location`, `contact-info`, `filter`) report that signature +structures exist and what they claim. **Tika does not cryptographically +validate signatures** — a well-formed `tk:signature:name` proves only that +signature metadata is present, not that it verifies, and +`pdf:incremental-update-count` > 0 alongside a signature means the file +changed after some revision was signed. Use a signature-validation tool for +validity; use Tika to know there's something to validate. + +## Digests: pin down what you examined + +```bash +java -jar tika-app.jar --digest=sha256 -j suspect.file +# adds tk:digest:SHA-256 to the metadata +``` + +Record the digest alongside findings so they're tied to exact bytes — +malware family lookups, dedup across an evidence set, and "is this the same +file I looked at yesterday" all start here. (`md5`, `sha1`, `sha384`, +`sha512` also available; SHA3 requires configuring the BouncyCastle digester +in a JSON config.) + +## Demo files + +The `demo/` directory in this skill ships four small real files with real +findings — a PDF with revision history, a macro-bearing Word document, a docx with +embedded content, and an xlsx whose metadata records the absolute path where +it was last saved — plus a README of suggested questions. They're +for showing a human what this skill does on files where the default parse +looks unremarkable, and for smoke-testing your setup end-to-end. + +## Showing the human the evidence + +Terminal `jq` quotes are fine for single answers; for "walk me through what +you found," give the human something browsable: + +- If an interactive JSON viewer is installed, use it: `jless + suspect.meta.json` or `fx`, or `visidata` (the rmeta array is naturally + tabular — rows are entries, columns are keys). +- **Better: generate a report.** (Run the embedded-file extraction first if + the report will link to extracted items.) Write a self-contained static + HTML file next to the evidence — summary and digests up top, a claims-vs-observations + table per entry, a collapsible `<details>` section per embedded item, + relative links to the extracted files — and open it in the browser. No + server, no dependencies, everything embedded (a page that phones out is + not an evidence artifact). Regenerate it as the investigation deepens; it + doubles as the deliverable the human keeps. +- If your agent host supports publishing pages/artifacts, that can make the + report nicely viewable — **but read this first: publishing uploads the + evidence to an external service.** Document content, metadata, extracted + file names and paths leave the machine; anything shared can be forwarded, + cached, or indexed beyond your control, and "deleted" rarely means gone. + For anything sensitive — client documents, case material, unreleased + content, anything under privilege or NDA — keep the report as a local + HTML file and do not publish it. Publish only when the human has + explicitly confirmed the content is safe to leave the machine. + +## Evidence discipline + +- Work on a copy; mount read-only in Docker. Tika doesn't modify inputs, + but the discipline should not depend on that. +- Save the full rmeta JSON per file and record the exact command and Tika + version (`--version`) with it — findings should be reproducible. +- Report inconsistencies as inconsistencies ("claimed creation 2019, claimed + creator tool released 2021"), not conclusions ("forged"). +- This is content analysis, not chain-of-custody forensics: no write + blocking, no acquisition hashing, no court-grade custody trail. For + matters likely to face legal scrutiny, treat Tika as the analysis layer + inside a proper forensic process, not the process. diff --git a/.skills/file-forensics/demo/README.md b/.skills/file-forensics/demo/README.md new file mode 100644 index 0000000000..5592344f40 --- /dev/null +++ b/.skills/file-forensics/demo/README.md @@ -0,0 +1,21 @@ +# Demo files — try the skill on these + +Real files with real findings to discover (Apache Tika test corpus, +ALv2-licensed, renamed for realism). Ask your agent, for example: + +| File | Try asking | +|---|---| +| `contract.pdf` | "Has this PDF been modified since it was first written? Show me what it said before." | +| `quarterly-report.docm` | "Does this document contain macros? Show me the code." | +| `memo.docx` | "What did the author delete or comment on? Show me content this file still carries but doesn't display." | +| `budget.xlsx` | "Where was this spreadsheet last saved, and what does that path reveal about its author's machine?" | + +Each is small and safe, and each carries a genuine non-obvious finding — +revision history you can extract as openable prior PDFs, real macro source, +metadata that leaks a local username and path. (Note: tika-app's single-file +mode pre-enables several forensics switches, so some findings appear even +without the skill's config; the skill explains which, and the capture-once / +query workflow is where the investigation value lives.) + +Original fixture names (provenance): testPDF_incrementalUpdates.pdf, +testWORD_macros.docm, testWORD_embedded_pics.docx, testEXCEL_big_numbers.xlsx. diff --git a/.skills/file-forensics/demo/budget.xlsx b/.skills/file-forensics/demo/budget.xlsx new file mode 100644 index 0000000000..ce5dd8e1f5 Binary files /dev/null and b/.skills/file-forensics/demo/budget.xlsx differ diff --git a/.skills/file-forensics/demo/contract.pdf b/.skills/file-forensics/demo/contract.pdf new file mode 100644 index 0000000000..8494cc8396 Binary files /dev/null and b/.skills/file-forensics/demo/contract.pdf differ diff --git a/.skills/file-forensics/demo/memo.docx b/.skills/file-forensics/demo/memo.docx new file mode 100644 index 0000000000..1a63e6f585 Binary files /dev/null and b/.skills/file-forensics/demo/memo.docx differ diff --git a/.skills/file-forensics/demo/quarterly-report.docm b/.skills/file-forensics/demo/quarterly-report.docm new file mode 100644 index 0000000000..a9153100f4 Binary files /dev/null and b/.skills/file-forensics/demo/quarterly-report.docm differ diff --git a/.skills/file-forensics/file-forensics-config.json b/.skills/file-forensics/file-forensics-config.json new file mode 100644 index 0000000000..8df9c3ab97 --- /dev/null +++ b/.skills/file-forensics/file-forensics-config.json @@ -0,0 +1,46 @@ +{ + "server": {}, + "parsers": [ + { + "default-parser": {} + }, + { + "pdf-parser": { + "extractActions": true, + "extractFontNames": true, + "parseIncrementalUpdates": true, + "extractInlineImages": true + } + }, + { + "office-parser": { + "extractMacros": true, + "includeDeletedContent": true, + "includeMoveFromContent": true, + "includeMissingRows": true + } + }, + { + "ooxml-parser": { + "extractMacros": true, + "includeDeletedContent": true, + "includeMoveFromContent": true, + "includeMissingRows": true + } + }, + { + "jsoup-parser": { + "extractScripts": true + } + } + ], + "parse-context": { + "commons-digester-factory": { + "digests": [ + { + "algorithm": "SHA256" + } + ] + } + } +} diff --git a/.skills/file-to-markdown-docker/SKILL.md b/.skills/file-to-markdown-docker/SKILL.md new file mode 100644 index 0000000000..fd0bc8fdcb --- /dev/null +++ b/.skills/file-to-markdown-docker/SKILL.md @@ -0,0 +1,223 @@ +--- +name: file-to-markdown-docker +description: > + Run Apache Tika as a Docker container when you need guaranteed OCR (scanned + PDFs, images) or geospatial raster support with zero local install — + `apache/tika:<version>-full` bundles Tesseract, GDAL, ImageMagick, and + fonts. Also covers the minimal image, port/volume/memory conventions, the + path-identity mount gotcha, and how to confirm OCR actually ran rather than + silently returning no text. Powered by Apache Tika. Use when a local `tika-app`/`tika-server` + doesn't have Tesseract installed, or you want a disposable, self-contained + parsing environment. Companion to the `file-to-markdown` skill, which covers the + parsing calls themselves once a server is up. +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Running Apache Tika via Docker + +Two images on Docker Hub: `apache/tika` (REST server, port 9998) and +`apache/tika-grpc` (gRPC, port 9090). This skill covers `apache/tika`; the +`file-to-markdown` companion skill covers the parsing calls in more depth once the +container is up (same HTTP API either way), but the examples below are +enough to parse on their own. + +The Docker route needs **no local Java at all** — the container brings its +own. If the host lacks Java 17+ (Tika 4.x's requirement), this is the +easiest path, not just the OCR path. This skill describes Tika 4.x images; +on a 3.x image the default output is XHTML, not Markdown. + +## Minimal vs `-full` — the choice that matters + +``` +apache/tika:<version> # JRE + tika-server-standard, pure-Java parsers only +apache/tika:<version>-full # + Tesseract OCR, GDAL, ImageMagick, font sets +``` + +**Take `-full` if the task involves OCR (scanned PDFs, photos of documents, +image-only PDFs) or geospatial rasters — otherwise take minimal.** A scanned +PDF parsed against the minimal image doesn't error; it just silently returns +little or no text, because there's no Tesseract to run. If output looks +suspiciously short for a document that's clearly a scan, that's the signal +you're on the wrong image, not that the file has no text — see **Confirming +OCR actually ran**, below. + +Tag forms: `<version>` rolls forward on rebuild of the same release, +`<version>-<N>` is immutable (pin this in anything long-lived), `latest` +tracks newest stable. Published for `linux/amd64`, `linux/arm64`, +`linux/s390x` — `docker pull`/`docker run` picks the right one automatically. + +## Starting it + +```bash +docker run -d -p 127.0.0.1:9998:9998 apache/tika:latest-full +# or pin a real version: apache/tika:4.0.0-full +curl -T document.pdf http://localhost:9998/tika +``` + +**Bind to `127.0.0.1`, not `0.0.0.0` or a bare port mapping, unless you +specifically mean to expose it.** Docker writes its own iptables rules, so +`-p 9998:9998` (no host part) can publish the server past your host +firewall onto the network — a real, easy-to-hit surprise, not a theoretical +one. Tika parses untrusted input by design; the server itself does no +authentication, so treat network exposure as a deliberate decision, not a +default. + +Give it a few seconds to start before the first request — `curl -sf +http://localhost:9998/version` is a simple readiness check to poll in a +script. + +## Mounting files: get the path identity right + +Plain `curl -T` uploads need **no mount at all** — the document travels in +the HTTP body. Mounts matter when the container must read paths itself: a +`-c` config file, pipes fetchers reading from a directory, extra jars. The +path-identity rule below applies to **directories of documents that callers +reference by path**; a single config or jar mounted at a fixed container +path (`-v .../my.json:/my.json:ro`) is fine and normal — nothing translates +those paths back and forth. When a caller (an agent, a script) passes +filesystem paths, mount the directory at the **same absolute path inside +the container** that the caller uses outside it: + +```bash +docker run -d -p 127.0.0.1:9998:9998 \ + -v "/home/me/project:/home/me/project:ro" \ + apache/tika:<version>-full +``` + +Don't remap to something like `/data` — if you do, every path the caller +passes needs translating before Tika can see it, and every response +(embedded-file paths from `/unpack`, resource names) needs translating back. +Matching the host path exactly makes the container a transparent stand-in +for a locally-installed Tika: paths just work in both directions. `:ro` is +worth defaulting to — Tika only needs to read the input, and a read-only +mount is a real containment boundary if a parse goes wrong, on top of (not +instead of) the process isolation Tika's own forked workers already give +you. + +The container runs as a **non-root user, UID/GID `35002:35002`**. Mounted +input files must be readable, and any directory Tika writes to must be +writable, by that UID — a mount that's `0600` owned by your host user will +fail inside the container even though it works fine outside it. + +## Confirming OCR actually ran + +Don't assume the `-full` image means OCR fired on a given file — confirm it, +especially the first time you stand one up. + +**For PDFs**, `pdf:ocr-page-count` is a verified, code-confirmed signal — +non-zero means Tesseract processed that many pages: + +```bash +curl -T scanned.pdf http://localhost:9998/rmeta | jq '.[0]."pdf:ocr-page-count"' +``` + +Secondary evidence: the same entry's `tk:parsed-by-full-set` lists +`org.apache.tika.parser.ocr.TesseractOCRParser` when Tesseract ran. + +`0` or `null` means OCR didn't run on this PDF — check you're on `-full` (not +minimal), that the PDF is actually image-only (a PDF with a real text layer +correctly skips OCR — that's not a bug), and that no mounted config sets +`skipOcr: true` (see below). If the key exists but under a **camelCase name** +(`pdf:ocrPageCount`) — or you see `X-TIKA:*` keys — your image predates the +4.0.0 metadata-key renames; upgrade the image rather than adapting to the old +spellings. + +**For standalone images** (not embedded in a PDF), there's no single +verified flag confirmed here — this skill doesn't have a code-checked answer +for that case yet. The pragmatic fallback: compare extracted-text length +against a file you know is a genuine scan; suspiciously empty output on a +visibly text-bearing image is the same "wrong image" signal as above. + +## Configuration (turning OCR off, or anything else) + +Mount a `tika-config.json` and point `-c` at it — anything after the image +name is appended to the entry point, which already sets `-h 0.0.0.0` (don't +pass `-h` again): + +```bash +docker run -d -p 127.0.0.1:9998:9998 \ + -v "$(pwd)/tika-config.json:/tika-config.json" \ + apache/tika:<version>-full -c /tika-config.json +``` + +To keep `-full`'s other parsers but disable OCR specifically (e.g. you only +wanted GDAL): + +```json +{ + "parsers": [ + { "default-parser": {} }, + { "tesseract-ocr-parser": { "skipOcr": true } } + ] +} +``` + +## Memory + +Size the container's `--memory` limit; don't pass `-Xmx` — the JVM sizes its +heap from the container's own limit. Tika Pipes forks additional JVMs +*inside* the same container for isolation, and each fork's heap comes out of +that same limit, so size for the forks, not just the parent: + +```bash +docker run -d -p 127.0.0.1:9998:9998 --memory 4g apache/tika:<version>-full +``` + +## Disposable / one-shot use + +For a short-lived session, name the container so you can stop it +deterministically — backgrounding `docker run` does NOT stop the container +when your script exits, and a leaked container keeps port 9998 occupied for +your next attempt: + +```bash +docker run -d --rm --name tika-tmp -p 127.0.0.1:9998:9998 apache/tika:latest-full +until curl -sf http://localhost:9998/version >/dev/null; do sleep 1; done +curl -T document.pdf http://localhost:9998/tika +docker stop tika-tmp # --rm removes it on stop +docker ps --filter name=tika-tmp # verify: should list nothing +``` + +For many parses in a session, start it once and reuse it — the JVM warm-up +cost is worth paying only once, not per file. + +## Troubleshooting + +- **`500` with an empty body** from `/tika` or `/rmeta`: the response tells + you nothing — go straight to `docker logs <container>`. A log line like + `Unknown fetcher type: 'file-system-fetcher' ... Available types: []` plus + a pf4j `No 'plugins' root` warning means the server can't find its plugins + directory — a symptom of an outdated or hand-built image whose working + directory isn't the distribution root. Official 4.0.0+ images set the + working directory correctly; upgrade the image (workaround for a broken + one: add `--workdir /opt/tika-server`). +- **Old metadata key spellings** (`pdf:ocrPageCount`, `X-TIKA:*`): image + predates 4.0.0 — upgrade (see the OCR section). +- **Connection refused right after start**: the JVM is still booting; poll + `curl -sf http://localhost:9998/version` rather than sleeping a fixed time. +- **Permission denied reading a mounted file**: the container runs as UID + 35002 — see the mount section. + +## What's not built yet + +There is no `apache/tika-app` image and no MCP mode for the Tika Docker +images as of this writing — the container always runs `tika-server`. If a +task specifically wants a stdio-spoken tool (MCP) rather than an HTTP server, +that isn't available through Docker today; use the `file-to-markdown` skill's +tika-app path directly on the host instead. diff --git a/.skills/file-to-markdown/SKILL.md b/.skills/file-to-markdown/SKILL.md new file mode 100644 index 0000000000..52253471b8 --- /dev/null +++ b/.skills/file-to-markdown/SKILL.md @@ -0,0 +1,316 @@ +--- +name: file-to-markdown +description: > + Turn almost any file into Markdown plus metadata — PDF, Office, HTML, + email, archives, images, audio/video, 1000+ formats — powered by Apache + Tika, either via + the tika-app CLI (zero setup, one file) or a running tika-server (curl, + warm process, many calls). Leads with rmeta (structured, embedded-item-aware + output) as the default operation rather than flat concatenated text, since + you can't tell whether a file has embedded content from its extension. + Covers metadata-only triage, type and language detection, OCR, and + output-size discipline for agent context. Use whenever a task involves + reading the content of a file whose format you don't want to hand-parse. +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Using Apache Tika from an agent + +Apache Tika turns almost any document into text you can read, and reports +per-document content as **Markdown** by default (4.x) — the format you +actually want, not raw XML or a wall of HTML tags. See below for why to +reach for the structured `rmeta` view rather than a flat blob, even when the +file looks simple. + +This skill is for reading files, one at a time, inside your normal working +loop. It is not a batch pipeline — see **Batch processing**, below, for why. + +**When NOT to reach for Tika:** if your host can already read the file +natively (many agent environments read PDFs and images directly) and you +only need its visible text, use that — it's one step, not three. Tika earns +its place for everything else: `.docx`/`.xlsx`/`.pptx`, email (`.eml`, +`.msg`) and its attachments, archives, embedded content inside anything, +metadata (authors, dates, edit history), OCR, and the long tail of ~1000 +formats nothing else opens. + +**Requirements:** Tika 4.x needs **Java 17+** (`java -version` to check). +No Java 17? In order of least pain: + +1. **Docker installed?** Use the Docker route — zero local Java (see the + `file-to-markdown-docker` companion skill, or + `docker run apache/tika:latest-full`). +2. **No Docker either? Offer to install a user-local Java — ask the user + first; never install software silently.** The least invasive option is a + Temurin JRE unpacked into a directory the user owns: ~45 MB, no admin + rights, no PATH changes, uninstall = delete the directory. + + ```bash + mkdir -p ~/tika-jre && cd ~/tika-jre + curl -L "https://api.adoptium.net/v3/binary/latest/21/ga/linux/x64/jre/hotspot/normal/eclipse"; | tar xz + ~/tika-jre/*/bin/java -jar tika-app.jar --version + ``` + + Swap `linux`/`x64` in the URL for `mac`|`windows` and `aarch64` as + needed (Windows: fetch the `.zip` variant and unzip). If the user prefers + a managed install, the system package works too (`temurin-21-jre` via + apt, `brew install --cask temurin@21`, `winget install + EclipseAdoptium.Temurin.21.JRE`) — that needs admin rights and touches + system state, so it's their call, not the default. +This skill describes **Tika 4.x**; on a 3.x install the defaults differ +(3.x outputs XHTML, not Markdown, and some flags changed) — check +`java -jar tika-app.jar --version` or `curl localhost:9998/version` if +behavior doesn't match what's described here. + +## Which surface: tika-app or tika-server? + +First gate: **no Java 17 on this machine?** Then this choice is moot — use +the Docker route (`file-to-markdown-docker` skill), or offer the user-local +JRE install (Requirements, above). With Java 17+, two ways to reach Tika — +pick based on how many files you expect to touch in this session. + +- **`tika-app` (CLI)** — zero setup, no process to manage, one file per + invocation. Each call pays a JVM startup cost (roughly a second). Use this + for "read this one file" or a handful of files. +- **`tika-server` (curl)** — a warm process you either already have running + (ask, or check `curl -s localhost:9998/version`) or start yourself. No + per-call JVM cost once it's up. Use this if you're about to read many files + in the same session, or a server is already available. + +If unsure and doing more than two or three files, start a server: + +```bash +java -jar tika-server-standard-<version>.jar & +# starts on localhost:9998 +``` + +**Getting either one:** download the **zip** distribution from +https://tika.apache.org/download.html and run the jar from inside the +unzipped directory. Do NOT grab just the jar from Maven Central: 4.x jars +are thin launchers that need the `lib/` directory sitting next to them, and +fail with `NoClassDefFoundError` on their own. If a jar you've been pointed +at already works, it's inside a proper distribution — leave it where it is. + +Both surfaces are driven by the same parsing engine — output is identical +either way for the same handler/format choice. + +## The core operation: parse one file with `rmeta` + +Default to **`-J` (tika-app) / `/rmeta` (tika-server)**, not plain `/tika`. + +```bash +java -jar tika-app.jar -J document.pdf # JSON array, markdown content per entry +curl -T document.pdf http://localhost:9998/rmeta # same shape +``` + +Result is a JSON array: entry 0 is the document itself, entries 1+ are +anything embedded in it (attachments, embedded images/objects, archive +members — if any). Read entry 0's `tk:content` field for the common +single-document case (note the `tk:` prefix — a plain `.content` lookup +returns nothing). + +**Why rmeta and not plain `/tika`, even for an ordinary-looking file:** you +cannot tell whether a file has embedded content from its extension or format +— a `.pdf` can carry attachments, a `.docx` can carry embedded objects, a +`.png` can carry XMP-embedded sidecar data, and even a file you're sure is +"flat" you can't actually confirm without parsing it. Plain `/tika` doesn't +avoid this — it still recursively parses and includes any embedded text, +just concatenated into one blob with no boundaries, no per-item metadata, +and no visibility into whether an individual embedded item failed. `rmeta` +is the *same parse*, structured, at no real extra cost. Reach for plain +`/tika`/`-t` deliberately (see below), not as the default. + +`-J` combines with `-x`/`-h`/`-t`/`-m` to pick the content format used inside +each entry (default markdown); the server's equivalent is `/rmeta/<handler>` +(`/rmeta/text`, `/rmeta/html`, ...). + +To pull embedded items out as actual files (not just their extracted text): +`/unpack` on the server, or on tika-app `-z`/`--extract` (direct +attachments, depth 1) or `-Z` (recursive, all depths), with +`--extract-dir=<dir>` for the destination. This is not +format-specific to office documents — it works on anything Tika can find +embedded content in: email attachments (`.eml`, `.msg`), archive members +(`.zip`), PDF attachments, embedded objects/images in any office format +(an inline pasted picture counts), and so on. + +What lands on disk: a `<name>-embed/` directory of the embedded files +**renumbered** (`00000001.jpg`, ...) — original names are not preserved on +disk; they're in the rmeta output's `tk:resource-name` per entry, so keep +the sibling `<name>.json` (an rmeta-shaped metadata dump `-z` also writes) +if you need to map numbers back to names. + +`-z`/`-Z` route through Tika Pipes mode rather than the fast synchronous +path the other flags use — expect several seconds and a burst of +plugin/forked-JVM startup logging on stderr; that's normal, not a hang or +an error. + +Read the extracted content selectively (grep, or Read with offset/limit) +rather than dumping it into your context wholesale — see **Output +discipline** below. + +## Flat concatenated text — when you deliberately don't need structure + +If you specifically want one blob of body text and don't care about +per-document boundaries, embedded-item metadata, or per-item exception +visibility — a quick keyword search, a rough skim — plain output is cheaper: + +```bash +java -jar tika-app.jar document.pdf > document.md # markdown is the default +curl -T document.pdf http://localhost:9998/tika > document.md +# note: the HTTP response Content-Type header says text/plain even when the +# body is Markdown — the endpoint you called, not the header, tells you the format +``` + +This still includes embedded document text (see above) — it's just flattened +in. And its damage behavior is the strongest reason to prefer `rmeta`: a +*fatal* container exception gets a `422` here (partial content, no detail), +but a **recoverable** parse problem — the common case for a damaged or +truncated file — returns a plain `200` with **silently truncated content +and no signal at all**: no status change, no header, nothing in the body. +`rmeta` on the same file returns `200` too, but *shows* the damage: a +`tk:exception:warn` (or `tk:exception:container-exception`) entry with the +stack trace, and possibly fewer array entries than the intact file would +produce. Status codes cannot be relied on to detect damage — inspecting +`tk:exception:*` keys in rmeta output is the detection mechanism. + +## Metadata only (fast triage before committing to full content) + +Cheaper than a full parse when you just need to know what a file is — +author, dates, page count, content-type — before deciding whether to read it. + +```bash +java -jar tika-app.jar -j document.pdf # JSON metadata, no content +curl -T document.pdf http://localhost:9998/meta # JSON by default +``` + +This doesn't tell you whether the file has embedded content — `/meta`/`-j` +covers only the container document's own metadata. Some formats surface a +hint (Office's `msoffice:has-comments`/`has-track-changes`), but the only +reliable way to know is `rmeta`'s array length: +`java -jar tika-app.jar -J file | jq 'length'` — `1` means no embedded +items; each entry past the first is one embedded item (its +`tk:embedded-resource-type` says how it's embedded, e.g. `INLINE` for a +pasted-in image vs `ATTACHMENT`). + +## Detection: what kind of file is this, without parsing it + +```bash +java -jar tika-app.jar -d document # prints the media type +curl -T document http://localhost:9998/detect # text/plain media type + +java -jar tika-app.jar -l document.pdf # language only +curl -T document.pdf http://localhost:9998/language +``` + +`/detect` and `/language` work even on files with no extension or a +misleading one — detection is content-based. + +## OCR (scanned PDFs, images, screenshots) + +Text-layer parsing does nothing for a scanned page or a photo of text — you +need OCR, which requires Tesseract to be present. `tika-app`/local +`tika-server` only OCR if Tesseract is installed on the host; there is no +bundled fallback. + +**Guaranteed OCR, no local install:** run the `-full` tika-server Docker +image, which bundles Tesseract, GDAL, and fonts: + +```bash +docker run -d -p 127.0.0.1:9998:9998 apache/tika:<version>-full +curl -T scanned.pdf http://localhost:9998/tika # OCR runs automatically +``` + +If a parse of an image-heavy PDF comes back suspiciously short, that's the +signal you're missing OCR, not that the file has no text. For mount/path +setup, confirming OCR actually ran, and other Docker specifics, see the +`file-to-markdown-docker` skill. + +## Output discipline — don't flood your own context + +The most common mistake using Tika from inside an agent loop: piping a large +document's full Markdown/JSON straight into context. A single PDF can produce +megabytes of output. Instead: + +1. **Redirect to a file, not a variable or inline output.** `> extracted.json`, + then read what you need with offset/limit or grep — don't capture full + stdout into your working context by default. +2. **Triage with metadata or detection first** (above) when you're deciding + *whether* to read a file, not just what's in it. +3. **`rmeta`/`-J` output is compact single-line JSON — use `jq`, not grep, to + isolate one entry** (e.g. `jq '.[0]."tk:content"'`); a bare grep can find a + string but can't tell you which array entry it came from. Add + `-r`/`--pretty-print` first if you want it grep-friendly instead. A file + with many embedded items (a large `.eml`, a nested archive) can produce a + long array either way. +4. **Ask for `text`/`txt` as the per-entry handler** (`/rmeta/text`, + `-J -t`) when you only need body text, not formatting — smaller output, + same information for most downstream uses (search, keyword extraction, + summarization prompts you control the framing of). + +## Batch processing — not this skill + +If the task is "parse thousands of files," stop reaching for per-call +tika-app/curl inside your loop — that's the wrong shape (slow, and each +result would flood your context in turn). Use tika-app's Tika Pipes mode +instead, which is designed for it and writes results to an output directory +(or a configured emitter) rather than back to you: + +```bash +java -jar tika-app.jar -i /path/to/input -o /path/to/output +# one JSON-array (rmeta-shaped) output file per input document; +# add --handler m --content-only for bare .md files instead +``` + +This runs out-of-band; check the output directory or configured emitter for +results rather than expecting them in your context. Configuring fetchers, +emitters, and worker count is beyond this skill's scope — see the Tika Pipes +documentation at https://tika.apache.org/docs if you need to set this up. + +## Error handling (tika-server) + +- **`429`** — the server's worker pool is saturated, not broken. Back off and + retry (`Retry-After` header tells you how long). +- **`503`** with `TIMEOUT`/`OOM`/`UNSPECIFIED_CRASH` — that specific parse + failed (the file may be hostile or malformed); the server itself is fine. + Retrying the same file will likely fail the same way — move on rather than + loop. +- **`422`** on the raw endpoints (`/tika`, `/tika/text`, etc.) — a *fatal* + container exception; partial content is still in the body. Recoverable + parse problems do NOT get a 422 — they return `200` with silently + truncated content (see the flat-text section above); only `/rmeta` reveals + those, via `tk:exception:*`. +- **`400`** — malformed request: an unrecognized handler name on + `/rmeta/<handler>` (the message lists the valid set), or an unknown + fetcher/emitter on `/pipes`. Fix the request; retrying unchanged won't + help. **Exception:** a wrong handler under `/tika/...` returns a bare + `404`, because `/tika` has only four literal handler routes. +- Handler names differ per family: `/rmeta/<handler>` accepts `text`, `txt`, + `html`, `xml`, `body`, `markdown`, `md`, `ignore`; `/tika/<handler>` is + only `text`, `html`, `xml`, `md` (plus `json`). `/tika/markdown` is a 404 + even though `/rmeta/markdown` works. +- **`429`/`503`** (below) are documented behavior you can't easily reproduce + with clean small files — take them on faith until you hit them under load. + +## Trust note + +Tika parses untrusted files safely when it runs in a forked/isolated process +— which `tika-server` and `tika-app`'s `-f`/`--fork` mode both do. Calling +the library directly in-process on a file you don't trust has no such +protection: a hostile file can exhaust memory/CPU or crash the process. If +you're not sure a file is safe, use tika-server or `-f`, not an embedded +parser call. diff --git a/AGENTS.md b/AGENTS.md index 3603210829..a858197a5b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -23,6 +23,10 @@ the ground rules: build with `./mvnw` (always `clean`, `-Pfast` for quick builds), never run `git commit`/`git push` or write to GitHub, code and test conventions, pre-commit checks. +## Working on Tika + +Contributor-facing — building, testing, and releasing this codebase. + | Skill | Use when | |-------|----------| | `.skills/dev/SKILL.md` | Any development task — load at session start | @@ -32,3 +36,14 @@ conventions, pre-commit checks. | `.skills/tika-eval-encoding-regression/SKILL.md` | Charset-detector regression hunts | | `.skills/tika-eval-h2-query/SKILL.md` | Querying the tika-eval H2 database directly | | `.skills/update-site-for-release/SKILL.md` | Updating tika.apache.org for a release | + +## Using Tika + +For any agent that wants Tika as a tool — not specific to this repo, useful +whether or not you're working on Tika's own source. + +| Skill | Use when | +|-------|----------| +| `.skills/file-to-markdown/SKILL.md` | Turning a file (PDF, Office, email, archives, images, ...) into Markdown + metadata via tika-app or tika-server | +| `.skills/file-to-markdown-docker/SKILL.md` | Need guaranteed OCR/GDAL with no local install, or a disposable containerized Tika — running tika-server via Docker | +| `.skills/file-forensics/SKILL.md` | What a file claims vs. contains: provenance, tamper signals, hidden/embedded content, macros, digests — evidence, not verdicts | diff --git a/README.md b/README.md index f8f27b7a5c..06c6ab157f 100644 --- a/README.md +++ b/README.md @@ -5,12 +5,29 @@ [](https://ci-builds.apache.org/job/Tika/job/tika-main-jdk17/lastBuild/testReport/) [](http://search.maven.org/#search|ga|1|g%3A%22org.apache.tika%22) -Apache Tika(TM) is a toolkit for detecting and extracting metadata and structured text content from various documents using existing parser libraries. +Apache Tika(TM) detects and extracts metadata and text from over a thousand +file types. As of 4.0.0 it emits **Markdown by default** — output shaped for +LLM and RAG pipelines — parses in **crash-isolated forked processes**, and +adds vision-language-model parsers (Claude, Gemini, OpenAI) for documents OCR +can't read. Tika is a project of the [Apache Software Foundation](https://www.apache.org). Apache Tika, Tika, Apache, the Apache feather logo, and the Apache Tika project logo are trademarks of The Apache Software Foundation. +## Using Tika from AI Agents + +Tika 4.x is built for agent pipelines: Markdown output by default, structured +recursive extraction (`-J` / `/rmeta`), and process isolation so a hostile +document takes down a fork, not your service. + +Ready-to-use agent skills live in [`.skills/`](.skills/): +[`file-to-markdown`](.skills/file-to-markdown/SKILL.md) (parsing via tika-app +or tika-server) and +[`file-to-markdown-docker`](.skills/file-to-markdown-docker/SKILL.md) +(containerized Tika with guaranteed OCR). They are standalone — copy them into +any agent's skill directory; nothing in them requires this repository. + ## Quick Start **Parse a file in Java:** @@ -29,7 +46,11 @@ launcher that loads the parsers from the adjacent `lib/`; on its own it fails wi `NoClassDefFoundError`. ```bash -java -jar tika-app-<version>.jar --text document.pdf +java -jar tika-app-<version>.jar document.pdf # Markdown (the 4.x default) +java -jar tika-app-<version>.jar --text document.pdf # plain text +java -jar tika-app-<version>.jar -J document.pdf # structured JSON: metadata + + # content for the document AND + # anything embedded in it ``` **Maven dependency:**
