[
https://issues.apache.org/jira/browse/TIKA-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131025#comment-18131025
]
ASF GitHub Bot commented on TIKA-4766:
--------------------------------------
krickert commented on PR #2961:
URL: https://github.com/apache/tika/pull/2961#issuecomment-6369389865
I went over the copilot review and issued the fixes
Here's a summary, followed by the more detailed summary produced by AI:
* Fixed the booleans fall back to string.
* Fixed the edge case of a skip vs failed and updated the javadoc
* Better error handling when paring closed early
AI specifics:
> Addressed the Copilot review in ffe0add7:
> - **Unrecognized booleans fall back to strings** (`MetadataTagger`).
`parseBoolean` now returns `null` for anything other than true/false, yes/no or
1/0 (any case). `tagByDeclaredType(BOOLEAN, ...)` then returns `null`, so the
whole entry falls back to strings, as integers, numbers and dates already do,
instead of arriving as a guessed `false`. Tests:
`booleanParsingAcceptsCommonSpellings` now also checks that `maybe` and `N/A`
give `null`, and the new `unrecognizedBooleanFallsBackToStrings` covers the
fallback.
> - **`PARSE_EXCEPTION_NO_EMIT` maps to `FAILED`** (`DocumentBuilder`).
tika-grpc sends `ON_PARSE_EXCEPTION.SKIP`, so this status means the parse
threw. It comes back with only the stack trace and no metadata list, so there
is nothing partial to return. The Javadoc is updated, and the new
`parseExceptionWithNothingEmittedIsFailedNotPartial` covers it.
> - **An interrupted FetchAndParse closes the call with `UNAVAILABLE`**
(`TikaGrpcServerImpl`, `TikaGrpcV2ServerImpl`). When `runFetchAndParse` is
interrupted it has no outcome. The unary and server-streaming calls used to
complete anyway with no reply, and a unary client saw `INTERNAL: No value
received for unary call`. `fetchAndParseImpl` now closes the call through a
shared `closeInterrupted` and returns `false`, and the caller skips
`onCompleted()`. This applies to v1 and v2, and to the bidi streams. The new
`testInterruptedFetchAndParseClosesWithUnavailable` interrupts the handler
thread before the call (in-process transport, direct executor) and checks that
v1 and v2 both send no reply, no completion and one `UNAVAILABLE` error.
> Tested with `./mvnw test -pl tika-grpc-mapper,tika-grpc`: 33/33 and 29/29
pass, and checkstyle is clean.
> Replace tika-grpc fields map with a typed Document parse contract
> -----------------------------------------------------------------
>
> Key: TIKA-4766
> URL: https://issues.apache.org/jira/browse/TIKA-4766
> Project: Tika
> Issue Type: New Feature
> Components: tika-pipes
> Affects Versions: 4.0.0
> Reporter: Kristian Rickert
> Priority: Major
> Labels: grpc, pipes, protobuf
>
> Replace the flat {{FetchAndParseReply.fields}} map ({{map<string,string>}})
> with a typed parse result.
> Approach (reshaped from the original {{ParseResponse}} design after review in
> [PR #2916|https://github.com/apache/tika/pull/2916]): a single small
> {{Document}} proto (~200 lines) rather than per-format metadata messages.
> * *Content*: a structured markdown block tree -- headings, paragraphs, lists,
> tables, code blocks, inline runs (CommonMark + GFM) -- plus the rendered
> markdown string {{ToMarkdownContentHandler}} already produces (TIKA-4730).
> * *Typed common metadata*: title, authors, keywords, languages,
> created/modified as {{Timestamp}}s, page/word/character counts, dimensions,
> rights.
> * *Lossless tagged tail*: every remaining metadata key,
> multivalue-preserving, typed only where Tika's own {{Property}} declares a
> type, string otherwise -- never guessed.
> * *Embedded documents* recurse as fully typed child {{Document}}s.
> * Format specifics live in per-parser {{DocumentTransformer}} code
> ({{tika-grpc-mapper}}), never in the wire contract, so metadata churn never
> forces a client rebuild.
> Modules: {{tika-grpc-api}} (proto + generated messages + bundled
> {{FileDescriptorSet}}), {{tika-grpc-mapper}}, {{tika-grpc}} integration.
> Breaking change for clients reading {{fields}} (field number reserved).
> PR: [https://github.com/apache/tika/pull/2921] (supersedes
> [#2916|https://github.com/apache/tika/pull/2916])
> Follow-ups will be tracked in separate issues: pluggable external parsers
> (opaque {{Any}} extension results from registered gRPC services), and a
> Markdown input parser.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
