krickert commented on PR #3451:
URL: https://github.com/apache/tika/pull/3451#issuecomment-6473964932
First - much better than what I did.
Second - I think we can avoid the string conversion completely
Here's a short version that builds on what you have so far:
Protobuf already has a value type, `google.protobuf.Value` (plus `ListValue`
and `Struct`). I'd use that instead of a custom `Values`. It's a first-class
type that every client + proto-nerd already knows, and you get real numbers,
booleans and lists on the wire instead of strings with hints. The struct is
smaller too. However yours is superior in one way - all numbers in Value are
doubles - so integers can lose precision.
Instead - how about this?
In java, every protobuf message extends `Message`. So the plug-in point can
be one interface, found through SPI:
```java
public interface MessageParser<T extends Message> {
Class<T> messageType();
List<Property> properties(); // the keys it writes and their types
List<Metadata> parse(T message) throws TikaException;
}
```
So you'd have to just make a plugin jar ships its own generated stubs and
one of these. The parser maps the message into Metadata the same way any Tika
parser does, in plain Java. Tika never sees the .proto, and there's no mapping
config or reflection. The service takes a `google.protobuf.Any` and picks the
parser by its type URL. It returns the same document list as FetchAndParse,
with unary, streaming and bidi calls just like yours. I have it working on top
of your PR and can share the branch.
The problem with swapping strings are the weirdo things that happen - dates
and times are annoying :) And converting to strings is really heavy on the CPU
under heavy load.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
