gansheer commented on issue #9392:
URL: https://github.com/apache/camel-quarkus/issues/9392#issuecomment-6583940206
**Environment:** current `main`, Quarkus 4.0.0.Beta1, Mandrel jdk-25 builder
image, `camel-quarkus-tika` with Tika as brought by `camel-tika` 4.23.0.
### Registering the models is required
```java
if
(QuarkusClassLoader.isClassPresentAtRuntime("org.apache.tika.ml.chardetect.MojibusterEncodingDetector"))
{
resourceDirectory.produce(new
NativeImageResourceDirectoryBuildItem("org/apache/tika/ml/chardetect"));
}
```
The class-present guard keeps the directory out of builds that don't use
`tika-parser-text-module`. With this, the detector loads. But it is not enough.
Once the detector is active, native plain text and XML parsing fail:
```
java.nio.charset.UnsupportedCharsetException: windows-1252
```
`MojibusterEncodingDetector` resolves charsets by name at runtime:
`windows1252Fallback()` is unguarded and is hit on ASCII input, no-match and
empty-pool paths; `NaiveBayesBigramEncodingDetector` resolves the cohort
charset labels from the model during construction; `StructuralEncodingRules`
needs the ISO-2022 charsets. Native images bundle only a small subset of the
JDK charsets by default, so the suggested change alone converts a logged
warning into an HTTP 500 on the default parsing path.
### Charset handling
There is no extension-facing API for charsets in Quarkus 4.0.0.Beta1;
`quarkus.native.add-all-charsets` is an application property only. Since the
detector is pulled in transitively by the extension, applications can't
reasonably be expected to configure it. The runtime jar therefore ships
`META-INF/native-image/org.apache.camel.quarkus/camel-quarkus-tika/native-image.properties`
with `Args = -H:+AddAllCharsets`.
Native integration test results with a temporary `/charset/{name}` probe:
- no property file: only UTF-8 available; `windows-1252`, `ISO-8859-2`,
`Shift_JIS`, `KOI8-R`, `Big5`, `EUC-JP`, `GBK`, `ISO-2022-JP`, `IBM437` all
unsupported; parse fails with the `UnsupportedCharsetException` above
- with the property file: all of the above supported, parse succeeds, full
native suite 10/10
- with the property file plus `quarkus.native.add-all-charsets=true`: same
result
The jar-level `Args` win over Quarkus's own `-H:-AddAllCharsets` because the
native-image driver processes jar properties when it reaches `-jar
<runner>.jar`, which Quarkus places last. I also verified that
`quarkus.native.additional-build-args=-H:-AddAllCharsets`,
`additional-build-args-append` and the `NATIVE_IMAGE_OPTIONS` environment
variable all land before `-jar`, so none of them can turn it off.
Cost: +6.5 MB (156.03 MB -> 162.58 MB) on the integration test application.
### Proposed change
- register `org/apache/tika/ml/chardetect` under the class-present guard
- ship the `native-image.properties` with `-H:+AddAllCharsets`
- document both in the Tika usage page and the generated reference page,
including that the option can't be disabled
- the existing native `TikaIT.testXml` already guards the charset
requirement, so no new test
I'm happy to send a PR if forcing the charset set from the extension is
acceptable. The alternative, registering the resources only and documenting
`quarkus.native.add-all-charsets` for applications, leaves native plain
text/XML parsing broken by default until the user opts in, which is why I went
the other way.
_Writen with the help of AI, because it produces far more beautiful text
than me._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
