[
https://issues.apache.org/jira/browse/TIKA-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18141322#comment-18141322
]
Hudson commented on TIKA-5027:
------------------------------
SUCCESS: Integrated in Jenkins build Tika ยป tika-main-jdk17 #1788 (See
[https://ci-builds.apache.org/job/Tika/job/tika-main-jdk17/1788/])
TIKA-5027: Junk retrain v113 (#3571) (github:
[https://github.com/apache/tika/commit/74249f86e6ad0bd69a1bb073312758bddb90a25b])
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/resources/org/apache/tika/ml/chardetect/hashed-ngram.hng
* (edit) docs/modules/ROOT/pages/advanced/charset-detection-design.adoc
* (edit)
tika-ml/tika-ml-junkdetect/src/main/resources/org/apache/tika/ml/junkdetect/junkdetect.bin.provenance
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/test/resources/org/apache/tika/ml/chardetect/hashed-ngram-parity.tsv
* (edit) CHANGES.txt
* (edit) tika-ml/tika-ml-junkdetect-tools/train-junk-model.sh
* (edit)
tika-ml/tika-ml-junkdetect/src/test/java/org/apache/tika/ml/junkdetect/HashedArbitrationTest.java
* (add)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/test/java/org/apache/tika/ml/chardetect/MacRomanDetectionTest.java
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/test/java/org/apache/tika/ml/chardetect/HashedNgramEncodingDetectorTest.java
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/resources/org/apache/tika/ml/chardetect/hashed-ngram.hng.provenance
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/java/org/apache/tika/ml/chardetect/MojibusterEncodingDetector.java
* (edit)
tika-ml/tika-ml-junkdetect/src/main/java/org/apache/tika/ml/junkdetect/HashedArbitration.java
* (edit)
tika-ml/tika-ml-junkdetect-tools/src/main/java/org/apache/tika/ml/junkdetect/tools/JunkFamilies.java
* (edit)
tika-ml/tika-ml-junkdetect/src/main/resources/org/apache/tika/ml/junkdetect/junkdetect.bin
> Improve junk detector
> ---------------------
>
> Key: TIKA-5027
> URL: https://issues.apache.org/jira/browse/TIKA-5027
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Major
> Fix For: 4.3.0
>
>
> In looking at charset detection, I found a number of things we could improve
> in junk detection.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
