[
https://issues.apache.org/jira/browse/TIKA-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18144043#comment-18144043
]
Hudson commented on TIKA-5062:
------------------------------
SUCCESS: Integrated in Jenkins build Tika ยป tika-main-jdk17 #1792 (See
[https://ci-builds.apache.org/job/Tika/job/tika-main-jdk17/1792/])
TIKA-5062: classify magic-less content as text or binary with
TextBinaryClassifier (#3598) (github:
[https://github.com/apache/tika/commit/fee8015695e795424a17383098695db3bd74fe67])
* (add) tika-core/src/main/java/org/apache/tika/detect/TextBinaryClassifier.java
* (edit) CHANGES.txt
* (edit) tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java
* (edit) tika-core/src/test/java/org/apache/tika/mime/MimeDetectionTest.java
* (add)
tika-core/src/test/java/org/apache/tika/detect/TextBinaryClassifierTest.java
> Improve text/binary classification
> ----------------------------------
>
> Key: TIKA-5062
> URL: https://issues.apache.org/jira/browse/TIKA-5062
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
>
> Since the beginning of time, Tika has used a heuristic fallback TextDetector
> if MimeTypes couldn't find a match. The heuristics were based on ascii and
> utf-8, not legacy encoded text.
> We should experiment to see if we can get a small model to outperform the
> current heuristic and be robust against legacy encoded text.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
