[
https://issues.apache.org/jira/browse/TIKA-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Allison updated TIKA-2937:
------------------------------
Priority: Minor (was: Major)
> Improve legacy HTML charset detector by replicating Standard's behavior for
> UTF-16
> ----------------------------------------------------------------------------------
>
> Key: TIKA-2937
> URL: https://issues.apache.org/jira/browse/TIKA-2937
> Project: Tika
> Issue Type: Improvement
> Reporter: Tim Allison
> Priority: Minor
>
> Our newer standards based detector follows the standard and automatically
> modifies a charset header of "UTF-16" (and some variants) to "UTF-8". This
> appears to help quite a bit.
> ||LegacyHTMLDetector||StandardHTMLDetector||NumCommonTokensLegacy||NumCommonTokensStandard||Delta||
> |UTF-16LE|UTF-8|10|27351|27341|
> |UTF-16BE|UTF-8|14|43565|43551|
> |UTF-16|UTF-8|6099|748024|741925|
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
