Pascal Essiembre created TIKA-2219: -------------------------------------- Summary: CharsetDetector no longer detects windows-1252 charset Key: TIKA-2219 URL: https://issues.apache.org/jira/browse/TIKA-2219 Project: Tika Issue Type: Bug Components: parser Affects Versions: 1.14 Environment: Any. Reporter: Pascal Essiembre Priority: Minor
Starting with Tika 1.14, windows-1252 is no longer detected, as ISO-8859-1 is always detected instead. While not tested, this likely affects other windows-125* encodings. I tracked it down to a change in the {{CharsetRecog_sbcs.CharsetRecog_8859_1#getName()}} method. Now it always returns "ISO-8859-1" whereas before it was: {{return haveC1Bytes ? "windows-1252" : "ISO-8859-1";}} Now that condition has been moved to the {{match(CharsetDetector det)}} method so that the returned CharsetMatch has the proper name. The problem with that is {{CharsetDetector#detectAll()}} method overwrites the correct match with a new one that will return the value of {{#getName()}} from the {{CharsetRecognizer}} instead (which is always "ISO-8859-1" in this case). There might be legitimate reasons why the {{CharsetMatch}} instances in {{detectAll()}} method are replaced with new ones, but changing this code in that method appears to work for me: // Remove this: // CharsetMatch m = new CharsetMatch(this, csr, confidence); // matches.add(m); // Add this instead: matches.add(charsetMatch); -- This message was sent by Atlassian JIRA (v6.3.4#6332)