[
https://issues.apache.org/jira/browse/TIKA-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041186#comment-17041186
]
Tim Allison commented on TIKA-3048:
-----------------------------------
Thank you for sharing that file. I can't read Chinese, but I _think_ that the
encoding is actually UTF-8. When I open the file as utf-8 and run it through
google translate, I get: "On a dark night of the day, two dead oysters came out
to play."
When I open the file in Chrome, and then translate the text there, I get "
€ € 戠 殑 澶 戠 殑 澶 戠 殑 澶 戠 殑 澶 戠 殑 澶 戠 殑 澶 戠 殑 澶 戠 殑 澶 戠 殑 澶 镄 勭 墶 铔 庡 嚭 鏉
ョ 帺 銆 镄 勭 墶 铔 庡 嚭 鏉 ョ 帺 銆 镄 勭 墶 铔 庡 嚭 鏉 ョ 帺 銆 镄 勭 墶 铔 庡 嚭 鏉 ョ 帺 銆 镄 勭 墶 铔 庡 嚭 鏉
ョ 帺 銆 镄 勭 墶 铔 庡 嚭 鏉 ョ 帺 銆 镄 勭 墶 铔 庡 嚭 鏉 ョ 帺 銆" WHEN ENGLISH IS THE TARGET
LANGUAGE.
> Tika unable to parse html files with non UTF-8 charset
> ------------------------------------------------------
>
> Key: TIKA-3048
> URL: https://issues.apache.org/jira/browse/TIKA-3048
> Project: Tika
> Issue Type: Bug
> Components: parser
> Affects Versions: 1.9
> Reporter: Akash
> Priority: Major
> Attachments: ChineseFile.html
>
>
> Tika is returning junk characters when parsing chinese characters present
> inside html file. Html file have charset mentioned as GB2312 explicitly.
> <head><meta http-equiv=Content-Type content="text/html; charset=gb2312"><meta
> name=Generator content="Microsoft Word 15 (filtered medium)">
>
> If we remove this charset from the html meta tag, then parsing works fine.
>
> Similar issue is observed for Arabic, Russain, Korean, Japanese, Hungarian
> and Spanish languages.
> Charset mentioned for each languages -
> Hungarian - iso-8859-1
> Chinese - gb2312
> Spanish - iso-8859-1
> Russian - koi8-r
> Korean - ks_c_5601-1987
> Japanese - iso-2022-jp
> Arabic - windows-1256
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
