[
https://issues.apache.org/jira/browse/FLUME-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663508#comment-15663508
]
PIPELINE edited comment on FLUME-3022 at 11/15/16 9:23 AM:
-----------------------------------------------------------
inputCharset = UTF-8
deserializer.outputCharset = UTF-8
-----------------------------------------------------------------------
may be i am wrong, it persist as hex.
-----------------------------------------------------------------------
log file:::
{"cTime":1478709134982,"content":"😄哈喽","first":true,"lgId":"867196022102501_1477932364878","lt":"apptalk","lv":"v1","os":"android","pId":0,"recieverId":296134,"sTime":1478709133893,"uId":6384385,"uType":0}
channel file:::
{"cTime":1478709134982,"content":"ð<9f><98><84>å<93><88>å<96>½","first":true,"lgId":"867196022102501_1477932364878","lt":"apptalk","lv":"v1","os":"android","pId":0,"recieverId":296134,"sTime":1478709133893,"uId":6384385,"uType":0}
hdfs file:::
{"cTime":1478709134982,"content":"ð<9f><98><84>å<93><88>å<96>½","first":true,"lgId":"867196022102501_1477932364878","lt":"apptalk","lv":"v1","os":"android","pId":0,"recieverId":296134,"sTime":1478709133893,"uId":6384385,"uType":0}
was (Author: pipe):
inputCharset = UTF-8
deserializer.outputCharset = UTF-8
may be i am wrong, it persist as hex.
log file:::
{"cTime":1478709134982,"content":"😄哈喽","first":true,"lgId":"867196022102501_1477932364878","lt":"apptalk","lv":"v1","os":"android","pId":0,"recieverId":296134,"sTime":1478709133893,"uId":6384385,"uType":0}
channel file:::
{"cTime":1478709134982,"content":"ð<9f><98><84>å<93><88>å<96>½","first":true,"lgId":"867196022102501_1477932364878","lt":"apptalk","lv":"v1","os":"android","pId":0,"recieverId":296134,"sTime":1478709133893,"uId":6384385,"uType":0}
hdfs file:::
{"cTime":1478709134982,"content":"ð<9f><98><84>å<93><88>å<96>½","first":true,"lgId":"867196022102501_1477932364878","lt":"apptalk","lv":"v1","os":"android","pId":0,"recieverId":296134,"sTime":1478709133893,"uId":6384385,"uType":0}
> SpoolDirSource can not work well on Chinese log, and translating more slow
> than 1.5.2 or 1.6.0 with the same conf.
> ------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-3022
> URL: https://issues.apache.org/jira/browse/FLUME-3022
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.7.0
> Environment: jdk1.7
> Reporter: PIPELINE
>
> Cause 1.5.2 or 1.6.0 has spoolDirSource bug with emoji, i tried 1.7.0.
> well ,it resolved emoji bug, but may create new bugs. such as:
> 1. can not work well on Chinese log files.
> 2.translating more slow than 1.5.2 or 1.6.0 with the same conf.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
