[ https://issues.apache.org/jira/browse/HIVE-10983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xiaowei wang updated HIVE-10983: -------------------------------- Summary: SerDeUtils bug ,when Text is reused (was: LazySimpleSerDe bug ,when Text is reused ) > SerDeUtils bug ,when Text is reused > ------------------------------------- > > Key: HIVE-10983 > URL: https://issues.apache.org/jira/browse/HIVE-10983 > Project: Hive > Issue Type: Bug > Components: API, CLI > Affects Versions: 0.14.0 > Environment: Hadoop 2.3.0-cdh5.0.0 > Hive 0.14 > Reporter: xiaowei wang > Assignee: xiaowei wang > Priority: Critical > Labels: patch > Fix For: 0.14.1, 1.2.0 > > Attachments: HIVE-10983.1.patch.txt, HIVE-10983.2.patch.txt > > > When i query data from a lzo table , I found in results : the length of the > current row is always largr than the previous row, and sometimes,the current > row contains the contents of the previous row。 For example ,i execute a sql > ,"select * from web_searchhub where logdate=2015061003", the result of sql > see blow.Notice that ,the second row content contains the first row content. > INFO [03:00:05.589] HttpFrontServer::FrontSH > msgRecv:Remote=/10.13.193.68:42098,session=3151,thread=254 2015061003 > INFO [03:00:05.594] <18941e66-9962-44ad-81bc-3519f47ba274> > session=901,thread=223ession=3151,thread=254 2015061003 > The content of origin lzo file content see below ,just 2 rows. > INFO [03:00:05.635] <b88e0473-7530-494c-82d8-e2d2ebd2666c_forweb> > session=3148,thread=285 > INFO [03:00:05.635] HttpFrontServer::FrontSH > msgRecv:Remote=/10.13.193.68:42095,session=3148,thread=285 > I think this error is caused by the Text reuse,and I found the solutions . > Addicational, table create sql is : > CREATE EXTERNAL TABLE `web_searchhub`( > `line` string) > PARTITIONED BY ( > `logdate` string) > ROW FORMAT DELIMITED > FIELDS TERMINATED BY '\\U0000' > WITH SERDEPROPERTIES ( > 'serialization.encoding'='GBK') > STORED AS INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat" > OUTPUTFORMAT > "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"; > LOCATION > 'viewfs://nsX/user/hive/warehouse/raw.db/web/web_searchhub' ; -- This message was sent by Atlassian JIRA (v6.3.4#6332)