[
https://issues.apache.org/jira/browse/HIVE-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695243#action_12695243
]
Zheng Shao commented on HIVE-266:
---------------------------------
A simple "INSERT SELECT *" shows with the pending changes (and HIVE-375,
HIVE-356) we saved 23% or running time:
Both input and output tables are gzip-block-compressed sequence files. The
saving could be bigger for uncompressed tables.
{code}
task new(sec) old(sec) ratio
0 137 183 0.74863388
1 135 173 0.780346821
2 131 172 0.761627907
3 127 165 0.76969697
4 136 180 0.755555556
5 142 170 0.835294118
6 128 170 0.752941176
7 133 177 0.751412429
8 143 173 0.826589595
9 126 178 0.707865169
total 1338 1741 0.768523837
{code}
> Improve SerDe performance by using Text instead of String
> ---------------------------------------------------------
>
> Key: HIVE-266
> URL: https://issues.apache.org/jira/browse/HIVE-266
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.2.0
> Reporter: Zheng Shao
> Priority: Critical
> Attachments: HIVE-266.1.patch
>
>
> A recent performance study showed that 2 places in Hive code has exhibited
> large cpu usage percentage:
> 1. String.getBytes() (UTF-8 encoding)
> 2. String.split()
> We should replace String with Text object to:
> 1. Avoid UTF-8 decoding and encoding
> 2. Reuse the Text object and avoid creating new objects for each column in
> each row like in String.split()
> This is expected to give a big (20%+) performance improvement to Hive.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
