[ https://issues.apache.org/jira/browse/AVRO-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570398#comment-13570398 ]
Joseph Adler commented on AVRO-1241: ------------------------------------ One other note: this change speeds up data loading by 3-4x in our environment. > improve trevni performance on string deserialization > ---------------------------------------------------- > > Key: AVRO-1241 > URL: https://issues.apache.org/jira/browse/AVRO-1241 > Project: Avro > Issue Type: Improvement > Components: java > Affects Versions: 1.7.3 > Reporter: Joseph Adler > Assignee: Joseph Adler > Fix For: 1.7.4 > > Attachments: AVRO-1241 > > > I have been trying to implement a storage function for Apache Pig that writes > data in Trevni format. I found that the storage function was very slow when > reading whole records. > I did some profiling (with Yourkit) and found that most of the CPU time was > being spent in org.apache.trevni.InputBuffer$readString() (specifically in > the String() method). I changed to java.nio.charset.CharsetDecoder.decode for > deserialization and saw a big improvement. Changes are included in the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira