[
https://issues.apache.org/jira/browse/HCATALOG-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414269#comment-13414269
]
Rajesh Balamohan commented on HCATALOG-448:
-------------------------------------------
Thanks Mithun and Rohini,
My initial analysis reveals the following
1. Following codepath could be problematic HCatBaseStorer --> putNext() -->
FileRecordWriterContainer.write().
2. In FileRecordWriterContainer.write(), localJobInfo, localContext are created
very often (almost for every record written using Hcat storer).
3. This internally creates new JobConf and much of the time is spent in there.
This could be a redundant operation for every record.
Removing this unwanted jobconf creation can give better performance. I will
check it out and post the findings.
~Rajesh.B
> HCatStorer performance is 4x slower in HCat 0.4 than HCat 0.2
> -------------------------------------------------------------
>
> Key: HCATALOG-448
> URL: https://issues.apache.org/jira/browse/HCATALOG-448
> Project: HCatalog
> Issue Type: Bug
> Affects Versions: 0.4.1
> Reporter: Rohini Palaniswamy
> Assignee: Mithun Radhakrishnan
> Priority: Critical
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
