[ https://issues.apache.org/jira/browse/HIVE-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629247#comment-13629247 ]
Kevin Wilfong commented on HIVE-4318: ------------------------------------- It looks like the performance cost of those methods was recognized shortly after they were added https://issues.apache.org/jira/browse/HIVE-768 It's possible the config hive.task.progress isn't as effective at countering it as expected. Can you make sure it is set to false in your run. > OperatorHooks hit performance even when not used > ------------------------------------------------ > > Key: HIVE-4318 > URL: https://issues.apache.org/jira/browse/HIVE-4318 > Project: Hive > Issue Type: Bug > Components: Query Processor > Environment: Ubuntu LXC (64 bit) > Reporter: Gopal V > Assignee: Gunther Hagleitner > Attachments: HIVE-4318.1.patch > > > Operator Hooks inserted into Operator.java cause a performance hit even when > it is not being used. > For a count(1) query tested with & without the operator hook calls. > {code:title=with} > 2013-04-09 07:33:58,920 Stage-1 map = 100%, reduce = 100%, Cumulative CPU > 84.07 sec > Total MapReduce CPU Time Spent: 1 minutes 24 seconds 70 msec > OK > 28800991 > Time taken: 40.407 seconds, Fetched: 1 row(s) > {code} > {code:title=without} > 2013-04-09 07:33:02,355 Stage-1 map = 100%, reduce = 100%, Cumulative CPU > 68.48 sec > ... > Total MapReduce CPU Time Spent: 1 minutes 8 seconds 480 msec > OK > 28800991 > Time taken: 35.907 seconds, Fetched: 1 row(s) > {code} > The effect is multiplied by the number of operators in the pipeline that has > to forward the row - the more operators there are the, the slower the query. > The modification made to test this was > {code:title=Operator.java} > --- ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java > +++ ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java > @@ -526,16 +526,16 @@ public void process(Object row, int tag) throws > HiveException { > return; > } > OperatorHookContext opHookContext = new OperatorHookContext(this, row, > tag); > - preProcessCounter(); > - enterOperatorHooks(opHookContext); > + //preProcessCounter(); > + //enterOperatorHooks(opHookContext); > processOp(row, tag); > - exitOperatorHooks(opHookContext); > - postProcessCounter(); > + //exitOperatorHooks(opHookContext); > + //postProcessCounter(); > } > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira