[ https://issues.apache.org/jira/browse/HIVE-20868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16679235#comment-16679235 ]
Gopal V commented on HIVE-20868: -------------------------------- LGTM - +1 on the .2 patch. > SMB Join fails intermittently when TezDummyOperator has child op in > getFinalOp in MapRecordProcessor > ---------------------------------------------------------------------------------------------------- > > Key: HIVE-20868 > URL: https://issues.apache.org/jira/browse/HIVE-20868 > Project: Hive > Issue Type: Bug > Reporter: Deepak Jaiswal > Assignee: Deepak Jaiswal > Priority: Major > Attachments: HIVE-20868.1.patch, HIVE-20868.2.patch > > > In MapRecordProcessor::getFinalOp() due to external cause(not known), the > TezDummyStoreOperator may have MergeJoin Op as child intermittently. Due to > this, the fetchDone remains set to true for the DummyOp which was set by > previous task. Ideally, fetchDone should be reset for each task. This > eventually leads to the join op skip rows from that dummy op resulting in > wrong results. > Good init order > {code} > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = TS[3] (core) > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = FIL[24] > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = SEL[5] > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = DUMMY_STORE[45] > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: Iterating > children of dummy op DUMMY_STORE[45] > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp returns DUMMY_STORE[45] > 2018-11-01 21:42:33,677 [INFO] [TezChild] |tez.MapRecordProcessor|: > InitProcessor : setting fetchDone to false > {code} > Bad init order > {code} > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = TS[3] (core) > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = FIL[24] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = SEL[5] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = DUMMY_STORE[45] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > Iterating children of dummy op DUMMY_STORE[45] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: Child of > Dummy Op MERGEJOIN[44] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = MERGEJOIN[44] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = SEL[13] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp child Ops = RS[14] > 2018-11-01 21:42:33,304 [INFO] [TezChild] |tez.MapRecordProcessor|: > getFinalOp returns RS[14] > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)