[
https://issues.apache.org/jira/browse/HIVE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712016#comment-13712016
]
Phabricator commented on HIVE-4502:
-----------------------------------
navis has commented on the revision "HIVE-4502 [jira] NPE - subquery smb joins
fails".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java:619 Ah sorry. This was
throwing NPE when debugging.
ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:72 Ok, sure.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRRedSink1.java:70 I think
it's not possible to have 2 or more children for RS, but might be possible for
Tez. I'll do that.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:261 ok
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:281 No.
Will be changed to package local.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:310
MapRedTask in fact. If any of MapRedTask need BucketizedInputFormat, use
BucketizedInputFormat. It's fix for
https://issues.apache.org/jira/browse/HIVE-4502?focusedCommentId=13679262&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13679262
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:851 ok.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:942 done
ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out:165 If Task-A is
merged into Task-B, task-A will not be visible anywhere. I've changed not to
walk further when all parents are not walked, which changed stage-ids in result.
TS1 TS2
SMB
RS
GBY
RS
GBY
FS
before :
TS1(stage-1)-SMB-RS-GBY[split]RS(stage-2)-GBY-FS
TS2(stage-3)-SMB[merge into stage-1]-RS-GBY-RS-GBY-RS-FS
after :
TS1(stage-1)-SMB
TS2(stage-2)-SMB[merged into stage-1]-RS-GBY[split]RS(stage-3)-GBY-FS
I'll make a issue for rearranging Stage-IDs. (and order of explain output,
too)
ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinFactory.java:228 It's
just a refactoring. setupBucketMapJoinInfo() was called both in
initMapJoinPlan() and joinMapJoinPlan().
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:318 ok
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:949 It's
called in join-checked block.
if (reducer.getClass() == JoinOperator.class) {
....
cplan.setNeedsTagging(true);
}
REVISION DETAIL
https://reviews.facebook.net/D10695
BRANCH
HIVE-4502
ARCANIST PROJECT
hive
To: JIRA, ashutoshc, navis
Cc: brock, vikram
> NPE - subquery smb joins fails
> ------------------------------
>
> Key: HIVE-4502
> URL: https://issues.apache.org/jira/browse/HIVE-4502
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.11.0
> Reporter: Vikram Dixit K
> Assignee: Navis
> Attachments: HIVE-4502.D10695.1.patch, HIVE-4502.D10695.2.patch,
> HIVE-4502.D10695.3.patch, smb_mapjoin_25.q, smb_mapjoin_25.q
>
>
> Found this issue while running some SMB joins. Attaching test case that
> causes this error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
