[
https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539247#comment-13539247
]
Namit Jain commented on HIVE-3784:
----------------------------------
[~vinodkv], I agree that with change, it will be 2 map-only jobs instead of 1
map job.
I haven't tested it myself, but seems likely.
But, the current code has way too much complexity to handle this special case.
Ideally, this change
should be done as part of converting join tasks into conditional join tasks.
That layer should be smarter
to see that there is no need of a conditional task, and a map-only task is
possible. Also, another layer needs
to be written to merge consecutive map-only tasks.
Although, for this special case, we are taking a hit, I still believe this is
the right long term way to go.
> de-emphasize mapjoin hint
> -------------------------
>
> Key: HIVE-3784
> URL: https://issues.apache.org/jira/browse/HIVE-3784
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Namit Jain
> Attachments: hive.3784.1.patch, hive.3784.2.patch, hive.3784.3.patch,
> hive.3784.4.patch, hive.3784.5.patch
>
>
> hive.auto.convert.join has been around for a long time, and is pretty stable.
> When mapjoin hint was created, the above parameter did not exist.
> The only reason for the user to specify a mapjoin currently is if they want
> it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin.
> Eventually, that should also go away, but that may take some time to
> stabilize.
> There are many rules in SemanticAnalyzer to handle the following trees:
> ReduceSink -> MapJoin
> Union -> MapJoin
> MapJoin -> MapJoin
> This should not be supported anymore. In any of the above scenarios, the
> user can get the mapjoin behavior by setting hive.auto.convert.join to true
> and not specifying the hint. This will simplify the code a lot.
> What does everyone think ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
