[
https://issues.apache.org/jira/browse/HIVE-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13775940#comment-13775940
]
Aaron Mitchell commented on HIVE-3816:
--------------------------------------
Hmmm... this is a lot of boolean logic. Lets simplify your solution by using De
Morgan's law.
if(!runningViaChild) ->
!(!ShimLoader.getHadoopShims().isLocalMode(conf) ||
conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD)
ShimLoader.getHadoopShims().isLocalMode(conf) &&
!conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD)
Your solution would always run jobs via a child VM in distributed mode. Is this
intentional? I would think running a distributed job via a JobClient in
ExecDriver would be ideal instead. If you want to simply debug a child jvm then
set HIVE_DEBUG_RECURSIVE according to HIVE-2500.
I would like to run a Job in the same JVM due to unit/integration test concerns
(I'm using an in-memory mock Accumulo). Could anyone shed some light regarding
why runningViaChild takes local mode into account and isn't simply to
SUBMITVIACHILD i.e.
runningViaChild = conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD); (git
history took me back to HADOOP-3601)
> Typo in the MapRedTask? Which may affect the debugging purpose with IDE.
> ------------------------------------------------------------------------
>
> Key: HIVE-3816
> URL: https://issues.apache.org/jira/browse/HIVE-3816
> Project: Hive
> Issue Type: Bug
> Components: CLI
> Affects Versions: 0.9.1
> Reporter: Cheng Hao
> Priority: Trivial
> Fix For: 0.9.1
>
>
> There may be a typo in the java file
> {code:title=org.apache.hadoop.hive.ql.exec.MapRedTask|borderStyle=solid}
> // Origin: here may have a typo?
> // runningViaChild = ShimLoader.getHadoopShims().isLocalMode(conf) ||
> // conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
> // New: should be the "!"
> runningViaChild = !ShimLoader.getHadoopShims().isLocalMode(conf) ||
> conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
> if(!runningViaChild) {
> // we are not running this mapred task via child jvm
> // so directly invoke ExecDriver
> return super.execute(driverContext);
> }
> {code}
> Per my understanding, there would be 3 cases:
> 1) Run the job in a NEW Child process(maybe pseudo hadoop mode) in the same
> node
> mapred.job.tracker=local
> hive.exec.submitviachild=true
> 2) Run the job in the same process in the same node (very helpful in
> debugging)
> mapred.job.tracker=local
> hive.exec.submitviachild=false
> 3) Run the job in a real Hadoop Cluster specified
> mapred.job.tracker!=local
> hive.exec.submitviachild=true/false
> The typo may cause the second case NEVER happens, which we need in developing
> (debugging);
> Sorry, not sure if I understand that correctly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
