[jira] [Commented] (HIVE-5054) Remove unused property submitviachild

[Aaron Mitchell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13776396#comment-13776396
 ] 

Aaron Mitchell commented on HIVE-5054:
--

Regarding [~appodictic]'s comment
{quote}Could it be useful in hive server 2 type scenarios?
hive.exec.submitviachild Determines whether the map/reduce jobs should be 
submitted through a separate jvm in the non local mode. false - By default jobs 
are submitted through the same jvm as the compiler{quote}

I think submitviachild would be useful to control whether jobs are are 
submitted through a separate jvm in both local and distributed mode. Please 
take a look at HIVE-3816.

 Remove unused property submitviachild
 -

 Key: HIVE-5054
 URL: https://issues.apache.org/jira/browse/HIVE-5054
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
 Attachments: HIVE-5054.patch, HIVE-5054.patch


 This property only exist in HiveConf and is always set to false. Lets get rid 
 of dead code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3816) Typo in the MapRedTask? Which may affect the debugging purpose with IDE.

[Aaron Mitchell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13775940#comment-13775940
 ] 

Aaron Mitchell commented on HIVE-3816:
--

Hmmm... this is a lot of boolean logic. Lets simplify your solution by using De 
Morgan's law. 
if(!runningViaChild) -
!(!ShimLoader.getHadoopShims().isLocalMode(conf) || 
conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD)
ShimLoader.getHadoopShims().isLocalMode(conf)  
!conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD)

Your solution would always run jobs via a child VM in distributed mode. Is this 
intentional? I would think running a distributed job via a JobClient in 
ExecDriver would be ideal instead. If you want to simply debug a child jvm then 
set HIVE_DEBUG_RECURSIVE according to HIVE-2500. 

I would like to run a Job in the same JVM due to unit/integration test concerns 
(I'm using an in-memory mock Accumulo). Could anyone shed some light regarding 
why runningViaChild takes local mode into account and isn't simply to 
SUBMITVIACHILD  i.e.
runningViaChild = conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);  (git 
history took me back to HADOOP-3601)

 Typo in the MapRedTask? Which may affect the debugging purpose with IDE.
 

 Key: HIVE-3816
 URL: https://issues.apache.org/jira/browse/HIVE-3816
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 0.9.1
Reporter: Cheng Hao
Priority: Trivial
 Fix For: 0.9.1


 There may be a typo in the java file
 {code:title=org.apache.hadoop.hive.ql.exec.MapRedTask|borderStyle=solid}
   // Origin: here may have a typo?
// runningViaChild = ShimLoader.getHadoopShims().isLocalMode(conf) ||
//   conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
   // New: should be the !
   runningViaChild = !ShimLoader.getHadoopShims().isLocalMode(conf) ||
 conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
   if(!runningViaChild) {
 // we are not running this mapred task via child jvm
 // so directly invoke ExecDriver
 return super.execute(driverContext);
   }
 {code} 
 Per my understanding, there would be 3 cases:
 1) Run the job in a NEW Child process(maybe pseudo hadoop mode) in the same 
 node
 mapred.job.tracker=local
 hive.exec.submitviachild=true
 2) Run the job in the same process in the same node (very helpful in 
 debugging)
 mapred.job.tracker=local
 hive.exec.submitviachild=false
 3) Run the job in a real Hadoop Cluster specified
 mapred.job.tracker!=local
 hive.exec.submitviachild=true/false
 The typo may cause the second case NEVER happens, which we need in developing 
 (debugging);
 Sorry, not sure if I understand that correctly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira