[ https://issues.apache.org/jira/browse/HIVE-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021535#comment-16021535 ]
Laurel Hale commented on HIVE-16735: ------------------------------------ [~leftylev] Thanks, but I think I must be missing something? I checked both places and neither mentions that the value should be set in MBs. I cleared my browser cache. This is what I see for the first link above: ---------------------------------------------- https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.mapred.local.mem hive.mapred.local.mem Default Value: 0 Added In: Hive 0.3.0 For local mode, memory of the mappers/reducers. --------------------------------------------- And here is what I see for the second link listed above: --------------------------------------------- https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Hive,Map-ReduceandLocal-Mode Note that there may be differences in the runtime environment of Hadoop server nodes and the machine running the Hive client (because of different jvm versions or different software libraries). This can cause unexpected behavior/errors while running in local mode. Also note that local mode execution is done in a separate, child jvm (of the Hive client). If the user so wishes, the maximum amount of memory for this child jvm can be controlled via the option hive.mapred.local.mem. By default, it's set to zero, in which case Hive lets Hadoop determine the default memory limits of the child jvm. --------------------------------------------- Thanks for being patient. Best, Laurel > Please add that hive.mapred.local.mem must be set in MBs in the documentation > ----------------------------------------------------------------------------- > > Key: HIVE-16735 > URL: https://issues.apache.org/jira/browse/HIVE-16735 > Project: Hive > Issue Type: Task > Components: Documentation > Reporter: Laurel Hale > Assignee: Lefty Leverenz > Priority: Minor > > Hi Lefty, > One of my developers noticed this and we do not document the local mode. > Thought I'd let you know. > Here's a good place where you could put this: > https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution > There's already a small subsection where hive.mapred.local.mem is documented, > but there isn't any information about how to set the value. My developer > discovered that when you set it in bytes, it throws the following error: > 2017-03-12 09:24:31,835 ERROR [HiveServer2-Background-Pool: Thread-22163()]: > mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(341)) - Exception: > java.lang.NumberFormatException: For input string: "4294967296" > java.lang.NumberFormatException: For input string: "4294967296" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Integer.parseInt(Integer.java:495) > at java.lang.Integer.parseInt(Integer.java:527) > at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1259) > at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:2415) > at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:2424) > at > org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeInChildVM(MapredLocalTask.java:228) > Instead, this value should be set in MBs for success. > Thanks, > Laurel -- This message was sent by Atlassian JIRA (v6.3.15#6346)