[jira] Commented: (MAPREDUCE-834) When TaskTracker config use old memory management values its memory monitoring is diabled.
[
https://issues.apache.org/jira/browse/MAPREDUCE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745384#action_12745384
]
Sreekanth Ramakrishnan commented on MAPREDUCE-834:
--
The patch does not change any part on {{JobTracker}} or streaming api's.
The issue with {{TestRecoveryManager}} timing out is reported on MAPREDUCE-880
and the streaming test case failures are also a known reported issue on hudson.
> When TaskTracker config use old memory management values its memory
> monitoring is diabled.
> --
>
> Key: MAPREDUCE-834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Karam Singh
>Assignee: Sreekanth Ramakrishnan
> Attachments: mapreduce-834-1.patch, mapreduce-834-2.patch,
> mapreduce-834-3.patch, mapreduce-834-4.patch
>
>
> TaskTracker memory config values -:
> mapred.tasktracker.vmem.reserved=8589934592
> mapred.task.default.maxvmem=2147483648
> mapred.task.limit.maxvmem=4294967296
> mapred.tasktracker.pmem.reserved=2147483648
> TaskTracker start as -:
>2009-08-05 12:39:03,308 WARN
> org.apache.hadoop.mapred.TaskTracker: The variable
> mapred.tasktracker.vmem.reserved is no longer used
> 2009-08-05 12:39:03,308 WARN
> org.apache.hadoop.mapred.TaskTracker: The variable
> mapred.tasktracker.pmem.reserved is no longer used
> 2009-08-05 12:39:03,308 WARN
> org.apache.hadoop.mapred.TaskTracker: The variable
> mapred.task.default.maxvmem is no longer used
> 2009-08-05 12:39:03,308 WARN
> org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.limit.maxvmem
> is no longer used
> 2009-08-05 12:39:03,308 INFO
> org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for
> all reduce tasks on
> 2009-08-05 12:39:03,309 INFO
> org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin :
> org.apache.hadoop.util.linuxmemorycalculatorplu...@19be4777
> 2009-08-05 12:39:03,311 WARN
> org.apache.hadoop.mapred.TaskTracker: TaskTracker's
> totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-834) When TaskTracker config use old memory management values its memory monitoring is diabled.
[ https://issues.apache.org/jira/browse/MAPREDUCE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745375#action_12745375 ] Hadoop QA commented on MAPREDUCE-834: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417108/mapreduce-834-4.patch against trunk revision 806065. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/console This message is automatically generated. > When TaskTracker config use old memory management values its memory > monitoring is diabled. > -- > > Key: MAPREDUCE-834 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-834 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Karam Singh >Assignee: Sreekanth Ramakrishnan > Attachments: mapreduce-834-1.patch, mapreduce-834-2.patch, > mapreduce-834-3.patch, mapreduce-834-4.patch > > > TaskTracker memory config values -: > mapred.tasktracker.vmem.reserved=8589934592 > mapred.task.default.maxvmem=2147483648 > mapred.task.limit.maxvmem=4294967296 > mapred.tasktracker.pmem.reserved=2147483648 > TaskTracker start as -: >2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable > mapred.tasktracker.vmem.reserved is no longer used > 2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable > mapred.tasktracker.pmem.reserved is no longer used > 2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable > mapred.task.default.maxvmem is no longer used > 2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.limit.maxvmem > is no longer used > 2009-08-05 12:39:03,308 INFO > org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for > all reduce tasks on > 2009-08-05 12:39:03,309 INFO > org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin : > org.apache.hadoop.util.linuxmemorycalculatorplu...@19be4777 > 2009-08-05 12:39:03,311 WARN > org.apache.hadoop.mapred.TaskTracker: TaskTracker's > totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-834) When TaskTracker config use old memory management values its memory monitoring is diabled.
[ https://issues.apache.org/jira/browse/MAPREDUCE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744915#action_12744915 ] Hemanth Yamijala commented on MAPREDUCE-834: Few comments: - Memory allotted for slot based on old configuration should not be based on getMaxVirtualMemoryForTask(), but on JobConf.MAPRED_TASK_DEFAULT_MAXVMEM_PROPERTY. Also note that this value will be in bytes, while the system maintains everything else in MB. So, it should be converted to MB. - testTaskMemoryMonitoringWithDeprecatedConfiguration should also set the TT configuration for JobConf.MAPRED_TASK_DEFAULT_MAXVMEM_PROPERTY in bytes instead of MAPRED_TASK_MAXVMEM_PROPERTY. > When TaskTracker config use old memory management values its memory > monitoring is diabled. > -- > > Key: MAPREDUCE-834 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-834 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Karam Singh > Attachments: mapreduce-834-1.patch > > > TaskTracker memory config values -: > mapred.tasktracker.vmem.reserved=8589934592 > mapred.task.default.maxvmem=2147483648 > mapred.task.limit.maxvmem=4294967296 > mapred.tasktracker.pmem.reserved=2147483648 > TaskTracker start as -: >2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable > mapred.tasktracker.vmem.reserved is no longer used > 2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable > mapred.tasktracker.pmem.reserved is no longer used > 2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable > mapred.task.default.maxvmem is no longer used > 2009-08-05 12:39:03,308 WARN > org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.limit.maxvmem > is no longer used > 2009-08-05 12:39:03,308 INFO > org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for > all reduce tasks on > 2009-08-05 12:39:03,309 INFO > org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin : > org.apache.hadoop.util.linuxmemorycalculatorplu...@19be4777 > 2009-08-05 12:39:03,311 WARN > org.apache.hadoop.mapred.TaskTracker: TaskTracker's > totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
