[
https://issues.apache.org/jira/browse/MAPREDUCE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797001#action_12797001
]
Vinod K V commented on MAPREDUCE-1218:
--------------------------------------
Looked at that latest patch. Coming out good. Some comments, most of them are
finishing touches.
- LinuxResourceCalculatorPlugin.java
-- The method {{#getCpuUsage()}} in itself doesn't update the cpu-usage.
So, if one makes two calls to this method separated by a time interval, the
result won't reflect the updated cpu-usage unless calls to
{{#getCumulativeCpuTime()}} are explicitly made in between. This should be
fixed by calling {{#readProcStatFile()}} in this method also. The main method
should be modified to call this method twice and the tests should also verify
this.
-- You've changed {{CPU_TIME_FORMAT}} from _"^cpu[0-9]*[ \t]*([0-9]*)[
\t]*([0-9]*)[ \t]*([0-9]*)[ \t].*"_ to _"^cpu[ \t]*([0-9]*)[ \t]*([0-9]*)[
\t]*([0-9]*)[ \t].*"_. I guess the earlier is correct with cpu names not having
any space/tab?
-- Why even have MINIMUM_UPDATE_INTERVAL in updateCpuUsage()? If this is
mainly used for making sure sampleTime is not equal to lastSampleTime, then we
can do so directly and remove MINIMUM_UPDATE_INTERVAL altogether.
- TaskTracker.java
-- To handle the deprecation of
{{mapreduce.tasktracker.memorycalculatorplugin}} in TaskTracker, for memory
calculations we should first try to use the class denoted by this configuration
if present, otherwise only we should fall back to the new resource-calculator.
To facilitate this we will also retain a deprecated
{{TTConfig.TT_MEMORY_CALCULATOR_PLUGIN}} constant.
-- Nit: +661 "LOG.info(" Using MemoryCalculatorPlugin : " +
resourceCalculatorPlugin);" should instead be "LOG.info(" Using
ResourceCalculatorPlugin : " + resourceCalculatorPlugin);"
- Please annotate Dummy{Resource|Memory}CalculatorPlugin classes as
@InterfaceAudience.Private because they both are only test specific.
- We should document {{mapreduce.tasktracker.resourcecalculatorplugin}} in
mapred-default.xml and remove the documentation for
{{mapreduce.tasktracker.memorycalculatorplugin}} from the same.
- Please convert TestLinuxResourceCalculatorPlugin and TestTTResourceReporting
into Junit 4 testcases.
> Collecting cpu and memory usage for TaskTrackers
> ------------------------------------------------
>
> Key: MAPREDUCE-1218
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1218
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Environment: linux
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1218-rename.sh, MAPREDUCE-1218-v2.patch,
> MAPREDUCE-1218-v3.patch, MAPREDUCE-1218-v4.patch, MAPREDUCE-1218.patch
>
>
> The information can be used for resource aware scheduling.
> Note that this is related to MAPREDUCE-220. There the per task resource
> information is collected.
> This one collects the per machine information.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
