[
https://issues.apache.org/jira/browse/MAPREDUCE-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iyappan Srinivasan updated MAPREDUCE-1871:
------------------------------------------
Attachment: 1871-ydist-security-patch.txt
Thanks for implementing the class container instead of an array. Now with that
in place do you still need to bother about the order of the statistics in
getInfoFromAllClientsForAllTaskType()? Seems like it is not necessary any more
(you have getters in place) as well as the switch statement seems like an
overkill, is it?
- [Iyappan] - No. this info has to be sequential, till the time it is
collected in the statisticsCollectionHandler object. After that, it is
independent of the sequence(getters method can be used).
in the testTaskTrackerInfoTaskFailure the first collection of statistics seems
to be unused before the second one is actually done. Is it really should be
there? [Iyappan] - The first Collection of statistics collects the
TaskTracker statistics before a job is executed. The second Collection of
statistics collects the TaskTracker statistics after the job is executed. This
difference in numbers is verified for correct Task collection.
Also, the second collection creates a bunch of temp. int's which are later used
for assertions. Why don't you replace the temps with queries? Lesser code is
usually better. Same goes for all other tests in
TestTaskTrackerInfoSuccessfulFailedJobs and TestTaskTrackerInfoTTProcess
classes. [Iyappan] - The information needs to be collected as int as it needs
to be added with total tasks that are exectued in a job and then compared with
the final output (which is also an int) in the end. I have used the minimum
code possible.
replace "mapreduce.job.complete.cancel.delegation.tokens" with proper config
key from MR [Iyappan] - Done in trunk. In 20.1.xxx, this is not available.
break the lines like this + setSinceStartTotalTasks(totalInfoValues);break; in
two: it's hard to read otherwise [Iyappan] Done
make this name more comprehensive like getTTClientInstance + public TTClient
getTTClientIns(TaskInfo taskInfo); in the same method, why the loop goes up to
60? {{+ while (counter < 60) { }} [Iyappan] Changed the name. it goes to 60
because it waits till that job assigns a task to a tasktracker and after which
it will get a TTClientInstance of that running task. It waits for one minute
for this. If after 60 seconds also, no task gets assigned for that job, then
null is returned.
seems to be an unneccesary line break
+ static final Log LOG = LogFactory.
+ getLog(TTClient.class);
[Iyappan] - Done
> Create automated test scenario for "Collect information about number of tasks
> succeeded / total per time unit for a tasktracker"
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1871
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1871
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: test
> Reporter: Iyappan Srinivasan
> Assignee: Iyappan Srinivasan
> Attachments: 1871-ydist-security-patch.txt,
> 1871-ydist-security-patch.txt, 1871-ydist-security-patch.txt,
> 1871-ydist-security-patch.txt, 1871-ydist-security-patch.txt,
> 1871-ydist-security-patch.txt, 1871-ydist-security-patch.txt,
> MAPREDUCE-1871.patch, MAPREDUCE-1871.patch, MAPREDUCE-1871.patch
>
>
> Create automated test scenario for "Collect information about number of tasks
> succeeded / total per time unit for a tasktracker"
> 1) Verification of all the above mentioned fields with the specified TTs.
> Total no. of tasks and successful tasks should be equal to the corresponding
> no. of tasks specified in TTs logs
> 2) Fail a task on tasktracker. Node UI should update the status of tasks on
> that TT accordingly.
> 3) Kill a task on tasktracker. Node UI should update the status of tasks on
> that TT accordingly
> 4) Positive Run simultaneous jobs and check if all the fields are populated
> with proper values of tasks. Node UI should have correct valiues for all the
> fields mentioned above.
> 5) Check the fields across one hour window Fields related to hour should be
> updated after every hour
> 6) Check the fields across one day window fields related to hour should be
> updated after every day
> 7) Restart a TT and bring it back. UI should retain the fields values.
> 8) Positive Run a bunch of jobs with 0 maps and 0 reduces simultanously.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
