[
https://issues.apache.org/jira/browse/MAPREDUCE-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862623#action_12862623
]
Iyappan Srinivasan commented on MAPREDUCE-1741:
-----------------------------------------------
What is the reason we create manually 100 files, please provide the
documentation, looking at the code it is not evident to me.
100 files are created for the test scenario in which the movemnt of history
files to done directory should still happen, even iftehre are 100 dummy files
in that directory.
In the multiple job scenario are the jobs launches simultaneously or
sequentially one after the other ?
What is the reason we create manually 100 files, please provide the
documentation, looking at the code it is not evident to me. 2. In the multiple
job scenario are the jobs launches simultaneously or sequentially one after the
other ?
- It is sequential, with a difference of few seconds. The test scenario is
when a bunch of testcases finish almost at the same time, still ,the history
files have to be moved to done directory
Test names are bit confusing for me. Can you please change to some appropriate
names?
Here is my suggestion.
testRetiredJobsCompletedLocation() - > testRetiredJobsHistoryLocation()
testRetiredMultipleJobsCompletedLocation() ->
testRetiredMultipleJobsHistoryLocation()
- Done
String hadoopLogDirString = jconf.get("hadoop.log.dir");
jobHistoryDonePathString =
jconf.get("mapred.job.tracker.history.completed.location");
String jobHistoryPathString = jconf.get("hadoop.job.history.location");
Can please check the above three statements with assert condition. Otherwise if
any of these attributes are not available in the configuration
then test fails w/ NPE w/out any message. So please check the above three
values before proceeding to further steps
- hadoop.log.dir is not required . Removed it.
- Added a Assert statement for mapred.job.tracker.history.completed.location
- hadoop.job.history.location is a optional parm , which may or may not be
present. And it si checked for null, before usage. So, no need for assert.
How do you confirm whether its moved to retired job state or not after
completion of job ?
I don't see any specific condition in the code for retire job. Is there any
default interval available in the configuration for moving jobs to retire?
- The retired jbo movement is done as a inline method and will happen soon
after job is completed. So, the code waits for a maximum of 30 seconds
to make absolutely sure that it is moved to retired.
Put the assert statement like below after completion of job.So that user will
have clear information whenever test fails.
Assert.assertTrue("Job history files are not available in history location
after job retired.", jobFileFound);
- It is checking in the done directory. Job history file should be found here.
If it si not found, then it asserts. I have elaborated the message as "Job
History
File is not found in the done directory"
Can you please put the annotation @Test after java doc comments instead of top.
[ Show ยป ] Vinay Kumar Thota added a comment - 30/Apr/10 02:06 AM
Test names are bit confusing for me. Can you please change to some appropriate
names?
Here is my suggestion.
testRetiredJobsCompletedLocation() - > testRetiredJobsHistoryLocation()
testRetiredMultipleJobsCompletedLocation() ->
testRetiredMultipleJobsHistoryLocation()
String hadoopLogDirString = jconf.get("hadoop.log.dir");
jobHistoryDonePathString =
jconf.get("mapred.job.tracker.history.completed.location");
String jobHistoryPathString = jconf.get("hadoop.job.history.location");
Can please check the above three statements with assert condition. Otherwise if
any of these attributes are not available in the configuration
then test fails w/ NPE w/out any message. So please check the above three
values before proceeding to further steps
Assert.assertTrue("jobFileFound is false", jobFileFound);
You are checking the above condition while running the job right, in this case
the above statement would be wrong because
the jobFileFound should be false and it should be like below.
Assert.assertFalse("Job history files available in history location for running
job itself.",jobFileFound)
How do you confirm whether its moved to retired job state or not after
completion of job ? I don't see any specific condition in the code for retire
job. Is there any default interval available in the configuration for moving
jobs to retire?
Put the assert statement like below after completion of job.So that user will
have clear information whenever test fails.
Assert.assertTrue("Job history files are not available in history location
after job retired.", jobFileFound);
Even I have same question that Balaji has asked regarding creating 1000 files
in done location. Can you please elaborate more on this in your comments. Can
you please put the annotation @Test after java doc comments instead of top.
- Done
> Automate the test scenario of job related files are moved from history
> directory to done directory
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1741
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1741
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: test
> Affects Versions: 0.22.0
> Reporter: Iyappan Srinivasan
> Fix For: 0.22.0
>
> Attachments: TestJobHistoryLocation.patch
>
>
> Job related files are moved from history directory to done directory, when
> 1) Job succeeds
> 2) Job is killed
> 3) When 100 files are put in the done directory
> 4) When multiple jobs are completed at the same time, some successful, some
> failed.
> Also, two files, conf.xml and job files should be present in the done
> directory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
