[
https://issues.apache.org/jira/browse/MAPREDUCE-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hemanth Yamijala updated MAPREDUCE-1834:
----------------------------------------
Attachment: MAPREDUCE-1834.patch
I spent some time looking at this problem. The timeout was occurring most
likely because of the usage of Process.waitFor(), which has been problematic in
cases where the launched process writes to the error / output streams. Note
that diff will spew out differences when they do exist, and so
Process.waitFor() hangs. The standard pattern in Hadoop has been to replace
this with ShellCommandExecutor.execute(). The attached patch has this
modification. Unfortunately, the test fails now everytime.
As far as I can understand, the test is launching two identical runs of
mapreduce jobs and is trying to diff the output history directories to make
sure they have identical content. But history operations can happen
asynchronously. When I ran the test, I could see all sorts of differences
between the directories being diff'ed. Contents and file sizes of history files
were differing, as well as which files were moved to the DONE folder were
differing. I think the operation of moving to DONE folder is certainly
asynchronous. I don't know if some buffering of history data is happening that
is causing the history files to have differing contents as well.
Given all this I doubt if this test case will ever pass. I don't know enough
about the intent though to fix it. Anyone can help ?
> TestSimulatorDeterministicReplay timesout on trunk
> --------------------------------------------------
>
> Key: MAPREDUCE-1834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/mumak
> Affects Versions: 0.21.0
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1834.patch,
> TestSimulatorDeterministicReplay.log
>
>
> TestSimulatorDeterministicReplay timesout on trunk.
> See hudson patch build
> http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/216/testReport/org.apache.hadoop.mapred/TestSimulatorDeterministicReplay/testMain/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
