[
https://issues.apache.org/jira/browse/MAPREDUCE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039830#comment-13039830
]
Todd Lipcon commented on MAPREDUCE-1347:
----------------------------------------
- for the guava dependency, it should probably be conf="mapred->default" rather
than releaseaudit (otherwise the jars won't end up in the built package)
- the constant for version should be "guava.version" to match the naming
convention
- this test case seems unlikely to actually fail even if the bug persisted --
if you revert the fix and keep the test case, does it fail more than 50% of the
time? Would it be possible to directly instantiate the outputformat with some
mocks without having to go through running a whole job? Then you could hit it
harder since you wouldn't have to touch disk.
- maybe add a comment explaining that the use of the computing map is to make
sure this code is thread-safe?
> Missing synchronization in MultipleOutputFormat
> -----------------------------------------------
>
> Key: MAPREDUCE-1347
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1347
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2, 0.21.0, 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Attachments: MAPREDUCE-1347.r2.diff, MAPREDUCE-1347.r3.diff,
> mapreduce.1347.r1.diff
>
>
> MultipleOutputFormat's RecordWriter implementation doesn't use
> synchronization when accessing the recordWriters member. When using
> multithreaded mappers or reducers, this can result in problems where two
> threads will both try to create the same file, causing
> AlreadyBeingCreatedException. Doing this more fine-grained than just
> synchronizing the whole method is probably a good idea, so that multithreaded
> mappers can actually achieve parallelism writing into separate output streams.
> From what I can tell, the new API's MultipleOutputs seems not to have this
> issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
