[ https://issues.apache.org/jira/browse/MAPREDUCE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-1347: ------------------------------- Target Version/s: 2.3.0 (was: 2.0.3-alpha) > Missing synchronization in MultipleOutputFormat > ----------------------------------------------- > > Key: MAPREDUCE-1347 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1347 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client > Affects Versions: 1.0.0 > Reporter: Todd Lipcon > Assignee: Harsh J > Labels: concurrency > Attachments: MAPREDUCE-1347.r10.diff, mapreduce.1347.r1.diff, > MAPREDUCE-1347.r2.diff, MAPREDUCE-1347.r3.diff, MAPREDUCE-1347.r4.diff, > MAPREDUCE-1347.r5.diff, MAPREDUCE-1347.r6.diff, MAPREDUCE-1347.r7.diff, > MAPREDUCE-1347.r8.diff, MAPREDUCE-1347.r9.diff > > > MultipleOutputFormat's RecordWriter implementation doesn't use > synchronization when accessing the recordWriters member. When using > multithreaded mappers or reducers, this can result in problems where two > threads will both try to create the same file, causing > AlreadyBeingCreatedException. Doing this more fine-grained than just > synchronizing the whole method is probably a good idea, so that multithreaded > mappers can actually achieve parallelism writing into separate output streams. > From what I can tell, the new API's MultipleOutputs seems not to have this > issue. -- This message was sent by Atlassian JIRA (v6.1#6144)