I found the solution to the problem! As Harsh said I didn't use FileOutputCommiter because this Commiter is not used in TableOutputFormat. I changed the TableOutputFormat class to use this commiter and now everything works great!
Thank you for you help!!! From: [email protected] To: [email protected] Subject: RE: MultipleOutputs Files remain in temporary folder Date: Tue, 31 May 2011 05:37:27 +0300 I forgot to mention that there is no _SUCCESS folder in the output directory. Thanks for your replies! I use TableOutputFormat to delete entries from a HBase Table and MO for HFileOutputFormat. Until yesterday I used normal HFileOutputFormat output (not MO) and the files appeared in the output directory and not in the temporary folder. I performed the deletes manually inside the reducer. Then I decided to use MO for the HFiles and TableOutputFormat for the deletes. Now the Hfiles remain in the temporary folder. I think that TableOutputFormat uses its own committer and I do not explicitly change the commiter. Do you have anything to suggest? > From: [email protected] > Date: Tue, 31 May 2011 01:20:57 +0530 > Subject: Re: MultipleOutputs Files remain in temporary folder > To: [email protected] > > Panayotis, > > I've not seen this happen yet. I've regularly used MO to write my > files and both TextFileO/F and NullO/F have worked fine despite me not > writing a byte to their collectors. In fact, the test case for MO too > passes when I modify it to never emit to the default output sink. > > Are you using the default OutputCommitter (FileOutputCommitter)? > > 2011/5/30 Panayotis Antonopoulos <[email protected]>: > > Hello, > > I just noticed that the files that are created using MultipleOutputs remain > > in the temporary folder into attempt sub-folders when there is no normal > > output (using context.write(...)). > > > > Has anyone else noticed that? > > Is there any way to change that and make the files appear in the output > > directory? > > > > Thank you in advance! > > Panagiotis. > > > > > > -- > Harsh J
