If you are using TextOutputFormat for your job, getRecordWriter() (i.e RecordWriter<K, V> org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.getRecordWriter(TaskAttemptContext job) throws IOException, InterruptedException) method uses FileOutputFormat.getDefaultWorkFile() for generating the file names. It uses $output/_temporary/$taskid/part-[mr]-$id format to generate the o/p path name for task in temp dir.
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.html#getDefaultWorkFile(org.apache.hadoop.mapreduce.TaskAttemptContext, java.lang.String) If you want to change the o/p path name for the task, you need to override this method for your Job accordingly whichever you want. If you want change only base output name for the output file(default value is "part"), you can use "mapreduce.output.basename" configuration. Thanks&Regards Devaraj K From: Maysam Hossein Yabandeh [mailto:myaban...@qf.org.qa] Sent: 12 June 2013 21:16 To: user@hadoop.apache.org Cc: Maysam Hossein Yabandeh Subject: Assigning the same partition number to the mapper output Hi, I was wondering if it is possible in hadoop to assign the same partition numbers to the map outputs. I am running a map-only job (with zero reducers) and hadoop shuffles the partitions in the output: i.e. input/part-m-0000X is processed by task number Y and hence generates output/part-m-0000Y (where X != Y). Thanks Maysam CONFIDENTIALITY NOTICE: This email and any attachments transmitted with it are confidential and intended for the use of individual or entity to which it is addressed. If you have received this email in error, please delete it immediately and inform the sender. Unless you are the intended recipient, you may not use, disclose, copy or distribute this email or any attachments included. The contents of the emails including any attachments may be subjected to copyrights law, In such case the contents may not be copied, adapted, distributed or transmitted without the consent of the copyright owner.
