Hi,

I am using now hadoop version 0.21.0.

AYK, this version supports to use MultipleOutputs class to reduce outputs in several files.

but, in my case, there is nothing in files. (just empty files)

here is my code.

main class)
....
MultipleOutputs.addNamedOutput(job, FeederConfig.INSERT_OUTPUT_NAME, TextOutputFormat.class, Text.class, Text.class); MultipleOutputs.addNamedOutput(job, FeederConfig.DELETE_OUTPUT_NAME, TextOutputFormat.class, Text.class, Text.class); MultipleOutputs.addNamedOutput(job, FeederConfig.UPDATE_OUTPUT_NAME, TextOutputFormat.class, Text.class, Text.class); MultipleOutputs.addNamedOutput(job, FeederConfig.NOTCHANGE_OUTPUT_NAME, TextOutputFormat.class, Text.class, Text.class);
....

mapper)
nothing to do for this job.
just write keys and values

reducer)
...
multipleOutputs.write(getOutputFileName(code), new Text(key), new Text(value));
    context.write(new Text(key), new Text(value));
...
    private String getOutputFileName(String code) {
        String retFileName = "";

        if (code.equals(EPComparedResult.INSERT.getCode())) {
            retFileName = FeederConfig.INSERT_OUTPUT_NAME;
        } else if (code.equals(EPComparedResult.DELETE.getCode())) {
            retFileName = FeederConfig.DELETE_OUTPUT_NAME;
        } else if (code.equals(EPComparedResult.UPDATE.getCode())) {
            retFileName = FeederConfig.UPDATE_OUTPUT_NAME;
        } else {
            retFileName = FeederConfig.NOTCHANGE_OUTPUT_NAME;
        }

        return retFileName;
    }
...


result)
$ hadoop fs -ls output
11/02/07 13:09:13 INFO security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000 11/02/07 13:09:13 WARN conf.Configuration: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
Found 4 items
-rw-r--r-- 2 irteam supergroup 0 2011-01-31 19:59 /user/test/output/DELETE-r-00000 -rw-r--r-- 2 irteam supergroup 0 2011-01-31 19:59 /user/test/output/INSERT-r-00000 -rw-r--r-- 2 irteam supergroup 0 2011-01-31 18:53 /user/test/output/_SUCCESS -rw-r--r-- 2 irteam supergroup 649622 2011-01-31 18:53 /user/test/output/part-r-00000

--
Junyoung Kim (juneng...@gmail.com)

Reply via email to