Ok, help. I am trying to create local task outputs in my reduce job,
and they get created, then go poof when the job's done.
My first take was to use FileOutputFormat.getWorkOutputPath, and
create directories in there for my outputs (which are Lucene
indexes). Exasperated, I then wrote a small OutputFormat/RecordWriter
pair to write the indexes. In each case, I can see directories being
created in attempt_foo/_temporary, but when the task is over they're
gone.
I've stared at TextOutputFormat and I can't figure out why it's files
survive and mine don't. Help! Again, this is 0.18.3.
Thanks,
Ian
- Task files in _temporary not getting promoted out Ian Soboroff
-