Greetings, I'm trying to get a handle on job history logging. According to the documentation in 'hadoop-defaul.xml' the 'hadoop.job.history.user.location' determines where job history logs are written. If not specified these logs go into '<output-dir>/_logs/history'. This can cause problems with applications that don't know about this convention. It would also be nicer in my opinion to keep logs and data separate.
It seems to me that a nice way to handle this would be to put logs in '/logs/<job-id>/history' or something. Can this be done? Is there a need for the "job-id" folder? If this can't be done, are there alternatives that work well. -lincoln -- lincolnritter.com
