Johan Gustavsson created MAPREDUCE-6988:
-------------------------------------------
Summary: Let JHS support different file systems for
intermediate_done and done
Key: MAPREDUCE-6988
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6988
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Reporter: Johan Gustavsson
Priority: Minor
Currently JHS uses filecontext to move files from intermediate_done to done
folder. Since filecontext limits the use to 1 filesystem it makes it harder to
use s3 as a storage for jhist files. By moving this to filesystem interface we
can set hdfs for intermediate storage and s3 as long term storage therefore
reducing the number of puts to s3 and removing the need for all M/R containers
to carry a s3 sdk.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
