[
https://issues.apache.org/jira/browse/MAPREDUCE-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko updated MAPREDUCE-6988:
-------------------------------------------
Fix Version/s: (was: 2.7.5)
> Let JHS support different file systems for intermediate_done and done
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6988
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6988
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver
> Affects Versions: 2.7.4
> Reporter: Johan Gustavsson
> Priority: Minor
> Attachments: MAPREDUCE-6988.000.patch, MAPREDUCE-6988.001.patch,
> MAPREDUCE-6988.002.patch, MAPREDUCE-6988.003.patch
>
>
> Currently JHS uses filecontext to move files from intermediate_done to done
> folder. Since filecontext limits the use to 1 filesystem it makes it harder
> to use s3 as a storage for jhist files. By moving this to filesystem
> interface we can set hdfs for intermediate storage and s3 as long term
> storage therefore reducing the number of puts to s3 and removing the need for
> all M/R containers to carry a s3 sdk.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]