[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Johan Gustavsson updated MAPREDUCE-6988:
----------------------------------------
        Fix Version/s: 2.8.3
                       3.0.0
                       2.7.5
                       2.9.0
    Affects Version/s: 3.0.0
                       2.9.0
                       2.8.0
                       2.7.4
     Target Version/s: 2.8.1, 2.7.4, 2.9.0, 3.0.0
               Status: Patch Available  (was: Open)

> Let JHS support different file systems for intermediate_done and done
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6988
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6988
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver
>    Affects Versions: 2.7.4, 2.8.0, 2.9.0, 3.0.0
>            Reporter: Johan Gustavsson
>            Priority: Minor
>             Fix For: 2.9.0, 2.7.5, 3.0.0, 2.8.3
>
>         Attachments: MAPREDUCE-6988.000.patch
>
>
> Currently JHS uses filecontext to move files from intermediate_done to done 
> folder. Since filecontext limits the use to 1 filesystem it makes it harder 
> to use s3 as a storage for jhist files. By moving this to filesystem 
> interface we can set hdfs for intermediate storage and s3 as long term 
> storage therefore reducing the number of puts to s3 and removing the need for 
> all M/R containers to carry a s3 sdk.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to