[ https://issues.apache.org/jira/browse/HADOOP-15087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-15087: ------------------------------------ Affects Version/s: 3.0.0 Issue Type: New Feature (was: Improvement) > S3A to support writing directly to the destination dir without creating temp > directory to avoid rename > ------------------------------------------------------------------------------------------------------- > > Key: HADOOP-15087 > URL: https://issues.apache.org/jira/browse/HADOOP-15087 > Project: Hadoop Common > Issue Type: New Feature > Components: fs/s3 > Affects Versions: 3.0.0 > Reporter: Yonger > > Rename in workloads like Teragen/Terasort who use Hadoop default > outputcommitters really hurt performance a lot. > Stocator announce it doesn't create the temporary directories any all, and > still preserves Hadoop's fault tolerance. I add a switch when creating file > via integrating it's code into s3a, I got 5x performance gain in Teragen and > 15% performance improvement in Terasort. > -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org