[ https://issues.apache.org/jira/browse/HADOOP-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455609#comment-13455609 ]
Dong Xiang commented on HADOOP-8800: ------------------------------------ Looking forward to hearing more details about this proposed feature. > Dynamic Compress Stream > ----------------------- > > Key: HADOOP-8800 > URL: https://issues.apache.org/jira/browse/HADOOP-8800 > Project: Hadoop Common > Issue Type: New Feature > Components: io > Affects Versions: 2.0.1-alpha > Reporter: yankay > Labels: patch > Original Estimate: 168h > Remaining Estimate: 168h > > We use compress in MapReduce in some case because It use CPU to improve IO > throughput. > But we can only set one compress algorithm in configure file. The hadoop > cluster is changing every time. So a compress algorithm may not work well in > all case. > Why not provide an algorithm named dynamic. It can change compress level and > algorithm dynamicly based on performance. Like tcp, it starts up slowly, and > try run faster and faster. It can make the io faster by choose a more > suitable compress algorithm. > I would write a detail design here, and try to submit a patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira