+hdfs/common/mr On Thu, Aug 17, 2017 at 1:28 PM, Wangda Tan <wheele...@gmail.com> wrote:
> Hi all, > > I want to hear your thoughts of merging YARN resource profile branch into > trunk in the next few weeks. The goal is to get it in for Hadoop 3.0 beta1. > > *Regarding to testing:* > We did extensive tests for the feature in the last several months. > Comparing to latest trunk. > - For SLS benchmark: We didn't see observable performance gap from > simulated test based on 8K nodes SLS traces (1 PB memory). We got 3k+ > containers allocated per second. > - For microbenchmark: We use performance test cases added by YARN-6775, it > shows around 5% performance regression comparing to trunk. > > *Regarding to API stability: * > Most new added @Public APIs are @Unstable (We're going to convert some new > added @Public/@Evolving to @Unstable in the cleanup JIRA as well), we want > to get this included by beta1 so we get some feedbacks before declaring > stable API. > > There're few pending cleanups under YARN-3926 umbrella JIRA. Besides these > cleanups, this feature works from end-to-end, we will do another iteration > of end-to-end tests after cleanup patches got committed. > > We would love to get your thoughts before opening a voting thread. > > Special thanks to a team of folks who worked hard and contributed towards > this efforts including design discussion / patch / reviews, etc.: Varun > Vasudev, Sunil Govind, Daniel Templeton, Vinod Vavilapalli, Yufei Gu, > Karthik Kambatla, Jason Lowe, Arun Suresh. > > Thanks, > Wangda Tan >
