Opening a JIRA would be fine. This makes it easier for people to obtain the patch(es).
Cheers > On Nov 18, 2016, at 11:35 PM, Anoop John <anoop.hb...@gmail.com> wrote: > > Because of some compatibility issues, we decide that this will be done > in 2.0 only.. Ya as Andy said, it would be great to share the 1.x > backported patches. Is it a mega patch at ur end? Or issue by issue > patches? Latter would be best. Pls share patches in some place and a > list of issues backported. I can help with verifying the issues once > so as to make sure we dont miss any... > > -Anoop- > >> On Sat, Nov 19, 2016 at 12:32 AM, Enis Söztutar <enis....@gmail.com> wrote: >> Thanks for sharing this. Great work. >> >> I don't see any reason why we cannot backport to branch-1. >> >> Enis >> >> On Fri, Nov 18, 2016 at 9:37 AM, Andrew Purtell <andrew.purt...@gmail.com> >> wrote: >> >>> Yes, please, the patches will be useful to the community even if we decide >>> not to backport into an official 1.x release. >>> >>> >>>>> On Nov 18, 2016, at 12:25 PM, Bryan Beaudreault < >>>> bbeaudrea...@hubspot.com> wrote: >>>> >>>> Is the backported patch available anywhere? Not seeing it on the >>> referenced >>>> JIRA. If it ends up not getting officially backported to branch-1 due to >>>> 2.0 around the corner, some of us who build our own deploy may want to >>>> integrate into our builds. Thanks! These numbers look great >>>> >>>>> On Fri, Nov 18, 2016 at 12:20 PM Anoop John <anoop.hb...@gmail.com> >>> wrote: >>>>> >>>>> Hi Yu Li >>>>> Good to see that the off heap work help you.. The perf >>>>> numbers looks great. So this is a compare of on heap L1 cache vs off >>> heap >>>>> L2 cache(HBASE-11425 enabled). So for 2.0 we should make L2 off heap >>>>> cache ON by default I believe. Will raise a jira for that we can >>> discuss >>>>> under that. Seems like L2 off heap cache for data blocks and L1 cache >>> for >>>>> index blocks seems a right choice. >>>>> >>>>> Thanks for the backport and the help in testing the feature.. You were >>>>> able to find some corner case bugs and helped community to fix them.. >>>>> Thanks goes to ur whole team. >>>>> >>>>> -Anoop- >>>>> >>>>> >>>>>> On Fri, Nov 18, 2016 at 10:14 PM, Yu Li <car...@gmail.com> wrote: >>>>>> >>>>>> Sorry guys, let me retry the inline images: >>>>>> >>>>>> Performance w/o offheap: >>>>>> >>>>>> >>>>>> Performance w/ offheap: >>>>>> >>>>>> >>>>>> Peak Get QPS of one single RS during Singles' Day (11/11): >>>>>> >>>>>> >>>>>> >>>>>> And attach the files in case inline still not working: >>>>>> >>>>>> Performance_without_offheap.png >>>>>> < >>>>> https://drive.google.com/file/d/0B017Q40_F5uwbWEzUGktYVIya3JkcXVjRkFvVG >>> NtM0VxWC1n/view?usp=drive_web >>>>>> >>>>>> >>>>>> Performance_with_offheap.png >>>>>> < >>>>> https://drive.google.com/file/d/0B017Q40_F5uweGR2cnJEU0M1MWwtRFJ5YkxUeF >>> VrcUdPc2ww/view?usp=drive_web >>>>>> >>>>>> >>>>>> Peak_Get_QPS_of_Single_RS.png >>>>>> < >>>>> https://drive.google.com/file/d/0B017Q40_F5uwQ2FkR2k0ZmEtRVNGSFp5RUxHM3 >>> F6bHpNYnJz/view?usp=drive_web >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Best Regards, >>>>>> Yu >>>>>> >>>>>>> On 18 November 2016 at 19:29, Ted Yu <yuzhih...@gmail.com> wrote: >>>>>>> >>>>>>> Yu: >>>>>>> With positive results, more hbase users would be asking for the >>> backport >>>>>>> of offheap read path patches. >>>>>>> >>>>>>> Do you think you or your coworker has the bandwidth to publish >>> backport >>>>>>> for branch-1 ? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>>> On Nov 18, 2016, at 12:11 AM, Yu Li <car...@gmail.com> wrote: >>>>>>>> >>>>>>>> Dear all, >>>>>>>> >>>>>>>> We have backported read path offheap (HBASE-11425) to our customized >>>>>>> hbase-1.1.2 (thanks @Anoop for the help/support) and run it online for >>>>> more >>>>>>> than a month, and would like to share our experience, for what it's >>>>> worth >>>>>>> (smile). >>>>>>>> >>>>>>>> Generally speaking, we gained a better and more stable >>>>>>> throughput/performance with offheap, and below are some details: >>>>>>>> 1. QPS become more stable with offheap >>>>>>>> >>>>>>>> Performance w/o offheap: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Performance w/ offheap: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> These data come from our online A/B test cluster (with 450 physical >>>>>>> machines, and each with 256G memory + 64 core) with real world >>>>> workloads, >>>>>>> it shows using offheap we could gain a more stable throughput as well >>> as >>>>>>> better performance >>>>>>>> >>>>>>>> Not showing fully online data here because for online we published >>> the >>>>>>> version with both offheap and NettyRpcServer together, so no >>> standalone >>>>>>> comparison data for offheap >>>>>>>> >>>>>>>> 2. Full GC frequency and cost >>>>>>>> >>>>>>>> Average Full GC STW time reduce from 11s to 7s with offheap. >>>>>>>> >>>>>>>> 3. Young GC frequency and cost >>>>>>>> >>>>>>>> No performance degradation observed with offheap. >>>>>>>> >>>>>>>> 4. Peak throughput of one single RS >>>>>>>> >>>>>>>> On Singles Day (11/11), peak throughput of one single RS reached >>> 100K, >>>>>>> among which 90K from Get. Plus internet in/out data we could know the >>>>>>> average result size of get request is ~1KB >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Offheap are used on all online machines (more than 1600 nodes) >>> instead >>>>>>> of LruCache, so the above QPS is gained from offheap bucketcache, >>> along >>>>>>> with NettyRpcServer(HBASE-15756). >>>>>>>> >>>>>>>> Just let us know if any comments. Thanks. >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Yu >>>
