Actually sending in a separate thread - since it does not really compare different versions of HBase but one version of Block Cache vs FS Cache(through hdfs).
On Sat, Jun 29, 2013 at 12:00 PM, Varun Sharma <[email protected]> wrote: > I did some tests yesterday, on this. I will send them in a separate thread. > > > On Sat, Jun 29, 2013 at 5:10 AM, lars hofhansl <[email protected]> wrote: > >> In my measurements 0.94 has been getting faster with each release in both >> read and write performance. >> I wonder how representative PE is after all; it only tests via the local >> FS layer (not HDFS), among other issues. >> >> -- Lars >> >> >> >> ----- Original Message ----- >> From: Jean-Marc Spaggiari <[email protected]> >> To: [email protected]; lars hofhansl <[email protected]> >> Cc: >> Sent: Friday, June 28, 2013 8:03 PM >> Subject: Re: 30% random performance in 0.95+ >> >> I think we should do that on 0.94 as well. I don't see any good reason >> to not do it. >> >> JM >> >> 2013/6/28 lars hofhansl <[email protected]>: >> > Yep. >> > Now the question is: Make these changes to 0.94 as well? Or just >> document these better. >> > >> > -- Lars >> > >> > >> > >> > ----- Original Message ----- >> > From: Andrew Purtell <[email protected]> >> > To: "[email protected]" <[email protected]>; lars hofhansl < >> [email protected]> >> > Cc: >> > Sent: Friday, June 28, 2013 2:08 PM >> > Subject: Re: 30% random performance in 0.95+ >> > >> > I've been thinking about how to periodically search through some of our >> > parameter space to see what changes to defaults are better all the way >> > around. Probably will so something based on Bigtop. >> > >> > >> > On Friday, June 28, 2013, lars hofhansl wrote: >> > >> >> And indeed just this makes a tremendous difference. Unpatched 0.94 with >> >> 40% block cache configured is actually faster than 0.95 with the same >> block >> >> cache size. >> >> >> >> -- Lars >> >> >> >> >> >> >> >> ----- Original Message ----- >> >> From: lars hofhansl <[email protected] <javascript:;>> >> >> To: "[email protected] <javascript:;>" <[email protected] >> <javascript:;> >> >> > >> >> Cc: >> >> Sent: Friday, June 28, 2013 1:34 PM >> >> Subject: Re: 30% random performance in 0.95+ >> >> >> >> Thanks JM, >> >> >> >> HBASE-8450 (r1485562) is interesting. It increases (among other things) >> >> the block cache percentage from 24 to 40%, which would lead to a higher >> >> probability of a future random read to hit an already cached block. >> >> >> >> >> >> -- Lars >> >> >> >> >> >> >> >> ----- Original Message ----- >> >> From: Jean-Marc Spaggiari <[email protected] <javascript:;>> >> >> To: [email protected] <javascript:;>; lars hofhansl < >> [email protected]<javascript:;> >> >> > >> >> Cc: >> >> Sent: Friday, June 28, 2013 1:18 PM >> >> Subject: Re: 30% random performance in 0.95+ >> >> >> >> I have the script done to run over a list of "svn releases", so if >> >> required, just give me a bunch of them or a range and I can restart. >> >> Just keep me posted. >> >> >> >> JM >> >> >> >> 2013/6/28 lars hofhansl <[email protected] <javascript:;>>: >> >> > I did a few more test (on my laptop, which is not quite >> representative), >> >> and found only a 2-3% improvement from HBASE-8001+HBASE-8012 in the >> end. >> >> > I'll look through the issues that you identified. >> >> > >> >> > -- Lars >> >> > >> >> > >> >> > >> >> > ----- Original Message ----- >> >> > From: Jean-Marc Spaggiari <[email protected] <javascript:;>> >> >> > To: [email protected] <javascript:;> >> >> > Cc: >> >> > Sent: Friday, June 28, 2013 12:51 PM >> >> > Subject: Re: 30% random performance in 0.95+ >> >> > >> >> > Sorry folks, >> >> > >> >> > I'm a bit late to run the tests... 0.94.8 and 0.94.9 are currently >> >> > running, but here is what I have been able to capture so far for 0.95 >> >> > over the last year: >> >> > r1357480 1513196 >> >> > r1367009 1440244.4 >> >> > r1375812 1287143.5 >> >> > r1381671 1287200.2 >> >> > r1388620 1295262.6 >> >> > r1394335 1022140.2 >> >> > r1403898 884171.9 >> >> > r1410631 804229.9 >> >> > r1419787 846816.9 >> >> > r1426557 853535.3 >> >> > r1433514 873265.1 >> >> > r1438972 840666.9 >> >> > r1446106 877432.2 >> >> > r1452661 883974.8 >> >> > r1458421 882233.3 >> >> > r1464267 847000.8 >> >> > r1478964 877433.5 >> >> > r1485868 744905.5 >> >> > r1494869 765105.9 >> >> > >> >> > So seems that there was some improvements between r1367009 and >> >> > r1403898 but they are old. Also another major improvement between >> >> > r1478964 and r1485868... >> >> > >> >> > Let me know if you want me to dig further and I will be very happy >> to do >> >> so. >> >> > >> >> > JM >> >> > >> >> > 2013/6/28 Stack <[email protected] <javascript:;>>: >> >> >> On Fri, Jun 28, 2013 at 10:53 AM, lars hofhansl <[email protected] >> <javascript:;>> >> >> wrote: >> >> >> >> >> >>> I partially tracked this down to HBASE-8001 and HBASE-8012 by >> looking >> >> at >> >> >>> the call stacks in a profiling session. >> >> >>> HBASE-8767 is a backport of both patched to 0.94. >> >> >>> >> >> >> >> >> >> Sounds like nice work by Raymond Liu... >> >> >> St.Ack >> >> > >> >> >> > >> > >> > -- >> > Best regards, >> > >> > - Andy >> > >> > Problems worthy of attack prove their worth by hitting back. - Piet Hein >> > (via Tom White) >> > >> >> >
