Probably not. The number for on cluster are just so close that it looks like the only large differences in perf were on standalone installs. Since everywhere that talks about standalone talks about how it's not to be used as a basis for performance evaluation I think things are fine. 0.94 looks great and I would +1 it if I had a vote.
Also testing on 0.90 is harder to get everything set up; There's no presplit in pe, so I'm not 100% sure that the numbers would be reliable. On Mon, May 7, 2012 at 1:43 PM, Enis Söztutar <e...@hortonworks.com> wrote: > Elliot, any plan on running the same on 0.90.x? > > Enis > > On Mon, May 7, 2012 at 11:07 AM, Elliott Clark <ecl...@stumbleupon.com > >wrote: > > > Sorry everything is in elapsed time as reported by Elapsed time in > > milliseconds. So higher is worse. > > > > The standard deviation on 0.92.1 writes is 4,591,384 so Write 5 is a > little > > outside of 1 std dev. Not really sure what happened on that test, but it > > does appear that PE is very noisy. > > > > On Mon, May 7, 2012 at 10:47 AM, Todd Lipcon <t...@cloudera.com> wrote: > > > > > Is higher better or worse? :) Any idea what happened on the "Write 5" > > test? > > > > > > On Mon, May 7, 2012 at 10:42 AM, Elliott Clark <ecl...@stumbleupon.com > > > > > wrote: > > > > > http://www.scribd.com/eclark847297/d/92715238-0-94-0-RC3-Cluster-Perf > > > > > > > > On Fri, May 4, 2012 at 7:42 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > >> 0.94 also has LoadTestTool (from FB) > > > >> > > > >> I have used it to do some cluster load testing. > > > >> > > > >> Just FYI > > > >> > > > >> On Fri, May 4, 2012 at 3:14 PM, Elliott Clark < > ecl...@stumbleupon.com > > > >> >wrote: > > > >> > > > >> > With the cluster size that I'm testing YCSB was stressing the > client > > > >> > machine more than the cluster. I was saturating the network of > the > > > test > > > >> > machine. So I switched over to pe; while it doesn't have a > > realistic > > > >> work > > > >> > load it is better than nothing. > > > >> > > > > >> > On Fri, May 4, 2012 at 3:07 PM, Ted Yu <yuzhih...@gmail.com> > wrote: > > > >> > > > > >> > > Thanks for the update, Elliot. > > > >> > > > > > >> > > If I read your post correctly, you're using PE. ycsb is better > > > >> measuring > > > >> > > performance, from my experience. > > > >> > > > > > >> > > Cheers > > > >> > > > > > >> > > On Fri, May 4, 2012 at 3:04 PM, Elliott Clark < > > > ecl...@stumbleupon.com > > > >> > > >wrote: > > > >> > > > > > >> > > > So I got 94.0rc3 up on a cluster and tried to break it, > Killing > > > >> masters > > > >> > > and > > > >> > > > killing rs. Everything seems good. hbck reports everything is > > > good. > > > >> > And > > > >> > > > all my reads succeed. > > > >> > > > > > > >> > > > I'll post cluster benchmark numbers once they are done > running. > > > >> Should > > > >> > > > only be a couple more hours of pe runs. > > > >> > > > > > > >> > > > Looks great to me. > > > >> > > > On Thu, May 3, 2012 at 10:36 AM, Elliott Clark < > > > >> ecl...@stumbleupon.com > > > >> > > > >wrote: > > > >> > > > > > > >> > > > > I agree it was just a micro benchmark with no guarantee that > > it > > > >> > relates > > > >> > > > to > > > >> > > > > real world. With it just being standalone I didn't think > > anyone > > > >> > should > > > >> > > > take > > > >> > > > > the numbers as 100% representative. Really I was just > trying > > to > > > >> > shake > > > >> > > > out > > > >> > > > > any weird behaviors and the fact that we got a big speed up > > was > > > >> > > > > interesting. > > > >> > > > > > > > >> > > > > > > > >> > > > > On Thu, May 3, 2012 at 12:15 AM, Mikael Sitruk < > > > >> > > mikael.sit...@gmail.com > > > >> > > > >wrote: > > > >> > > > > > > > >> > > > >> Hi guys > > > >> > > > >> Looking at the posted slide/pictures for the benchmark the > > > >> > > > >> following intriguing me: > > > >> > > > >> 1. The recordcount is only 100,000 > > > >> > > > >> 2. workoloada is: read 50%, update 50% and zipfian > > distribution > > > >> even > > > >> > > > with > > > >> > > > >> 5M operations count, the same keys are updated again and > > again. > > > >> > > > >> 3. heap size 10G > > > >> > > > >> > > > >> > > > >> Therefore it might be that the dataset is too small (even > > with > > > 3 > > > >> > > > versions > > > >> > > > >> configured we have = 3(version)*100,000(keys)*1KB (size of > > > >> record) = > > > >> > > 300 > > > >> > > > >> MB > > > >> > > > >> of "live" dataset ? > > > >> > > > >> And approximately the number of store files will be 5x10^6 > > (op > > > >> > > > >> count)*1KB(record size)/256MB(max store file size > > > (Default))=>20 > > > >> > store > > > >> > > > >> file, even taking factor of 10 for metadata (record key, in > > > store > > > >> > > files) > > > >> > > > >> we > > > >> > > > >> will get 200 files. > > > >> > > > >> if a major compaction is running it will shrink all the > > > storefile > > > >> > to a > > > >> > > > >> single small one. > > > >> > > > >> What I try to say is - if the maths are correct - (please > > note > > > >> that > > > >> > i > > > >> > > > did > > > >> > > > >> not take into account compression which just make things > > > better), > > > >> > can > > > >> > > we > > > >> > > > >> relate on such scenario for performance benchmark with such > > > small > > > >> > > > dataset > > > >> > > > >> and such distribution? > > > >> > > > >> > > > >> > > > >> Regards > > > >> > > > >> Mikael.S > > > >> > > > >> > > > >> > > > >> On Thu, May 3, 2012 at 1:10 AM, Ted Yu < > yuzhih...@gmail.com> > > > >> wrote: > > > >> > > > >> > > > >> > > > >> > I am surprised to see 0.92.1 exhibit such unfavorable > > > >> performance > > > >> > > > >> profile. > > > >> > > > >> > Let's see whether cluster testing gives us similar > results. > > > >> > > > >> > > > > >> > > > >> > On Wed, May 2, 2012 at 3:07 PM, Elliott Clark < > > > >> > > ecl...@stumbleupon.com > > > >> > > > >> > >wrote: > > > >> > > > >> > > > > >> > > > >> > > Sure, sorry about that. > > > >> > > > >> > > > > > >> > > > >> > > http://imgur.com/waxlS > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > http://www.scribd.com/eclark847297/d/92151092-Hbase-0-94-0-RC3-Local-YCSB-Perf > > > >> > > > >> > > > > > >> > > > >> > > On Wed, May 2, 2012 at 3:01 PM, Ted Yu < > > > yuzhih...@gmail.com> > > > >> > > wrote: > > > >> > > > >> > > > > > >> > > > >> > > > Elliot: > > > >> > > > >> > > > Thanks for the report. > > > >> > > > >> > > > Can you publish results somewhere else ? > > > >> > > > >> > > > Attachments were stripped off. > > > >> > > > >> > > > > > > >> > > > >> > > > On Wed, May 2, 2012 at 2:59 PM, Elliott Clark < > > > >> > > > >> ecl...@stumbleupon.com > > > >> > > > >> > > > >wrote: > > > >> > > > >> > > > > > > >> > > > >> > > > > I ran some tests of local filesystem YCSB. I used > the > > > 0.90 > > > >> > > > client > > > >> > > > >> for > > > >> > > > >> > > > > 0.90.6. For the rest of the tests I used 0.92 > > clients. > > > >> The > > > >> > > > >> results > > > >> > > > >> > are > > > >> > > > >> > > > > attached. > > > >> > > > >> > > > > > > > >> > > > >> > > > > 0.90 -> 0.94.0RC3 13% faster > > > >> > > > >> > > > > 0.92 -> 0.94.0RC3 50% faster > > > >> > > > >> > > > > > > > >> > > > >> > > > > This seems to be a pretty large performance > > > improvement. > > > >> > > I'll > > > >> > > > >> run > > > >> > > > >> > > some > > > >> > > > >> > > > > tests on a cluster later today. > > > >> > > > >> > > > > > > > >> > > > >> > > > > On Tue, May 1, 2012 at 10:20 PM, lars hofhansl < > > > >> > > > >> lhofha...@yahoo.com > > > >> > > > >> > > > >wrote: > > > >> > > > >> > > > > > > > >> > > > >> > > > >> Thanks Todd. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> I agree with doing source code releases going > > forward. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> For that, would it be sufficient to just vote > > against > > > an > > > >> > SVN > > > >> > > > tag? > > > >> > > > >> > > > >> Tarballs can then be pulled straight from that > tag. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> -- Lars > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> ----- Original Message ----- > > > >> > > > >> > > > >> From: Todd Lipcon <t...@cloudera.com> > > > >> > > > >> > > > >> To: dev@hbase.apache.org; lars hofhansl < > > > >> > lhofha...@yahoo.com > > > >> > > > > > > >> > > > >> > > > >> Cc: > > > >> > > > >> > > > >> Sent: Tuesday, May 1, 2012 9:35 PM > > > >> > > > >> > > > >> Subject: Re: ANN: The third hbase 0.94.0 release > > > >> candidate > > > >> > is > > > >> > > > >> > > available > > > >> > > > >> > > > >> for download > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> +1 from me, I took it for a spin on the local > > > filesystem > > > >> > with > > > >> > > > >> some > > > >> > > > >> > > YCSB > > > >> > > > >> > > > >> load. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> Here is my signature on the non-secure tarball. > > > >> > > > >> > > > >> -----BEGIN PGP SIGNATURE----- > > > >> > > > >> > > > >> Version: GnuPG v1.4.10 (GNU/Linux) > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > iEYEABECAAYFAk+guTIACgkQXkPKua7Hfq9YSQCeMnCQ4XFqLjw+PF8IXNPDug+t > > > >> > > > >> > > > >> h90AoJ+q4YSg4JbfiCmaXenadWSRU1of > > > >> > > > >> > > > >> =CdfZ > > > >> > > > >> > > > >> -----END PGP SIGNATURE----- > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> I didn't check out the secure tarball. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> I think for future releases we should do the > voting > > > >> > against a > > > >> > > > >> source > > > >> > > > >> > > tar > > > >> > > > >> > > > >> (ie an svn export) since we now produce multiple > > > >> binaries, > > > >> > > and > > > >> > > > >> it's > > > >> > > > >> > > > easier > > > >> > > > >> > > > >> to verify that a source tar matches SVN, etc. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> -Todd > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> On Tue, May 1, 2012 at 4:26 PM, lars hofhansl < > > > >> > > > >> lhofha...@yahoo.com> > > > >> > > > >> > > > >> wrote: > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > The third 0.94.0 RC is available for download > > here: > > > >> > > > >> > > > >> > > http://people.apache.org/~larsh/hbase-0.94.0-rc3/ > > > >> > > > >> > > > >> > (My gpg key is available from pgp.mit.edu. Key > > id: > > > >> > > 7CA45750) > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > HBase 0.94 is a performance release, and there > are > > > some > > > >> > > > >> > interesting > > > >> > > > >> > > > new > > > >> > > > >> > > > >> > features as well. > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > It is wire compatible with 0.92.x. 0.92 clients > > > should > > > >> > work > > > >> > > > >> with > > > >> > > > >> > > 0.94 > > > >> > > > >> > > > >> > servers and vice versa. > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > You can do a rolling restart to get your 0.92.x > > > HBase > > > >> up > > > >> > on > > > >> > > > >> this > > > >> > > > >> > > > >> 0.94.0RC. > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > The full list of changes is available here: > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12316419 > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > Please take this RC for a spin, check out the > doc, > > > etc, > > > >> > and > > > >> > > > >> vote > > > >> > > > >> > > +1/-1 > > > >> > > > >> > > > >> by > > > >> > > > >> > > > >> > May 8th on whether we should release this as > > 0.94.0. > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > Thanks. > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > -- Lars > > > >> > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> -- > > > >> > > > >> > > > >> Todd Lipcon > > > >> > > > >> > > > >> Software Engineer, Cloudera > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > -- > > > Todd Lipcon > > > Software Engineer, Cloudera > > > > > >
