Hi guys Looking at the posted slide/pictures for the benchmark the following intriguing me: 1. The recordcount is only 100,000 2. workoloada is: read 50%, update 50% and zipfian distribution even with 5M operations count, the same keys are updated again and again. 3. heap size 10G
Therefore it might be that the dataset is too small (even with 3 versions configured we have = 3(version)*100,000(keys)*1KB (size of record) = 300 MB of "live" dataset ? And approximately the number of store files will be 5x10^6 (op count)*1KB(record size)/256MB(max store file size (Default))=>20 store file, even taking factor of 10 for metadata (record key, in store files) we will get 200 files. if a major compaction is running it will shrink all the storefile to a single small one. What I try to say is - if the maths are correct - (please note that i did not take into account compression which just make things better), can we relate on such scenario for performance benchmark with such small dataset and such distribution? Regards Mikael.S On Thu, May 3, 2012 at 1:10 AM, Ted Yu <yuzhih...@gmail.com> wrote: > I am surprised to see 0.92.1 exhibit such unfavorable performance profile. > Let's see whether cluster testing gives us similar results. > > On Wed, May 2, 2012 at 3:07 PM, Elliott Clark <ecl...@stumbleupon.com > >wrote: > > > Sure, sorry about that. > > > > http://imgur.com/waxlS > > > > > http://www.scribd.com/eclark847297/d/92151092-Hbase-0-94-0-RC3-Local-YCSB-Perf > > > > On Wed, May 2, 2012 at 3:01 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > Elliot: > > > Thanks for the report. > > > Can you publish results somewhere else ? > > > Attachments were stripped off. > > > > > > On Wed, May 2, 2012 at 2:59 PM, Elliott Clark <ecl...@stumbleupon.com > > > >wrote: > > > > > > > I ran some tests of local filesystem YCSB. I used the 0.90 client for > > > > 0.90.6. For the rest of the tests I used 0.92 clients. The results > are > > > > attached. > > > > > > > > 0.90 -> 0.94.0RC3 13% faster > > > > 0.92 -> 0.94.0RC3 50% faster > > > > > > > > This seems to be a pretty large performance improvement. I'll run > > some > > > > tests on a cluster later today. > > > > > > > > On Tue, May 1, 2012 at 10:20 PM, lars hofhansl <lhofha...@yahoo.com > > > >wrote: > > > > > > > >> Thanks Todd. > > > >> > > > >> I agree with doing source code releases going forward. > > > >> > > > >> For that, would it be sufficient to just vote against an SVN tag? > > > >> Tarballs can then be pulled straight from that tag. > > > >> > > > >> -- Lars > > > >> > > > >> > > > >> > > > >> ----- Original Message ----- > > > >> From: Todd Lipcon <t...@cloudera.com> > > > >> To: dev@hbase.apache.org; lars hofhansl <lhofha...@yahoo.com> > > > >> Cc: > > > >> Sent: Tuesday, May 1, 2012 9:35 PM > > > >> Subject: Re: ANN: The third hbase 0.94.0 release candidate is > > available > > > >> for download > > > >> > > > >> +1 from me, I took it for a spin on the local filesystem with some > > YCSB > > > >> load. > > > >> > > > >> Here is my signature on the non-secure tarball. > > > >> -----BEGIN PGP SIGNATURE----- > > > >> Version: GnuPG v1.4.10 (GNU/Linux) > > > >> > > > >> iEYEABECAAYFAk+guTIACgkQXkPKua7Hfq9YSQCeMnCQ4XFqLjw+PF8IXNPDug+t > > > >> h90AoJ+q4YSg4JbfiCmaXenadWSRU1of > > > >> =CdfZ > > > >> -----END PGP SIGNATURE----- > > > >> > > > >> I didn't check out the secure tarball. > > > >> > > > >> I think for future releases we should do the voting against a source > > tar > > > >> (ie an svn export) since we now produce multiple binaries, and it's > > > easier > > > >> to verify that a source tar matches SVN, etc. > > > >> > > > >> -Todd > > > >> > > > >> > > > >> On Tue, May 1, 2012 at 4:26 PM, lars hofhansl <lhofha...@yahoo.com> > > > >> wrote: > > > >> > > > >> > The third 0.94.0 RC is available for download here: > > > >> > http://people.apache.org/~larsh/hbase-0.94.0-rc3/ > > > >> > (My gpg key is available from pgp.mit.edu. Key id: 7CA45750) > > > >> > > > > >> > HBase 0.94 is a performance release, and there are some > interesting > > > new > > > >> > features as well. > > > >> > > > > >> > It is wire compatible with 0.92.x. 0.92 clients should work with > > 0.94 > > > >> > servers and vice versa. > > > >> > > > > >> > You can do a rolling restart to get your 0.92.x HBase up on this > > > >> 0.94.0RC. > > > >> > > > > >> > The full list of changes is available here: > > > >> > > > > >> > > > > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12316419 > > > >> > > > > >> > Please take this RC for a spin, check out the doc, etc, and vote > > +1/-1 > > > >> by > > > >> > May 8th on whether we should release this as 0.94.0. > > > >> > > > > >> > Thanks. > > > >> > > > > >> > -- Lars > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> Todd Lipcon > > > >> Software Engineer, Cloudera > > > >> > > > >> > > > > > > > > > >
