HBASE-19338 has committed, do you want to update the master branch and test it again?
2017-11-29 10:32 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > And one thing may effect performance is that, now we rely on > CompletableFuture more and more in our code base, but at least before > jdk8u131 there is a performance regression for CompletableFuture. So > consider moving to the newest jdk if you are still on an older version. > > Thanks. > > 2017-11-29 3:35 GMT+08:00 Mike Drob <md...@apache.org>: > > > Ted, > > > > To clarify, I'm talking about a hypothetical bugfix that impacted > > performance, not to imply that I know of a specific such change. > > > > I've seen it often enough before that performance is blazing fast at the > > expense of accuracy and people are surprised when correctness takes > longer. > > > > > > Mike > > > > On Tue, Nov 28, 2017 at 1:01 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > Mike: > > > Which JIRA was the important bug-fix ? > > > > > > Thanks > > > > > > On Tue, Nov 28, 2017 at 9:16 AM, Mike Drob <md...@apache.org> wrote: > > > > > > > Eshcar - do you have time to try the other alpha releases and see > where > > > > exactly we introduced the regressions? > > > > > > > > Also, I'm worried that the performance regression may be related to > an > > > > important bug-fix, where before we may have had fast writes but also > > > risked > > > > incorrect behavior somehow. > > > > > > > > Mike > > > > > > > > On Tue, Nov 28, 2017 at 2:48 AM, Eshcar Hillel > <esh...@oath.com.invalid > > > > > > > wrote: > > > > > > > > > I agree, so will wait till we focus on performance. > > > > > Just one more update, I also ran the same experiment (write-only) > > with > > > > > banch-2 beta-1.Here is a summary of the throughput I see in each > > > > tag/branch: > > > > > ------------------------------- | BASIC | NONE | > > > > > -------------------------------2-alpha-1| 110K | 80K | > > > > > 2-beta-1 | 81K | 62K | > > > > > master | 60K | 55K |------------------------------- > > > > > This means there are multiple sources for the regression. > > > > > > > > > > Thanks > > > > > > > > > > On Saturday, November 25, 2017, 7:44:01 AM GMT+2, 张铎(Duo > Zhang) < > > > > > palomino...@gmail.com> wrote: > > > > > > > > > > I think first we need a release plan on when we will begin to > focus > > on > > > > the > > > > > performance issue? > > > > > > > > > > I do not think it is a good time to focus on performance issue now > as > > > we > > > > > haven’t stabilized our build yet. The performance regression may > come > > > > back > > > > > again after some bug fixes and maybe we use a wrong way to increase > > > > > performance and finally we find that it is just a bug... > > > > > > > > > > Of course I do not mean we can not do any performance related > issues > > > now, > > > > > for example, HBASE-19338 is a good catch and can be fixed right > now. > > > > > > > > > > And also, for AsyncFSWAL and in memory compaction, we need to > > consider > > > > the > > > > > performance right now as they are born for performance, but let’s > > focus > > > > on > > > > > the comparison to other policies, not a previous release so we can > > find > > > > the > > > > > correct things to fix. > > > > > > > > > > Of course, if there is a big performance downgrading comparing to > the > > > > > previous release and we find it then we should tell others, just > like > > > > this > > > > > email. An earlier notification is always welcomed. > > > > > > > > > > Thanks. > > > > > > > > > > Stack <st...@duboce.net>于2017年11月25日 周六13:22写道： > > > > > > > > > > > On Thu, Nov 23, 2017 at 7:35 AM, Eshcar Hillel > > > <esh...@oath.com.invalid > > > > > > > > > > > wrote: > > > > > > > > > > > > > Happy Thanksgiving all, > > > > > > > > > > > > > > > > > > > And to you Eshcar. > > > > > > > > > > > > > > > > > > > > > > > > > In recent benchmarks I ran in HBASE-18294 I discovered major > > > > > performance > > > > > > > degradation of master code w.r.t 2-alpha-1 code.I am running > > > > write-only > > > > > > > workload (similar to the one reported in HBASE-16417). I am > using > > > the > > > > > > same > > > > > > > hardware and same configuration settings (specifically, I > testes > > > both > > > > > > basic > > > > > > > memstore compaction with optimal parameters, and no memsore > > > > > > > compaction).While in 2-alpha-1 code I see throughput of > ~110Kops > > > for > > > > > > basic > > > > > > > compaction and ~80Kops for no compaction, in the master code I > > get > > > > only > > > > > > > 60Kops and 55Kops, respectively. *This is almost 50% reduction > in > > > > > > > performance*. > > > > > > > (1) Did anyone else noticed such degradation?(2) Do we have any > > > > > > systematic > > > > > > > automatic/semi-automatic method to track the sources of this > > > > > performance > > > > > > > issue? > > > > > > > Thanks,Eshcar > > > > > > > > > > > > > > > > > > > > > > > > > On #1, no. I've not done perf compare. I wonder if later alpha > > > versions > > > > > > include the regression (I'll have to check and see). > > > > > > > > > > > > On #2, again no. I intend to do a bit of perf tuning and compare > > > before > > > > > > release. > > > > > > > > > > > > If you don't file an issue, I will do so later for myself as a > task > > > to > > > > > > compare at least to alpha-1. > > > > > > > > > > > > Thanks Eshcar, > > > > > > > > > > > > St.Ack > > > > > > > > > > > > > > > > > > > > > -- *Best Regards,* lijin bin
