After applying HBASE-22577 the performance is good enough now. And the YCSB workloada also shows that there are no big differences on performance. Please see the comments on HBASE-22564 for more details.
Will get HBASE-22577 in soon, and resolve the pending issues such as HBASE-22237. Any other concerns? Thanks. 张铎(Duo Zhang) <palomino...@gmail.com> 于2019年6月14日周五 上午11:27写道： > It’s hadoop-2.8.5. > > But I do not think it makes much difference, we just use the same server > setup, only different clients implementation. > > Sakthi <sakthivel.azh...@gmail.com>于2019年6月14日 周五10:18写道： > >> Duo, >> >> What version on Hadoop did you use in the 5 node cluster for your >> comparisons? For what it's worth, I would also like to try out the PE/LTT >> comparison of both the clients. Or, if any other form of comparison would >> be helpful then I'm open for suggestions and would like to give it a try. >> >> Sakthi >> >> On Thu, Jun 13, 2019 at 4:55 PM 张铎(Duo Zhang) <palomino...@gmail.com> >> wrote: >> >> > It will be transparent, mostly, the interface is still kept. There are >> some >> > incompatible behaviors, for example, now admin.split will wait till the >> > split is actually done, while in the old time it will return immediately >> > after we send the request to master. >> > >> > Jean-Marc Spaggiari <jean-m...@spaggiari.org>于2019年6月13日 周四23:26写道： >> > >> > > Hi, >> > > >> > > Is this going to change the way the client should be called? Or it >> will >> > be >> > > mostly transparent replacement? >> > > >> > > Thanks, >> > > >> > > JMS >> > > >> > > Le jeu. 13 juin 2019 à 02:13, 张铎(Duo Zhang) <palomino...@gmail.com> a >> > > écrit : >> > > >> > > > Josh Elser <els...@apache.org> 于2019年6月12日周三 下午10:00写道： >> > > > >> > > > > Nice perf results! >> > > > > >> > > > > https://issues.apache.org/jira/browse/HBASE-22237 looks like it's >> > also >> > > > > good to be resolved, given >> > > > > >> > > > > >> > > > >> > > >> > >> https://builds.apache.org/job/HBASE%20Nightly/job/HBASE-21512/279/testReport/ >> > > > > (TestLogLevel will be fixed on your rebase/merge). >> > > > > >> > > > > Poking through the PR, it looks like the big change is that we're >> > also >> > > > > defaulting over to use the [sync]ConnectionOverAsyncConnection. >> Good >> > to >> > > > > do it now to help iron things out more. Calling it out to make >> sure >> > > > > others see this. Is it still possible to use the old Connection >> impl? >> > > (I >> > > > > think the answer is "no"). >> > > > > >> > > > No, all the code have been purged... >> > > > >> > > > > >> > > > > Only other question: are there updates for the book that should >> > happen >> > > > > before you move past this? What about "knobs" for configuring >> > retries, >> > > > > internal thread pool(s)? Anything like that you think would be >> > > important >> > > > > for people to tweak? >> > > > > >> > > > Will fill a 'fat' release note soon. I think there will be less >> > > parameters >> > > > to tune, as we do not need any thread pools unless you are using >> > > > coprocessor related methods(which are deprecated and we recommend >> users >> > > to >> > > > use the ones in async client interface). The retry config is still >> the >> > > same >> > > > with the old sync client. >> > > > >> > > > > >> > > > > +1 >> > > > > >> > > > > On 6/11/19 5:48 AM, 张铎(Duo Zhang) wrote: >> > > > > > Filed https://issues.apache.org/jira/browse/HBASE-22564 >> > > > > > >> > > > > > 张铎(Duo Zhang) <palomino...@gmail.com> 于2019年6月11日周二 下午3:53写道： >> > > > > > >> > > > > >> Let me do a YCSB test about the performance. >> > > > > >> >> > > > > >> Stack <st...@duboce.net> 于2019年6月11日周二 下午1:15写道： >> > > > > >> >> > > > > >>> +1 on merge from me. >> > > > > >>> >> > > > > >>> It removes the complicated multi-threaded edifice we'd built >> > > > > client-side >> > > > > >>> to >> > > > > >>> fake an async behavior replacing it with an actual async >> > > > > implementation. >> > > > > >>> Users will immediately notice a radical plummet in working >> thread >> > > > > count on >> > > > > >>> the client side. >> > > > > >>> >> > > > > >>> For the cleanup of old idioms alone, in test code in >> particular, >> > > the >> > > > > patch >> > > > > >>> is worth merging. >> > > > > >>> >> > > > > >>> Any perf numbers to share comparing old sync and async? >> > > > > >>> >> > > > > >>> What about difference in operation? Is there any commentary or >> > doc >> > > or >> > > > > >>> release note to point at? >> > > > > >>> >> > > > > >>> Thanks, >> > > > > >>> S >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> On Mon, Jun 10, 2019 at 6:59 PM 张铎(Duo Zhang) < >> > > palomino...@gmail.com >> > > > > >> > > > > >>> wrote: >> > > > > >>> >> > > > > >>>> https://issues.apache.org/jira/browse/HBASE-21512 >> > > > > >>>> >> > > > > >>>> "Reimplement sync client based on async client" >> > > > > >>>> >> > > > > >>>> The jira title tells everything. This is what I promised >> when I >> > > > first >> > > > > >>>> introduced the async client in HBase, about three years ago, >> > that >> > > > the >> > > > > >>> sync >> > > > > >>>> client can be implemented on top of the async client, so we >> can >> > > > remove >> > > > > >>> the >> > > > > >>>> old sync client implementation, which can reduce our client >> code >> > > > base >> > > > > a >> > > > > >>>> lot. >> > > > > >>>> >> > > > > >>>> I've already opened a PR here, and received several >> > > feedback(thanks >> > > > > >>> stack!) >> > > > > >>>> >> > > > > >>>> https://github.com/apache/hbase/pull/287 >> > > > > >>>> >> > > > > >>>> It shows that we add 8,663 lines and remove 31,386 lines. >> > > > > >>>> >> > > > > >>>> This is the flaky dashboard for this branch >> > > > > >>>> >> > > > > >>>> >> > > > > >>>> >> > > > > >>> >> > > > > >> > > > >> > > >> > >> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/HBASE-21512/lastSuccessfulBuild/artifact/dashboard.html >> > > > > >>>> >> > > > > >>>> With the recent efforts I think it is getting better. >> > > > > >>>> >> > > > > >>>> Will fill the release note soon, it will be a fat one. >> > > > > >>>> >> > > > > >>>> Please vote >> > > > > >>>> >> > > > > >>>> [] +1 >> > > > > >>>> [] +0/-0 >> > > > > >>>> [] -1 Do not merge the branch back because ... >> > > > > >>>> >> > > > > >>>> Thanks. Any suggestions are welcomed. >> > > > > >>>> >> > > > > >>> >> > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> >
