Re: [Nfs-ganesha-devel] rpcping comparison nfs-server

2018-03-28 Thread William Allen Simpson
On 3/27/18 9:34 AM, William Allen Simpson wrote: On 3/25/18 1:44 PM, William Allen Simpson wrote: On 3/23/18 1:30 PM, William Allen Simpson wrote: Ran some apples to apples comparisons today V2.7-dev.5: Without the client-side rbtrees, rpcping works a lot better: Thought of a small tweak to

Re: [Nfs-ganesha-devel] rpcping comparison nfs-server

2018-03-27 Thread William Allen Simpson
On 3/25/18 1:44 PM, William Allen Simpson wrote: On 3/23/18 1:30 PM, William Allen Simpson wrote: Ran some apples to apples comparisons today V2.7-dev.5: Without the client-side rbtrees, rpcping works a lot better: Thought of a small tweak to the list adding routine, so it doesn't kick the e

Re: [Nfs-ganesha-devel] rpcping profile

2018-03-25 Thread Matt Benjamin
With N=10 and num_calls=100, on Lemon, test_rbt averages 2.8M reqs/s. That's about half the rate when N=1, which I think is expected. If this is really an available rbt in-order search-remove-insert retire rate when N is 10, my intuition would be it's sufficiently fast not to be t

Re: [Nfs-ganesha-devel] rpcping profile

2018-03-25 Thread Matt Benjamin
1 What is the peak outstanding size of outstanding calls 1.1 if e.g. > 100k is that correct: as last week, why would a sensible client issue more than e.g. 1000 calls without seeing replies? 1.3 if outstanding calls is <= 1, why can test_rbt retire millions of duty cycles / s in that scenario

Re: [Nfs-ganesha-devel] rpcping comparison nfs-server

2018-03-25 Thread William Allen Simpson
On 3/23/18 1:30 PM, William Allen Simpson wrote: Ran some apples to apples comparisons today V2.7-dev.5: Without the client-side rbtrees, rpcping works a lot better: Ganesha (worst, best): rpcping tcp localhost count=1000 threads=1 workers=5 (port=2049 program=13 version=3 procedure=0)

Re: [Nfs-ganesha-devel] rpcping profile

2018-03-25 Thread William Allen Simpson
On 3/24/18 7:50 AM, William Allen Simpson wrote: Noting that the top problem is exactly my prediction by knowledge of the code:   clnt_req_callback() opr_rbtree_insert() The second is also exactly as expected:   svc_rqst_expire_insert() opr_rbtree_insert() svc_rqst_expire_cmpf() These are bo

[Nfs-ganesha-devel] rpcping profile

2018-03-24 Thread William Allen Simpson
Using local file tests/rpcping. Using local file ../profile. Total: 989 samples 321 32.5% 32.5% 321 32.5% svc_rqst_expire_cmpf 149 15.1% 47.5% 475 48.0% opr_rbtree_insert 139 14.1% 61.6% 140 14.2% __writev 56 5.7% 67.2% 66 6.7% __GI___pthread

[Nfs-ganesha-devel] rpcping comparison nfs-server

2018-03-23 Thread William Allen Simpson
Ran some apples to apples comparisons today V2.7-dev.5: Ganesha (worst, best): rpcping tcp localhost count=1000 threads=1 workers=5 (port=2049 program=13 version=3 procedure=0): mean 33950.1556, total 33950.1556 rpcping tcp localhost count=1000 threads=1 workers=5 (port=2049 program=13

Re: [Nfs-ganesha-devel] rpcping

2018-03-15 Thread Daniel Gryniewicz
100k is a much more accurate measurement. I haven't gotten any crashes since the fixes from yesterday, but I can keep trying. On Thu, Mar 15, 2018 at 12:10 PM, William Allen Simpson wrote: > On 3/15/18 10:23 AM, Daniel Gryniewicz wrote: >> >> Can you try again with a larger count, like 100k? 5

Re: [Nfs-ganesha-devel] rpcping

2018-03-15 Thread William Allen Simpson
On 3/15/18 10:23 AM, Daniel Gryniewicz wrote: Can you try again with a larger count, like 100k? 500 is still quite small for a loop benchmark like this. In the code, I commented that 500 is minimal. I've done a pile of 100, 200, 300, and they perform roughly the same as 500. rpcping tcp loca

Re: [Nfs-ganesha-devel] rpcping

2018-03-15 Thread Daniel Gryniewicz
Can you try again with a larger count, like 100k? 500 is still quite small for a loop benchmark like this. Daniel On Thu, Mar 15, 2018 at 9:02 AM, William Allen Simpson wrote: > On 3/14/18 3:33 AM, William Allen Simpson wrote: >> >> rpcping tcp localhost threads=1 count=500 (port=2049 program=1

Re: [Nfs-ganesha-devel] rpcping

2018-03-15 Thread William Allen Simpson
On 3/14/18 3:33 AM, William Allen Simpson wrote: rpcping tcp localhost threads=1 count=500 (port=2049 program=13 version=3 procedure=0): mean 51285.7754, total 51285.7754 DanG pushed the latest code onto ntirpc this morning, and I'll submit a pullup for Ganesha later today. I've changed t

Re: [Nfs-ganesha-devel] rpcping

2018-03-14 Thread Matt Benjamin
Hi Bill, I was not (not intentionally, and, I think, not at all) imputing sentiment to Daniel nor myself. I was paraphrasing statements Daniel made not only informally to me, but publically in this week's nfs-ganesha call, in which you were present. I'm not denigrating your work, but I did dispu

Re: [Nfs-ganesha-devel] rpcping

2018-03-14 Thread William Allen Simpson
On 3/14/18 7:27 AM, Matt Benjamin wrote: Daniel doesn't think you've measured much accurately yet, but at least the effort (if not the discussion) aims to. I'm sure Daniel can speak for himself. At your time of writing, Daniel had not yet arrived in the office after my post this am. So I'm as

Re: [Nfs-ganesha-devel] rpcping

2018-03-14 Thread Matt Benjamin
Daniel doesn't think you've measured much accurately yet, but at least the effort (if not the discussion) aims to. On Wed, Mar 14, 2018 at 2:54 AM, William Allen Simpson wrote: Matt -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.c

Re: [Nfs-ganesha-devel] rpcping

2018-03-14 Thread William Allen Simpson
On 3/13/18 1:58 PM, Daniel Gryniewicz wrote: rpcping was not thread safe.  I have fixes for it incoming. With DanG's significant help, we now have better timing results. There was an implicit assumption in the ancient code that it was calling single threaded tirpc, while ntirpc is multi-thread

Re: [Nfs-ganesha-devel] rpcping

2018-03-13 Thread William Allen Simpson
On 3/13/18 8:27 AM, Matt Benjamin wrote: On Tue, Mar 13, 2018 at 2:38 AM, William Allen Simpson wrote: but if we assume xids retire in xid order also, They do. Should be no variance. Eliminating the dupreq caching -- also using the rbtree -- significantly improved the timing. It's certain

Re: [Nfs-ganesha-devel] rpcping

2018-03-13 Thread Daniel Gryniewicz
rpcping was not thread safe. I have fixes for it incoming. Daniel On 03/13/2018 12:13 PM, William Allen Simpson wrote: On 3/13/18 2:38 AM, William Allen Simpson wrote: In my measurements, using the new CLNT_CALL_BACK(), the client thread starts sending a stream of pings.  In every case, it pe

Re: [Nfs-ganesha-devel] rpcping

2018-03-13 Thread William Allen Simpson
On 3/13/18 2:38 AM, William Allen Simpson wrote: In my measurements, using the new CLNT_CALL_BACK(), the client thread starts sending a stream of pings.  In every case, it peaks at a relatively stable rate. DanG suggested that timing was dominated by the system time calls. The previous numbers

Re: [Nfs-ganesha-devel] rpcping

2018-03-13 Thread Matt Benjamin
On Tue, Mar 13, 2018 at 2:38 AM, William Allen Simpson wrote: > On 3/12/18 6:25 PM, Matt Benjamin wrote: >> >> If I understand correctly, we always insert records in xid order, and >> xid is monotonically increasing by 1. I guess pings might come back >> in any order, > > > No, they always come b

Re: [Nfs-ganesha-devel] rpcping

2018-03-12 Thread William Allen Simpson
On 3/12/18 6:25 PM, Matt Benjamin wrote: If I understand correctly, we always insert records in xid order, and xid is monotonically increasing by 1. I guess pings might come back in any order, No, they always come back in order. This is TCP. I've gone to some lengths to fix the problem that

Re: [Nfs-ganesha-devel] rpcping

2018-03-12 Thread Matt Benjamin
That's certainly suggestive. I found it hard to believe the red-black tree performance could be that bad, at a loading of 10K items--even inserting, searching, and removing in-order. Then again, I never benchmarked the opr_rbtree code. If I understand correctly, we always insert records in xid o

Re: [Nfs-ganesha-devel] rpcping

2018-03-12 Thread William Allen Simpson
[These are with a Ganesha that doesn't dupreq cache the null operation.] Just how slow is this RB tree? Here's a comparison of 1000 entries versus 100 entries in ops per second: rpcping tcp localhost threads=5 count=1000 (port=2049 program=13 version=3 procedure=0): average 2963.2517, tota

Re: [Nfs-ganesha-devel] rpcping

2018-03-12 Thread William Allen Simpson
One of the limiting factors in our Ganesha performance is that the NULL operation is going through the dupreq code. That can be easily fixed with a check that jumps to nocache. One of the limiting factors in our ntirpc performance seems to be the call_replies tree that stores the xid of calls to

Re: [Nfs-ganesha-devel] rpcping

2018-03-12 Thread William Allen Simpson
[top post] Matt produced a new tcp-only variant that skipped rpcbind. I tried it, and immediately got crashes. So I've pushed out a few bug fixes With my fixes, here are the results on my desktop. First and foremost, I compared with my prior results against rpcbind, and they were comparabl

Re: [Nfs-ganesha-devel] rpcping

2018-03-08 Thread William Allen Simpson
On 3/8/18 12:33 PM, William Allen Simpson wrote: Still having no luck.  Instead of relying on RPC itself, checked with Ganesha about what it registers, and tried some of those. Without running Ganesha, rpcinfo reports portmapper services by default on my machine. Can talk to it via localhost (

[Nfs-ganesha-devel] rpcping

2018-03-08 Thread William Allen Simpson
Still having no luck. Instead of relying on RPC itself, checked with Ganesha about what it registers, and tried some of those. The default procedure is 0, that according to every RFC is reserved for do nothing. But rpcbind is not finding program and version. To be honest, I'm not sure how this