Just a note that I'm seeing unexpected improvements from just the interrupt processing. For some unknown timing reason, client responses are better aggregated in buffers, allowing faster inline processing. Note the change from non-INLINE to INLINE....
OF course, I don't know why we are supporting NFSv4 over UDP? But I see the improvement for a lot of calls. V2.5 as of this weekend: [main] Register_program :DISP :INFO :Registering NFS V4/UDP [main] rpc :TIRPC :DEBUG :clnt_vc_ncreatef: fd 29 connected [main] rpc :TIRPC :DEBUG :svc_xprt_lookup:145 locking [main] rpc :TIRPC :DEBUG :makefd_xprt:422 unlocking @svc_xprt_lookup:145 [main] rpc :TIRPC :DEBUG :makefd_xprt() 0xe67680 xp_refs 1 fd 29 af 0 port 4294967295 @ makefd_xprt:423 [main] rpc :TIRPC :DEBUG :svc_fd_ncreatef() 0xe67680 xp_refs 1 fd 29 af 1 port 0 @ svc_fd_ncreatef:309 [main] rpc :TIRPC :DEBUG :clnt_vc_ncreatef: fd 29 completed [main] rpc :TIRPC :DEBUG :clnt_vc_control:494 locking [main] rpc :TIRPC :DEBUG :clnt_vc_control:616 unlocking @clnt_vc_control:494 [main] rpc :TIRPC :DEBUG :rpc_ctx_alloc:106 locking [main] rpc :TIRPC :DEBUG :clnt_vc_call:306 unlocking @rpc_ctx_alloc:106 [main] rpc :TIRPC :DEBUG :xdr_ioq_reset() xioq 0xe67080 head 0xe6d760 wh_pos 0 [main] rpc :TIRPC :DEBUG :svc_ref_it() 0xe67680 xp_refs 2 fd 29 af 1 port 0 @ svc_ioq_write_submit:290 [main] rpc :TIRPC :DEBUG :clnt_vc_call:350 locking [main] rpc :TIRPC :DEBUG :svc_rqst_hook_events: 0xe67680 epoll add fd 29 sr_rec 0xe60f70 epoll_fd 32 control fd pair (30:31) (0, 107) [main] rpc :TIRPC :DEBUG :ev_sig: fd 30 sig 0 [main] rpc :TIRPC :DEBUG :svc_rqst_evchan_reg: 0xe67680 xp_refs 2 chan_id 6 refcnt 2 [0x7f5c9ca5dee0] rpc :TIRPC :DEBUG :work_pool_thread() svc_work_pool task 0xe670e8 [0x7f5c9ca5dee0] rpc :TIRPC :DEBUG :svc_release_it() 0xe67680 xp_refs 1 fd 29 af 1 port 0 @ svc_ioq_write:229 [0x7f5c9ca5dee0] rpc :TIRPC :DEBUG :work_pool_thread() svc_work_pool waiting for task [main] rpc :TIRPC :DEBUG :xdr_reply_decode:253 non-INLINE [main] rpc :TIRPC :DEBUG :xdr_reply_decode:286 SUCCESS [main] rpc :TIRPC :DEBUG :clnt_vc_call:450 unlocking @clnt_vc_call:350 [main] rpc :TIRPC :DEBUG :clnt_vc_destroy: cl_destroy 0xe679e0 cl_refcnt 0 [main] rpc :TIRPC :DEBUG :svc_release_it() 0xe67680 xp_refs 0 fd 29 af 1 port 0 @ clnt_vc_destroy:692 [main] rpc :TIRPC :DEBUG :svc_rqst_unhook_events: 0xe67680 epoll del fd 29 sr_rec 0xe60f70 epoll_fd 32 control fd pair (30:31) (0, 107) [main] rpc :TIRPC :DEBUG :svc_rqst_unreg: 0xe67680 xp_refs 0 chan_id 6 refcnt 2 [main] rpc :TIRPC :DEBUG :svc_xprt_clear:218 locking [main] rpc :TIRPC :DEBUG :svc_xprt_clear:229 unlocking @svc_xprt_clear:218 [main] rpc :TIRPC :DEBUG :svc_vc_destroy_it() 0xe67680 xp_refs 0 should actually destroy things @ clnt_vc_destroy:692 Today's v2.6 build: [main] Register_program :DISP :INFO :Registering NFS V4/UDP [main] rpc :TIRPC :DEBUG :clnt_vc_ncreatef: fd 29 connected [main] rpc :TIRPC :DEBUG :svc_xprt_lookup:164 locking [main] rpc :TIRPC :DEBUG :makefd_xprt:412 unlocking @svc_xprt_lookup:164 [main] rpc :TIRPC :DEBUG :makefd_xprt() 0x802c00 xp_refs 1 fd 29 af 0 port 4294967295 @ makefd_xprt:413 [main] rpc :TIRPC :DEBUG :svc_fd_ncreatef() 0x802c00 xp_refs 1 fd 29 af 1 port 0 @ svc_fd_ncreatef:304 [main] rpc :TIRPC :DEBUG :clnt_vc_ncreatef: fd 29 completed [main] rpc :TIRPC :DEBUG :clnt_vc_control:437 locking [main] rpc :TIRPC :DEBUG :clnt_vc_control:559 unlocking @clnt_vc_control:437 [main] rpc :TIRPC :DEBUG :clnt_vc_call:309 locking [main] rpc :TIRPC :DEBUG :clnt_vc_call:326 unlocking @clnt_vc_call:309 [main] rpc :TIRPC :DEBUG :xdr_ioq_reset() xioq 0x7fede0 head 0x805b78 wh_pos 0 [main] rpc :TIRPC :DEBUG :svc_ref_it() 0x802c00 xp_refs 2 fd 29 af 1 port 0 @ svc_ioq_write_submit:280 [main] rpc :TIRPC :DEBUG :clnt_vc_call:367 locking [main] rpc :TIRPC :DEBUG :svc_rqst_hook_events: 0x802c00 epoll add fd 29 sr_rec 0x7f9820 epoll_fd 32 control fd pair (30:31) [main] rpc :TIRPC :DEBUG :ev_sig: fd 30 sig 0 [main] rpc :TIRPC :DEBUG :svc_rqst_evchan_reg: 0x802c00 xp_refs 2 chan_id 6 refcnt 3 [0x7fffeeba8ee0] rpc :TIRPC :DEBUG :work_pool_thread() svc_work_pool task 0x7fee48 [main] rpc :TIRPC :DEBUG :rpc_ctx_wait_reply:136 unlocking @clnt_vc_call:367 [main] rpc :TIRPC :DEBUG :rpc_ctx_wait_reply: call ctx 0x7fffffffd1d0 (xid 1 client 0x805330) [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_rqst_epoll_event: wakeup fd 31 (sr_rec 0x7f9820) [0x7fffeeba8ee0] rpc :TIRPC :DEBUG :svc_release_it() 0x802c00 xp_refs 1 fd 29 af 1 port 0 @ svc_ioq_write:219 [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_rqst_epoll_event: after consume sig fd 31 (sr_rec 0x7f9820) [0x7fffeeba8ee0] rpc :TIRPC :DEBUG :work_pool_thread() svc_work_pool waiting for task [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_rqst_epoll_loop: before epoll_wait fd 32 [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_rqst_epoll_event: 0x802c00 event 1 [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_ref_it() 0x802c00 xp_refs 2 fd 29 af 1 port 0 @ svc_rqst_epoll_event:783 [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_vc_recv:660 locking [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_rqst_rearm_events: 0x802c00 epoll arm fd 29 sr_rec 0x7f9820 epoll_fd 32 control fd pair (30:31) [0x7fffeebcaee0] rpc :TIRPC :DEBUG :work_pool_thread() svc_work_pool task 0x7f9850 [0x7fffeebdbee0] nfs_rpc_decode_request :DISP :DEBUG :0x802c00 context 0x802d90 [0x7fffeebcaee0] rpc :TIRPC :DEBUG :svc_rqst_epoll_loop: before epoll_wait fd 32 [0x7fffeebdbee0] rpc :TIRPC :DEBUG :xdr_reply_decode:248 INLINE [0x7fffeebdbee0] rpc :TIRPC :DEBUG :xdr_reply_decode:286 SUCCESS [0x7fffeebdbee0] rpc :TIRPC :DEBUG :rpc_ctx_xfer_replymsg: call ctx 0x7fffffffd1d0 acknowledged (xid 1) [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_vc_stat:644 unlocking @svc_vc_recv:660 [main] rpc :TIRPC :DEBUG :rpc_ctx_wait_reply: call ctx 0x7fffffffd1d0 replied (xid 1 client 0x805330) [0x7fffeebdbee0] nfs_rpc_decode_request :DISP :F_DBG :SVC_DECODE on socket 29 (/run/rpcbind.sock) xid=1 returned XPRT_IDLE [main] rpc :TIRPC :DEBUG :rpc_ctx_wait_reply:150 locking [0x7fffeebdbee0] rpc :TIRPC :DEBUG :svc_release_it() 0x802c00 xp_refs 1 fd 29 af 1 port 0 @ svc_rqst_xprt_task:751 [main] rpc :TIRPC :DEBUG :clnt_vc_call:391 unlocking @rpc_ctx_wait_reply:150 [0x7fffeebdbee0] rpc :TIRPC :DEBUG :work_pool_thread() svc_work_pool waiting for task [main] rpc :TIRPC :DEBUG :clnt_vc_destroy: cl_destroy 0x805330 cl_refcnt 0 [main] rpc :TIRPC :DEBUG :svc_release_it() 0x802c00 xp_refs 0 fd 29 af 1 port 0 @ clnt_vc_destroy:635 [main] rpc :TIRPC :DEBUG :svc_rqst_unhook_events: 0x802c00 epoll del fd 29 sr_rec 0x7f9820 epoll_fd 32 control fd pair (30:31) (8362016, 8399872) [main] rpc :TIRPC :DEBUG :svc_rqst_unreg: 0x802c00 xp_refs 0 chan_id 6 refcnt 3 [main] rpc :TIRPC :DEBUG :svc_xprt_clear:215 locking [main] rpc :TIRPC :DEBUG :svc_xprt_clear:227 unlocking @svc_xprt_clear:215 [main] rpc :TIRPC :DEBUG :svc_vc_destroy_it() 0x802c00 xp_refs 0 should actually destroy things @ clnt_vc_destroy:635 ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
