On 8/30/2019 8:43 AM, Xing Zhengjun wrote:


On 8/7/2019 3:56 PM, Xing Zhengjun wrote:


On 7/24/2019 1:17 PM, Xing Zhengjun wrote:


On 7/12/2019 2:42 PM, Xing Zhengjun wrote:
Hi Trond,

     I attached perf-profile part big changes, hope it is useful for analyzing the issue.

Ping...

ping...

ping...

ping...




In testcase: fsmark
on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
with following parameters:

         iterations: 20x
         nr_threads: 64t
         disk: 1BRD_48G
         fs: xfs
         fs2: nfsv4
         filesize: 4M
         test_size: 80G
         sync_method: fsyncBeforeClose
         cpufreq_governor: performance

test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload.
test-url: https://sourceforge.net/projects/fsmark/

commit:
   e791f8e938 ("SUNRPC: Convert xs_send_kvec() to use iov_iter_kvec()")
   0472e47660 ("SUNRPC: Convert socket page send code to use iov_iter()")

e791f8e9380d945e 0472e476604998c127f3c80d291
---------------- ---------------------------
          %stddev     %change         %stddev
              \          |                \
     527.29           -22.6%     407.96        fsmark.files_per_sec
       1.97 ± 11%      +0.9        2.88 ±  4% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry        0.00            +0.9        0.93 ±  4% perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xs_sendpages        2.11 ± 10%      +0.9        3.05 ±  4% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary        5.29 ±  2%      +1.2        6.46 ±  7% perf-profile.calltrace.cycles-pp.svc_recv.nfsd.kthread.ret_from_fork        9.61 ±  5%      +3.1       12.70 ±  2% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork        9.27 ±  5%      +3.1       12.40 ±  2% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork       34.52 ±  4%      +3.3       37.78 ±  2% perf-profile.calltrace.cycles-pp.ret_from_fork       34.52 ±  4%      +3.3       37.78 ±  2% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork        0.00            +3.4        3.41 ±  4% perf-profile.calltrace.cycles-pp.memcpy_erms.memcpy_from_page._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg        0.00            +3.4        3.44 ±  4% perf-profile.calltrace.cycles-pp.memcpy_from_page._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg        0.00            +3.5        3.54 ±  4% perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xs_sendpages        2.30 ±  5%      +3.7        6.02 ±  3% perf-profile.calltrace.cycles-pp.__rpc_execute.rpc_async_schedule.process_one_work.worker_thread.kthread        2.30 ±  5%      +3.7        6.02 ±  3% perf-profile.calltrace.cycles-pp.rpc_async_schedule.process_one_work.worker_thread.kthread.ret_from_fork        1.81 ±  4%      +3.8        5.59 ±  4% perf-profile.calltrace.cycles-pp.call_transmit.__rpc_execute.rpc_async_schedule.process_one_work.worker_thread        1.80 ±  3%      +3.8        5.59 ±  3% perf-profile.calltrace.cycles-pp.xprt_transmit.call_transmit.__rpc_execute.rpc_async_schedule.process_one_work        1.73 ±  4%      +3.8        5.54 ±  4% perf-profile.calltrace.cycles-pp.xs_tcp_send_request.xprt_transmit.call_transmit.__rpc_execute.rpc_async_schedule        1.72 ±  4%      +3.8        5.54 ±  4% perf-profile.calltrace.cycles-pp.xs_sendpages.xs_tcp_send_request.xprt_transmit.call_transmit.__rpc_execute        0.00            +5.4        5.42 ±  4% perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xs_sendpages.xs_tcp_send_request        0.00            +5.5        5.52 ±  4% perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.xs_sendpages.xs_tcp_send_request.xprt_transmit        0.00            +5.5        5.53 ±  4% perf-profile.calltrace.cycles-pp.sock_sendmsg.xs_sendpages.xs_tcp_send_request.xprt_transmit.call_transmit        9.61 ±  5%      +3.1       12.70 ±  2% perf-profile.children.cycles-pp.worker_thread        9.27 ±  5%      +3.1       12.40 ±  2% perf-profile.children.cycles-pp.process_one_work        6.19            +3.2        9.40 ±  4% perf-profile.children.cycles-pp.memcpy_erms       34.53 ±  4%      +3.3       37.78 ±  2% perf-profile.children.cycles-pp.ret_from_fork       34.52 ±  4%      +3.3       37.78 ±  2% perf-profile.children.cycles-pp.kthread        0.00            +3.5        3.46 ±  4% perf-profile.children.cycles-pp.memcpy_from_page        0.00            +3.6        3.56 ±  4% perf-profile.children.cycles-pp._copy_from_iter_full        2.47 ±  4%      +3.7        6.18 ±  3% perf-profile.children.cycles-pp.__rpc_execute        2.30 ±  5%      +3.7        6.02 ±  3% perf-profile.children.cycles-pp.rpc_async_schedule        1.90 ±  4%      +3.8        5.67 ±  3% perf-profile.children.cycles-pp.call_transmit        1.89 ±  3%      +3.8        5.66 ±  3% perf-profile.children.cycles-pp.xprt_transmit        1.82 ±  4%      +3.8        5.62 ±  3% perf-profile.children.cycles-pp.xs_tcp_send_request        1.81 ±  4%      +3.8        5.62 ±  3% perf-profile.children.cycles-pp.xs_sendpages        0.21 ± 17%      +5.3        5.48 ±  4% perf-profile.children.cycles-pp.tcp_sendmsg_locked        0.25 ± 18%      +5.3        5.59 ±  3% perf-profile.children.cycles-pp.tcp_sendmsg        0.26 ± 16%      +5.3        5.60 ±  3% perf-profile.children.cycles-pp.sock_sendmsg        1.19 ±  5%      +0.5        1.68 ±  3% perf-profile.self.cycles-pp.get_page_from_freelist        6.10            +3.2        9.27 ±  4% perf-profile.self.cycles-pp.memcpy_erms


On 7/9/2019 10:39 AM, Xing Zhengjun wrote:
Hi Trond,

On 7/8/2019 7:44 PM, Trond Myklebust wrote:
I've asked several times now about how to interpret your results. As far as I can tell from your numbers, the overhead appears to be entirely contained in the NUMA section of your results. IOW: it would appear to be a scheduling overhead due to NUMA. I've been asking whether or not that is a correct interpretation of the numbers you published.
Thanks for your feedback. I used the same hardware and the same test parameters to test the two commits:     e791f8e938 ("SUNRPC: Convert xs_send_kvec() to use iov_iter_kvec()")     0472e47660 ("SUNRPC: Convert socket page send code to use iov_iter()")

If it is caused by NUMA, why only commit 0472e47660 throughput is decreased? The filesystem we test is NFS, commit 0472e47660 is related with the network, could you help to check if have any other clues for the regression. Thanks.






--
Zhengjun Xing

Reply via email to