> On Aug 30, 2018, at 7:15 AM, w.kruzel via openzfs-developer > <developer@lists.open-zfs.org> wrote: > > The flamegraphs are here: > https://drive.google.com/open?id=1vM-5wy4s-QhV2D3hBVh5bPgaqPqEKsMa > <https://drive.google.com/open?id=1vM-5wy4s-QhV2D3hBVh5bPgaqPqEKsMa> > > There are 11 of them. > Files out.svg to out4.svg are dtrace flamegraphs of reading when L2ARC has > been in use. > out5.svg to out10.svg are dtrace flamegraphs of usign nvmecontrol command - > in read mode either using single thread or multiple threads. > > So, what I noticed is, that only when I used nvmecontrol with multiple > threads i.e: > # nvmecontrol perftest -n 4 -o read -s 65536 -t 10 nvme0ns1 > I can then find this process "kernel`nvme_qpair_process_completions" - just > search for nvme in the graph. > It's hard to select it in some of them. > kernel`nvme_qpair_process_completions > kernel`intr_event_execute_handlers > kernel`nvme_qpair_submit_request > kernel`nvme_qpair_complete_tracker > kernel`nvme_ctrlr_submit_io_request > > Is this queueing system for access to nvme disk? See out10 and out6 and out7 > for the nvme processes. > All I know is that when arc_read runs, it does not talk to these nvme > processes.
flamegraphs sample stacks executing on CPUs. They are useless for the analysis you're looking for. ZFS knows nothing about NVMe, SATA, SCSI, or any other low-level block protocol. Nor does it care. To get to your answer, look at the block interface boundary. -- richard > > I haven't tried using the nvme as regular disk and then looking at the output > yet, as we are currently using the file server quite extensively. > > I have also tried iostats -x, which is very useful besides that it looks at > nvd and not nvme (small detail) so it will not notice the nvmecontrol reads. > > openzfs <https://openzfs.topicbox.com/latest> / openzfs-developer / see > discussions <https://openzfs.topicbox.com/groups/developer> + participants > <https://openzfs.topicbox.com/groups/developer/members> + delivery options > <https://openzfs.topicbox.com/groups/developer/subscription>Permalink > <https://openzfs.topicbox.com/groups/developer/Tf62628db027682f7-Mb0fb155dbf4c71e1b0d4531f> ------------------------------------------ openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/Tf62628db027682f7-M19d34ae166b5cee602311d8b Delivery options: https://openzfs.topicbox.com/groups/developer/subscription
