RE: arm-smmu-v3 high cpu usage for NVMe

2020-05-24 Thread Song Bao Hua (Barry Song)
> Subject: Re: arm-smmu-v3 high cpu usage for NVMe > > On 20/03/2020 10:41, John Garry wrote: > > + Barry, Alexandru > > >>>>>     PerfTop:   85864 irqs/sec  kernel:89.6%  exact:  0.0% lost: > >>>>> 0/34434

Re: arm-smmu-v3 high cpu usage for NVMe

2020-05-22 Thread John Garry
On 20/03/2020 10:41, John Garry wrote: + Barry, Alexandru     PerfTop:   85864 irqs/sec  kernel:89.6%  exact:  0.0% lost: 0/34434 drop: 0/40116 [4000Hz cycles],  (all, 96 CPUs)

Re: arm-smmu-v3 high cpu usage for NVMe

2020-04-06 Thread John Garry
On 02/04/2020 13:10, John Garry wrote: On 18/03/2020 20:53, Will Deacon wrote: As for arm_smmu_cmdq_issue_cmdlist(), I do note that during the testing our batch size is 1, so we're not seeing the real benefit of the batching. I can't help but think that we could improve this code to try to

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-25 Thread John Garry
FWIW I believe is is still on the plan for someone here to dust off the PMU pNMI patches at some point. Cool. Well I can try to experiment with what Julien had at v4 for now. JFYI, I have done some more perf record capturing, and updated the "annotate" and "report" output here

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-24 Thread John Garry
On 24/03/2020 12:07, Robin Murphy wrote: On 2020-03-24 11:55 am, John Garry wrote: On 24/03/2020 10:43, Marc Zyngier wrote: On Tue, 24 Mar 2020 09:18:10 + John Garry  wrote: On 23/03/2020 09:16, Marc Zyngier wrote: + Julien, Mark Hi Marc, Time to enable pseudo-NMIs in the PMUv3

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-24 Thread Robin Murphy
On 2020-03-24 11:55 am, John Garry wrote: On 24/03/2020 10:43, Marc Zyngier wrote: On Tue, 24 Mar 2020 09:18:10 + John Garry  wrote: On 23/03/2020 09:16, Marc Zyngier wrote: + Julien, Mark Hi Marc, Time to enable pseudo-NMIs in the PMUv3 driver... Do you know if there is any plan

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-24 Thread John Garry
On 24/03/2020 10:43, Marc Zyngier wrote: On Tue, 24 Mar 2020 09:18:10 + John Garry wrote: On 23/03/2020 09:16, Marc Zyngier wrote: + Julien, Mark Hi Marc, Time to enable pseudo-NMIs in the PMUv3 driver... Do you know if there is any plan for this? There was. Julien Thierry has a

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-24 Thread Marc Zyngier
On Tue, 24 Mar 2020 09:18:10 + John Garry wrote: > On 23/03/2020 09:16, Marc Zyngier wrote: > > + Julien, Mark > > Hi Marc, > > >>> Time to enable pseudo-NMIs in the PMUv3 driver... > >>> > >> > >> Do you know if there is any plan for this? > > > > There was. Julien Thierry has a bunch

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-24 Thread John Garry
On 23/03/2020 09:16, Marc Zyngier wrote: + Julien, Mark Hi Marc, Time to enable pseudo-NMIs in the PMUv3 driver... Do you know if there is any plan for this? There was. Julien Thierry has a bunch of patches for that [1], but they needs reviving. So those patches still apply cleanly

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-23 Thread Marc Zyngier
On 2020-03-23 09:03, John Garry wrote: On 20/03/2020 16:33, Marc Zyngier wrote: JFYI, I've been playing for "perf annotate" today and it's giving strange results for my NVMe testing. So "report" looks somewhat sane, if not a worryingly high % for arm_smmu_cmdq_issue_cmdlist():     55.39% 

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-23 Thread John Garry
On 20/03/2020 16:33, Marc Zyngier wrote: JFYI, I've been playing for "perf annotate" today and it's giving strange results for my NVMe testing. So "report" looks somewhat sane, if not a worryingly high % for arm_smmu_cmdq_issue_cmdlist():     55.39%  irq/342-nvme0q1  [kernel.kallsyms]  [k]

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-20 Thread Marc Zyngier
Hi John, On 2020-03-20 16:20, John Garry wrote: I've run a bunch of netperf instances on multiple cores and collecting SMMU usage (on TaiShan 2280). I'm getting the following ratio pretty consistently. - 6.07% arm_smmu_iotlb_sync - 5.74% arm_smmu_tlb_inv_range 5.09%

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-20 Thread John Garry
I've run a bunch of netperf instances on multiple cores and collecting SMMU usage (on TaiShan 2280). I'm getting the following ratio pretty consistently. - 6.07% arm_smmu_iotlb_sync - 5.74% arm_smmu_tlb_inv_range 5.09% arm_smmu_cmdq_issue_cmdlist 0.28% __pi_memset

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-20 Thread Jean-Philippe Brucker
On Fri, Mar 20, 2020 at 10:41:44AM +, John Garry wrote: > On 19/03/2020 18:43, Jean-Philippe Brucker wrote: > > On Thu, Mar 19, 2020 at 12:54:59PM +, John Garry wrote: > > > Hi Will, > > > > > > > > > > > On Thu, Jan 02, 2020 at 05:44:39PM +, John Garry wrote: > > > > > And for the

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-20 Thread John Garry
On 19/03/2020 18:43, Jean-Philippe Brucker wrote: On Thu, Mar 19, 2020 at 12:54:59PM +, John Garry wrote: Hi Will, On Thu, Jan 02, 2020 at 05:44:39PM +, John Garry wrote: And for the overall system, we have: PerfTop: 85864 irqs/sec kernel:89.6% exact: 0.0% lost: 0/34434

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-19 Thread Jean-Philippe Brucker
On Thu, Mar 19, 2020 at 12:54:59PM +, John Garry wrote: > Hi Will, > > > > > On Thu, Jan 02, 2020 at 05:44:39PM +, John Garry wrote: > > > And for the overall system, we have: > > > > > >PerfTop: 85864 irqs/sec kernel:89.6% exact: 0.0% lost: 0/34434 > > > drop: > > > 0/40116

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-19 Thread John Garry
Hi Will, On Thu, Jan 02, 2020 at 05:44:39PM +, John Garry wrote: And for the overall system, we have: PerfTop: 85864 irqs/sec kernel:89.6% exact: 0.0% lost: 0/34434 drop: 0/40116 [4000Hz cycles], (all, 96 CPUs)

Re: arm-smmu-v3 high cpu usage for NVMe

2020-03-18 Thread Will Deacon
Hi John, On Thu, Jan 02, 2020 at 05:44:39PM +, John Garry wrote: > And for the overall system, we have: > > PerfTop: 85864 irqs/sec kernel:89.6% exact: 0.0% lost: 0/34434 drop: > 0/40116 [4000Hz cycles], (all, 96 CPUs) >