Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 07/04/2017 02:55 PM, Brian King wrote: > On 07/01/2017 11:43 AM, Jens Axboe wrote: >> Now: >> >> http://git.kernel.dk/cgit/linux-block/commit/?h=mq-inflight=87f73ef2b9edb6834001df8f7cb48c7a116e8cd3 >> >>> And updated the branch here: >>> >>>

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 07/03/2017 08:20 PM, Ming Lei wrote: >> Right. So the two modes I'm running in are: >> >> 1. 80 null_blk devices, each with one submit_queue, with one fio job per >> null_blk device, >>so 80 threads total. 80 logical CPUs >> 2. 1 null_blk device, with 80 submit_queues, 80 fio jobs, 80

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 07/01/2017 11:43 AM, Jens Axboe wrote: > Now: > > http://git.kernel.dk/cgit/linux-block/commit/?h=mq-inflight=87f73ef2b9edb6834001df8f7cb48c7a116e8cd3 > >> And updated the branch here: >> >> http://git.kernel.dk/cgit/linux-block/log/?h=mq-inflight >> >> to include that, and be based on top of

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On Sat, Jul 1, 2017 at 10:18 AM, Brian King wrote: > On 06/30/2017 06:26 PM, Jens Axboe wrote: >> On 06/30/2017 05:23 PM, Ming Lei wrote: >>> Hi Bian, >>> >>> On Sat, Jul 1, 2017 at 2:33 AM, Brian King >>> wrote: On 06/30/2017 09:08 AM,

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

Hi Bian, On Sat, Jul 1, 2017 at 2:33 AM, Brian King wrote: > On 06/30/2017 09:08 AM, Jens Axboe wrote: > Compared with the totally percpu approach, this way might help 1:M or > N:M mapping, but won't help 1:1 map(NVMe), when hctx is mapped to > each

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/30/2017 10:17 PM, Jens Axboe wrote: > On 06/30/2017 08:08 AM, Jens Axboe wrote: >> On 06/30/2017 07:05 AM, Brian King wrote: >>> On 06/29/2017 09:17 PM, Jens Axboe wrote: On 06/29/2017 07:20 PM, Ming Lei wrote: > On Fri, Jun 30, 2017 at 2:42 AM, Jens Axboe wrote:

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/30/2017 08:08 AM, Jens Axboe wrote: > On 06/30/2017 07:05 AM, Brian King wrote: >> On 06/29/2017 09:17 PM, Jens Axboe wrote: >>> On 06/29/2017 07:20 PM, Ming Lei wrote: On Fri, Jun 30, 2017 at 2:42 AM, Jens Axboe wrote: > On 06/29/2017 10:00 AM, Jens Axboe wrote:

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/30/2017 06:26 PM, Jens Axboe wrote: > On 06/30/2017 05:23 PM, Ming Lei wrote: >> Hi Bian, >> >> On Sat, Jul 1, 2017 at 2:33 AM, Brian King wrote: >>> On 06/30/2017 09:08 AM, Jens Axboe wrote: >>> Compared with the totally percpu approach, this way might help

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/30/2017 05:23 PM, Ming Lei wrote: > Hi Bian, > > On Sat, Jul 1, 2017 at 2:33 AM, Brian King wrote: >> On 06/30/2017 09:08 AM, Jens Axboe wrote: >> Compared with the totally percpu approach, this way might help 1:M or >> N:M mapping, but won't help 1:1

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/30/2017 09:08 AM, Jens Axboe wrote: Compared with the totally percpu approach, this way might help 1:M or N:M mapping, but won't help 1:1 map(NVMe), when hctx is mapped to each CPU(especially there are huge hw queues on a big system), :-( >>> >>> Not disagreeing with that,

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/30/2017 07:05 AM, Brian King wrote: > On 06/29/2017 09:17 PM, Jens Axboe wrote: >> On 06/29/2017 07:20 PM, Ming Lei wrote: >>> On Fri, Jun 30, 2017 at 2:42 AM, Jens Axboe wrote: On 06/29/2017 10:00 AM, Jens Axboe wrote: > On 06/29/2017 09:58 AM, Jens Axboe wrote:

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/29/2017 09:17 PM, Jens Axboe wrote: > On 06/29/2017 07:20 PM, Ming Lei wrote: >> On Fri, Jun 30, 2017 at 2:42 AM, Jens Axboe wrote: >>> On 06/29/2017 10:00 AM, Jens Axboe wrote: On 06/29/2017 09:58 AM, Jens Axboe wrote: > On 06/29/2017 02:40 AM, Ming Lei wrote:

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On Fri, Jun 30, 2017 at 2:42 AM, Jens Axboe wrote: > On 06/29/2017 10:00 AM, Jens Axboe wrote: >> On 06/29/2017 09:58 AM, Jens Axboe wrote: >>> On 06/29/2017 02:40 AM, Ming Lei wrote: On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: > On 06/28/2017

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On Thu, Jun 29, 2017 at 12:31:05PM -0500, Brian King wrote: > On 06/29/2017 11:25 AM, Ming Lei wrote: > > On Thu, Jun 29, 2017 at 11:58 PM, Jens Axboe wrote: > >> On 06/29/2017 02:40 AM, Ming Lei wrote: > >>> On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: >

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On Thu, Jun 29, 2017 at 11:58 PM, Jens Axboe wrote: > On 06/29/2017 02:40 AM, Ming Lei wrote: >> On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: >>> On 06/28/2017 03:12 PM, Brian King wrote: This patch converts the in_flight counter in struct hd_struct

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/29/2017 07:20 PM, Ming Lei wrote: > On Fri, Jun 30, 2017 at 2:42 AM, Jens Axboe wrote: >> On 06/29/2017 10:00 AM, Jens Axboe wrote: >>> On 06/29/2017 09:58 AM, Jens Axboe wrote: On 06/29/2017 02:40 AM, Ming Lei wrote: > On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/29/2017 10:00 AM, Jens Axboe wrote: > On 06/29/2017 09:58 AM, Jens Axboe wrote: >> On 06/29/2017 02:40 AM, Ming Lei wrote: >>> On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: On 06/28/2017 03:12 PM, Brian King wrote: > This patch converts the in_flight counter

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/29/2017 11:25 AM, Ming Lei wrote: > On Thu, Jun 29, 2017 at 11:58 PM, Jens Axboe wrote: >> On 06/29/2017 02:40 AM, Ming Lei wrote: >>> On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: On 06/28/2017 03:12 PM, Brian King wrote: > This patch

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/29/2017 09:58 AM, Jens Axboe wrote: > On 06/29/2017 02:40 AM, Ming Lei wrote: >> On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: >>> On 06/28/2017 03:12 PM, Brian King wrote: This patch converts the in_flight counter in struct hd_struct from a pair of atomics

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/29/2017 02:40 AM, Ming Lei wrote: > On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: >> On 06/28/2017 03:12 PM, Brian King wrote: >>> This patch converts the in_flight counter in struct hd_struct from a >>> pair of atomics to a pair of percpu counters. This eliminates a

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 05:19 PM, Jens Axboe wrote: > On 06/28/2017 04:07 PM, Brian King wrote: >> On 06/28/2017 04:59 PM, Jens Axboe wrote: >>> On 06/28/2017 03:54 PM, Jens Axboe wrote: On 06/28/2017 03:12 PM, Brian King wrote: > -static inline int part_in_flight(struct hd_struct *part) >

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On Thu, Jun 29, 2017 at 5:49 AM, Jens Axboe wrote: > On 06/28/2017 03:12 PM, Brian King wrote: >> This patch converts the in_flight counter in struct hd_struct from a >> pair of atomics to a pair of percpu counters. This eliminates a couple >> of atomics from the hot path. When

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 04:07 PM, Brian King wrote: > On 06/28/2017 04:59 PM, Jens Axboe wrote: >> On 06/28/2017 03:54 PM, Jens Axboe wrote: >>> On 06/28/2017 03:12 PM, Brian King wrote: -static inline int part_in_flight(struct hd_struct *part) +static inline unsigned long part_in_flight(struct

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 04:59 PM, Jens Axboe wrote: > On 06/28/2017 03:54 PM, Jens Axboe wrote: >> On 06/28/2017 03:12 PM, Brian King wrote: >>> -static inline int part_in_flight(struct hd_struct *part) >>> +static inline unsigned long part_in_flight(struct hd_struct *part) >>> { >>> - return

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 04:49 PM, Jens Axboe wrote: > On 06/28/2017 03:12 PM, Brian King wrote: >> This patch converts the in_flight counter in struct hd_struct from a >> pair of atomics to a pair of percpu counters. This eliminates a couple >> of atomics from the hot path. When running this on a Power

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 03:54 PM, Jens Axboe wrote: > On 06/28/2017 03:12 PM, Brian King wrote: >> -static inline int part_in_flight(struct hd_struct *part) >> +static inline unsigned long part_in_flight(struct hd_struct *part) >> { >> -return atomic_read(>in_flight[0]) + >>

Re: [dm-devel] [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 03:12 PM, Brian King wrote: > This patch converts the in_flight counter in struct hd_struct from a > pair of atomics to a pair of percpu counters. This eliminates a couple > of atomics from the hot path. When running this on a Power system, to > a single null_blk device with 80