Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Fri, Jan 2, 2015 at 3:22 AM, Ming Lei wrote:
> On Thu, Jan 1, 2015 at 11:14 PM, Sedat Dilek wrote:
>> On Thu, Jan 1, 2015 at 1:29 AM, Ming Lei wrote:
>> [...]
How did you test with fio (your fio lines)?
>>>
>>> Your fio command line is basically same with my fio config, and you
>>> can attach one image to loop via: losetup -f file_name. Looks your
>>> randread result is good, and I can observe ~80 IOPS vs. ~200 IOPS
>>> on my slow HDD. in the randread test too.
>>>
>>> #fio config##
>>> [global]
>>> direct=1
>>> size=128G
>>> bsrange=4k-4k
>>> timeout=30
>>> numjobs=1
>>> ioengine=libaio
>>> iodepth=64
>>> filename=/dev/loop0
>>> group_reporting=1
>>>
>>> [f]
>>> rw=${RW}
>>>
>>
>> Thanks for your fio config-file.
>>
>> When a 1GiB is created on my ext4-system, the IOPS raised up from 515 to 981.
>>
>> # egrep 'iops=|Laying out IO file' fio-test*next20141231-1-loopmq-small*
>> fio-test1-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
>> io=1024.0MB, bw=2063.4KB/s, iops=515 , runt=508182msec
>> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
>> out IO file(s) (1 file(s) / 1024MB)
>> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
>> io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
>>
>> I made some more testing with next-20141231 plus block-loop-mq-v3 and
>> block.git#for-linus on top (see attached full patch).
>
> I will submit v4 after block.git#for-linus is merged for avoiding
> the conflict, and blk_mq_freeze_queue() still need to be
> exported.
>
>>
>> This increases IOPS from 981 up to 2103.
>
> Looks there isn't fancy patch for performance boost in block.git#for-linus,
> I am wondering it may be caused by page cache, and before each test,
> I always drop cache via 'echo 3 > /proc/sys/vm/drop_caches' for avoiding
> the effect.
>
OK, good to know.
I played a bit with the --size option, below 1GiB I get some fantastic results.
But I will retest with dropped-caches.
>>
>> # egrep 'iops=|Laying out IO file'
>> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt
>> fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt
>> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
>> out IO file(s) (1 file(s) / 1024MB)
>> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
>> io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
>> fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt:randread: Laying
>> out IO file(s) (1 file(s) / 1024MB)
>> fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt: read :
>> io=1024.0MB, bw=8414.9KB/s, iops=2103 , runt=124611msec
>>
>> If you like you can have a closer look at the fio benchmark results,
>
> Anyway both us can observe big improvement on randread about
> the patch.
>
>> I also have added two patches which made it easy for me to cope with
>> the merge conflicts within block/loop stuff.
>
> Thanks for your report on this conflict.
>
BTW, I played also with plain block-linus and your patchset on top
against a vanilla Linux v3.19-rc2.
v4 much appreciated.
Feel free to add my Tested-by.
- Sedat -
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Thu, Jan 1, 2015 at 11:14 PM, Sedat Dilek wrote:
> On Thu, Jan 1, 2015 at 1:29 AM, Ming Lei wrote:
> [...]
>>> How did you test with fio (your fio lines)?
>>
>> Your fio command line is basically same with my fio config, and you
>> can attach one image to loop via: losetup -f file_name. Looks your
>> randread result is good, and I can observe ~80 IOPS vs. ~200 IOPS
>> on my slow HDD. in the randread test too.
>>
>> #fio config##
>> [global]
>> direct=1
>> size=128G
>> bsrange=4k-4k
>> timeout=30
>> numjobs=1
>> ioengine=libaio
>> iodepth=64
>> filename=/dev/loop0
>> group_reporting=1
>>
>> [f]
>> rw=${RW}
>>
>
> Thanks for your fio config-file.
>
> When a 1GiB is created on my ext4-system, the IOPS raised up from 515 to 981.
>
> # egrep 'iops=|Laying out IO file' fio-test*next20141231-1-loopmq-small*
> fio-test1-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
> io=1024.0MB, bw=2063.4KB/s, iops=515 , runt=508182msec
> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
> out IO file(s) (1 file(s) / 1024MB)
> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
> io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
>
> I made some more testing with next-20141231 plus block-loop-mq-v3 and
> block.git#for-linus on top (see attached full patch).
I will submit v4 after block.git#for-linus is merged for avoiding
the conflict, and blk_mq_freeze_queue() still need to be
exported.
>
> This increases IOPS from 981 up to 2103.
Looks there isn't fancy patch for performance boost in block.git#for-linus,
I am wondering it may be caused by page cache, and before each test,
I always drop cache via 'echo 3 > /proc/sys/vm/drop_caches' for avoiding
the effect.
>
> # egrep 'iops=|Laying out IO file'
> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt
> fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt
> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
> out IO file(s) (1 file(s) / 1024MB)
> fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
> io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
> fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt:randread: Laying
> out IO file(s) (1 file(s) / 1024MB)
> fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt: read :
> io=1024.0MB, bw=8414.9KB/s, iops=2103 , runt=124611msec
>
> If you like you can have a closer look at the fio benchmark results,
Anyway both us can observe big improvement on randread about
the patch.
> I also have added two patches which made it easy for me to cope with
> the merge conflicts within block/loop stuff.
Thanks for your report on this conflict.
Thanks,
Ming Lei
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Fri, Jan 2, 2015 at 3:22 AM, Ming Lei tom.leim...@gmail.com wrote:
On Thu, Jan 1, 2015 at 11:14 PM, Sedat Dilek sedat.di...@gmail.com wrote:
On Thu, Jan 1, 2015 at 1:29 AM, Ming Lei tom.leim...@gmail.com wrote:
[...]
How did you test with fio (your fio lines)?
Your fio command line is basically same with my fio config, and you
can attach one image to loop via: losetup -f file_name. Looks your
randread result is good, and I can observe ~80 IOPS vs. ~200 IOPS
on my slow HDD. in the randread test too.
#fio config##
[global]
direct=1
size=128G
bsrange=4k-4k
timeout=30
numjobs=1
ioengine=libaio
iodepth=64
filename=/dev/loop0
group_reporting=1
[f]
rw=${RW}
Thanks for your fio config-file.
When a 1GiB is created on my ext4-system, the IOPS raised up from 515 to 981.
# egrep 'iops=|Laying out IO file' fio-test*next20141231-1-loopmq-small*
fio-test1-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
io=1024.0MB, bw=2063.4KB/s, iops=515 , runt=508182msec
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
out IO file(s) (1 file(s) / 1024MB)
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
I made some more testing with next-20141231 plus block-loop-mq-v3 and
block.git#for-linus on top (see attached full patch).
I will submit v4 after block.git#for-linus is merged for avoiding
the conflict, and blk_mq_freeze_queue() still need to be
exported.
This increases IOPS from 981 up to 2103.
Looks there isn't fancy patch for performance boost in block.git#for-linus,
I am wondering it may be caused by page cache, and before each test,
I always drop cache via 'echo 3 /proc/sys/vm/drop_caches' for avoiding
the effect.
OK, good to know.
I played a bit with the --size option, below 1GiB I get some fantastic results.
But I will retest with dropped-caches.
# egrep 'iops=|Laying out IO file'
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt
fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
out IO file(s) (1 file(s) / 1024MB)
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt:randread: Laying
out IO file(s) (1 file(s) / 1024MB)
fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt: read :
io=1024.0MB, bw=8414.9KB/s, iops=2103 , runt=124611msec
If you like you can have a closer look at the fio benchmark results,
Anyway both us can observe big improvement on randread about
the patch.
I also have added two patches which made it easy for me to cope with
the merge conflicts within block/loop stuff.
Thanks for your report on this conflict.
BTW, I played also with plain block-linus and your patchset on top
against a vanilla Linux v3.19-rc2.
v4 much appreciated.
Feel free to add my Tested-by.
- Sedat -
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Thu, Jan 1, 2015 at 11:14 PM, Sedat Dilek sedat.di...@gmail.com wrote:
On Thu, Jan 1, 2015 at 1:29 AM, Ming Lei tom.leim...@gmail.com wrote:
[...]
How did you test with fio (your fio lines)?
Your fio command line is basically same with my fio config, and you
can attach one image to loop via: losetup -f file_name. Looks your
randread result is good, and I can observe ~80 IOPS vs. ~200 IOPS
on my slow HDD. in the randread test too.
#fio config##
[global]
direct=1
size=128G
bsrange=4k-4k
timeout=30
numjobs=1
ioengine=libaio
iodepth=64
filename=/dev/loop0
group_reporting=1
[f]
rw=${RW}
Thanks for your fio config-file.
When a 1GiB is created on my ext4-system, the IOPS raised up from 515 to 981.
# egrep 'iops=|Laying out IO file' fio-test*next20141231-1-loopmq-small*
fio-test1-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
io=1024.0MB, bw=2063.4KB/s, iops=515 , runt=508182msec
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
out IO file(s) (1 file(s) / 1024MB)
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
I made some more testing with next-20141231 plus block-loop-mq-v3 and
block.git#for-linus on top (see attached full patch).
I will submit v4 after block.git#for-linus is merged for avoiding
the conflict, and blk_mq_freeze_queue() still need to be
exported.
This increases IOPS from 981 up to 2103.
Looks there isn't fancy patch for performance boost in block.git#for-linus,
I am wondering it may be caused by page cache, and before each test,
I always drop cache via 'echo 3 /proc/sys/vm/drop_caches' for avoiding
the effect.
# egrep 'iops=|Laying out IO file'
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt
fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt:randread: Laying
out IO file(s) (1 file(s) / 1024MB)
fio-test2-3.19.0-rc2-next20141231-1-loopmq-small.txt: read :
io=1024.0MB, bw=3924.3KB/s, iops=981 , runt=267206msec
fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt:randread: Laying
out IO file(s) (1 file(s) / 1024MB)
fio-test2-3.19.0-rc2-next20141231-3-loopmq-small.txt: read :
io=1024.0MB, bw=8414.9KB/s, iops=2103 , runt=124611msec
If you like you can have a closer look at the fio benchmark results,
Anyway both us can observe big improvement on randread about
the patch.
I also have added two patches which made it easy for me to cope with
the merge conflicts within block/loop stuff.
Thanks for your report on this conflict.
Thanks,
Ming Lei
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Thu, Jan 1, 2015 at 8:18 AM, Sedat Dilek wrote:
> On Thu, Jan 1, 2015 at 1:01 AM, Ming Lei wrote:
>> Hi Sedat,
>>
>> On Thu, Jan 1, 2015 at 6:32 AM, Sedat Dilek wrote:
>>> Forgot to CC LKML and linux-fsdevel.
>>>
>>> - Sedat -
>>
>>>
>>> OK, I have installed fio (1.59-1) and libaio1 (0.3.109-2ubuntu1) here.
>>>
>>> You say in [1]:
>>>
>>> "In the following test:
>>> - base: v3.19-rc2-2041231
>>> - loop over file in ext4 file system on SSD disk
>>> - bs: 4k, libaio, io depth: 64, O_DIRECT, num of jobs: 1
>>> - throughput: IOPS"
>>>
>>> I tried to reproduce that inspired by [2]...
>>>
>>> root# fio --name=randread --rw=randread --bs=4k --ioengine=libaio
>>> --iodepth=64 --direct=1 --numjobs=1 --size=1G
>>>
>>> ...you had no size given (here: 1GiB) - fio requires that parameter to run.
>>>
>>> This results in 165 VS. 515 IOPS here.
>>
>> Thanks for your test.
>>
>> Also if your disk is quick enough, you will observe improvement on
>> read test too.
>>
>
> This is no SSD here.
>
> # dmesg | egrep -i 'hitachi|ata1|sda'
> [0.457892] ata1: SATA max UDMA/133 abar m2048@0xf0708000 port
> 0xf0708100 irq 25
> [0.777445] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [0.778759] ata1.00: ATA-8: Hitachi HTS545050A7E380, GG2OA6C0, max UDMA/133
> [0.778778] ata1.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32),
> AA
> [0.780154] ata1.00: configured for UDMA/133
> [0.780970] scsi 0:0:0:0: Direct-Access ATA Hitachi
> HTS54505 A6C0 PQ: 0 ANSI: 5
> [0.782050] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks:
> (500 GB/465 GiB)
> [0.782058] sd 0:0:0:0: [sda] 4096-byte physical blocks
> [0.782255] sd 0:0:0:0: [sda] Write Protect is off
> [0.782262] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
> [0.782339] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
> enabled, doesn't support DPO or FUA
> [0.800644] sda: sda1 sda2 sda3
> [0.802029] sd 0:0:0:0: [sda] Attached SCSI disk
>
> How did you test with fio (your fio lines)?
Your fio command line is basically same with my fio config, and you
can attach one image to loop via: losetup -f file_name. Looks your
randread result is good, and I can observe ~80 IOPS vs. ~200 IOPS
on my slow HDD. in the randread test too.
#fio config##
[global]
direct=1
size=128G
bsrange=4k-4k
timeout=30
numjobs=1
ioengine=libaio
iodepth=64
filename=/dev/loop0
group_reporting=1
[f]
rw=${RW}
Thanks,
Ming Lei
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Thu, Jan 1, 2015 at 1:01 AM, Ming Lei wrote: > Hi Sedat, > > On Thu, Jan 1, 2015 at 6:32 AM, Sedat Dilek wrote: >> Forgot to CC LKML and linux-fsdevel. >> >> - Sedat - > >> >> OK, I have installed fio (1.59-1) and libaio1 (0.3.109-2ubuntu1) here. >> >> You say in [1]: >> >> "In the following test: >> - base: v3.19-rc2-2041231 >> - loop over file in ext4 file system on SSD disk >> - bs: 4k, libaio, io depth: 64, O_DIRECT, num of jobs: 1 >> - throughput: IOPS" >> >> I tried to reproduce that inspired by [2]... >> >> root# fio --name=randread --rw=randread --bs=4k --ioengine=libaio >> --iodepth=64 --direct=1 --numjobs=1 --size=1G >> >> ...you had no size given (here: 1GiB) - fio requires that parameter to run. >> >> This results in 165 VS. 515 IOPS here. > > Thanks for your test. > > Also if your disk is quick enough, you will observe improvement on > read test too. > This is no SSD here. # dmesg | egrep -i 'hitachi|ata1|sda' [0.457892] ata1: SATA max UDMA/133 abar m2048@0xf0708000 port 0xf0708100 irq 25 [0.777445] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [0.778759] ata1.00: ATA-8: Hitachi HTS545050A7E380, GG2OA6C0, max UDMA/133 [0.778778] ata1.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA [0.780154] ata1.00: configured for UDMA/133 [0.780970] scsi 0:0:0:0: Direct-Access ATA Hitachi HTS54505 A6C0 PQ: 0 ANSI: 5 [0.782050] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB) [0.782058] sd 0:0:0:0: [sda] 4096-byte physical blocks [0.782255] sd 0:0:0:0: [sda] Write Protect is off [0.782262] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00 [0.782339] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [0.800644] sda: sda1 sda2 sda3 [0.802029] sd 0:0:0:0: [sda] Attached SCSI disk How did you test with fio (your fio lines)? - Sedat - >> # grep "iops=" test-* >> test-1-next20141231.txt: read : io=1024.0MB, bw=678578 B/s, iops=165 >> , runt=1582340msec >> test-2-next20141231-block-mq-v3.txt: read : io=1024.0MB, >> bw=2063.4KB/s, iops=515 , runt=508182msec >> >> Full fio-logs and some other useful configs/logs/patches attached. >> >> - Sedat - >> >> [1] http://marc.info/?l=linux-kernel=142003220301459=2 >> [2] >> http://wiki.mikejung.biz/Benchmarking#Fio_Random_Write_Test_using_libaio_and_direct_flags > > > > Thanks, > Ming Lei -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
Hi Sedat, On Thu, Jan 1, 2015 at 6:32 AM, Sedat Dilek wrote: > Forgot to CC LKML and linux-fsdevel. > > - Sedat - > > OK, I have installed fio (1.59-1) and libaio1 (0.3.109-2ubuntu1) here. > > You say in [1]: > > "In the following test: > - base: v3.19-rc2-2041231 > - loop over file in ext4 file system on SSD disk > - bs: 4k, libaio, io depth: 64, O_DIRECT, num of jobs: 1 > - throughput: IOPS" > > I tried to reproduce that inspired by [2]... > > root# fio --name=randread --rw=randread --bs=4k --ioengine=libaio > --iodepth=64 --direct=1 --numjobs=1 --size=1G > > ...you had no size given (here: 1GiB) - fio requires that parameter to run. > > This results in 165 VS. 515 IOPS here. Thanks for your test. Also if your disk is quick enough, you will observe improvement on read test too. > # grep "iops=" test-* > test-1-next20141231.txt: read : io=1024.0MB, bw=678578 B/s, iops=165 > , runt=1582340msec > test-2-next20141231-block-mq-v3.txt: read : io=1024.0MB, > bw=2063.4KB/s, iops=515 , runt=508182msec > > Full fio-logs and some other useful configs/logs/patches attached. > > - Sedat - > > [1] http://marc.info/?l=linux-kernel=142003220301459=2 > [2] > http://wiki.mikejung.biz/Benchmarking#Fio_Random_Write_Test_using_libaio_and_direct_flags Thanks, Ming Lei -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v3 0/5] block: loop: convert to blk-mq
Hi, The following 5 patches convert current loop driver into blk-mq: - loop's throughput of read and randread can get improved much because of the work queue implementation, also there is opportunity for blk-mq to merge I/O from loop block - loop driver gets quite simplified with blk-mq because there is the whole request based framework and something like flush mechanism, and the conversion can be throught as cleanup V3: - rebased on v3.19-rc2-next20141231 - submit I/O via workqueue - don't handle write requests concurrently - set flag of BLK_MQ_F_SG_MERGE - fix failure path in patch 2/5 V2: - replace work queue with kthread worker because wq may cause regression on some fast devices(such as loop over null_blk, tmpfs file, ...), in these cases, loop thread becomes CPU bound actually, so work queue will introduce lots of unnecessary context switch, and decrease throughput a lot one this cases - introduce prepare_flush_rq_fn callback and its pair in blk-mq ops for setting up flush rq's pdu correctly - move kernel aio patches into another patchset as suggested by Christoph - no change in the other 4 patches V1: - improve failure path in aio_kernel_submit() block/blk-mq.c |4 +- block/blk-mq.h |1 - drivers/block/loop.c | 417 drivers/block/loop.h | 17 +- include/linux/blk-mq.h |2 + 5 files changed, 221 insertions(+), 220 deletions(-) Thanks, Ming Lei -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
Hi Sedat, On Thu, Jan 1, 2015 at 6:32 AM, Sedat Dilek sedat.di...@gmail.com wrote: Forgot to CC LKML and linux-fsdevel. - Sedat - OK, I have installed fio (1.59-1) and libaio1 (0.3.109-2ubuntu1) here. You say in [1]: In the following test: - base: v3.19-rc2-2041231 - loop over file in ext4 file system on SSD disk - bs: 4k, libaio, io depth: 64, O_DIRECT, num of jobs: 1 - throughput: IOPS I tried to reproduce that inspired by [2]... root# fio --name=randread --rw=randread --bs=4k --ioengine=libaio --iodepth=64 --direct=1 --numjobs=1 --size=1G ...you had no size given (here: 1GiB) - fio requires that parameter to run. This results in 165 VS. 515 IOPS here. Thanks for your test. Also if your disk is quick enough, you will observe improvement on read test too. # grep iops= test-* test-1-next20141231.txt: read : io=1024.0MB, bw=678578 B/s, iops=165 , runt=1582340msec test-2-next20141231-block-mq-v3.txt: read : io=1024.0MB, bw=2063.4KB/s, iops=515 , runt=508182msec Full fio-logs and some other useful configs/logs/patches attached. - Sedat - [1] http://marc.info/?l=linux-kernelm=142003220301459w=2 [2] http://wiki.mikejung.biz/Benchmarking#Fio_Random_Write_Test_using_libaio_and_direct_flags Thanks, Ming Lei -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Thu, Jan 1, 2015 at 1:01 AM, Ming Lei tom.leim...@gmail.com wrote: Hi Sedat, On Thu, Jan 1, 2015 at 6:32 AM, Sedat Dilek sedat.di...@gmail.com wrote: Forgot to CC LKML and linux-fsdevel. - Sedat - OK, I have installed fio (1.59-1) and libaio1 (0.3.109-2ubuntu1) here. You say in [1]: In the following test: - base: v3.19-rc2-2041231 - loop over file in ext4 file system on SSD disk - bs: 4k, libaio, io depth: 64, O_DIRECT, num of jobs: 1 - throughput: IOPS I tried to reproduce that inspired by [2]... root# fio --name=randread --rw=randread --bs=4k --ioengine=libaio --iodepth=64 --direct=1 --numjobs=1 --size=1G ...you had no size given (here: 1GiB) - fio requires that parameter to run. This results in 165 VS. 515 IOPS here. Thanks for your test. Also if your disk is quick enough, you will observe improvement on read test too. This is no SSD here. # dmesg | egrep -i 'hitachi|ata1|sda' [0.457892] ata1: SATA max UDMA/133 abar m2048@0xf0708000 port 0xf0708100 irq 25 [0.777445] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [0.778759] ata1.00: ATA-8: Hitachi HTS545050A7E380, GG2OA6C0, max UDMA/133 [0.778778] ata1.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA [0.780154] ata1.00: configured for UDMA/133 [0.780970] scsi 0:0:0:0: Direct-Access ATA Hitachi HTS54505 A6C0 PQ: 0 ANSI: 5 [0.782050] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB) [0.782058] sd 0:0:0:0: [sda] 4096-byte physical blocks [0.782255] sd 0:0:0:0: [sda] Write Protect is off [0.782262] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00 [0.782339] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [0.800644] sda: sda1 sda2 sda3 [0.802029] sd 0:0:0:0: [sda] Attached SCSI disk How did you test with fio (your fio lines)? - Sedat - # grep iops= test-* test-1-next20141231.txt: read : io=1024.0MB, bw=678578 B/s, iops=165 , runt=1582340msec test-2-next20141231-block-mq-v3.txt: read : io=1024.0MB, bw=2063.4KB/s, iops=515 , runt=508182msec Full fio-logs and some other useful configs/logs/patches attached. - Sedat - [1] http://marc.info/?l=linux-kernelm=142003220301459w=2 [2] http://wiki.mikejung.biz/Benchmarking#Fio_Random_Write_Test_using_libaio_and_direct_flags Thanks, Ming Lei -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v3 0/5] block: loop: convert to blk-mq
On Thu, Jan 1, 2015 at 8:18 AM, Sedat Dilek sedat.di...@gmail.com wrote:
On Thu, Jan 1, 2015 at 1:01 AM, Ming Lei tom.leim...@gmail.com wrote:
Hi Sedat,
On Thu, Jan 1, 2015 at 6:32 AM, Sedat Dilek sedat.di...@gmail.com wrote:
Forgot to CC LKML and linux-fsdevel.
- Sedat -
OK, I have installed fio (1.59-1) and libaio1 (0.3.109-2ubuntu1) here.
You say in [1]:
In the following test:
- base: v3.19-rc2-2041231
- loop over file in ext4 file system on SSD disk
- bs: 4k, libaio, io depth: 64, O_DIRECT, num of jobs: 1
- throughput: IOPS
I tried to reproduce that inspired by [2]...
root# fio --name=randread --rw=randread --bs=4k --ioengine=libaio
--iodepth=64 --direct=1 --numjobs=1 --size=1G
...you had no size given (here: 1GiB) - fio requires that parameter to run.
This results in 165 VS. 515 IOPS here.
Thanks for your test.
Also if your disk is quick enough, you will observe improvement on
read test too.
This is no SSD here.
# dmesg | egrep -i 'hitachi|ata1|sda'
[0.457892] ata1: SATA max UDMA/133 abar m2048@0xf0708000 port
0xf0708100 irq 25
[0.777445] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[0.778759] ata1.00: ATA-8: Hitachi HTS545050A7E380, GG2OA6C0, max UDMA/133
[0.778778] ata1.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32),
AA
[0.780154] ata1.00: configured for UDMA/133
[0.780970] scsi 0:0:0:0: Direct-Access ATA Hitachi
HTS54505 A6C0 PQ: 0 ANSI: 5
[0.782050] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks:
(500 GB/465 GiB)
[0.782058] sd 0:0:0:0: [sda] 4096-byte physical blocks
[0.782255] sd 0:0:0:0: [sda] Write Protect is off
[0.782262] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[0.782339] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[0.800644] sda: sda1 sda2 sda3
[0.802029] sd 0:0:0:0: [sda] Attached SCSI disk
How did you test with fio (your fio lines)?
Your fio command line is basically same with my fio config, and you
can attach one image to loop via: losetup -f file_name. Looks your
randread result is good, and I can observe ~80 IOPS vs. ~200 IOPS
on my slow HDD. in the randread test too.
#fio config##
[global]
direct=1
size=128G
bsrange=4k-4k
timeout=30
numjobs=1
ioengine=libaio
iodepth=64
filename=/dev/loop0
group_reporting=1
[f]
rw=${RW}
Thanks,
Ming Lei
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v3 0/5] block: loop: convert to blk-mq
Hi, The following 5 patches convert current loop driver into blk-mq: - loop's throughput of read and randread can get improved much because of the work queue implementation, also there is opportunity for blk-mq to merge I/O from loop block - loop driver gets quite simplified with blk-mq because there is the whole request based framework and something like flush mechanism, and the conversion can be throught as cleanup V3: - rebased on v3.19-rc2-next20141231 - submit I/O via workqueue - don't handle write requests concurrently - set flag of BLK_MQ_F_SG_MERGE - fix failure path in patch 2/5 V2: - replace work queue with kthread worker because wq may cause regression on some fast devices(such as loop over null_blk, tmpfs file, ...), in these cases, loop thread becomes CPU bound actually, so work queue will introduce lots of unnecessary context switch, and decrease throughput a lot one this cases - introduce prepare_flush_rq_fn callback and its pair in blk-mq ops for setting up flush rq's pdu correctly - move kernel aio patches into another patchset as suggested by Christoph - no change in the other 4 patches V1: - improve failure path in aio_kernel_submit() block/blk-mq.c |4 +- block/blk-mq.h |1 - drivers/block/loop.c | 417 drivers/block/loop.h | 17 +- include/linux/blk-mq.h |2 + 5 files changed, 221 insertions(+), 220 deletions(-) Thanks, Ming Lei -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
