Hi Michal,
Did you get any chance to run the evaluation with this patchset?
Thanks, Anand
On 1/30/2021 9:08 AM, Anand Jain wrote:
500m is really small data size for such measurement
I reran the read policy tests with some changes in the fio command
options. Mainly to measure IOPS throughput and latency on the filesystem
with latency-policy and pid-policy.
Each of these tests was run for 3 iterations and the best and worst of
those 3 iterations are shown below.
These workloads are performing read-write which is the most commonly
used workload, on a single type of device (which is nvme here) and two
devices are configured for RAID1.
In all these read-write workloads, pid-policy performed ~25% better than
the latency-policy for both throughput and IOPS, and 3% better on the
latency parameter.
I haven't analyzed these read-write workloads on RAID1c3/RAID1c4 yet,
but RAID1 is more common than other types, IMO.
So I think pid-policy should remain as our default read policy.
However as shown before, pid-policy perform worst in the case of special
configs such as volumes with mixed types of devices. For those special
mixed types of devices, latency-policy performs better than pid-policy.
As tested before typically latency-policy provided ~175% better
throughput performance in the case of mixed types of devices (SSD and
nvme).
Feedbacks welcome.
Fio logs below.
IOPS focused readwrite workload:
fio --filename=/btrfs/foo --size=500GB --direct=1 --rw=randrw --bs=4k
--ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based
--group_reporting --name=iops-randomreadwrite --eta-newline=1
pid [latency] device roundrobin ( 00)
read: IOPS=40.6k, BW=159MiB/s (166MB/s)(18.6GiB/120002msec)
[pid] latency device roundrobin ( 00)
read: IOPS=50.7k, BW=198MiB/s (208MB/s)(23.2GiB/120001msec)
IOPS is 25% better with pid policy.
Throughput focused readwrite workload:
fio --filename=/btrfs/foo --size=500GB --direct=1 --rw=randrw --bs=64k
--ioengine=libaio --iodepth=64 --runtime=120 --numjobs=4 --time_based
--group_reporting --name=throughput-randomreadwrite --eta-newline=1
pid [latency] device roundrobin ( 00)
read: IOPS=8525, BW=533MiB/s (559MB/s)(62.4GiB/120003msec)
[pid] latency device roundrobin ( 00)
read: IOPS=10.7k, BW=670MiB/s (702MB/s)(78.5GiB/120005msec)
Throughput is 25% better with pid policy
Latency focused readwrite workload:
fio --filename=/btrfs/foo --size=500GB --direct=1 --rw=randrw --bs=4k
--ioengine=libaio --iodepth=1 --runtime=120 --numjobs=4 --time_based
--group_reporting --name=latency-randomreadwrite --eta-newline=1
pid [latency] device roundrobin ( 00)
read: IOPS=59.8k, BW=234MiB/s (245MB/s)(27.4GiB/120003msec)
lat (usec): min=68, max=826930, avg=1917.20, stdev=4210.90
[pid] latency device roundrobin ( 00)
read: IOPS=61.9k, BW=242MiB/s (253MB/s)(28.3GiB/120001msec)
lat (usec): min=64, max=751557, avg=1846.07, stdev=4082.97
Latency is 3% better with pid policy.
