Dear Experts,
I'm observing strange behavior on a cluster 2.1.20 during compactions. My setup is: 12 nodes m4.2xlarge (8 vCPU, 32G RAM) Ubuntu 16.04, 2T EBS gp2. Filesystem: XFS, blocksize 4k, device read-ahead - 4k /sys/block/vxdb/queue/nomerges = 0 SizeTieredCompactionStrategy After data loads when effectively nothing else is talking to the cluster and compactions is the only activity, I see something like this: $ iostat -dkx 1 ... Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util xvda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 xvdb 0.00 0.00 4769.00 213.00 19076.00 26820.00 18.42 7.95 1.17 1.06 3.76 0.20 100.00 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util xvda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 xvdb 0.00 0.00 6098.00 177.00 24392.00 22076.00 14.81 6.46 1.36 0.96 15.16 0.16 100.00 Writes are fine: 177 writes/sec <-> ~22Mbytes/sec, But for some reason compactions generate a huge number of small reads: 6098 reads/s <-> ~24Mbytes/sec. ===> Read size is 4k Why instead much smaller amount of large reads I'm getting huge number of 4k reads instead? What could be the reason? Thanks, Kyrill
