On Wed, Mar 31, 2021 at 11:56:21AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana <fdman...@suse.com> > > Currently a full send operation uses the standard btree readahead when > iterating over the subvolume/snapshot btree, which despite bringing good > performance benefits, it could be improved in a few aspects for use cases > such as full send operations, which are guaranteed to visit every node > and leaf of a btree, in ascending and sequential order. The limitations > of that standard btree readahead implementation are the following: > > 1) It only triggers readahead for for leaves that are physically close > to the leaf being read, within a 64K range; > > 2) It only triggers readahead for the next or previous leaves if the > leaf being read is not currently in memory; > > 3) It never triggers readahead for nodes. > > So add a new readahead mode that addresses all these points and use it > for full send operations. > > The following test script was used to measure the improvement on a box > using an average, consumer grade, spinning disk and with 16Gb of ram: > > $ cat test.sh > #!/bin/bash > > DEV=/dev/sdj > MNT=/mnt/sdj > MKFS_OPTIONS="--nodesize 16384" # default, just to be explicit > MOUNT_OPTIONS="-o max_inline=2048" # default, just to be explicit > > mkfs.btrfs -f $MKFS_OPTIONS $DEV > /dev/null > mount $MOUNT_OPTIONS $DEV $MNT > > # Create files with inline data to make it easier and faster to create > # large btrees. > add_files() > { > local total=$1 > local start_offset=$2 > local number_jobs=$3 > local total_per_job=$(($total / $number_jobs)) > > echo "Creating $total new files using $number_jobs jobs" > for ((n = 0; n < $number_jobs; n++)); do > ( > local start_num=$(($start_offset + $n * $total_per_job)) > for ((i = 1; i <= $total_per_job; i++)); do > local file_num=$((start_num + $i)) > local file_path="$MNT/file_${file_num}" > xfs_io -f -c "pwrite -S 0xab 0 2000" $file_path > /dev/null > if [ $? -ne 0 ]; then > echo "Failed creating file $file_path" > break > fi > done > ) & > worker_pids[$n]=$! > done > > wait ${worker_pids[@]} > > sync > echo > echo "btree node/leaf count: $(btrfs inspect-internal dump-tree -t 5 > $DEV | egrep '^(node|leaf) ' | wc -l)" > } > > initial_file_count=500000 > add_files $initial_file_count 0 4 > > echo > echo "Creating first snapshot..." > btrfs subvolume snapshot -r $MNT $MNT/snap1 > > echo > echo "Adding more files..." > add_files $((initial_file_count / 4)) $initial_file_count 4 > > echo > echo "Updating 1/50th of the initial files..." > for ((i = 1; i < $initial_file_count; i += 50)); do > xfs_io -c "pwrite -S 0xcd 0 20" $MNT/file_$i > /dev/null > done > > echo > echo "Creating second snapshot..." > btrfs subvolume snapshot -r $MNT $MNT/snap2 > > umount $MNT > > echo 3 > /proc/sys/vm/drop_caches > blockdev --flushbufs $DEV &> /dev/null > hdparm -F $DEV &> /dev/null > > mount $MOUNT_OPTIONS $DEV $MNT > > echo > echo "Testing full send..." > start=$(date +%s) > btrfs send $MNT/snap1 > /dev/null > end=$(date +%s) > echo > echo "Full send took $((end - start)) seconds" > > umount $MNT > > The durations of the full send operation in seconds were the following: > > Before this change: 217 seconds > After this change: 205 seconds (-5.7%) > > Signed-off-by: Filipe Manana <fdman...@suse.com>
Added to misc-next, thanks.