On 11/10/26 9:53 AM, Robert Webb wrote:
Not new to tape storage, but it has been many years since I have worked with 
them.

Just set up a new Bacula deployment and ran a large backup of around 45TB's. Files are mostly large video files and then a mix of of other files like photos. Noticed the backup settled out at around 114MB/s which I figured was good for what I was backing up.

Now trying to do a full restore and was perplexed as to why the restore was a bit slower.  Went through a lot of diagnosis on the Proxmox server setup with its ZFS storage, added a SLOG, and ultimately decided that storage was not the bottleneck, nor was the 10G network connection.

Through more reading, it seems that the default 64K block size being used to write to the LTO9 drive has a lot of overhead and probably causing the slower backups and restores. So what is the better minimum and maximum black sizes? I have read 256k to 512k is good for minimums and 1M to 2M for maximum. Looking for some confirmations before I make any changes and do another full backup once all the restores are complete.


Thanks...

Hello Robert,

I have done a lot of testing in Quantum's lab to certify Bacula Enterprise with 
their LTO8/9/10 drives.

A few years ago, after running the attached script, the best overall settings for the MaximumFileSize and MaximumBlockSize for SD Device settings were as follows:
----8<----
MaximumFileSize = 32GB
MaximumBlockSize = 2097152
----8<----

I let this script run over a weekend with more `file_sizes=` and `block_sizes=` variable settings than are in the attached script (edit as needed).

In the end, a colleague of mine took the results and imported them into a spreadsheet which generated tables and graphs and those settings where the overall best for LTO8/9. The performance was just as good when I was testing LTO10 this summer.

If you run this script and come up with different results (or the same ones :), 
I'd love to hear back from you!


Note: MinimumBlockSize should never be set!


Hope this helps,
Bill
--
Bill Arlofski
[email protected]
#!/bin/bash
#
# bill - 20220208 - Automatically tests all combinations of as
#                   many file size and block size settings that
#                   are added to the file_size and block_size
#                   variables.
#
#                 - Should be run as:
#
#                   # time ./tape_speed_tests.sh | tee /tmp/tapetest.log
#
# bill - 20240306 - Be sure to stop the Bacula SD, or to temporarily
#                   disable the drive you will use for testing.
#                   In bconsole, you can do this with:

#                   * disable storage=xxxx drive=0
# ----------------------------------------------------------------------
#
# Set theses three variables to the
# correct settings for your SD's device
# with a blank tape in it. VERY IMPORTANT!
# This script will BLINDLY use the devices
# you specify here. Make sure they are
# correct to avoid data loss!
# ----------------------------------------
drive_index="0"
changer_device="/dev/tape/by-id/xxxxxxxxxxxx"
archive_device="/dev/tape/by-id/yyyyyyyyyyyy-nst"

# Set these to the sizes you wish to test
# ---------------------------------------
# file_sizes="8 12 16 32"
# block_sizes="129024 131072 258048 262144 516096 524288 1048576 2097152"
# Limit the number of file and block sizes to test to speed up the process
# ------------------------------------------------------------------------
file_sizes="16 32"
block_sizes="516096 524288 1048576 2097152"

# ==================================================
# Nothing should need to be modified below this line
# ==================================================

now=$(date +%Y%m%d%H%M)
bt="/opt/bacula/bin/btape"
config="/tmp/BtapeTestDrive.cfg"
drive_device="BtapeTestDrive"

echo -e "\n-------------------"
echo "${now}"
echo -e "-------------------"
echo "Testing of drive device ${drive_device} started..."
for fs in ${file_sizes}; do
    for bs in ${block_sizes}; do
        echo -e "\n-------------------"
        # Create the required temporary config file
        # -----------------------------------------
        echo "Creating temporary config file ${config} using file size: ${fs}, block size: ${bs}"
        echo "Storage {" > ${config}
        echo "  Name = BtapeTestSD" >> ${config}
        echo "  PidDirectory = /tmp" >> ${config}
        echo "  WorkingDirectory = /tmp" >> ${config}
        echo "}" >> ${config}
        echo "Autochanger {" >> ${config}
        echo "  Name = BtapeTestLib" >> ${config}
        echo "  ChangerCommand = \"/opt/bacula/scripts/mtx-changer %c %o %S %a %d\"" >> ${config}
        echo "  ChangerDevice = ${changer_device}" >> ${config}
        echo "  Device = ${drive_device}" >> ${config}
        echo "}" >> ${config}
        echo "Device {" >> ${config}
        echo "  Name = ${drive_device}" >> ${config}
        echo "  DeviceType = Tape" >> ${config}
        echo "  DriveIndex = ${drive_index}" >> ${config}
        echo "  ArchiveDevice = ${archive_device}" >> ${config}
        echo "  AlwaysOpen = yes" >> ${config}
        echo "  MediaType = TAPE" >> ${config}
        echo "  Autochanger = yes" >> ${config}
        echo "  RandomAccess = no" >> ${config}
        echo "  RemovableMedia = yes" >> ${config}
        echo "  AutomaticMount = yes" >> ${config}
        echo "  MaximumFileSize = ${fs}GB" >> ${config}
        echo "  MaximumBlockSize = ${bs}" >> ${config}
        echo "} " >> ${config}

        echo "Running speed test on tape drive ${drive_device} using file size: ${fs}, block size: ${bs}"
        echo "speed file_size=${fs}" | ${bt} -v -c ${config} ${drive_device}
    done 
done

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Reply via email to