Hi, I am right now in a configuration phase of a Bareos system with an Autoloader + 2 LTO-8 drives (attached over SAS,hardware compression Enabled) using LTO-8 Type M tapes. i made a d2d2t test using data spooling and tested the system with a backup set of 9.7TB/778,294 files. I am wondering about the spooling speed - it took me 18 hours to spool this nearly 10 TB of files, during that time - while spooling - the bareos-client used only one core nearly 100% all the time an i hat a throughput of about 1,3-1,4 GBit/Sek aka 153,61 MB/s.
So - ist this the top speed i can accomplish? I thought maybey the usage of only one Thread/Core on the client could be the thing that slows me down - if this ist true - how to use more than one core? Is there something else that i could do to speed up? Is maybe the MD5 calculation the bottleneck? Thanks for any idea/help! ...and btw i got also this two warnings although the whole process finaly worked: "bareos-sd JobId 95: Error: backends/generic_tape_device.cc:146 Unable to open device "lto8-0" (/dev/tape/by-id/scsi-35000e111c23a90b5-nst): ERR=No medium found" ?!?! "bareos-sd JobId 95: 3997 Bad alert command: sh -c 'tapeinfo -f /dev/tape/by-id/scsi-35000e111c23a90b8 | grep TapeAlert': ERR=Child exited with code 1." ??!?!? Specs: *Bareos Server: Dual Xeon E5-2697 v2 (24Cores,48 Threads); 128GB Memory; SSD for boot, system, DB; Spooldrive: Hardware RAID-5 (writecache enabled) out of 12x12TB Seagate Exos x14, Intel 10 GBit Network conncetion Ubuntu 18.04.2; Bareos 18.2.5; PostgreSQL version 10.10 *Autoloader with 160 Slots, Barcode reader, filled with LTO-7 Tapes labeled and initialized als LTO-8 Type M, 2 IMB LTO-8 drives (attached over SAS, hardware compression Enabled) *Testclient: Dual Xeon CPU E5-2603 v4 (12 Cores,12 Threads), 64GByre Memory, SSD boot, system; Datadrive to backup is on a hardware RAID 6 (writecache enabled) out of 12 SATA III WD 3,5" 7.2k (512e) drives. Intel 10 GBit Network conncetion Debian 9.7 Stretch Installed the System, optimized PostgreSQL according to pgtune (https://pgtune.leopard.in.ua), labeling and initializing worked; Benchmarks shows that all harddrives are quiet fast anf should be way faster than the above speed! The 10Gbit Connection brings with iperf 9.4 GBits/s Testconfiguration: /bareos-sd.d/ /autochanager/ autochanger-0.conf BOF Autochanger { Name = "neoxl80" Changer Device = /dev/tape/by-id/scsi-35000e111c23a90b8 Device = lto8-0, lto8-1 Changer Command = "/usr/lib/bareos/scripts/mtx-changer %c %o %S %a %d" } EOF /device/ lto8-0.conf BOF Device { Name = "lto8-0" DeviceType = Tape DriveIndex = 0 ArchiveDevice = /dev/tape/by-id/scsi-35000e111c23a90b5-nst MediaType = LTO-8 Check Labels = yes AutoChanger = yes # default: no AutomaticMount = yes # default: no Maximum File Size = 50GB # default: 1000000000 (1GB) Always Open = yes Alert Command = "sh -c 'tapeinfo -f %c | grep TapeAlert'" Removable Media = yes # Media can be removed from the device - like tapes Random Access = no Spool Directory = /120TB/bareosspool Maximum Spool Size = 1109951162777600 # 100TB } EOF lto8-1.conf BOF Device { Name = "lto8-1" DeviceType = Tape DriveIndex = 1 ArchiveDevice = /dev/tape/by-id/scsi-35000e111c23a90bf-nst MediaType = LTO-8 Check Labels = yes AutoChanger = yes # default: no AutomaticMount = yes # default: no Maximum File Size = 50GB # default: 1000000000 (1GB) Always Open = yes Alert Command = "sh -c 'tapeinfo -f %c | grep TapeAlert'" Removable Media = yes # Media can be removed from the device - like tapes Random Access = no Spool Directory = /120TB/bareosspool Maximum Spool Size = 1109951162777600 # 100TB } EOF /bareos-dir.d/ /client/ demeter.conf BOF Client { Name = demeter Address = xxx.xxx.xxx.xxx Password = xxx } EOF /director/ bareos-dir.conf BOF Director { Name = bareos-dir QueryFile = "/usr/lib/bareos/scripts/query.sql" Maximum Concurrent Jobs = 10 Password = "xxxxxxxxxx" # Console password Messages = Daemon Auditing = yes Optimize For Speed = yes } EOF /fileset/ FSetDemeterSharedfolders.conf BOF FileSet { Name = "FSetDemeterSharedfolders" Description = "Backup all regular filesystems, determined by filesystem type." Include { Options { Signature = MD5 # calculate md5 checksum per file One FS = No # change into other filessytems FS Type = btrfs FS Type = ext2 # filesystems of given types will be backed up FS Type = ext3 # others will be ignored FS Type = ext4 FS Type = reiserfs FS Type = jfs FS Type = xfs FS Type = zfs # noatime - if supported by the OS, the read time won't be adapted # this would generate a bunch of writes for no reason on the client machine. noatime = yes } File = /sharedfolders # Datafolder on Demeter } # Things that usually have to be excluded # You have to exclude /var/lib/bareos/storage # on your bareos server Exclude { File = /var/lib/bareos File = /var/lib/bareos/storage File = /proc File = /tmp File = /var/tmp File = /.journal File = /.fsck } } EOF /job/ DemeterTestJob.conf BOF Job { Name = "DemeterTestJob" Type = Backup Level = Full Client = "demeter" FileSet = "FSetDemeterSharedfolders" Storage = "Tape" Pool = "Archiv2d-2" #2days pool :) Messages = "Standard" Spool Data = yes } EOF /pool/ Archiv2d-2.conf BOF Pool { Name = Archiv2d-2 Pool Type = Backup Recycle = yes # Bareos can automatically recycle Volumes AutoPrune = yes # Prune expired volumes Recycle Pool = Scratch Scratch Pool = Scratch Maximum Block Size = 1048576 Volume Retention = 2 days # Maximum Volume Bytes = 15 TB # Limit Volume size to something reasonable Maximum Volumes = 100 } EOF /storage/ Tape.conf BOF Storage { Name = Tape Address = lemgol # N.B. Use a fully qualified name here (do not use "localhost" here). Password = "xxxxxxxxxxxxxx" Device = neoxl80 Media Type = LTO-8 Auto Changer = yes } EOF -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/301a4606-92e9-4242-89c8-8c0df5f962f9%40googlegroups.com.
