Re: multiple LTOs in the same library
Someone is reading it. Maybe nobody knows the answer... Diego Il 16/05/2023 10:40, ASV ha scritto: This mailing list seems dead. Is anybody actually still reading it? ___ Hello everyone, I have an Amanda setup nicely working with a SpectraStack. The drives are LTO8 but I've just figured out that there are several LTO7 too in the library. Now I've both types defined in amanda.conf however I cannot find a way to tell Amanda that tape labelled XXX or with the barcode 101010 is an LTO7 while another is an LTO8 instead. I've seen something around like specifying the blocksize in the tapelist file but didn't convince me moreover I didn't manage to find one example of such case despite I don't think this is such a rare case with libraries with a lot of tapes. I'm using amanda 3.3.3. I know is terribly old but we are going to do a major upgrade within months. In the meantime I'd like to implement this configuration, if possible. Any hint would be highly appreciated. Many thanks in advance. -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: New Amanda Community release 3.5.2 Has Arrived! -- email bounce
Il 14/09/2022 19:00, gene heskett ha scritto: Waiting for 2T SSD's to become affordable, if they ever do.Depends on your definition of "affordable". I bought 2 2TB NVMes at about 300€ each... But SSDs and NVMes have a quite "limited" life. Look at TBW numbers. Many SSDs and NVMes have some cache RAM for sectors that get rewritten very often, but backups tendo to be sequential so the cache is nearly useless. Once upon a time there were flash-backed ramdrives: controller + RAM + SSD + battery, all the flash contents fitted in the RAM, so the flash was never written unless the battery got discharged enough. Safe assumption: battery dicharging = host off = no ongoing writes. Needless to say the cost was really high and they where usually reserved for (parts of) databases. -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: Ordering the dumps?
Whoops... Found your answer in the spam folder. That's a very interesting document, packed with very useful informations. Probably something changed in the meantime, but it's worth reading anyway. Best regards, Diego Il 28/10/2021 10:57, Stefan G. Weichinger ha scritto: Am 28.10.21 um 10:42 schrieb Diego Zuccato: Il 28/10/2021 10:02, Stefan G. Weichinger ha scritto: btw: it's recommended NOT to use "localhost" in disklist. There's even an FAQ for that. Sorry, I can't find it. Could you please post a link? Since I have some 'localhost' entries I'd probably have to change 'em. I expected this question ;-) Had to google myself, it seems not to be on wiki.zmanda.com Found it here, wrote that years ago, I don't know if everything still is true. https://docs.huihoo.com/amanda/2.5.x/topten.html#id2578555 - full docs also there: https://docs.huihoo.com/amanda/2.5.x/index.html I remember the amount of work converting and formatting, oh my ... -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: Problem to back up docker volumes
Try setting an explicit ACL for 'amanda' user instead of just giving o+x. And check permissions on the full path: too often the problem is in the permissions of an ancestor folder but you keep looking at the file... Been there, done that. Too many times :( Il 18/11/2021 05:06, Olivier ha scritto: Hi, I know it is a bit off topic, but I have a problem to back up my docker volumes (.../docker/volumes): sudo -u amanda amcheck -c normal puffer1000 Amanda Backup Client Hosts Check ERROR: puffer1000: [puffer1000: Could not access /home2/docker/volumes (/home2/docker/volumes): Perm ission denied] I suspect it is something to do with some access control list on Linux (I am more used to FreeBSD); I checked but saw nothing glaring to my face: on@puffer:~$ sudo getfacl /home2/docker/volumes [sudo] password for on: getfacl: Removing leading '/' from absolute path names # file: home2/docker/volumes # owner: root # group: root user::rwx group::--- other::--x I even tried to create a test directory, I gave it the same ACL and could amcheck and amdump that test directory. I must say that I am at lost here on what could be the main cause for this issue. and idea or pointer will be very welcome. TIA, Olivier -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: Ordering the dumps?
Il 28/10/2021 10:02, Stefan G. Weichinger ha scritto: btw: it's recommended NOT to use "localhost" in disklist. There's even an FAQ for that. Sorry, I can't find it. Could you please post a link? Since I have some 'localhost' entries I'd probably have to change 'em. Tks. -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: LVM for vtapes
A different approach here. I configured a single MD-RAID6 of 16 4TB disks and the vtapes are simple folders. I did it mostly because I didn't know the exact space requirements our backups would have. Today I'd go with a smaller RAID (probably just 10 disks, so to have 32TB and a sane stripe size of 4k). I didn't like the idea of having "hard-sized" vtapes because of the wasted space. I don't like having to remove SAS/SATA HDDs because the connector is usually specced for about 500 mating cycles. If you need frequent swaps it's better to use a caddy with its own connectors (like ESATA/USB external enclosure, one per removable disk). With USB3 the performance shouldn't be an issue. Maybe the CPU load, but a recent system should be able to handle it quite well. HIH. Il 21/09/2021 05:17, Olivier ha scritto: Jon, Interesting discussion in other threads got me wondering whether I should have made some other choices when setting up my vtape environment. Particularly whether I should have used LVM (Logical Volume Management) to create one large filesystem covering my multiple dedicated disks. Its a topic I do not recall being discussed, pros & cons. I am using 7 disks of 3 (or is that 4) and 6 TB (should upgrade them all to 6TB soon) almost dedicated to vtapes (the last disk also has a copy of the deleted accounts). I have them configured as individual disks. The size of my vtapes is also about 100GB and I am using a small chunck size, so my disks end up being 80% full at least. When I designed my vtape architecture, I decided to keep each disk individual so that it can be put offline after usage. My idea was to have a system that could prompt an operator to "mount a disk" before the backup and the disk could be manually unmounted and safe stored each day. It is taking advantage of the automount service on FreeBSD. Mounting could be USD disk, or hot-swap. I never went very far in the implementation. I wrote all that many years ago when vtapes were new and limited to a single directory, that is why I wrote my own tape changer. I knew about the risk of loosing a disk and it being a good portion of consecutive backups. But what I had in mind was: - have the system as simple and as portable as possible, so I can shove a disk in another machine and extract contents manually (during the great flood of Bangkok in 2011, I moved all the servers and also took all my hard disks from Amanda backup, but I did not need to move the rack mounted server itself); - a side advantage of my own tape changer is that I can keep the older disks (each disk has an individual label, like any vtape has a label) (I have updated them from 500GB to 1TB to 3TB and soon to 6TB) and the vtapes are still known into tapelist (they are marked noreuse). If the need arise, I can still remount that old disk. So far (10+ years) the only disk I had failling was the disk having the holding partition, I guess it was because of excessive usage. I understand that vtapes have evolved since I started using them, but my system works for me so I never took the time to search any further. Best regards, Olivier -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: DLE splitting
I'm not an expert but I'd set a tapechanger script that wakes the vtape by reading or writing something to the disk. Il 20/09/2021 05:07, Jon LaBadie ha scritto: is there a way to run a user supplied program just before amanda's taper opens each tape file? I.e., before it actually opens and creates a new file on the tape. Do I recall correctly that it opens a tmp file on the vtape and when complete renames it? Or am I thinking of the chunks on the holding disk? jon -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: amgtar: Operation not permitted
Il 19/01/21 01:53, Tom Robinson ha scritto: > I now get a lot of permission warnings and errors. Of particular concern > are the 'Operation not permitted' messages: Maybe you're running SELinux on the clients and amgtar is not allowed to access everything? -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: How's amanda feeling these days?
Il 30/09/20 05:10, Olivier ha scritto: > But if you don't use RAID, you can double the number of backups you are > keeping, that includes older versions. That is a risk calculation > between having secure backups and loosing the backups data as same time > as the primary data. We are using a 3U enclosure with 16 hot-swap bays. The SSD for SO is mounted internally and all the hot swap bays are populated w/ 4TB WD RED disks configured in RAID6 via mdadm (I don't trust "HW RAID", since in case of a controller failure your data is usually toast, but you can always find some way to connect a bunch of drives to a mobo). In case of a disk failure the only real problem is identify which is the failed disk (just run "md5sum /dev/mdX" and see which one isn't working, unless you know how to handle enclosure LEDs). > No, but I never dug that very much. i need to replace a disk or mount > and older disk about twice a year, so I can reboot at that time. I always prefer hot-swap bays. The only thing I should have changed is not to use all the slots: a free slot is useful both for connecting an archive volume for offline storage and to rebuild the array in a safer way (in some cases, the 'bad' disk is still mostly readable, so md can copy data from it instead of having to read all the other disks, which could expose another failing disk). > Developpers can also be told to let their workstation on all the time, > they can understand the reason why, then you can use > Amanda. Administrtive staff do not always understand these details. Administratives can be taught (but usually only the hard way) that the only recoverable documents are the ones they put on the dedicated network share. ]:) -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: native LTO encryption with Amanda
Il 19/05/20 23:56, Chris Hoogendyk ha scritto: > is it possible to configure amanda to handle keys for > the native LTO drive encryption? I rarely trust some HW features like encryption, RAID and compression... They tend to render the devices way less interoperable and make recovery way harder. A tape drive must only store exactly what I throw at it in the most reliable way. It must be readable on a similar drive even if from another supplier and with a different FW version (I've seen RAID cards unable to recognize the disks after a FW upgrade!). Crypto is fragile: it's easy to do it wrong. Hence I prefer a dedicated piece of software (that can easily be replaced/upgraded) to some HW that implements unknown crypto (for someone, XOR with key is enough to say "I support crypto")... -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: smbclient
Il 17/04/20 20:51, Nuno Dias ha scritto:
> I'm trying to use the amsamba plugin, put fails everytime in the
> estimate.
Probably unrelated, but...
> A search in google got me some results saying that some time ago the
> samba version 4.x was incompatible with amanda, anyone knows if this is
> still true?
I have had to apply this patch:
root@nas1:/usr/lib/amanda/application# diff -u amsamba.ori amsamba
--- amsamba.ori 2019-07-16 09:09:28.430585521 +0200
+++ amsamba 2019-07-18 10:44:37.257853728 +0200
@@ -462,6 +462,7 @@
debug("stderr: " . $_);
next if /^Domain=/;
next if /^WARNING/g;
+ next if /^Unable to initialize messaging context/;
# message if samba server is configured with 'security = share'
next if /Server not using user level security and no password
supplied./;
$self->print_to_server("smbclient: $_",
@@ -554,6 +555,7 @@
next if /^Domain=/;
next if /dumped \d+ files and directories/;
next if /^WARNING/g;
+ next if /^Unable to initialize messaging context/;
# message if samba server is configured with 'security = share'
next if /Server not using user level security and no password
supplied./;
debug("stderr: $_");
@@ -758,6 +760,7 @@
return if $line =~ /tarmode is now /;
return if $line =~ /tar_re_search set/;
return if $line =~ /WARNING/g;
+ return if $line =~ /^Unable to initialize messaging context/;
if ($line =~ /dumped (\d+) files and directories/) {
$nb_files = $1;
return;
--
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786
Re: bad tape?
Il 28/08/19 03:34, Olivier ha scritto: > Or write another device for Amanda to use, it would not be vtape, it > would be ... something. Could be 'rawdisk'. :) But better plan for some redundancy to compensate for silent corruption. And consider that SATA connectors have a limited life (about 500 mating cycles, IIRC): for frequently-removed disks it's better to use a dedicated caddy, maybe an eSata one. -- Diego Zuccato Servizi Informatici Dip. di Fisica e Astronomia (DIFA) - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it
Re: bad tape?
Il 27/08/19 05:55, Olivier ha scritto: > But a disk dedicated to vtapes should be doing a lot of sequetial > accesses: once it has been formatted and the slots have been assigned, > it is writting files the size of one Amanda's chunk. In fact, that would > be worth a study: the disk usage for vtapes vs. normal disk usage. That's the *perfect* use-case for SMR drives. But they'd need either no filesystem (like tapes :) ) or a dedicated one. Is it possible to use raw devices as vtapes? -- Diego Zuccato Servizi Informatici Dip. di Fisica e Astronomia (DIFA) - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it
Re: amanda-client: HOST localhost ERROR: //192.168.0.250/ftp: Application 'amsamba': exited with status 1
Il 17/07/19 19:04, Jean-Louis Martineau ha scritto: > Add the same pattern near line 757: > return if $line =~ /^Unable to initialize messaging context/; Tks! Applied. Hope it gets included in next update! -- Diego Zuccato Servizi Informatici Dip. di Fisica e Astronomia (DIFA) - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it
Re: amanda-client: HOST localhost ERROR: //192.168.0.250/ftp: Application 'amsamba': exited with status 1
Il 12/07/19 15:23, Diego Zuccato ha scritto:
> define dumptype win {
> auth "local"
> #estimate client
> #compress server custom
> #server-custom-compress "/usr/bin/pigz"
> program "APPLICATION"
> application {
> plugin "amsamba"
> property "AMANDAPASS" "/etc/amanda/daily/pass"
> }
> }
> (note the commented lines), I get a different error:
> FAILURE DUMP SUMMARY:
> localhost //192.168.48.41/port lev 0 FAILED [data timeout]
> localhost //192.168.48.41/port lev 0 FAILED [Got empty header]
> localhost //192.168.48.41/port lev 0 FAILED [data timeout]
> localhost //192.168.48.41/port lev 0 FAILED [Got empty header]
I finally managed to find the issue: seems newer amsamba does not like
local auth!
When changed to bsdtcp, it all started working again.
I'd prefer using local, but if bsdtcp is needed I'll use it :)
--
Diego Zuccato
Servizi Informatici
Dip. di Fisica e Astronomia (DIFA) - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786
mail: diego.zucc...@unibo.it
Re: amanda-client: HOST localhost ERROR: //192.168.0.250/ftp: Application 'amsamba': exited with status 1
Il 16/07/19 08:46, Stefan G. Weichinger ha scritto:
> I never figured out the correct patch ;-)
> Pls show your change, I want to give it another try.
Here it is.
-8<--
# diff -u amsamba.ori amsamba
--- amsamba.ori 2019-07-16 09:09:28.430585521 +0200
+++ amsamba 2019-07-10 11:02:13.352987193 +0200
@@ -462,6 +462,7 @@
debug("stderr: " . $_);
next if /^Domain=/;
next if /^WARNING/g;
+ next if /^Unable to initialize messaging context/g;
# message if samba server is configured with 'security = share'
next if /Server not using user level security and no password
supplied./;
$self->print_to_server("smbclient: $_",
@@ -554,6 +555,7 @@
next if /^Domain=/;
next if /dumped \d+ files and directories/;
next if /^WARNING/g;
+ next if /^Unable to initialize messaging context/g;
# message if samba server is configured with 'security = share'
next if /Server not using user level security and no password
supplied./;
debug("stderr: $_");
-8<--
But backups are still marked "strange". Seems it's not enough to ignore
it in amsamba... Maybe someone else can find the right fix...
--
Diego Zuccato
Servizi Informatici
Dip. di Fisica e Astronomia (DIFA) - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786
mail: diego.zucc...@unibo.it
Re: amanda-client: HOST localhost ERROR: //192.168.0.250/ftp: Application 'amsamba': exited with status 1
Il 10/07/19 14:23, Diego Zuccato ha scritto:
>> Too bad amdump fails after quite a long time with:
>> FAILED [data timeout]
>> FAILED [Error accepting incoming connection: Connection timed out]
>> Which incoming connection is it referring to???
I tried trplacing tcpd (default dep of Debian package) with xinetd but
the error remains the same.
If I change slightly the config for that dumptype to
define dumptype win {
auth "local"
#estimate client
#compress server custom
#server-custom-compress "/usr/bin/pigz"
program "APPLICATION"
application {
plugin "amsamba"
property "AMANDAPASS" "/etc/amanda/daily/pass"
}
}
(note the commented lines), I get a different error:
FAILURE DUMP SUMMARY:
localhost //192.168.48.41/port lev 0 FAILED [data timeout]
localhost //192.168.48.41/port lev 0 FAILED [Got empty header]
localhost //192.168.48.41/port lev 0 FAILED [data timeout]
localhost //192.168.48.41/port lev 0 FAILED [Got empty header]
I also see two gzip processes instead of just one:
30897 backup 20 0 2540 1496 1216 S 0.0 0.0 0:00.08 /bin/gzip
--fast
30900 backup 20 0 2540 756 692 S 0.0 0.0 0:00.00 /bin/gzip
--best
30902 backup 20 0 48472 31732 13740 S 0.0 0.4 0:00.28
/usr/bin/perl /usr/lib/amanda/application/amsamba backup --message line
--config daily --host localhost --disk //192.168.48.41/port --device
//192.168.48.41/port --level 0 --index line --record --amandapass
/etc/amanda/daily/pass
30905 backup 20 0 72244 17492 14916 S 0.0 0.2 0:00.14
/usr/bin/smbclient //192.168.48.41/portinerie -d 0 -U Backup -E -W MGMT
-c tarmode full reset hidden system quiet; tar c -
30906 backup 20 0 5612 1040 896 S 0.0 0.0 0:00.00 /bin/tar
-tf -
And when pigz is configured I still see a gzip process...
> Any hint?
Please help!
--
Diego Zuccato
Servizi Informatici
Dip. di Fisica e Astronomia (DIFA) - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786
mail: diego.zucc...@unibo.it
Re: amanda-client: HOST localhost ERROR: //192.168.0.250/ftp: Application 'amsamba': exited with status 1
Il 10/07/19 14:17, Diego Zuccato ha scritto: > Too bad amdump fails after quite a long time with: > FAILED [data timeout] > FAILED [Error accepting incoming connection: Connection timed out] > > Which incoming connection is it referring to??? Forgot to say that: 1) if I place the wrong password in the pass file, it fails immediatly 2) launching smbclient from command line works: # sudo -u backup smbclient //192.168.48.41/port -d 0 -U Backup -E -W MGMT -c ls\;quit Unable to initialize messaging context Enter MGMT\Backup's password: . D0 Wed Jun 12 08:17:51 2019 .. D0 Wed Jun 12 08:17:51 2019 [...files and folders...] 314572543 blocks of size 4096. 48702724 blocks available Any hint? Tks. -- Diego Zuccato Servizi Informatici Dip. di Fisica e Astronomia (DIFA) - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it
Re: amanda-client: HOST localhost ERROR: //192.168.0.250/ftp: Application 'amsamba': exited with status 1
Il 09/07/19 18:22, Tobias Koeck ha scritto: > It seems to break only 'amcheck a2g-2' right now. The 'amdump' command > seems to be dumping and writing it anway so it doesn't seem that super > urgent. I just upgraded to Debian 10, too, to check if that would fix my issues with amsamba. Too bad the patch given here: http://www.backupcentral.com/forum/14/292396/smbclient_usage_fails_with__unable_to_initialize_messaging_context_ is not in packaged version (3.5.1) yet. Adding the two lines, in my case, fixes the amcheck issue. Too bad amdump fails after quite a long time with: FAILED [data timeout] FAILED [Error accepting incoming connection: Connection timed out] Which incoming connection is it referring to??? -- Diego Zuccato Servizi Informatici Dip. di Fisica e Astronomia (DIFA) - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it
Re: Amanda 3.5.1 on Debian: how to start?
On 10/06/19 19:23, Jean-Francois Malouin wrote: > This would be a new installation, not an upgrade, and on Debian/Jessie. > I can always help debug! Jessie, IIRC, is already EOL and will not receive further updates: is it wise to use it for a new install? amanda-3.5.1 is already packaged in Buster (testing), so you could have a Stretch base with amanda from Buster -- at least you'll have a supported distro. Too bad I can't make amsamba work in such a setup, so no backup of Samba shares :( -- Diego Zuccato Servizi Informatici Dip. di Fisica e Astronomia (DIFA) - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it
