Re: New version of amandatape script.
On Tue, Feb 14, 2006 at 10:10:51PM +0100, Josef Wolf wrote: Sorry, I wrote my mail into the script and sent an empty message ;-) Now here comes the real mail: Hello! Here comes a new version of the amandatape program I posted about two years ago. This are the changes made since the last release: - Adopt to new logfile format with chunked tapings. - Add possibility to limit output to specified DLEs. - Split output to multiple paper sheets if it doesn't fit onto one page. - Report correct tape with -l option when more than one tape was used in a single amdump/amflush run. - Output total number of DLEs and number of DLEs found on current tape. - Omit error messages on EOT when taper continues writing on next tape. Please let me know if there are bugs or if you like it. Here is the script: #! /usr/bin/perl # amandatape -- a utility to print amanda tape labels for DAT and CD. # # 2004-02-12 Josef Wolf ([EMAIL PROTECTED]) # # Portions of this program which I authored may be used for any purpose # so long as this notice is left intact. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # I wrote this program because I was dissatisfied with the original label # printing program that comes with the amanda distribution. I wanted to see # from one glance on the newest tape which tapes in which order I need to # recover a specific DLE. # # This program prints tapelabels for the amanda backup system. The output # can be in plain ASCII or in postscript. The postscript output is formatted # so that it can be folded to fit into a DAT case or into a CD jewel case. # # An example ASCII output (somewhat stripped to make it fit into 80 columns) # is attached below. Here is an explanation of the example output: # # The columns in the output have following meanings: # # date: This name seems to be intuitive, but unfortunately, it is somewhat #misleading. Actually, this is the name of the logfile that provided #the corresponding information. # label: The label of the tape. # fn:File number on the tape. # fm:Filemark # Osize: Original (that is, uncompressed) size of the dump(s). # Dsize: Size of the dump(s). This is usually pretty close to Tsize so it #is of very little interest. # Tsize: The size of dump(s) on the tape. # Dtime: Dumper time. # Ttime: Taper Time. # Dspd: Dumper speed. # Tspd: Taper speed. # DLE: Disk list entry. # lv:Dump-level. # dpl: "Dumps per level". This is a list of dump levels (starting with #level 0). # error: An error message. # # The output is split into four sections: # # The first section (if present) lists errors. In the example below we can see # that there were two taper errors and a warning that a DLE must be flushed to # tape. # # The second section contains only one line with four fields: # - date # - tape label # - total amount of data on that tape. # - number of DLEs on that tape and total number of DLEs. # # The third section lists all the tapes that are needed to recover all # DLEs. In the example below we can see that three level0, four level1 # and one level3 from tape VOL01 are needed to do a full restore of all # DLEs. In addition, three level0, one level1 and one level2 from VOL09 # are still needed. Everything else on VOL09 is obsoleted by the dumps # on VOL01. VOL08 contains two level0 and one leve1 that are not obsoleted # by newer tapes. Finally, VOL07 contain one level0 dump that is not # obsoleted by newer tapes. # Tapes that contain only obsoleted data are not mentioned at all unless you # supply the -t command line option. # # The fourth section is the main section and is itself split into multiple # sections, one for each DLE. In this section we can see which tapes we # need to recover a specific DLE. For example, we can see that in order # to recover raven:/u4 we need file 1 from VOL07, file 6 from VOL08, # file 7 from VOL09 and file 1 from VOL01, in this order. # # The Ordering of the sections can be different depending on the choosen # output format. # # Due to lack of space, there is a special handling when output is formatted # for DAT: The fourth section is printed in such a way that non-important data # (everything on the left from the Osize column) is cut off from the label. # # Here comes the example output: # # date Tsize label fm lv error # 2004-02-20.0 1499M VOL06 8 ? writing file: No space left on device # 2004-02-23.0 1499M VOL01 9 ? writing file: No space left on device # 2004-02-23.0 670M??? ? 0 raven:/m/u1 not on tape yet #
New version of amandatape script.
Re: Still get ERROR [host fw.my.co.uk: port 62679 not secure] after I added my ipchain rule:
Paul Bijnens wrote: On 02/14/2006 04:56 PM, Chuck Amadi Systems Administrator wrote: I have just edited my firewall and added a ipchain rule but I still got an error as below: Amanda Backup Client Hosts Check ERROR: server.my.co.uk: [host fw.smtl.co.uk: port 62679 not secure] This seems to be a result of the NAT in ipchains: it changes the source port to someting over 6. Here is my take on the scenario: let's concentrate on the amdump part for the time being. 1) your Amanda Backup server is a package from SuSE, cannot be recompiled. So first you need to find out if --with-udpportrange is compiled in with the SuSE package. To find out, do: amadmin configname version |grep --with-udpportrange If --with-udpportrange is compiled in, you need to make sure the Amanda Backup server can use those ports to connect to the Amanda Backup client. >> ERROR: server.my.co.uk: [host fw.smtl.co.uk: port 62679 not secure] this indicates that the server is trying to connect to the client using udp port 62679. 2) there could be a NAT issue, but we need to resolve 1) first. --Kevin However, why is the name "fw.smtl.co.uk"? I did not know that ipchains used uses NAT for traffic to the firewall itself too? Make really really sure that the amandaserver does bind to a port from the udp-port range: In one window start as root: # tcpdump port 10080 In another window, to the "amcheck". And verify the that port on the amandaserver is one from 1001-1009. This could also happen when amcheck lost the suid root bit (but I believe that it would complain about that before you get that far). A possible workaround here is to recompile the software on the client to not fail on a "non secure" port. That notion of "secure port" (ports < 1024 require root priviledge to open), is in these days not a strong security check anyway, where anyone can install a workstation or boot from a live-CD and be root to open any port < 1024. I have setup my fw rules as below: # Amanda Client - Enterprise random udp forks to Nemesis Server ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX 1001:1009 -j ACCEPT ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX 10080:10083 -j ACCEPT Outgoing packets are allowed from behind our firewall and all forwaded to our main file server that is the same server for amanda backup tape server I do not remember anymore, but maybe there is a possibility to not do NAT for a certain portrange/host ? I re compiled amanda client as below: ./configure --with-user=amanda --with-group=disk --with-configdir=/etc/amanda --with-udpportrange=1001, 1009 --with-tcpportrange=11000, 11300 -- Thank you! Kevin Till Amanda documentation: http://wiki.zmanda.com Amanda forums:http://forums.zmanda.com
Re: Release of amanda-2.5.0b2
On Mon, Feb 13, 2006 at 01:58:09PM -0800, Kevin Till wrote: > Josef Wolf wrote: > >>1. VOL05:1 (this is the newest non-broken available dump) is _not_ > >> considered for retrieval at all. > >> > >>2. Instead, amfetchdump _tries_ to get the (broken) VOL04:7. > >> > >>3. But instead of VOL04:7 it gets the (older) VOL04:1. There seems to be > >> no attempt to further search for VOL04:7 > >> > >>4. The order of tapes seems to be wired. I would have expected > >> VOL05 VOL02 VOL03 VOL10 (how they were sceduled) > >> or VOL05 VOL04 VOL10 (last available for every dumpdate) > >> or VOL05 VOL10 VOL01 VOL08 VOL09 (first available for every dumpdate) > >> or some such. > >> > >>5. When trying to append the second chunk to the first one, amfetchdump > >> fails with "Bad file descriptor". The resulting dump (uncompressed) > >> is 527620009 bytes long. > >> > >>6. Next problem is with amrecover, but it seems to be closely related > >> with the "Bad file descriptor" problem. Unfortunately, I don't have a > >> transcript for this problem, because the system crashed. Here's the > >> description: > >> > >> When I tried to retrieve the above mentioned DLE mentioned in line c > >> with amrecover, the system (Athlon 1800+, 500MB RAM, 2G swap, > >> suse-10.0) freezed, but vterm switching and pinging from a different > >> host worked. This reminds me of overcommitments caused by memory-hogs. > >> > >> After reboot, I noticed following file in the slot-directory > >> of the vtape directory: > >> > >> -rw--- 1 amanda disk 527630347 Feb 7 07:52 info > >> > >> Notice that the length is almost the same as in 5. This file starts with > >> following contents: > > can you make sure you have restore-src/restore.c revision 1.19 or above? > One fix went it on r1.19 which resolved one file descriptor problem. Thanks Kevin! I have tried with newest (1.23) restore-src/restore.c. With this, bullet 5 seems to be gone. Bullet 6 don't crash the system anymore, but still don't seem to work properly: Extracting files using tape drive changer on host host.do.main. Load tape VOL01 now Continue [?/Y/n/s/t]? The following tapes are needed: VOL01 amrecover: short block 0 bytes UNKNOWN file amrecover: Can't read file header amrecover: Extractor child exited with status 1 extract_list - child returned non-zero status: 1 Continue [?/Y/n/r]? The other issues (1..4) are (of course) still present.
Re: Still get ERROR [host fw.my.co.uk: port 62679 not secure] after I added my ipchain rule:
On 02/14/2006 04:56 PM, Chuck Amadi Systems Administrator wrote: I have just edited my firewall and added a ipchain rule but I still got an error as below: Amanda Backup Client Hosts Check ERROR: server.my.co.uk: [host fw.smtl.co.uk: port 62679 not secure] This seems to be a result of the NAT in ipchains: it changes the source port to someting over 6. However, why is the name "fw.smtl.co.uk"? I did not know that ipchains used uses NAT for traffic to the firewall itself too? Make really really sure that the amandaserver does bind to a port from the udp-port range: In one window start as root: # tcpdump port 10080 In another window, to the "amcheck". And verify the that port on the amandaserver is one from 1001-1009. This could also happen when amcheck lost the suid root bit (but I believe that it would complain about that before you get that far). A possible workaround here is to recompile the software on the client to not fail on a "non secure" port. That notion of "secure port" (ports < 1024 require root priviledge to open), is in these days not a strong security check anyway, where anyone can install a workstation or boot from a live-CD and be root to open any port < 1024. I have setup my fw rules as below: # Amanda Client - Enterprise random udp forks to Nemesis Server ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX 1001:1009 -j ACCEPT ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX 10080:10083 -j ACCEPT Outgoing packets are allowed from behind our firewall and all forwaded to our main file server that is the same server for amanda backup tape server I do not remember anymore, but maybe there is a possibility to not do NAT for a certain portrange/host ? I re compiled amanda client as below: ./configure --with-user=amanda --with-group=disk --with-configdir=/etc/amanda --with-udpportrange=1001, 1009 --with-tcpportrange=11000, 11300 -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: Still get ERROR [host fw.my.co.uk: port 62679 not secure] after I added my ipchain rule:
Hi List I would like to mention that the selected port range udp 1001,1009 and tcp 11000,11300 have only been re compiled on the Amanda client, thus I haven't been opened on both amanda client and amanda server ends of the firewall. I didn't want to re compile a productive amanda tape server plus I used the default software within SuSE Linux Enterprise Server 9. So if I have to open the selected port range on the amanda tape server can I just edit /etc/services and add the 1001 and 1009 systems privileged ports. or have I got to run the --with-udpportrange=1001,1009 thus having to start from scratch which is not really feasible. amanda 1001/udp # Amanda amanda 1009/udp # Amanda Cheers On Tue, 2006-02-14 at 15:56 +, Chuck Amadi Systems Administrator wrote: > Hi all > > I have just edited my firewall and added a ipchain rule but I still got > an error as below: > > Amanda Backup Client Hosts Check > > ERROR: server.my.co.uk: [host fw.smtl.co.uk: port 62679 not secure] > Client check: 4 hosts checked in 10.780 seconds, 1 problem found > > Here is also my Amanda Debug file: > less /tmp/amanda/amandad.20060214163540.debug > > Amanda 2.4 REQ HANDLE 003-D0990808 SEQ 1139931009 > SECURITY USER amanda > SERVICE noop > OPTIONS features=ecfffeff9ffe0f; > > > amandad: time 0.000: sending ack: > > Amanda 2.4 ACK HANDLE 003-D0990808 SEQ 1139931009 > > > amandad: time 0.006: sending REP packet: > > Amanda 2.4 REP HANDLE 003-D0990808 SEQ 1139931009 > ERROR [host fw.my.co.uk: port 62679 not secure] > > > amandad: time 0.007: got packet: > > Amanda 2.4 ACK HANDLE 003-D0990808 SEQ 1139931009 > > > I have setup my fw rules as below: > > # Amanda Client - Enterprise random udp forks to Nemesis Server > > ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX > 1001:1009 -j ACCEPT > > ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX > 10080:10083 -j ACCEPT > > Outgoing packets are allowed from behind our firewall and all forwaded > to our main file server that is the same server for amanda backup tape > server > > I re compiled amanda client as below: > > ./configure --with-user=amanda --with-group=disk > --with-configdir=/etc/amanda --with-udpportrange=1001, 1009 > --with-tcpportrange=11000, 11300 > > I haven't edited the /etc/services as I had read this does not effect > initial UDP request made from the amanda tape server. > > I have read and digested learnt a few things but I am still having > issues using Amanda between hosts separated by a firewall using > ipchains. > > Cheers for your help. > > > -- Unix/ Linux Systems Administrator Chuck Amadi The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Email chuck.smtl.co.uk Tel: +44 1656 752820 Fax: +44 1656 752830
Re: Still get ERROR [host fw.my.co.uk: port 62679 not secure] after I added my ipchain rule:
Hi all I have just edited my firewall and added a ipchain rule but I still got an error as below: Amanda Backup Client Hosts Check ERROR: server.my.co.uk: [host fw.smtl.co.uk: port 62679 not secure] Client check: 4 hosts checked in 10.780 seconds, 1 problem found Here is also my Amanda Debug file: less /tmp/amanda/amandad.20060214163540.debug Amanda 2.4 REQ HANDLE 003-D0990808 SEQ 1139931009 SECURITY USER amanda SERVICE noop OPTIONS features=ecfffeff9ffe0f; amandad: time 0.000: sending ack: Amanda 2.4 ACK HANDLE 003-D0990808 SEQ 1139931009 amandad: time 0.006: sending REP packet: Amanda 2.4 REP HANDLE 003-D0990808 SEQ 1139931009 ERROR [host fw.my.co.uk: port 62679 not secure] amandad: time 0.007: got packet: Amanda 2.4 ACK HANDLE 003-D0990808 SEQ 1139931009 I have setup my fw rules as below: # Amanda Client - Enterprise random udp forks to Nemesis Server ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX 1001:1009 -j ACCEPT ipchains -A input -i $EXTERNAL_INTERFACE -p udp -s 193.XX.XX.XXX 10080:10083 -j ACCEPT Outgoing packets are allowed from behind our firewall and all forwaded to our main file server that is the same server for amanda backup tape server I re compiled amanda client as below: ./configure --with-user=amanda --with-group=disk --with-configdir=/etc/amanda --with-udpportrange=1001, 1009 --with-tcpportrange=11000, 11300 I haven't edited the /etc/services as I had read this does not effect initial UDP request made from the amanda tape server. I have read and digested learnt a few things but I am still having issues using Amanda between hosts separated by a firewall using ipchains. Cheers for your help. -- Unix/ Linux Systems Administrator Chuck Amadi The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Email chuck.smtl.co.uk Tel: +44 1656 752820 Fax: +44 1656 752830
Re: WARNING: server.my.co.uk: selfcheck request timed out. Host down? this is a private IP address.
Hi I have re compiled amanda client on the server that sits outside of
my LAN with the following port range. --with-udpportrange=1100,11030
Thus you stating that it must be a system udp port range i.e < 1023
If So I have Unassigned range -f 1001-1009 So I would use this when I re
compile a udp system port range --with-udpportrange=1001,1009 Does the
tcp port range stay the same. --with-tcpportrange=1100,11030 .
My IPChain example.
ipchains -A input -p udp -i -s $193.XXX.XXX.XXX{AMANDA_CLIENT} 1001:1009
--dport $192.168.1.XXX{AMANDA_SERVER} -j ACCEPT
ipchains -A input -p tcp -i -s $193.XXX.XXX.XXX {AMANDA_CLIENT}
10080:10083 --dport $192.168.1.XXX{AMANDA_SERVER} -j ACCEPT
CheersOn Tue, 2006-02-14 at 10:44 +0100, Paul Bijnens wrote:
> On 02/14/2006 10:25 AM, Chuck Amadi Systems Administrator wrote:
> > Hi Agian
> >
> > I am re compiling my amanda client thus after running the switch
> >
> > --with-udpportrange=11000,110030 and --with-tcpportrange=11000,110030
> >
> > I edit the /etc/services on the amanda client and add the following:
> >
> > # Leave this lot intact:
> > amanda 10080/tcp # Amanda
> the 10080/tcp is not used.
>
> > amanda 10080/udp # Amanda
> > amandaidx 10082/tcp
> > amidxtape 10083/tcp
> >
> > # Check that these port ranges are Unassigned.
> > amanda 11000-11030/tcp # Amanda
> > (--with-tcpportrange=11000,110030)
> > amanda 11000-11030/udp # Amanda
> > (--with-udpportrange=11000,110030)
>
> I think these two will give a syntax error in /etc/services :-)
>
> You cannot give a name to a range (and the name "amanda" must
> be for the 10080/udp port!)
>
> Moreover, it is not strictly needed that the range is unassigned.
> When Amanda wants to use a port in the range that is already in
> use by something else, it will simply skip that port.
> But because you need 3 tcp connections for each dump in parallel,
> ("inparallel" in amanda.conf) you must add some extra ports in
> the range to take into account the ports that are already in use.
>
>
>
--
Unix/ Linux Systems Administrator
Chuck Amadi
The Surgical Material Testing Laboratory (SMTL),
Princess of Wales Hospital
Coity Road
Bridgend,
United Kingdom, CF31 1RQ.
Email chuck.smtl.co.uk
Tel: +44 1656 752820
Fax: +44 1656 752830
Re: [Fwd: Re: WARNING: server.my.co.uk: selfcheck request timed out. Host down? this is a private IP address.]
On 02/14/2006 10:11 AM, Chuck Amadi Systems Administrator wrote: Hi Paul Bijnes Was the UDP also to be recompiled --with-udpportrange=11000,11030 thus the same port range as tcp. no, the only UDP connection is the one to 10080. But do allow the reply packet: in iptables, this is automatically, in ipchains this is not automatically (-- I believe -- it has been years since I used ipchains...). The server binds to a local port in the UDP-portrange, sends a packet to the client at port 10080, and must be able get the reply from client to server: you have to open the UDP-portrange on the firewall to allow the reply packet. The UDP-portrange must be < 1024, otherwise, the client will complain about "port not secure", and it will not work. The tcpportrange is used to send the data/mesg/index stream from client to server. You need 3 tcp connections for each host that is doing a backup in parallel. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: [Fwd: Re: WARNING: server.my.co.uk: selfcheck request timed out. Host down? this is a private IP address.]
Hi Paul Bijnes Was the UDP also to be recompiled --with-udpportrange=11000,11030 thus the same port range as tcp. Cheers On Tue, 2006-02-14 at 08:19 +, chuck.amadi wrote: > email message attachment (Re: WARNING: server.my.co.uk: selfcheck > request timed out. Host down? thisis a private IP address.) > On Tue, 2006-02-14 at 08:19 +, chuck.amadi wrote: -- Unix/ Linux Systems Administrator Chuck Amadi The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Email chuck.smtl.co.uk Tel: +44 1656 752820 Fax: +44 1656 752830
Re: WARNING: server.my.co.uk: selfcheck request timed out. Host down? this is a private IP address.
On 02/14/2006 02:25 AM, Kevin Till wrote: *Assume all outgoing packets are accepted/allowed.* For amdump to work, you need to open up: backup client : 10080(udp), a small range of tcp ports for data transfer e.g. 11000:11030 (recompile amanda with --with-tcpportrange=11000,11030) For amrecover to work, you need to open up: backup server: 10082(udp), 10083(udp), a small range of tcp ports for data transfer e.g. 11000:11030 (recompile amanda with --with-tcpportrange=11000,11030) I believe 10082 and 10083 are tcp, not udp. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
