Re: odd dump timeout symptoms

2005-08-17 Thread Jamie Wilkinson
This one time, at band camp, Jon LaBadie wrote:
>On Thu, Aug 18, 2005 at 10:16:48AM +1000, Jamie Wilkinson wrote:
>> 
>> Another data point though; this backup today started at 1:30am (it's now
>> 10:15am) and is only 35% through -- about 20GiB.  This is pretty abnormal
>> (well, compared back to when it used to run -- it would on a bad day be
>> finishing about now).
>> 
>
>Do you have network analysis capability?  One of the early things
>network-knowledgable people seem to ask is if there is a duplex
>(half/full) mismatch between the switch and the system.  Possibly
>some auto-negotiation did not get it right.  IIRC, it causes a lot
>of collisions and resends.

Yeah, I do.  Thanks for the pointer.


Re: odd dump timeout symptoms

2005-08-17 Thread Jon LaBadie
On Thu, Aug 18, 2005 at 10:16:48AM +1000, Jamie Wilkinson wrote:
> 
> Another data point though; this backup today started at 1:30am (it's now
> 10:15am) and is only 35% through -- about 20GiB.  This is pretty abnormal
> (well, compared back to when it used to run -- it would on a bad day be
> finishing about now).
> 

Do you have network analysis capability?  One of the early things
network-knowledgable people seem to ask is if there is a duplex
(half/full) mismatch between the switch and the system.  Possibly
some auto-negotiation did not get it right.  IIRC, it causes a lot
of collisions and resends.

-- 
Jon H. LaBadie  [EMAIL PROTECTED]
 JG Computing
 4455 Province Line Road(609) 252-0159
 Princeton, NJ  08540-4322  (609) 683-7220 (fax)


Re: odd dump timeout symptoms

2005-08-17 Thread Jamie Wilkinson
This one time, at band camp, Jon LaBadie wrote:
>As I noted, I was uncertain of the details, someone understanding more
>about the networking aspects can comment more.  But just for clarification
>of the way I worded that:
>
>- I did not say anything about the backup data stream over udp
>- Your logs showed it was the index backup stream having problem,
>  not the backup stream (they are separate)
>- I said the timeout setting "affected" the stream, I wasn't sure
>  if it was the stream itself or not
>- amanda does use some udp connections, perhaps for control communication

Thanks.

The only things I can find in google relating to the index stream are
conntrack-related timeouts; there is no explicit firewall running on the
server or client on their backup network interfaces (amanda is running over
a private segment) let alone NAT, so the amanda conntrack module isn't being
used.

Another data point though; this backup today started at 1:30am (it's now
10:15am) and is only 35% through -- about 20GiB.  This is pretty abnormal
(well, compared back to when it used to run -- it would on a bad day be
finishing about now).



Re: odd dump timeout symptoms

2005-08-17 Thread Jamie Wilkinson
This one time, at band camp, Scott R. Burns wrote:
>Can the version of GNU tar you are using handle single archives of this size
>? There were some older versions that used signed long internals that
>overflowed on me in the past and caused problems.

It's 1.13.25 from RHEL 3.  I havne't seen anything like this on other RHEL 3
machines doing similar sized backups.



Re: Has anyone a script for Monthly/Daily backups

2005-08-17 Thread Paul Bijnens

Paul Bijnens wrote:

== cut here == Version 2, using one subprocess only
#!/bin/sh
# First friday of the month
case `date +%d%a` in
[1-7]Fri) amdump monthly ;;
*)  amdump daily ;;
esac
== cut here


Oops, "date +%d" results in a zero padded two digit date, so
that should be:

> == cut here == Version 2, using one subprocess only
> #!/bin/sh
> # First friday of the month
> case `date +%d%a` in
> 0[1-7]Fri) amdump monthly ;;
> *)  amdump daily ;;
> esac
> == cut here


--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...*
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out  *
***


Re: Has anyone a script for Monthly/Daily backups

2005-08-17 Thread chuck.amadi
He is my previous Boss hack if you look at the third script would it be 
a simple process to just add amanda amcheck and amdump after the first 
two scripts have confirmed the true date being a daily or monthly backup.

Or would you suggest to keep it simple use paul bijnens script .

Please note I would prefer to still run amcheck prior to amdump

Thanks all. I am looking at ideas prior to startig my hack in work 2 mory.

-- --   First part of script-

#!/bin/sh
# script name dobackup.sh
# takes 1 arugument - monthly or daily

# script to run backup program
/path/to/program/backup.sh $1  2>&1 | /usr/bin/tee /path/to/backup-org/ 
'/bin/date +"%Y-%m-%d"'


-- End of Script 
--





- Second part of script 
-



#!/bin/sh
# scirpt name today_is_last_friday.sh
# Run this shell on a friday to check if it is the last friday in the month.
# used for backup system to run monthly backup on last friday in the 
month, daily otherwise.


DATEPROG=/bin/date
# exits with true (0) if it is the last friday, otherwise with 1

# Get the month for this friday
thisfri='$DATEPROG --date="this friday" "+%B" '

# Get the month for next friday
nextfri='$DATEPROG --date="next friday" "+%B" '

# now compare them  - if different, today is the last friday in the month

if [ "$thisfri" !=  "$nextfri"  ]
then
# echo "Today is the last friday of the month ! "
exit 0
else
# echo "Today is NOT the last friday of the month ! "
exit 1
fi


--End of 
script



--- Last script 



#!/bin/sh

# shell script to run on fridays to check  if it's the last friday in 
the month

# if so, do a complete monthly backup, otherwise do a usually daily backup.

cd /path/to/program

if ./today_is_last_friday.sh
then
# Thus could I run su - c "/usr/sbin/amcheck 
MonthlySet1";su - c "/usr/sbin/amdump MonthlySet1";

echo " Starting MonthlySet1 Backup"
/path/to/prgram/backup.sh monthly

else
# Thus could I run su - c "/usr/sbin/amcheck DailySet1";su 
- c "/usr/sbin/amdump DailySet1";

echo " Starting DailySet1 Backup"
/path/to/prgram/backup.sh monthly

fi



--- end of script 
-


Re: HD backup strategy ?

2005-08-17 Thread Mike Delaney
On Wed, Aug 17, 2005 at 03:03:59PM +0300, Toomas Aas wrote:
> Jon LaBadie wrote:
> 
> >Haven't seen anyone on the list mention using it, but Iomega
> >introduced some interesting hardware last year.  I think they
> >call it "Rev", basically a small, removalble hard drive
> >cartridge.  Think high capacity, tiny zip drive as it has
> >35GB native capacity and a builtin compressor.  Along with
> >their single slot device, they also introduce an autoloader
> >that can hold 10 cartridges.
> 
> Funny, I just stumbled upon this on another mailing list yesterday. 
> Looks like IOMega continues to live up to its reputation:
> 
> http://lists.freebsd.org/pipermail/freebsd-questions/2005-July/094114.html

Or someone didn't look hard enough.  While most systems device drivers
do detect REV drives as CD-ROMs, that doesn't prevent one from writing
to it.  And while the filesystem format isn't ISO-9660, it's not some
Iomega proprietary thing either: it's UDF.

Basically, from the system's perspective REV drives look and act like
high capacity DVD-RAM units.  I've been using one with SuSE Linux for
several months now.  



Re: Has anyone a script for Monthly/Daily backups

2005-08-17 Thread chuck.amadi
To all thanks I had a previous hack from our old Boss crafted backup 
hack solution I do know Python and I like shell scripting So I got a few 
scripts excellent list.


Cheers

Chuck


Re: Has anyone a script for Monthly/Daily backups

2005-08-17 Thread Frank Smith
--On Wednesday, August 17, 2005 17:58:30 +0100 "chuck.amadi" <[EMAIL 
PROTECTED]> wrote:

> Has anyone a script that will check for the last friday of the month and if 
> So run the monthly config backup or if not the usually daily config.
 
If you have gnu date available (the one used by Linux and sometimes
available on other OSes, or you could build it from source) , you could
run a script from cron every Friday that does something like:

if [ `date -d 'next friday' +%m` -ne `date +%m` ] ; then
  amdump archiveconfig
fi

If you don't have gnu date then it gets messy, since you have to figure
out if 7 days from now is still this month or next, worrying about the
length of the month and whether or not it's a leap year.

> I am currently looking at hacking something together as I was just going to 
> edit the crontab and comment and uncomment between daily and monthly.
> 
> I have a 10 tape run So a 2 week daily schedule but I to implement a monthly 
> that will run on the last fridat of each month.

Make sure your archive config dumptype has 'record no' in it or it will
mess up your daily schedule.

Frank

> 
> If anyone got a script please let me know ever - way I will post my hack if 
> possible.



-- 
Frank Smith  [EMAIL PROTECTED]
Sr. Systems Administrator   Voice: 512-374-4673
Hoover's Online   Fax: 512-374-4501



Re: Has anyone a script for Monthly/Daily backups

2005-08-17 Thread Paul Bijnens

chuck.amadi wrote:
Has anyone a script that will check for the last friday of the month and 
if So run the monthly config backup or if not the usually daily config.


I am currently looking at hacking something together as I was just going 
to edit the crontab and comment and uncomment between daily and monthly.


I have a 10 tape run So a 2 week daily schedule but I to implement a 
monthly that will run on the last fridat of each month.



Is the "last friday of the month" a solid requirement or could it
be "the FIRST friday of the month" instead.

If yes:

= cut here == Version 1
#!/bin/sh
# first friday of the month
if [ `date +%d` -lt 7 -a `date +%a` = "Fri" ]
then amdump monthly
else amdump daily
fi
== cut here

== cut here == Version 2, using one subprocess only
#!/bin/sh
# First friday of the month
case `date +%d%a` in
[1-7]Fri) amdump monthly ;;
*)  amdump daily ;;
esac
== cut here


If the requirement is necessarily "the last Friday of the month"
the script becomes a little more complicated because the last friday
could fall in the fifth week of a month.
A python script is already posted, a shell script is here:

 cut here 
#!/bin/sh
# Last friday of the month
if [ `cal | awk 'NF >= 6 {print $6}' | tail -1` -eq `date +%d` ]
then amdump monthtly
else amdump daily
fi
  cut here ===


--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out  *
***




Re: Backup too long with windows client

2005-08-17 Thread Paul Bijnens

tanguy yoann wrote:

Hi,

I have a problem with a backup:  it puts too much
time.  I do not backup many data but that takes a long
time.  I made this backup before and that lasted only
a few seconds.  I backup a Windows share by smbclient.
 I do not understand the problem. So I send to you


To be honest, I do not understand the problem either.
Try to reword the problem and be more specific in
what you expect and what you get (instead of "a few"
and "a long time").



files of debug to see whether you will be able to
understand what does not function correctly. Thanks a
lot.


...

GNUTAR //Yoann/backup 1 2005:8:18:7:54:1 -1 OPTIONS
|;bsd-auth;compress-best;index;


First candidate:
"compress client best" compresses only a few percentages
more, but takes twice or more time and cpu.
Use "compress client fast" instead.


...

amandad: time 0.000: running service
"/usr/lib/amanda/sendsize"
amandad: time 239.967: sending PREP packet:

Amanda 2.4 PREP HANDLE 000-18820608 SEQ 1124436519
OPTIONS features=feff9ffe7f;




Second wierd thing above:  why is amanda waiting 239
seconds before sending the first PREP (partial reply)
package?
On one of my slowest clients it takes 0.06 seconds.
This is one of the newer features in 2.4.5, which I'm
not completely familiar with.  If I understand the
code, then the first packet, containing only the
"OPTIONS feature=..." is send immediately after
the sendsize invocation.
If that is correct, than why does fork/exec of sendsize
take 239 seconds?



amandad: time 360.013: sending PREP packet:

...

amandad: time 360.070: pid  finish time Fri Aug 19
09:34:38 2005


And 120 seconds later the "sendsize" is complete.
Is 120 seconds a normal time to be expected here?
(Above you said "a few seconds".)




2)amandad sendbackup


...

amandad: time 0.000: running service
"/usr/lib/amanda/sendbackup"
amandad: time 0.003: sending REP packet:

Amanda 2.4 REP HANDLE 000-E8ED0608 SEQ 1124436518
CONNECT DATA 50084 MESG 50085 INDEX 50086
OPTIONS features=feff9ffe7f;


amandad: time 0.006: got packet:

Amanda 2.4 ACK HANDLE 000-E8ED0608 SEQ 1124436518


amandad: time 0.006: pid 6709 finish time Fri Aug 19
09:34:38 2005


Very weird that the "sendbackup" takes 0.003 seconds
while the sendsize took over 120 seconds (or even 360
seconds).





3)Sendsize


...

sendsize[6706]: time 359.946: spawning
/usr/bin/smbclient in pipeline

...

sendsize[6706]: time 360.008: Total number of bytes:
260218
sendsize[6706]: time 360.009: .
sendsize[6706]: estimate time for //Yoann/backup level
1: 0.063


The real estimate took a fraction of a second, as
expected (smbclient uses the internal "du" function
for estimates).



4)Sendbackup

...

sendbackup-gnutar: time 0.004: pid 6713: /bin/gzip
--best
sendbackup-gnutar: time 239.968: doing level 1 dump
from date: 2005-08-29  7:00:19 GMT
sendbackup-gnutar: time 359.950: backup of
\\Yoann\backup


Notice again the 239 seconds - 360 seconds milestones
before sendbackup actually does something useful.
Wierd, but you know your system better than I do...


--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out  *
***




Re: Has anyone a script for Monthly/Daily backups

2005-08-17 Thread Graeme Humphries

chuck.amadi wrote:

Has anyone a script that will check for the last friday of the month 
and if So run the monthly config backup or if not the usually daily 
config. 


I'm using the following. In crontab:

45 15   * * 5   root/root/monthly-date-check.py && 
/usr/sbin/amdump monthly


And then in /root/monthly-date-check.py:

#!/usr/bin/env python

# we use this with Amanda to see if it's the last friday and saturday of 
the month, for

# running monthly backups

import sys
import datetime   # a thing of beauty and a joy forever

FIRST = 0
SECOND = 1
THIRD = 2
FOURTH = FORTH = 3  # for people who have finger trouble
FIFTH = 4
LAST = -1
SECONDLAST = -2
THIRDLAST = -3

MONDAY = MON = 0
TUESDAY = TUE = TUES = 1
WEDNESDAY = WED = 2
THURSDAY = THU = THUR = 3
FRIDAY = FRI = 4
SATURDAY = SAT = 5
SUNDAY = SUN = 6

JANUARY = JAN = 1
FEBRUARY = FEB = 2
MARCH = MAR = 3
APRIL = APR = 4
MAY = 5
JUNE = JUN = 6
JULY = JUL = 7
AUGUST = AUG = 8
SEPTEMBER = SEP = 9
OCTOBER = OCT = 10
NOVEMBER = NOV = 11
DECEMBER = DEC = 12

def 
dow_date_finder(which_weekday_in_month=FIRST,day=MONDAY,month=JANUARY,year=2000):

   dt = datetime.date(year,month,1)
   dow_lst = []
   while dt.weekday() != day:
   dt = dt + datetime.timedelta(days=1)
   while dt.month == month:
   dow_lst.append(dt)
   dt = dt + datetime.timedelta(days=7)
   return dow_lst[which_weekday_in_month]  # may raise an exception 
if slicing is wrong


if __name__ == "__main__":
   now = datetime.date.today()
   fri = dow_date_finder(LAST, FRIDAY, now.month, now.year)
   sat = datetime.date.fromordinal(fri.toordinal() + 1)
   if now == fri or now == sat:
   sys.exit(0)
   else:
   sys.exit(1)


--
Graeme Humphries ([EMAIL PROTECTED])
(306) 955-7075 ext. 485

My views are not the views of my employers.



Has anyone a script for Monthly/Daily backups

2005-08-17 Thread chuck.amadi
Has anyone a script that will check for the last friday of the month and 
if So run the monthly config backup or if not the usually daily config.


I am currently looking at hacking something together as I was just going 
to edit the crontab and comment and uncomment between daily and monthly.


I have a 10 tape run So a 2 week daily schedule but I to implement a 
monthly that will run on the last fridat of each month.


If anyone got a script please let me know ever - way I will post my hack 
if possible.


Re: Duration Of Backup

2005-08-17 Thread Paul Bijnens

Jon LaBadie wrote:

On Wed, Aug 17, 2005 at 09:13:22AM -0400, Mangala Gunadasa wrote:


[...]

backup takes 9hrs to finish. We have tried to make it more efficient by
changing the parameters "interface", and "net usage" of the amanda.conf
and did not succeed( Probably those parameters do not matter). We would
like to perform Full backup every Day to a SDLT Tape drive.
 [...]



Seconding Dave's comments, figure out what pieces of the backup take time.


Thirding!  Oh well, you know what I mean.



One additional thought, see if the estimate phase is an appreciable
part of the total time.  As you are doing full backups daily, I presume
you have a pretty good idea of the amount of data daily.  In that case
newer versions of amanda have alternatives to the estimate phase that
can greatly speed up that part of the backup.



Indeed, especially when using gnutar the estimates can take
a lot of time.
I use these parameters in my archive backups that do
full backups only.  I do not want any incrementals at all, even
if the tape is full, and do not want to waste time on estimates
for incrementals either:

define dumptype global {
program "GNUTAR"
index yes
record no
compress client fast
skip-incr yes   # emphasize we only want full dumps
strategy noinc  # also reduce time to do estimates
   }


If you have have really really fast servers, then adding "maxdumps 2"
or even more, together with a careful indication of splindlenumbers in
the disklist allows amanda to start more than one dump on one host
at a time.

Other general tips:  make sure you have enough holdingdisk space.
Then the backups themselves can finish as fast as they can, without
tapedrive speed constraints.

Also the command "amplot" can greatly help pointing to bottlenecks.


--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out  *
***




Enterprise: Backup to tape

2005-08-17 Thread Seth Rothenberg

Can anyone point me to information (especially if it is in a FAQ)
We presently use Amanda to back up 60 disks on 11 servers,
distributed across 34 virtual hostnames.

Meanwhile, I have a colleague who shopping for an
enterprise Central-backup-to-disk solutionand
they are apparently willing to hear a "bid" from Amanda (Inc.:-)

I would like to hear from people backing up 
enterprises that have hundreds of servers,
5,000 PC's, several SANs, Disaster Recovery, etc.

The plan is to backup to SAN nightly, archive to SATA disk in 2nd data
center,
and copy to tape for offsite storage.

Thanks
Seth



Re: Duration Of Backup

2005-08-17 Thread Jon LaBadie
On Wed, Aug 17, 2005 at 09:13:22AM -0400, Mangala Gunadasa wrote:
> Greetings
> 
> We have been using amanda for many years. Our capacity had grown and
> currently we are backing up 60 file systems across 12 servers. The
> backup takes 9hrs to finish. We have tried to make it more efficient by
> changing the parameters "interface", and "net usage" of the amanda.conf
> and did not succeed( Probably those parameters do not matter). We would
> like to perform Full backup every Day to a SDLT Tape drive.
> We will be adding 2 IBM P570's and we are concerned about the duration
> of the backup. Can we find how the big sites manage this type of
> situation?. I appreciate any comments on this subject.

Seconding Dave's comments, figure out what pieces of the backup take time.

One additional thought, see if the estimate phase is an appreciable
part of the total time.  As you are doing full backups daily, I presume
you have a pretty good idea of the amount of data daily.  In that case
newer versions of amanda have alternatives to the estimate phase that
can greatly speed up that part of the backup.

-- 
Jon H. LaBadie  [EMAIL PROTECTED]
 JG Computing
 4455 Province Line Road(609) 252-0159
 Princeton, NJ  08540-4322  (609) 683-7220 (fax)


Re: Duration Of Backup

2005-08-17 Thread Dave Ewart
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wednesday, 17.08.2005 at 09:13 -0400, Mangala Gunadasa wrote:

> We have been using amanda for many years. Our capacity had grown and
> currently we are backing up 60 file systems across 12 servers. The
> backup takes 9hrs to finish. We have tried to make it more efficient
> by changing the parameters "interface", and "net usage" of the
> amanda.conf and did not succeed( Probably those parameters do not
> matter). We would like to perform Full backup every Day to a SDLT Tape
> drive.  We will be adding 2 IBM P570's and we are concerned about the
> duration of the backup. Can we find how the big sites manage this type
> of situation?. I appreciate any comments on this subject.

To figure out a way to reduce the run time of the backups, you need to
work out whether you are hitting a network bandwidth limit, a hardware
limitation or something else.

Some ideas:

1. Relaxing your condition of a full backup every day would reduce your
backup duration, since each day AMANDA would be doing full backups of
some systems and level 1 backups of others.  If your filesystems are
fairly static, this may make sense.  I obviously don't know your
circumstances, but do you *really* need a full backup of *everything*
every day?  If you have a largely static setup, then insisting on a full
backup only once every two days may reduce your run time from 9 hours
down to 5 or 6 hours.

2. Upgrade your network infrastructure or your AMANDA server.  For
example, if you're running 100megabit, can you switch to gigabit?  Can
you give your AMANDA server a faster holding disk?

3. Get another tape drive and run backups in parallel.  Note however,
that this will not reduce run time if your backup job is
network-bandwidth-bound.

4. Check your compression configuration parameters.  Are dumps being
compressed on the client, or the server, or on the tape drive?  For
example, are slow clients being made to compress stuff themselves?

Dave.
- -- 
Dave Ewart
[EMAIL PROTECTED]
Computing Manager, Cancer Epidemiology Unit
Cancer Research UK / Oxford University
PGP: CC70 1883 BD92 E665 B840 118B 6E94 2CFD 694D E370
N 51.7518, W 1.2016
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFDAzu1bpQs/WlN43ARAtLiAJ95fwMkh+awuET9OGw/7I7QfohiugCeNRXO
ccD6x1zIA5lMf2y9J7FXW1I=
=uaa+
-END PGP SIGNATURE-


Re: Duration Of Backup

2005-08-17 Thread Mangala Gunadasa
Greetings

We have been using amanda for many years. Our capacity had grown and
currently we are backing up 60 file systems across 12 servers. The
backup takes 9hrs to finish. We have tried to make it more efficient by
changing the parameters "interface", and "net usage" of the amanda.conf
and did not succeed( Probably those parameters do not matter). We would
like to perform Full backup every Day to a SDLT Tape drive.
We will be adding 2 IBM P570's and we are concerned about the duration
of the backup. Can we find how the big sites manage this type of
situation?. I appreciate any comments on this subject.


Thank You,

Mangala Gunadasa
EHIT/Montefiore Medical Center




Backup too long with windows client

2005-08-17 Thread tanguy yoann
Hi,

I have a problem with a backup:  it puts too much
time.  I do not backup many data but that takes a long
time.  I made this backup before and that lasted only
a few seconds.  I backup a Windows share by smbclient.
 I do not understand the problem. So I send to you
files of debug to see whether you will be able to
understand what does not function correctly. Thanks a
lot.

1)amandad sendsize

Amanda 2.4 REQ HANDLE 000-18820608 SEQ 1124436519
SECURITY USER backup
SERVICE sendsize
OPTIONS
features=feff9ffe7f;maxdumps=1;hostname=debian.neo.net;
GNUTAR //Yoann/backup 1 2005:8:18:7:54:1 -1 OPTIONS
|;bsd-auth;compress-best;index;
GNUTAR //Yoann/backup 2 2005:8:18:7:54:1 -1 OPTIONS
|;bsd-auth;compress-best;index;


amandad: time 0.000: sending ack:

Amanda 2.4 ACK HANDLE 000-18820608 SEQ 1124436519


amandad: time 0.000: bsd security: remote host
debian.neo.net user backup local user backup
amandad: time 0.000: amandahosts security check passed
amandad: time 0.000: running service
"/usr/lib/amanda/sendsize"
amandad: time 239.967: sending PREP packet:

Amanda 2.4 PREP HANDLE 000-18820608 SEQ 1124436519
OPTIONS features=feff9ffe7f;


amandad: time 360.013: sending PREP packet:

Amanda 2.4 PREP HANDLE 000-18820608 SEQ 1124436519
OPTIONS features=feff9ffe7f;
//Yoann/backup 1 SIZE 255


amandad: time 360.069: sending PREP packet:

Amanda 2.4 PREP HANDLE 000-18820608 SEQ 1124436519
OPTIONS features=feff9ffe7f;
//Yoann/backup 1 SIZE 255
//Yoann/backup 2 SIZE 255


amandad: time 360.069: sending REP packet:

Amanda 2.4 REP HANDLE 000-18820608 SEQ 1124436519
OPTIONS features=feff9ffe7f;
//Yoann/backup 1 SIZE 255
//Yoann/backup 2 SIZE 255


amandad: time 360.070: got packet:

Amanda 2.4 ACK HANDLE 000-18820608 SEQ 1124436519


amandad: time 360.070: pid  finish time Fri Aug 19
09:34:38 2005

2)amandad sendbackup

Amanda 2.4 REQ HANDLE 000-E8ED0608 SEQ 1124436518
SECURITY USER backup
SERVICE sendbackup
OPTIONS
features=feff9ffe7f;hostname=debian.neo.net;
GNUTAR //Yoann/backup  1 2005:8:18:7:54:1 OPTIONS
|;bsd-auth;compress-best;index;


amandad: time 0.000: sending ack:

Amanda 2.4 ACK HANDLE 000-E8ED0608 SEQ 1124436518


amandad: time 0.000: bsd security: remote host
debian.neo.net user backup local user backup
amandad: time 0.000: amandahosts security check passed
amandad: time 0.000: running service
"/usr/lib/amanda/sendbackup"
amandad: time 0.003: sending REP packet:

Amanda 2.4 REP HANDLE 000-E8ED0608 SEQ 1124436518
CONNECT DATA 50084 MESG 50085 INDEX 50086
OPTIONS features=feff9ffe7f;


amandad: time 0.006: got packet:

Amanda 2.4 ACK HANDLE 000-E8ED0608 SEQ 1124436518


amandad: time 0.006: pid 6709 finish time Fri Aug 19
09:34:38 2005

3)Sendsize

sendsize: debug 1 pid 6667 ruid 34 euid 34: start at
Fri Aug 19 09:28:38 2005
sendsize: version 2.4.5
sendsize[6706]: time 359.946: calculating for amname
'//Yoann/backup', dirname '//Yoann/backup', spindle -1
sendsize[6706]: time 359.946: getting size via
smbclient for //Yoann/backup level 1
sendsize[6706]: time 359.946: spawning
/usr/bin/smbclient in pipeline
sendsize[6706]: argument list: smbclient
\\Yoann\backup -d 0 -U backup -E -W NEOTIP -c "archive
1;recurse;du"
sendsize[6667]: time 359.947: waiting for any estimate
child: 1 running
sendsize[6706]: time 360.004: Domain=[YOANN]
OS=[Windows 5.1] Server=[Windows 2000 LAN Manager]
sendsize[6706]: time 360.008: 
sendsize[6706]: time 360.008:   38146 blocks of size
1048576. 9619 blocks available
sendsize[6706]: time 360.008: Total number of bytes:
260218
sendsize[6706]: time 360.009: .
sendsize[6706]: estimate time for //Yoann/backup level
1: 0.063
sendsize[6706]: estimate size for //Yoann/backup level
1: 255 KB
sendsize[6706]: time 360.009: waiting for
/usr/bin/smbclient "//Yoann/backup" child
sendsize[6706]: time 360.009: after /usr/bin/smbclient
"//Yoann/backup" wait
sendsize[6706]: time 360.009: getting size via
smbclient for //Yoann/backup level 2
sendsize[6706]: time 360.009: spawning
/usr/bin/smbclient in pipeline
sendsize[6706]: argument list: smbclient
\\Yoann\backup -d 0 -U backup -E -W NEOTIP -c "archive
1;recurse;du"
sendsize[6706]: time 360.061: Domain=[YOANN]
OS=[Windows 5.1] Server=[Windows 2000 LAN Manager]
sendsize[6706]: time 360.064: 
sendsize[6706]: time 360.064:   38146 blocks of size
1048576. 9619 blocks available
sendsize[6706]: time 360.064: Total number of bytes:
260218
sendsize[6706]: time 360.065: .
sendsize[6706]: estimate time for //Yoann/backup level
2: 0.055
sendsize[6706]: estimate size for //Yoann/backup level
2: 255 KB
sendsize[6706]: time 360.065: waiting for
/usr/bin/smbclient "//Yoann/backup" child
sendsize[6706]: time 360.065: after /usr/bin/smbclient
"//Yoann/backup" wait
sendsize[6706]: time 360.066: done with amname
'//Yoann/backup', dirname '//Yoann/backup', spindle -1
sendsize[6667]: time 360.066: child 6706 terminated
normall

Re: odd dump timeout symptoms

2005-08-17 Thread Jon LaBadie
On Wed, Aug 17, 2005 at 03:05:01PM +1000, Jamie Wilkinson wrote:
> This one time, at band camp, Jon LaBadie wrote:
> >Search back over the list archives for details that I don't remember.
> 
> Thanks :)
> 
> >I think some have had this symptom when there was some sort of network
> >timeout setting that affected the index stream.  Not certain, but I
> >think it was a UDP setting for how long a connection could be open.
> 
> UDP?  sendbackup streams back over TCP, surely?
> 

As I noted, I was uncertain of the details, someone understanding more
about the networking aspects can comment more.  But just for clarification
of the way I worded that:

- I did not say anything about the backup data stream over udp
- Your logs showed it was the index backup stream having problem,
  not the backup stream (they are separate)
- I said the timeout setting "affected" the stream, I wasn't sure
  if it was the stream itself or not
- amanda does use some udp connections, perhaps for control communication

-- 
Jon H. LaBadie  [EMAIL PROTECTED]
 JG Computing
 4455 Province Line Road(609) 252-0159
 Princeton, NJ  08540-4322  (609) 683-7220 (fax)


Re: HD backup strategy ?

2005-08-17 Thread Jon LaBadie
On Wed, Aug 17, 2005 at 03:03:59PM +0300, Toomas Aas wrote:
> Jon LaBadie wrote:
> 
> >Haven't seen anyone on the list mention using it, but Iomega
> >introduced some interesting hardware last year.  I think they
> >call it "Rev", basically a small, removalble hard drive
> >cartridge.  Think high capacity, tiny zip drive as it has
> >35GB native capacity and a builtin compressor.  Along with
> >their single slot device, they also introduce an autoloader
> >that can hold 10 cartridges.
> 
> Funny, I just stumbled upon this on another mailing list yesterday. 
> Looks like IOMega continues to live up to its reputation:
> 
> http://lists.freebsd.org/pipermail/freebsd-questions/2005-July/094114.html

A peek at the IOMygosh website confirms the articles comment that
it is windows only!  Time for me to stop even thinking of it as a
potential amanda backup device.

jl
-- 
Jon H. LaBadie  [EMAIL PROTECTED]
 JG Computing
 4455 Province Line Road(609) 252-0159
 Princeton, NJ  08540-4322  (609) 683-7220 (fax)


Re: HD backup strategy ?

2005-08-17 Thread Toomas Aas

Jon LaBadie wrote:


Haven't seen anyone on the list mention using it, but Iomega
introduced some interesting hardware last year.  I think they
call it "Rev", basically a small, removalble hard drive
cartridge.  Think high capacity, tiny zip drive as it has
35GB native capacity and a builtin compressor.  Along with
their single slot device, they also introduce an autoloader
that can hold 10 cartridges.


Funny, I just stumbled upon this on another mailing list yesterday. 
Looks like IOMega continues to live up to its reputation:


http://lists.freebsd.org/pipermail/freebsd-questions/2005-July/094114.html

--
Toomas Aas 
|arvutivõrgu peaspetsialist | head specialist on computer networks|
|Tartu Linnakantselei   | Tartu City Office   |
- +372 736 1274