Re: Backing up PostgreSQL?

[Martín Marqués]

On Vie 21 Jun 2002 20:32, Greg A. Woods wrote:
> [ On Friday, June 21, 2002 at 18:00:19 (-0300), Martín Marqués wrote: ]
>
> >
> > Other UNIX? If you could rely on it, why does Informix (and other
> > database servers, but this is the one I used) still use it's own FS to
> > write the database files?
>
> Because there are tremendous performance advantages to using RAW I/O if
> you're prepared to go to the trouble of dealing with all the niggling
> details
>
> (some of these advantages are supposedly less visible on modern hardware
> and modern systems, but they sure as heck were plainly visible even a
> few years ago)

You sure are right about this. That's why some comercial databases are 
changing to OS read/write instead of doing raw read/writes. :-)

-- 
Porqué usar una base de datos relacional cualquiera,
si podés usar PostgreSQL?
-
Martín Marqués  |[EMAIL PROTECTED]
Programador, Administrador, DBA |   Centro de Telematica
   Universidad Nacional
del Litoral
-

Re: Backing up PostgreSQL?

[Ragnar Kjørstad]

On Fri, Jun 21, 2002 at 08:43:27PM -0400, Greg A. Woods wrote:
> > There are no writes to the filesystem at all while the snapshot is in
> > progress. The LVM-driver will block all data-access until it is
> > completed. If not, it wouldn't be a snapshot, would it?
> 
> I realize that -- but it doesn't mean the snapshot couldn't begin
> between two inter-related writes to two separate files (or even two
> separate but inter-related writes to two separate pages in the same
> file).  

In which case postgreSQL must rely on it's log on startup to finnish the
set of write-operations.

> If both writes are necessary to result in a consistent DB then
> the snapshot will necessarily be inconsistent by definition. 

Well yes, if you define "consistent" DB to mean you don't have to use
the WAL. I find this definition a little strange, as what I care about
is if my data is still there (and still correct) when I restore the
database.

Anyway - we've covered this so let's not go there again.


> > Huh? Why would you want a seperate backup of the database transaction
> > log? The log is stored in a file together with the rest of the database,
> > and will be included in the snapshot and the backup.
> 
> Yeah, but I'm assuming that I'll have to manually UNDO any uncommitted
> transactions, as per the 7.2.1 manual.

It says the aborted transactions will occupy space - it doesn't say that
you need to manually UNDO them. It doesn't say specificly that you
_don't_, but it's the only thing that makes sense; primarely because
it's impossible to do manual UNDO. (what if a transaction updated an
index but didn't get around to update the table, or the other way around
- how do you fix that manually?)



-- 
Ragnar Kjørstad
Big Storage

Re: Backing up PostgreSQL?

[Ragnar Kjørstad]

On Fri, Jun 21, 2002 at 08:30:44PM -0400, Greg A. Woods wrote:
> > It should only require REDO.
> > No changes are made to the actual database-files until the transaction
> > is commited, written to the WAL and fsynced. At this point there is no
> > longer a need for UNDO.
> 
> Hmmm possibly.  I'm not that intimately familiar with the current
> WAL implementation, though what I have stated comes directly from the
> 7.2.1 manual.  If I'm wrong then so is the manual.  :-)

I suppose I ought to make it clear that I'm not a postgreSQL developer
(haven't even read the code) and my statement is purely based on the
manual and and general database knowledge.

The manual _does_ state that the main benefit from the WAL is
consistency, and it wouldn't be if it made changes to the real
database-files before the transaction commited. 

I totally agree the manual could be clearer on this point.

> > I really don't see why relying on WAL is any different from relying on
> > other postgresql features - and it is hard to run a postgresql database
> > without relying on postgresql
> 
> well, the WAL implementation is new, acknowledged to be incomplete, and
> so far as I can tell requires additional work on startup
> 
> Indeed re-loading a pg_dump will take lots more time, but that's why I
> want my DB backups to be done both as a ready-to-roll set of file images
> as well as a pg_dump  :-)

Yes, the more redundant backup-solutions the better :)



-- 
Ragnar Kjørstad
Big Storage

Re: Backing up PostgreSQL?

[Ragnar Kjørstad]

On Fri, Jun 21, 2002 at 09:01:17PM -0400, Michael H.Collins wrote:
> I thought i just read that postgresql on ext3 outran oracle on raw
> devices.

Yes, it's funny how all database engines are faster than all other
database engines, isn't it?

> ~Because there are tremendous performance advantages to using RAW I/O if
> ~you're prepared to go to the trouble of dealing with all the niggling
> ~details
> ~
> ~(some of these advantages are supposedly less visible on modern
> ~hardware and modern systems, but they sure as heck were plainly visible
> ~even a few years ago)

There are (As far as I can think of) two aspects of filesystem vs device
access for database-performance:
1. There is some overhead in doing the operations through the
   filesystem. I think this is neglectable with an extent-based
   filesystem; for inode-based filesystems it may be messurable, but
   I doubt it's an important factor.

2. Caching. 
   Usually a read operation on a file will cause the file to be cached
   in RAM, and if the database-engine caches the same data you're
   wasting RAM and slowing down the system. Modern operating system
   changes this in two ways:
   1. The caching is better, so it's possible for a db-engine to
  rely on the os-cache instead of implementing it's own. 
   2. It's possible to turn off OS cache with raw-io.

There are possible some performance advantages of going through the
filesystem as well. For instance some filesystems use knowledge about
the physical hardware (such as the number of spindels and the chunk-size
of the RAID) to optimize IO. Unless the db-engine duplicate theese
features the OS may be able to do the IO more effectively.

The main reason for switching to using a filesystem is operational
though. I once heard that one of the major db-companies (Oracle?) were
going to start doing this on the default configuration because it would
eliminate all the problems with clueless users that "found" a free
partition and started using it for something else :)



-- 
Ragnar Kjørstad
Big Storage

Re: new label templates from Ulrik Sandberg, where are they?

[Ulrik Sandberg]

On Mon, 17 Jun 2002, Patrick C. F. Ernzer wrote:

> Hi,
>
> on Fri Apr 19 Ulrik Sandberg posted about new templates, I fail to find
> these labels in the CVS and in the archives.
>
> Where can these templates be found?
>
> Are there plans to include these in the examples directory at some point?

Sorry for the delay. Holidays.

I decided to keep sending the templates individually to everyone who asked
for them, until someone told me they are stored somewhere safe.

I'll send you them off-list in a moment.

--
Ulrik Sandberg

unsubscribe

[Jörg Reiners]

unsubscribe [EMAIL PROTECTED]

unsubscribe

[manoj]

every run a new tape?

[Juanjo]

Why amanda uses a new tape on every night run?

Tapes are 40/80GB DLT1, and I dont understand why It dumps daily
incrementals to a new tape always, most of tapes are not even at 20%.

Any help? Is it any parameter I've missed out?

Thanks.

irix 6.5.14 and amanda

[Robert . Weber]

We upgraded(patched) irix on some of our sgi's and noticed core.pid files
started appearing in /.  From the amanda logs it appears that amanda
cannot sigterm the xfsdumps and has resulted in SIGKILL.  The kills are
apparently causing the xfsdumps to dump core.  Is anyone else seeing this?
I'm running 2.4.2

Robert Weber
University of Colorado

Yet another DDS question

[Dan Smith]

Is there anything stopping me from using DDS4 tapes in my DDS2 drive?
Seems like I could just get extra capacity from the drive by having a
longer tape.  Will this work?  Will it work if I turn off media
recognition?

Just curious.

--Dan

Re: Amanda won't dump one particular partition

[John R. Jackson]

>Yeah, still have a problem.  ...

Nuts!  I was hoping your problem would just go away on its own :-).

>Am I doing this on the troublesome client or on the tape host?

The client.

>  [EMAIL PROTECTED]

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

planner and sendsize

[Juanjo]

Here it goes something interesting too:

sendsize.log>

calculating for amname '/data0', dirname '/data0'
sendsize: getting size via dump for /data0 level 0
sendsize: running "/sbin/dump 0Ssf 1048576 - /dev/md0"
/dev/md0: Bad magic number in super-block while opening filesystem
  DUMP: The ENTIRE dump is aborted.
running /usr/local/amanda/libexec/killgrp
.

???

Tar patch

[Vijay Kumar]

hi!


How do i apply the Tar patfch in the patches directory?
I am using tar 1.13.19 . Do i still need the patch?

thanks for any replies,

Vijay

Re: What to do about the holidays?

[Anthony A. D. Talltree]

> Is this okay to do? 

Yep, I've done it when eg. I was snowed in for a week and couldn't get
in to change the tape.  It's a good argument for a large holding disk,
and this is a feature that I haven't found in other, rather expensive,
backup packages.

DLT

[David Flood]

Sorry if this subject is off topic but can you use a 40/80 DLT tape 
in a 35/70 DLT tape drive. Although we'd only be using it as a 35/70 
tape.

-
David Flood
Systems Administrator
[EMAIL PROTECTED]
Tel: +44 (0)1224 262721
The Robert Gordon University
School of Computing
St. Andrews Street
Aberdeen
-

Re: S.O.S.

[Stephen Carville]

runtapes 2

You have to have a working changer configuration.

On Fri, 11 Jan 2002, [iso-8859-15] Túlio Machado de Faria wrote:

- Help me, please.
-
-
- I need to use amanda with two units of tape in one backup.
-
- How?
-
-

-- 
-- Stephen Carville
UNIX and Network Administrator
Ace Flood USA
310-342-3602
[EMAIL PROTECTED]

--with-group?

[David Flood]

What is the significance of the group when configuring i.e. --with-
group.  Does it mean whoever is in this group can run the amanda 
utils as well as the amanda user?

-
David Flood
Systems Administrator
[EMAIL PROTECTED]
Tel: +44 (0)1224 262721
The Robert Gordon University
School of Computing
St. Andrews Street
Aberdeen
-

Re: Documentation

[Eric Trager]

On 19 Apr 2002, Marcos Teodoro Dias Junior wrote:

> Hello,
>
> Where I can find some documentation at how to use amanda with tape
> changers ?

In the source tree, look in docs/TAPE.CHANGERS .

- Eric

Warning - old messages being resent to list

[Frank Smith]

Before replying to a message, check the sent date.  A batch of old
messages is being re-injected into the list.

Frank


--
Frank Smith   [EMAIL PROTECTED]
Systems AdministratorVoice: 512-374-4673
Hoover's OnlineFax: 512-374-4501

multiple types of systems.

[dave]

Hello,
I've got amanda installed on a redhat 7.2 box. I'm wondering if it can
back up a mixture of different clients, a few various windows machines, two
FreeBSD clients and a slackware box?
Also, wondering if it works with an hp colorado internal IDE tape drive?
Thanks.
Dave.

--portrange questions

[David Hayes]

I need to back up a machine that's behind a firewall, so I'm going to
reconfigure/recompile my server and clients with:

  --with-udpportrange=850,855 --with-portrange=X,Y

What's the minimum number of ports I can allocate on my server?  I'm
backing up 6 Unix machines with approximately 50 disklist entries.  Is
there some formula that relates portrange with disklist entries,
clients, dumpers, etc?

Thanks.

--david

changer.conf and tpchanger for sun storedge 4mm dds-3 autoloader

[Ben Fallah]

I have a Sun StorEdge 4mm DDS-3 autoloader (magazine of 6 tapes loader)
http://www.sun.com/storage/dds/index.html
any one with correct changer.conf file and know what tpchanger be set to in
amanda.conf ?
Thank You
-
Ben Fallah
Senior System/Network Engineer

Re: amanda-users list problem

[Bernhard R. Erdmann]

> That got brought to my attention today, but unfortunately, my tape cycle
> is only eight days, so the last good backup of the file was erased
> quite some time ago.

What tapes do you use?

Re: amanda was getting confused, but apparently no more :-)

[John R. Jackson]

>I wonder how hard it would be to train amanda to rotate holding 
>areas (partitions) on a raid instead of tapes?  ...

That's already part of 2.4.3 (or the amanda-242-tapeio CVS branch).

>Gene

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: Simultaneous Multi-Tape Multi-Host Backups

[John R. Jackson]

>...  Can Amanda backup the 4 filesystems to the 4 tape-drives
>simultaneously from one config file?  ...

No.  That will take a major overhaul (although it's definitely needed).

>Aaron Rainwater

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

advfs dislist entry

[Eric Zylstra]

Should be a simple one...

What is the proper way to enter an advfs partition into the disklist?

I looked into the /etc/fdmns// directory, saw which 
partition was involved and put that into the disklist.  The report comes 
back:

FAIL planner   0 [disk  offline on ?]

Thanks,

EZ

Amanda and NT shares

[Mike Hendrix]

I am trying to figure something and am looking for assistance.  I have 
checked the newsgroups without success.

The question is:  When I backup an NT/2000 share does amanda/tar even touch 
the archive bit on a file??  Can someone explain exactly what happens??  

---
"I have not failed. I've just found 10,000 ways
that won't work." - Thomas Edison

Michael Hendrix   [EMAIL PROTECTED]
Systems Engineer / SysAdmin Team Leader
Logical Net / Capital Net (518) 292-4509

Re: Another amcheck problem

[John R. Jackson]

>Prior to the run a netstat -a | grep amanda shows
>
>udp0  0 *:amanda*:*
>...

That's normal.

>After the failure a netstat -a | grep amanda  doesn't return anything.

Then amandad is probably dieing.  The FAQ talks about how to debug this.

>Shawn

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

unsubscribe

[Monserrat Seisdedos Nuñez]

unsubscribe

Re: self check : request timed out

[Joshua Baker-LePain]

On Tue, 26 Mar 2002 at 6:35pm, Melvin Lobo wrote

>   i have just installed Amanda and when i give amcheck -c conf i get the following 
>though the system is not using 
> 
>  WARNING:  : selfcheck request timed out.  Host down? 

This is a very common problem when first getting started.  Have you gone 
through FAQ-O-Matic (available through www.amanda.org)?

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: amrecover: did not get a reserved port:

[Tom Van de Wiele]

How exactly would I apply this patch?




"John R. Jackson" wrote:
> 
> >amrecover: did not get a reserved port: 50846
> >
> >What IS this? how do I solve it?  ...
> 
> I answered this exact same question three days ago:
> 
>   http://groups.yahoo.com/group/amanda-users/message/34226
> 
> >Tom Van de Wiele
> 
> John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: Is this correct?

[Chris Dahn]

On Thursday 13 December 2001 07:40 am, Adolfo Manuel Pachón Rodríguez wrote:
> Hi all:
>
> I have 5 tapes for one HP24DAT SCSI unit. I want to use one tape per day,
> not including saturday/monday.
>
> Is this the correct configuration?
>
> dumpcycle 7 days
> runspercycle 5 days
> tapecycle 5 tapes
>
> thanks

  Looks good to me.

-- 

Chris Dahn, SERG Code Ninja
  3141 Chestnut St.
  Attn: MCS Department
  Philadephia, PA  19104
  Office: 215.895.0203
  Fax: 215.895.1582

<->Software Engineering Research Group<->
Feel the SERG!
http://serg.mcs.drexel.edu/
CAT 186, The Microwave
http://pgp.mit.edu:11371/pks/lookup?search=Christopher+Dahn&op=index

amrecover

[Dan Barnes]

Hi all,

I'm doing my amanda backups to a spare disk. I use gtar as the backup
method. The backup process works just fine, but when I go to restore with
amrecover I get:


Extracting files from holding disk on host cain.
The following files are needed: /opt/amanda/dumps/20020405/cain._.2

Restoring files into directory /tmp/restore
Continue? [Y/n]: y

Extracting from file /opt/amanda/dumps/20020405/cain._.2
EOF, check amidxtaped.debug file on cain.
amrecover: short block 0 bytes
UNKNOWN file
amrecover: Can't read file header
extract_list - child returned non-zero status: 1

There is no amidxtaped.debug to be found. 

Anyone have any ideas? This happens every time.

Thanks for any suggestions or help.

- db

skipping cruft file

[Steve]

Hi all.

I've installed amanda from the latest source after removing the SuSE 7.3 rpm 
version.

When I run amcheck it passes server test and dies on client:
WARNING: localhost: selfcheck request timed out. Host down?

I've gone through the amanda chapter in the book recommended, the same with 
INSTALL file and cannot see anything I've missed. So I ran amoverview and it 
reported:
bad date 20011205: in   20011205: found Amanda directory.
bad date .bashrc: in  .bashrc: skipping cruft file, perhaps you should delete 
it.
date11
hostdisk30
localhoshda30

(Missing t is not a typo)

I've search every file without finding any 'cruft' reference. What is going 
on? I got further than this on the rpm version...

-- 
 
Steve Szmidt
V.P. Information
Video Group Distributors, Inc.

Changers

[Ignacio Dosil Lago]

Hi, everybody!
I´m testing Amanda (without tape device) and shurely this will become my 
new backup suite.
My questions are: What tape changers do you un/recommend me?
What should I know about this kind of devices?
Thanks!

Chunksize

[dpf]

What does the chunksize in Amanda 2.4.2p2 control and can I just 
comment it out? I'm running Amanda server on Solaris connected 
to a DLT 35/70 So if I can't comment it out, what is the 
recommended chunksize for this size of disk?

Also would a incorrectly sized chunksize slow down the backups? 
My chunksize is currently set to 2GB which may be a totally 
ridiculous figure.

Thanks in advance

-
David Flood
Systems Administrator
[EMAIL PROTECTED]
Tel: +44 (0)1224 262721
Robert Gordon University
School of Computing
St. Andrews Street
Aberdeen
-

Single tape?

[Davide]

Hi,
   I'm new to Amanda and I want to use it because I have multiple
platforms (Sun, SGI, Linux and PCs) that I need to back up. Only the
server will be backup daily while users data will be backup weekly (or
something other than daily).
Anyway, my question is, Is there a way to 'force' amanda to use the same
tape (2 or 3 days in a raw) for incremental backups? Couldn't find
anything on this and yes, I know you are not suppose to use the same tape
over but this is not crucial data (not final data) and I have a
ton of data to backup and not enough tapes.

Thanks

David

Re: Permission _still_ denied

[Stephen Carville]

On Fri, 5 Apr 2002, Doug Silver wrote:

- Hi Stephen -
-
- At first glance this appears like you might have built the amanda client
- stuff with the tcp/udp portrange but not the amanda server.  I'm not sure
- since I'm using 2.4.2p2, so perhaps with this newer version the debug
- messages are different.  Does the amanda user have permission to
- open up privileged ports (<1024 -- I think)?  That almost seems the
- problem since if that user can't open up a connection on port 960 to
- listen, but it can on 59116.

I checked the config.status for both machines and I configured amanda
"--with-user=amanda --with-group=adm".  AFAIK, the default is no
portrange.

-- 
-- Stephen Carville
UNIX and Network Administrator
DPSI (formerly Ace USA Flood Services)
310-342-3602
[EMAIL PROTECTED]

Re: Starting Fresh?

[Eric Zylstra]

On Thursday, April 11, 2002, at 07:13 AM, Rick Jones wrote:

> We have  been configuring and re-configuring/testing Amanda 2.4.3  for 
> a few
> weeks now. Now that all is
> working, we would like to start fresh...(load new tapes, have Amanda 
> forget
> about what was done up till
> now). How do I set that up?

Check the FAQ-O-MATIC:


It has all the fine details.

EZ

Re: Tape eject

[John R. Jackson]

>oh, and in addition, I did NOT use the no-rewind tape device for the eject
>command, so that the ejected tape is already rewound and ready for re-use
>again.

Ummm, that doesn't matter.  Tapes are always rewound before being ejected.

If you're using 4mm or 8mm or some type of tape with two reels in the
removable cartridge, it may seem like it would be possible to eject the
tape with it positioned part way through (ala ejecting a tape from your
VCR in the middle of a program), but all computer grade drives I know
of ignore that "capability" and rewind first.

Try it and see.  Take an Amanda tape, skip out a bunch of files then
do an "mt offline" on the no-rewind device.  Look at the tape when it
comes out.  All the media will be to one side.

>stephen.

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

disk relabeling

[Hussain Ali]

I have mislabled a disk but amanda has done dump to it. If i remove and
relabel it, will i loose the backup? 

Hussain

Re: Is this correct?

[Joshua Baker-LePain]

On Fri, 14 Dec 2001 at 9:17am, Kurt Yoder wrote

> You probably want at least one more tape in "tapecycle", so at least 6. 
> Everyone is always saying you want extras in case there's a bad run.
> 
> (Not that I understand this; after all, if there's a bad run, why not 
> just increment "tapecycle" then?)

Because that bad run may clobber the only level 0 of a particular 
filesystem that you have on tape.  Upping tapecycle at that point is 
closing the barn door after the horses are gone...

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: Amanda tape index script

[Anthony A. D. Talltree]

>What would the advantages/disadvantages be of using GNU tar 
>for all your backup needs?  Is it less efficient than vendor
>dump utilities?  Unwanted side effects?

In general, it's significantly slower, and touches the read dates on all
your files.  On the other hand, it's possible to break up a large
filesystem into chunks that are easier for Amanda to swallow.