Re: GNU tar syntax (or How does amanda do that?)
"John R. Jackson" wrote: > > >/usr/local/bin/tar -cvf - /home | rsh linux-host dd of=/dev/nst0 > > > >(you may want to specify '-b blocksize' to the tar command... and > > 'bs=blocksize' to the dd) > > The only problem with using dd like this is that dd does not do output > reblocking. It takes whatever record size comes in (i.e. the size of > the read()) and that's what it writes. As far as I have used dd on Linux, I can recommend "obs=32k". It does the trick of reblocking. Once I had a slow network between a DLT drive and the host I wanted to backup from: ssh hostname "/sbin/dump 0af - /" | dd obs=32k of=$TAPE ...was a pain for the tape drive My first guess using a buffer with dd ssh host "/sbin/dump 0af - /" | dd bs=100M | dd obs=32k of=$TAPE didn't succeed, but when I used ssh host "/sbin/dump 0af - /" | dd obs=100M | dd obs=32k of=$TAPE the tape drive rocked! Even if the manpage tells you: "if you specify bs, ibs and obs will be set to bs", dd will do no reblocking in this case.
Re: GNU tar syntax (or How does amanda do that?)
>/usr/local/bin/tar -cvf - /home | rsh linux-host dd of=/dev/nst0
>
>(you may want to specify '-b blocksize' to the tar command... and
> 'bs=blocksize' to the dd)
The only problem with using dd like this is that dd does not do output
reblocking. It takes whatever record size comes in (i.e. the size of
the read()) and that's what it writes.
With networking in between, those read's can be all sorts of interesting
sizes, which in turn will write all sorts of interesting record sizes to
the tape, which in turn will be hard to read back later. Like at 2AM.
When you really, really don't need yet another problem :-).
Adding "-b blocksize" to the tar or "bs=blocksize" to the dd will have
no effect on this whatsoever.
I think some versions of dd have options to deal with this, but it's not
broadly supported, which has always been one of the major mysteries of
Unix to me.
You might be better using "cat", which will (hopefully) use stdio and
thus all the output write's will be the same size (BUFSIZ from stdio.h).
Maybe not a size you want, or one you can control, but at least all
the same.
I wrote a little perl script years ago to deal with this (appended --
I call it "reblock"). As I look at it, it could use some better error
handling, and I'm certain could be coded better, but it's never failed me
(and dd certainly has). Your example would then go like this:
rsh linux-host mt -f /dev/nst0 rewind
/usr/local/bin/tar -cvf - /home | rsh linux-host "reblock > /dev/nst0"
John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
#!/usr/local/bin/perl
# Perform I/O reblocking to force all output records (except the last)
# to be of the same size.
if ($#ARGV >= 0) {
$BlockSize = shift (@ARGV);
$BlockSize = $BlockSize*1024 if ( $BlockSize =~ m/[kK]$/ );
$BlockSize = $BlockSize*512 if ( $BlockSize =~ m/[bB]$/ );
$BlockSize = $BlockSize*2 if ( $BlockSize =~ m/[wW]$/ );
} else {
$BlockSize = 10 * 1024; # normal tar blocking size
}
$buf = '';
$b = 0; # data in the buffer
while (($r = sysread (STDIN, $buf, $BlockSize, $b)) > 0) {
$b += $r;
if ($b >= $BlockSize) {
syswrite (STDOUT, $buf, $BlockSize, 0);
$b -= $BlockSize;
if ($b > 0) {
$buf = substr ($buf, $BlockSize, $b);
}
}
}
if ($b > 0) {
syswrite (STDOUT, $buf, $b, 0);
}
exit (0);
Re: GNU tar syntax (or How does amanda do that?)
>Date: Fri, 11 May 2001 11:08:59 -0600 >From: Paul Brannigan <[EMAIL PROTECTED]> > Subject: GNU tar syntax (or How does amanda do that?) > [snip] > Yet when I try to do this at the command line I get > tar complaints. Am I using the wrong syntax? > > solaris-host>#/usr/local/bin/tar -cvf linux-host:/dev/nst0 /home > sh: unknown host > /usr/local/bin/tar: Cannot open linux-host:/dev/nst0: I/O error > /usr/local/bin/tar: Error is not recoverable: exiting now > > Why the odd error: "sh: unknown host" [snip] I don't think this has anything to do with GNU tar except that GNU tar is calling rmt/rsh. Try the following and see what happens... /usr/local/bin/tar -cvf - /home | rsh linux-host dd of=/dev/nst0 (you may want to specify '-b blocksize' to the tar command... and 'bs=blocksize' to the dd)
Re: GNU tar syntax (or How does amanda do that?)
>Amanda some how uses GNU tar to archive >files on a Solaris 2.6 host to a remote tape device that is >attached to a Linux (7.0) host. > >Yet when I try to do this at the command line I get >tar complaints. Am I using the wrong syntax? Yes. Amanda doesn't do things the way you're thinking. It does not pass the tape device name to GNU tar and have it write to the tape. Amanda runs all (not just GNU tar) backup programs in a mode that has them write their image to stdout. Amanda then gathers that and sends it across the wire to the server. The server may write it into the holding disk or directly to tape. But GNU tar (or ufsdump, etc) know nothing about this. >solaris-host>#/usr/local/bin/tar -cvf linux-host:/dev/nst0 /home >sh: unknown host >/usr/local/bin/tar: Cannot open linux-host:/dev/nst0: I/O error >/usr/local/bin/tar: Error is not recoverable: exiting now > >Why the odd error: "sh: unknown host" I just tried this between two of my machines and things worked fine. Then I tried deliberately entering a bad host name: $ gtar cvf xxx:/tmp/jrj/z . xxx: unknown host gtar: Cannot open xxx:/tmp/jrj/z: I/O error gtar: Error is not recoverable: exiting now which matches what you're seeing except "xxx" instead of "sh". So there is clearly something wrong in the host name part of what you tried to do, but unless you entered "sh" as the host name, I don't know exactly what. What happens if you do this: $ rsh linux-host pwd >I had read some posts about a buggy version of GNU tar. >The GNU mirror sites don't seem to specify any revisions, >just 1.13 not the rumored 1.13.19 or the 1.13.17 which is >running on my linux host. I don't know what they have. You need to get the .17/.19 from alpha.gnu.org. >Paul John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
GNU tar syntax (or How does amanda do that?)
Some times I would like to just use GNU tar at the command line for a quick archive to a tape device on a remote host. My Amanda system seems to do this very well. Amanda some how uses GNU tar to archive files on a Solaris 2.6 host to a remote tape device that is attached to a Linux (7.0) host. Yet when I try to do this at the command line I get tar complaints. Am I using the wrong syntax? solaris-host>#/usr/local/bin/tar -cvf linux-host:/dev/nst0 /home sh: unknown host /usr/local/bin/tar: Cannot open linux-host:/dev/nst0: I/O error /usr/local/bin/tar: Error is not recoverable: exiting now Why the odd error: "sh: unknown host" In every respect these two host know about each other. with wide open permissions for logins, .rhosts etc. I had read some posts about a buggy version of GNU tar. The GNU mirror sites don't seem to specify any revisions, just 1.13 not the rumored 1.13.19 or the 1.13.17 which is running on my linux host. But like I said, as long as I let Amanda do the tar all is well. I just wanted to be able to do a quick tar on the Solaris host without changing any of my amanda configurations. What say you good people? Paul begin:vcard n:Brannigan;Paul tel;pager:303-826-2365 tel;fax:303-245-1025 tel;work:303-245-1045 x-mozilla-html:TRUE adr:;;3434 47th St.;Boulder;Colorado;80301;USA version:2.1 email;internet:[EMAIL PROTECTED] title:Support Engineer x-mozilla-cpt:;-5024 fn:Paul Brannigan end:vcard
