Re: [gentoo-user] tar and huge tarballs used for back-ups
On 7/14/08, Daniel Iliev <[EMAIL PROTECTED]> wrote: >Arttu: > - Can you, please, post the command you used to make the test backups? During weekend I used pretty much the spells given on splitpipe's examples page [1], only changing the directory given to tar and the drive device (and speed for the older drive, I'm not sure if it can even *read* at this speed): tar clz /test | splitpipe -s cdr-80 -o 'cdrecord dev=/dev/cdrw speed=24 -tao -v -gracetime 2 driveropts=burnfree -data -' Last run yesterday was using star instead of tar, otherwise using that same line. Still no luck. Today I tried switching off burnfree and dramatically dropping speed for the newer drive as well: tar clz /test | splitpipe -s cdr-80 -o 'cdrecord dev=/dev/cdrom speed=4 -tao -v -gracetime 2 driveropts=noburnfree -data -' But the result was the same, otherwise ok contents, but some bytes crapped at the end of the first disc. Hmm, if this seems to be consistent, then maybe there is something calculated/const'ed wrong for sizes of CDRs in splitpipe? I think I caught a glimpse of "calculate correct sizes" somewhere in its TODOs. [1] http://ds9a.nl/splitpipe/examples.html > - Did you try splitpipe on DVDs or on CDs only? CDR-80s only, Samsung Pleomax CD-R 52X 700 MB/80 min as it says on top of the discs still in the spindle. I don't even have DVD media to try with, only couple 100 piece spindles of those same CD-Rs. Actually, my motivation with splitpipe is/was getting multivolume CDs working so I wouldn't have to move to DVDs (or tapes, the horrors!) for a while. :) >Dirk, Arttu: > - Should we get off-list on this subject or at least open a new > thread? If you move, I'll follow -- if I even have any reasonable input for the discussion (of which I'm not too sure). -- Arttu V. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Monday 14 July 2008 08:05:03 Daniel Iliev wrote: >Dirk, Arttu: > - Should we get off-list on this subject or at least open a new > thread? Let's take it to b.g.o. Bye... Dirk -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Sun, 13 Jul 2008 17:39:17 +0300
"Arttu V." <[EMAIL PROTECTED]> wrote:
> On 7/12/08, Arttu V. <[EMAIL PROTECTED]> wrote:
> > then see if I get around later to try with cdrtools instead of
> > wodim (cdrkit).
>
> Ok, my final conclusion on splitpipe: it compiles but won't work for
> me.
>
> I've produced about a dozen coasters while having tried some
> combinations of two different machines/arches (x86, amd64, both with
> stable core packages), two different burners (trusty last millennium
> HP CD-Writer+ 9100 series and newish el cheapo LG branded
> do-everything hyper-duper-combo thingie), two different tars (GNU tar
> and star) and two different pieces of burning software (cdrkit and
> cdrtools).
>
> Consistently, the failure is at the end of the first volume (disc),
> where splitpipe must have been writing some broken bytes at the end of
> the volume (possibly miscalculating something first?) -- which it
> realizes when it does the reading (as joinpipe) and then there's the
> barf ("Fatal:" probably from splitpipe) and then bad input propagates
> to tar/zip/others which join in in the mass-barfing contest:
>
> UUID of this session is '35a141883a219349 0c2b7efad2ffdaf7'
> joinpipe: volume was started on Sun Jul 13 16:38:38 EEST 2008
> joinpipe: found volume 1, as expected
> star: WARNING: skipping leading '/' on filenames.
> Fatal: during read of a stretch of input: Input/output error
> star: Tar file too small (amount: 101 bytes).
> star: Unexpected EOF on input.
> star: Cannot recover from error - exiting.
> star: 68552 blocks + 5733 bytes (total of 701978213 bytes =
> 685525.59k).
>
Dirk, Arttu,
Sorry for the late reply. I was out for the weekend.
I've tested splitpipe with DVDs on stable amd64 only. I had no
problems. Later today I'll give the improved ebuild a try.
Currently I have several blank DVDs only, so I can't send any CD burning
results.
I have some questions:
Dirk:
- can you post the compile error, please?
Arttu:
- Can you, please, post the command you used to make the test backups?
- Did you try splitpipe on DVDs or on CDs only?
Dirk, Arttu:
- Should we get off-list on this subject or at least open a new
thread?
--
Best regards,
Daniel
--
gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On 7/12/08, Arttu V. <[EMAIL PROTECTED]> wrote:
> then see if I get around later to try with cdrtools instead of wodim (cdrkit).
Ok, my final conclusion on splitpipe: it compiles but won't work for me.
I've produced about a dozen coasters while having tried some
combinations of two different machines/arches (x86, amd64, both with
stable core packages), two different burners (trusty last millennium
HP CD-Writer+ 9100 series and newish el cheapo LG branded
do-everything hyper-duper-combo thingie), two different tars (GNU tar
and star) and two different pieces of burning software (cdrkit and
cdrtools).
Consistently, the failure is at the end of the first volume (disc),
where splitpipe must have been writing some broken bytes at the end of
the volume (possibly miscalculating something first?) -- which it
realizes when it does the reading (as joinpipe) and then there's the
barf ("Fatal:" probably from splitpipe) and then bad input propagates
to tar/zip/others which join in in the mass-barfing contest:
UUID of this session is '35a141883a219349 0c2b7efad2ffdaf7'
joinpipe: volume was started on Sun Jul 13 16:38:38 EEST 2008
joinpipe: found volume 1, as expected
star: WARNING: skipping leading '/' on filenames.
Fatal: during read of a stretch of input: Input/output error
star: Tar file too small (amount: 101 bytes).
star: Unexpected EOF on input.
star: Cannot recover from error - exiting.
star: 68552 blocks + 5733 bytes (total of 701978213 bytes = 685525.59k).
--
Arttu V.
--
gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On 7/12/08, Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > Am Samstag, 12. Juli 2008 schrieb Arttu V.: > >> Actually, splitpipe compiled pretty much "as such" on amd64 for me > > Hmm, maybe a compiler issue. Which gcc version are you using? I'm using > 4.3.1. gcc version 4.1.2 (Gentoo 4.1.2 p1.1) -- Arttu V. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Samstag, 12. Juli 2008 schrieb Arttu V.: > Actually, splitpipe compiled pretty much "as such" on amd64 for me Hmm, maybe a compiler issue. Which gcc version are you using? I'm using 4.3.1. Bye... Dirk signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
On 7/12/08, Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > Am Freitag, 11. Juli 2008 schrieb Dirk Heinrichs: >> Am Freitag, 11. Juli 2008 schrieb Daniel Iliev: >> > Any help will be much appreciated. >> > >> > [1] http://bugs.gentoo.org/show_bug.cgi?id=230813 >> >> Will take a look this weekend. > > Don't even get it compiled manually :-( I took Daniel's ebuild from the bug and scrubbed it slightly. I'm no pro in ebuilds either (they always get rewritten by some real dev :) ), so I dare not claim that I would have made it any better either, only different. :) Actually, splitpipe compiled pretty much "as such" on amd64 for me and my only problems using the thing (writing files to media) were figuring out the right device and command. Ok, something was written correctly, but something wasn't. Upon restoring attempt, splitpipe/joinpipe choked at the end of the first disc: UUID of this session is '8fb1a646f13a1d02 204d506d196884ab' joinpipe: volume was started on Sat Jul 12 14:51:50 EEST 2008 joinpipe: found volume 1, as expected Fatal: during read of a stretch of input: Input/output error gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now Files before that last one on the first disc were apparently restored correctly, I checked by comparing with diff -r to the originals. However, the last file that is split across discs and any later ones were unrecoverable. I'll see where to dump the couple coasters I just became the happy owner of and then see if I get around later to try with cdrtools instead of wodim (cdrkit). I'm afraid that if someone won't try that, it'll be all FUD on cdrkit and bugs all over again (without us actually knowing whether it really was, e.g., a PEBKAC/ID-10T-problem at my end). Ebuild is attached in the bug (#230813) if anyone wants to try this further, e.g., on x86 or with dvds. I only tried on amd64 and with CDR-80s. -- Arttu V. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Freitag, 11. Juli 2008 schrieb Dirk Heinrichs: > Am Freitag, 11. Juli 2008 schrieb Daniel Iliev: > > Any help will be much appreciated. > > > > [1] http://bugs.gentoo.org/show_bug.cgi?id=230813 > > Will take a look this weekend. Don't even get it compiled manually :-( Bye... Dirk signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Freitag, 11. Juli 2008 schrieb Daniel Iliev: > Any help will be much appreciated. > > [1] http://bugs.gentoo.org/show_bug.cgi?id=230813 Will take a look this weekend. Bye... Dirk signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > Am Freitag, 11. Juli 2008 schrieb ext Daniel Iliev: > > > Ah! Now I got it. "tar -M". I've missed that part. I'm sorry. > > No need to be sorry. Depends I strongly recommend not tu use the GNU tar -M extension as GNU tar has a significant probility to not to accept followup volumes when trying to restore gtar multi volume archives. If you like to use a reliable multi-volume implementation, I recommend to use star. > > BTW there is a tool called "splipipe" [1] that can emulate the > > behaviour of "tar -M" without packet writing. I've used it and it > > worked perfectly. > > > > [1] http://ds9a.nl/splitpipe/ > > > > Please don't get me wrong. I'm not advising against packet writing. > > I'm merely sharing the methods for making backup on DVDs I'm aware of in > > hope someone could benefit from the info. Well, I see no relation to packet writing. Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Fri, 11 Jul 2008 08:12:50 +0200 Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > > Didn't know that tool, looks nice. Would you mind creating a "new > ebuild request" for it in b.g.o? > > So we've two methods now, great. > Actually I have already done this [1] a few days ago. Unfortunately it is the first time I'm trying to create an ebuild and I have no experience with the interface of B.G.O. therefore the result is quite unsatisfactory. I used an ebuild from forums.gentoo.org as a template. It installs the package in "/usr/local". Any help will be much appreciated. [1] http://bugs.gentoo.org/show_bug.cgi?id=230813 -- Best regards, Daniel -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Freitag, 11. Juli 2008 schrieb ext Daniel Iliev: > Ah! Now I got it. "tar -M". I've missed that part. I'm sorry. No need to be sorry. > BTW there is a tool called "splipipe" [1] that can emulate the > behaviour of "tar -M" without packet writing. I've used it and it > worked perfectly. > > [1] http://ds9a.nl/splitpipe/ > > Please don't get me wrong. I'm not advising against packet writing. > I'm merely sharing the methods for making backup on DVDs I'm aware of in > hope someone could benefit from the info. Didn't know that tool, looks nice. Would you mind creating a "new ebuild request" for it in b.g.o? So we've two methods now, great. Bye... Dirk -- Dirk Heinrichs | Tel: +49 (0)162 234 3408 Configuration Manager | Fax: +49 (0)211 47068 111 Capgemini Deutschland | Mail: [EMAIL PROTECTED] Wanheimerstraße 68 | Web: http://www.capgemini.com D-40468 Düsseldorf | ICQ#: 110037733 GPG Public Key C2E467BB | Keyserver: wwwkeys.pgp.net signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Thu, 10 Jul 2008 19:05:55 +0200 Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > Am Donnerstag, 10. Juli 2008 schrieb Daniel Iliev: > > On Thu, 10 Jul 2008 13:06:06 +0200 > > > > Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > > > Hmm, never tried it myself so I don't know wether it works or not, > > > but what about enabling CONFIG_CDROM_PKTCDVD in the kernel and let > > > tar write to the device directly? > > > > > > Like: > > > > > > modprobe pktcdvd > > > pktsetup backup /dev/cdrom > > > tar -cMvf /dev/pktcdvd/backup myfiles # Change disc when prompted > > > pktsetup -d backup > > > rmmod pktcdvd > > [-snip-] > > If you just wanted to have a file system (other than isofs) on a > > write-once *DVD*, you can do something like: > > No, why would I? The idea behind the above is to let _tar_ write to > the device directly, as it would do with tape devices. No fs > involved. Ah! Now I got it. "tar -M". I've missed that part. I'm sorry. > OTOH, if I wanted a filesystem, that would be UDF, which I > could still write to using the method above (+ create the fs with > cdrwtool before running pktsetup and then mount it, of course). > > > dd if=/dev/null of=test.fs bs=1M seek=4480 count=0 [-snip-] > > And that would give me a multi volume backup if the data doesn't fit > on one disc? > No, no. It has nothing to do with multi-volumes. As I said I didn't see the "-M" part from your message and merely showed that one can write files directly without packet writing and those files can be FS images. BTW there is a tool called "splipipe" [1] that can emulate the behaviour of "tar -M" without packet writing. I've used it and it worked perfectly. [1] http://ds9a.nl/splitpipe/ Please don't get me wrong. I'm not advising against packet writing. I'm merely sharing the methods for making backup on DVDs I'm aware of in hope someone could benefit from the info. -- Best regards, Daniel -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Donnerstag, 10. Juli 2008 schrieb Daniel Iliev: > On Thu, 10 Jul 2008 13:06:06 +0200 > > Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > > Hmm, never tried it myself so I don't know wether it works or not, > > but what about enabling CONFIG_CDROM_PKTCDVD in the kernel and let > > tar write to the device directly? > > > > Like: > > > > modprobe pktcdvd > > pktsetup backup /dev/cdrom > > tar -cMvf /dev/pktcdvd/backup myfiles # Change disc when prompted > > pktsetup -d backup > > rmmod pktcdvd > > AFAIK packet writing is intended for using RW media as a "normal" > block device. Yes, sure. > I have never tried it myself either. I did, but only with CD-RW discs, which doesn't work well. It's known to work better with DVD±RW. > If you just wanted to have a file system (other than isofs) on a > write-once *DVD*, you can do something like: No, why would I? The idea behind the above is to let _tar_ write to the device directly, as it would do with tape devices. No fs involved. OTOH, if I wanted a filesystem, that would be UDF, which I could still write to using the method above (+ create the fs with cdrwtool before running pktsetup and then mount it, of course). > dd if=/dev/null of=test.fs bs=1M seek=4480 count=0 > mkfs.ext2 test.fs > mkdir loopdir > mount -o loop test.fs loopdir > cp -a "some files & dirs" loopdir/ > umount test.fs > rm -r loopdir > pipebench -b 5000 < test.fs | growisofs -Z /dev/sr0=/dev/fd/0 > mount /dev/dvd /mnt/dvd And that would give me a multi volume backup if the data doesn't fit on one disc? Bye... Dirk signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Thu, 10 Jul 2008 13:06:06 +0200 Dirk Heinrichs <[EMAIL PROTECTED]> wrote: > Am Donnerstag, 10. Juli 2008 schrieb ext Dale: > > > Thought I would move this problem to a new thread. It may not be > > related to the DVD itself. This is what I am currently testing. I > > used Kbackup to create a tarball in my /backup directory. I have > > the most basic setup for Kbackup at the moment. It creates the > > tarball but does not compress the files themselves but creates > > a .tar file. I think it takes the files, places it in the tarball > > then compresses it or something. I'm not real sure how Kbackup > > does its thing. I have it set to create tarballs and the limit is > > 4.4Gbs. I set it just short of a full 4.7Gb. > > Hmm, never tried it myself so I don't know wether it works or not, > but what about enabling CONFIG_CDROM_PKTCDVD in the kernel and let > tar write to the device directly? > > Like: > > modprobe pktcdvd > pktsetup backup /dev/cdrom > tar -cMvf /dev/pktcdvd/backup myfiles # Change disc when prompted > pktsetup -d backup > rmmod pktcdvd > > Bye... > > Dirk AFAIK packet writing is intended for using RW media as a "normal" block device. I have never tried it myself either. If you just wanted to have a file system (other than isofs) on a write-once *DVD*, you can do something like: dd if=/dev/null of=test.fs bs=1M seek=4480 count=0 mkfs.ext2 test.fs mkdir loopdir mount -o loop test.fs loopdir cp -a "some files & dirs" loopdir/ umount test.fs rm -r loopdir pipebench -b 5000 < test.fs | growisofs -Z /dev/sr0=/dev/fd/0 mount /dev/dvd /mnt/dvd -- Best regards, Daniel -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Thu, 10 Jul 2008 14:07:17 +0200, Joerg Schilling wrote: > > I burn my backups onto bootable live DVDs that include dar, as well as > > anything else I may need. When x86 compatible hardware is close to > > extinction, I will have to rethink this, if I'm still alive. > > > > Having said that, I've never looked at star, maybe I should (I'm not > > bothered about a GUI for a job that's handled by a cron task) > What do you do when your DVD no longer boots on the only working > hardware? What do you do if you cannot compile dar on the OS that late > runs on your machines? I stop using dar on DVDs. I call them bootable DVDs because I know they boot. Or I use my offsite backups, I'm far too paranoid to trust to one backup method. > dar still does not compile out of the box because the configure srcipt > aborts and needs manual fixes, so dar cannot be called a piece of > highly portable software. It's a;ways worked fine for me, but, as I mentioned, I'll give star a go some time. I installed it this morning but the delay between installing a new piece of software and using it is highly variable. -- Neil Bothwick First Law of Laboratory Work: Hot glass looks exactly the same as cold glass. signature.asc Description: PGP signature
Re: [gentoo-user] tar and huge tarballs used for back-ups
Daniel Iliev <[EMAIL PROTECTED]> wrote: > On Thu, 10 Jul 2008 11:01:02 +0200 > [EMAIL PROTECTED] (Joerg Schilling) wrote: > > > Daniel Iliev <[EMAIL PROTECTED]> wrote: > > > > > > Since I didn't use a DVD at all in this process, is this a > > > > Kbackup problem or a tar problem? That is the only two programs > > > > I used so I assume it would be one of those two. Also note, I > > > > use ext3 on that partition. All my others are reiserfs but > > > > everything done was on ext3. > > Those are not my words. Please, quote correctly. I quote correclty, you would need to lean hot to quote. The rest of your mail was pure FUD, so I removed it. Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Thu, 10 Jul 2008 11:01:02 +0200 [EMAIL PROTECTED] (Joerg Schilling) wrote: > Daniel Iliev <[EMAIL PROTECTED]> wrote: > > > > Since I didn't use a DVD at all in this process, is this a > > > Kbackup problem or a tar problem? That is the only two programs > > > I used so I assume it would be one of those two. Also note, I > > > use ext3 on that partition. All my others are reiserfs but > > > everything done was on ext3. Those are not my words. Please, quote correctly. > > From other hints, this seems to be a problem on both sides: > > - The built in tar archiver is far too dumb and does not > support enough features for backups > > - GNU tar often does not unpack POSIX compliant tar achives > and even sometimes has problems with it's own non-standard > archives. Care to prove this claim? > > You may like to check the POSIX compliance of the tar archive you > created to verify at which side your problem is located: [--snip--] > > > Ah! One more thing. AFAIK Kbackup has nothing to do with GNU Tar. It > > utilizes KDE's KTar class, so I wouldn't seek the problem in GNU Tar > > (/bin/tar). Actually you can easyly make e test: > May I ask why did you snipped the quote exactly where the correct syntax was shown and replaced it with the incorrect one that follows!? > An important reason not to use kbackup. > > Some examples: > /*--*/ > gtar -cf - /etc/passwd | tartest > gtar: Removing leading `/' from member names > 0.014r 0.000u 0.000s 0% 0M 0+0k 0st 0+0io 0pf+0w > tartest 1.10 (i386-pc-solaris2.11) (^^^ Are you using Gentoo at all or you just love to abuse its mailing list?) > > Copyright (C) 2002 Jörg Schilling > This is free software; see the source for copying conditions. There > is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A > PARTICULAR PURPOSE. > > Testing for POSIX.1-1990 TAR compliance... > Warning: illegal character ' ' (0x20) found in field 't_magic[5]' > Warning: illegal character ' ' (0x20) found in field 't_version[0]' [--snip--] Examples of what? Of your incompetence to use GNU Tar? Check this out: tar -cf - --format=posix /etc | tartest tartest 1.12 (x86_64-unknown-linux-gnu) Copyright (C) 2002 J�rg Schilling This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Testing for POSIX.1-1990 TAR compliance... tar: Removing leading `/' from member names Archive uses POSIX.1-2001 extensions. The correctness of the size field cannot be checked for this reason. Found 1st EOF block at 266978 Found 2nd EOF block at 266979 No deviations from POSIX.1-1990 TAR standard found. It seems your own tool disagrees with you. Read this: http://www.gnu.org/software/automake/manual/tar/posix.html ...and stop spreading FUD! -- Best regards, Daniel -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Joerg Schilling wrote: Neil Bothwick <[EMAIL PROTECTED]> wrote: I burn my backups onto bootable live DVDs that include dar, as well as anything else I may need. When x86 compatible hardware is close to extinction, I will have to rethink this, if I'm still alive. Having said that, I've never looked at star, maybe I should (I'm not bothered about a GUI for a job that's handled by a cron task) What do you do when your DVD no longer boots on the only working hardware? What do you do if you cannot compile dar on the OS that late runs on your machines? dar still does not compile out of the box because the configure srcipt aborts and needs manual fixes, so dar cannot be called a piece of highly portable software. Jörg I have to say that Kdar worked fine before and installed fine. I just didn't like the fact that I couldn't restore say my OS from it. Dar wasn't on the Gentoo CD that I had. Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Dirk Heinrichs wrote: Am Donnerstag, 10. Juli 2008 schrieb ext Dirk Heinrichs: Hmm, never tried it myself so I don't know wether it works or not, but what about enabling CONFIG_CDROM_PKTCDVD in the kernel and let tar write to the device directly? Another option could be to use bacula, but that may be oversized. Bye... Dirk I wish I knew what the program was that I used years ago on a old Mac. It was a incremental backup and was neat as heck. That was many many years ago tho. May look at bacula but a GUI would be really really nice. Thanks Dale :-) :-) P.S. I'm getting a new printer and a 1Gb stick of ram today. I'll have 2Gbs then. Wooo Ooo. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Neil Bothwick <[EMAIL PROTECTED]> wrote: > I burn my backups onto bootable live DVDs that include dar, as well as > anything else I may need. When x86 compatible hardware is close to > extinction, I will have to rethink this, if I'm still alive. > > Having said that, I've never looked at star, maybe I should (I'm not > bothered about a GUI for a job that's handled by a cron task) What do you do when your DVD no longer boots on the only working hardware? What do you do if you cannot compile dar on the OS that late runs on your machines? dar still does not compile out of the box because the configure srcipt aborts and needs manual fixes, so dar cannot be called a piece of highly portable software. Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Donnerstag, 10. Juli 2008 schrieb ext Dirk Heinrichs: > Hmm, never tried it myself so I don't know wether it works or not, but > what about enabling CONFIG_CDROM_PKTCDVD in the kernel and let tar write > to the device directly? Another option could be to use bacula, but that may be oversized. Bye... Dirk -- Dirk Heinrichs | Tel: +49 (0)162 234 3408 Configuration Manager | Fax: +49 (0)211 47068 111 Capgemini Deutschland | Mail: [EMAIL PROTECTED] Wanheimerstraße 68 | Web: http://www.capgemini.com D-40468 Düsseldorf | ICQ#: 110037733 GPG Public Key C2E467BB | Keyserver: wwwkeys.pgp.net signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
Am Donnerstag, 10. Juli 2008 schrieb ext Dale: > Thought I would move this problem to a new thread. It may not be > related to the DVD itself. This is what I am currently testing. I used > Kbackup to create a tarball in my /backup directory. I have the most > basic setup for Kbackup at the moment. It creates the tarball but does > not compress the files themselves but creates a .tar file. I think it > takes the files, places it in the tarball then compresses it or > something. I'm not real sure how Kbackup does its thing. I have it set > to create tarballs and the limit is 4.4Gbs. I set it just short of a > full 4.7Gb. Hmm, never tried it myself so I don't know wether it works or not, but what about enabling CONFIG_CDROM_PKTCDVD in the kernel and let tar write to the device directly? Like: modprobe pktcdvd pktsetup backup /dev/cdrom tar -cMvf /dev/pktcdvd/backup myfiles # Change disc when prompted pktsetup -d backup rmmod pktcdvd Bye... Dirk -- Dirk Heinrichs | Tel: +49 (0)162 234 3408 Configuration Manager | Fax: +49 (0)211 47068 111 Capgemini Deutschland | Mail: [EMAIL PROTECTED] Wanheimerstraße 68 | Web: http://www.capgemini.com D-40468 Düsseldorf | ICQ#: 110037733 GPG Public Key C2E467BB | Keyserver: wwwkeys.pgp.net signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Thu, 10 Jul 2008 11:02:51 +0200, Joerg Schilling wrote: > > Kdar. > > If you use throw away backups, you may live with a non-standard archive > format. > > If you like to archive your backups, better use a standard compliant > archive format that grants you the ability to unpack even many years > later. I burn my backups onto bootable live DVDs that include dar, as well as anything else I may need. When x86 compatible hardware is close to extinction, I will have to rethink this, if I'm still alive. Having said that, I've never looked at star, maybe I should (I'm not bothered about a GUI for a job that's handled by a cron task) -- Neil Bothwick I am Speedy Gonzales of Borg: Prepare to be accelerated! signature.asc Description: PGP signature
Re: [gentoo-user] tar and huge tarballs used for back-ups
Neil Bothwick <[EMAIL PROTECTED]> wrote: > On Thu, 10 Jul 2008 01:11:29 -0500, Dale wrote: > > > OK. Question. What is a good program that allows me to select certain > > directories to backup and then create DVD slices that I can burn to > > DVD? > > Kdar. If you use throw away backups, you may live with a non-standard archive format. If you like to archive your backups, better use a standard compliant archive format that grants you the ability to unpack even many years later. Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
Daniel Iliev <[EMAIL PROTECTED]> wrote: > > Since I didn't use a DVD at all in this process, is this a Kbackup > > problem or a tar problem? That is the only two programs I used so I > > assume it would be one of those two. Also note, I use ext3 on that > > partition. All my others are reiserfs but everything done was on > > ext3. >From other hints, this seems to be a problem on both sides: - The built in tar archiver is far too dumb and does not support enough features for backups - GNU tar often does not unpack POSIX compliant tar achives and even sometimes has problems with it's own non-standard archives. You may like to check the POSIX compliance of the tar archive you created to verify at which side your problem is located: - Fetch and install the latest star-1.5 ftp://ftp.berlios.de/pub/star - call "tartest < test.tar" and read the messages. See below. > "NOTE:Due to a KDE3 limitation, it can only backup files each up to 4GB > in size (on 32bit platforms at least; therefore larger files are > skipped)" Note that tar in former times was a simple archive format (but this was in 1978). Today, the tar archive format has become complex in order to be able to support any extension and in order not to have limitations. If you e.g. like to have support for files > 8 GB and don't like file name length limitations, you need to use the POSIX.1-2001 version of the tar standard. Star is able to do this since the POSIX standard has been defined "stable" for the tar part (in Summer 2001) and star is able to archive virtually any file meta data (property) of any file. This is what you usually want for backups and this is what you don't get from GNU tar and kbackup. > Ah! One more thing. AFAIK Kbackup has nothing to do with GNU Tar. It > utilizes KDE's KTar class, so I wouldn't seek the problem in GNU Tar > (/bin/tar). Actually you can easyly make e test: An important reason not to use kbackup. Some examples: /*--*/ gtar -cf - /etc/passwd | tartest gtar: Removing leading `/' from member names 0.014r 0.000u 0.000s 0% 0M 0+0k 0st 0+0io 0pf+0w tartest 1.10 (i386-pc-solaris2.11) Copyright (C) 2002 Jörg Schilling This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Testing for POSIX.1-1990 TAR compliance... Warning: illegal character ' ' (0x20) found in field 't_magic[5]' Warning: illegal character ' ' (0x20) found in field 't_version[0]' Warning: illegal character '^@' (0x00) found in field 't_version[1]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[0]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[1]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[2]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[3]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[4]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[5]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[6]' Warning: non octal character '^@' (0x00) found in field 't_devmajor[7]' Warning: non octal character '^@' (0x00) found in field 't_devminor[0]' Warning: non octal character '^@' (0x00) found in field 't_devminor[1]' Warning: non octal character '^@' (0x00) found in field 't_devminor[2]' Warning: non octal character '^@' (0x00) found in field 't_devminor[3]' Warning: non octal character '^@' (0x00) found in field 't_devminor[4]' Warning: non octal character '^@' (0x00) found in field 't_devminor[5]' Warning: non octal character '^@' (0x00) found in field 't_devminor[6]' Warning: non octal character '^@' (0x00) found in field 't_devminor[7]' *** Failing Filename 'etc/passwd' Found 1st EOF block at 3 Found 2nd EOF block at 4 >>> Archive is not POSIX.1-1990 TAR standard compliant. gtar --format=ustar -cf - /etc/passwd | tartest gtar: Removing leading `/' from member names 0.004r 0.000u 0.000s 0% 0M 0+0k 0st 0+0io 0pf+0w tartest 1.10 (i386-pc-solaris2.11) Copyright (C) 2002 Jörg Schilling This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Testing for POSIX.1-1990 TAR compliance... Found 1st EOF block at 3 Found 2nd EOF block at 4 No deviations from POSIX.1-1990 TAR standard found. NOTE: The last one is a result of 15 years of nitpicking the GNU tar maintainers! star delivers POSIX compliance for a long time Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Thu, 10 Jul 2008 01:11:29 -0500, Dale wrote: > OK. Question. What is a good program that allows me to select certain > directories to backup and then create DVD slices that I can burn to > DVD? Kdar. Is there an echo in here? ;-) -- Neil Bothwick All things being equal, fat people use more soap. signature.asc Description: PGP signature
Re: [gentoo-user] tar and huge tarballs used for back-ups
Daniel Iliev wrote: http://www.kde-apps.org/content/show.php?content=44998 "NOTE:Due to a KDE3 limitation, it can only backup files each up to 4GB in size (on 32bit platforms at least; therefore larger files are skipped)" If your platform happens to be a 32bit one... Ah! One more thing. AFAIK Kbackup has nothing to do with GNU Tar. It utilizes KDE's KTar class, so I wouldn't seek the problem in GNU Tar (/bin/tar). Actually you can easyly make e test: tar cpf my_huge_archive.tar --format=posix "/path/to/many/big/files" tar xpf my_huge_archive.tar --format=posix -C "/path/to/test/dir/" I bet it would work. More info: http://www.gnu.org/software/tar/manual/html_section/Formats.html Cool, thanks for that info. You want to hear something funny? In the settings for Kbackup, it has a setting for 4.7Gb, 8.5Gb, 9.4Gb and 17.1Gb. Looks like it would disable/grey out the ones that don't work. At least I know it is not me going crazy. o_O You're more than welcome to keep wondering tho. LOL I had no idea on what Kbackup used to create the tarball but I did assume it used tar since it was a .tar file. Should have guessed KDE would have a Ktar version though. :/ OK. Question. What is a good program that allows me to select certain directories to backup and then create DVD slices that I can burn to DVD? I prefer a GUI program if at all possible. I looked at the website for reoback but not sure if it will do what I want to do either. I do system backups but am more concerned about my data files. Oh, I tried mondo-rescue before but it turned into a nightmare. Mostly package version nightmare. I also think it was command line only. Thanks for the link and info. At least I can keep it under 4Gb and make backups. Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] tar and huge tarballs used for back-ups
On Wed, 09 Jul 2008 22:26:54 -0500 Dale <[EMAIL PROTECTED]> wrote: > Hi, > > Thought I would move this problem to a new thread. It may not be > related to the DVD itself. This is what I am currently testing. I > used Kbackup to create a tarball in my /backup directory. I have the > most basic setup for Kbackup at the moment. It creates the tarball > but does not compress the files themselves but creates a .tar file. > I think it takes the files, places it in the tarball then compresses > it or something. I'm not real sure how Kbackup does its thing. I > have it set to create tarballs and the limit is 4.4Gbs. I set it > just short of a full 4.7Gb. > > I then tried to untar the file to a directory to test if it will > untar it or not. It failed. I guess you figured that since I am > posting this problem. I'm just going to post info as I go along > here. Feel free to ask for more if you need something else. This is > the files that Kbackup generates: > > -rw-r--r-- 1 dale users 4295682560 2008-07-09 20:15 > Data_2008.07.09-20.11.01_1.tar > -rw-r--r-- 1 dale users 4295361024 2008-07-09 20:52 > Data_2008.07.09-20.11.01_2.tar > > This is what I get when I try to untar the #1 tarball: > > [EMAIL PROTECTED] / # tar -xvf /backup/Data_2008.07.09-20.11.01_1.tar -C > /backup/test/ > data/ > data/39.gif > data/Address-book-2006-12-27.ldif > data/alegis-group-sears8.21.03.sxw > data/bookmarks.html > data/Camera-pics/ > data/Camera-pics/2006-Christmas/ > data/Camera-pics/2006-Christmas/Christmas-0001.jpeg > tar: Skipping to next header > tar: Error exit delayed from previous errors > [EMAIL PROTECTED] / # > > I did not shorten that. It is the whole thing. It did not do much > and that file is coming off the hard drive not a DVD. Also note, > there is a huge list of files that are missing from that tarball. > Basically, that tarball is about worthless. This is from the #2 > tarball: > > [EMAIL PROTECTED] / # tar -xvf /backup/Data_2008.07.09-20.11.01_2.tar -C > /backup/test/ > data/Gentoo-stuff/livecd-i686-installer-2007.0.iso > tar: Skipping to next header > < SNIP a long list of files > > tar: Error exit delayed from previous errors > [EMAIL PROTECTED] / # > > I didn't see any other errors in the looong list of files. Also, it > did untar a lot of files but I really don't know how to tell if it > did them all or not. It appears they are all there but I'm not 100% > sure. It certainly did better than the first tarball for sure. > > Since I didn't use a DVD at all in this process, is this a Kbackup > problem or a tar problem? That is the only two programs I used so I > assume it would be one of those two. Also note, I use ext3 on that > partition. All my others are reiserfs but everything done was on > ext3. > > Other info that may matter: > > [ebuild R ] app-cdr/cdrtools-2.01.01_alpha41 USE="unicode" 0 kB > [ebuild R ] app-cdr/cdrdao-1.2.2-r2 USE="encode -debug > -gcdmaster -pccts" 0 kB > [ebuild R ] app-cdr/k3b-1.0.5-r1 USE="alsa arts dvdr dvdread > encode hal mp3 vorbis -css -debug -emovix -ffmpeg -flac -musepack > -musicbrainz -sndfile -vcd -xinerama" LINGUAS="-af -ar -bg -br -bs > -ca -cs -cy -da -de -el -en_GB -es -et -eu -fa -fi -fr -ga -gl -he > -hi -hu -is -it -ja -ka -lt -mk -ms -nb -nds -nl -nn -pa -pl -pt > -pt_BR -ru -rw -se -sk -sr [EMAIL PROTECTED] -sv -ta -tr -uk -uz -zh_CN > -zh_TW" 0 kB > > [EMAIL PROTECTED] / # uname -r > 2.6.23-gentoo-r8 > [EMAIL PROTECTED] / # > > That is the latest kernel that I can get to work correctly. > 2.6.24-r8 makes KDE slow as leap year for some reason. No clue, just > went back to what worked for me. > > Thoughts? Opinions? Suggestions? > > Thanks > > Dale > > :-) :-) http://www.kde-apps.org/content/show.php?content=44998 "NOTE:Due to a KDE3 limitation, it can only backup files each up to 4GB in size (on 32bit platforms at least; therefore larger files are skipped)" If your platform happens to be a 32bit one... Ah! One more thing. AFAIK Kbackup has nothing to do with GNU Tar. It utilizes KDE's KTar class, so I wouldn't seek the problem in GNU Tar (/bin/tar). Actually you can easyly make e test: tar cpf my_huge_archive.tar --format=posix "/path/to/many/big/files" tar xpf my_huge_archive.tar --format=posix -C "/path/to/test/dir/" I bet it would work. More info: http://www.gnu.org/software/tar/manual/html_section/Formats.html -- Best regards, Daniel -- gentoo-user@lists.gentoo.org mailing list
[gentoo-user] tar and huge tarballs used for back-ups
Hi, Thought I would move this problem to a new thread. It may not be related to the DVD itself. This is what I am currently testing. I used Kbackup to create a tarball in my /backup directory. I have the most basic setup for Kbackup at the moment. It creates the tarball but does not compress the files themselves but creates a .tar file. I think it takes the files, places it in the tarball then compresses it or something. I'm not real sure how Kbackup does its thing. I have it set to create tarballs and the limit is 4.4Gbs. I set it just short of a full 4.7Gb. I then tried to untar the file to a directory to test if it will untar it or not. It failed. I guess you figured that since I am posting this problem. I'm just going to post info as I go along here. Feel free to ask for more if you need something else. This is the files that Kbackup generates: -rw-r--r-- 1 dale users 4295682560 2008-07-09 20:15 Data_2008.07.09-20.11.01_1.tar -rw-r--r-- 1 dale users 4295361024 2008-07-09 20:52 Data_2008.07.09-20.11.01_2.tar This is what I get when I try to untar the #1 tarball: [EMAIL PROTECTED] / # tar -xvf /backup/Data_2008.07.09-20.11.01_1.tar -C /backup/test/ data/ data/39.gif data/Address-book-2006-12-27.ldif data/alegis-group-sears8.21.03.sxw data/bookmarks.html data/Camera-pics/ data/Camera-pics/2006-Christmas/ data/Camera-pics/2006-Christmas/Christmas-0001.jpeg tar: Skipping to next header tar: Error exit delayed from previous errors [EMAIL PROTECTED] / # I did not shorten that. It is the whole thing. It did not do much and that file is coming off the hard drive not a DVD. Also note, there is a huge list of files that are missing from that tarball. Basically, that tarball is about worthless. This is from the #2 tarball: [EMAIL PROTECTED] / # tar -xvf /backup/Data_2008.07.09-20.11.01_2.tar -C /backup/test/ data/Gentoo-stuff/livecd-i686-installer-2007.0.iso tar: Skipping to next header < SNIP a long list of files > tar: Error exit delayed from previous errors [EMAIL PROTECTED] / # I didn't see any other errors in the looong list of files. Also, it did untar a lot of files but I really don't know how to tell if it did them all or not. It appears they are all there but I'm not 100% sure. It certainly did better than the first tarball for sure. Since I didn't use a DVD at all in this process, is this a Kbackup problem or a tar problem? That is the only two programs I used so I assume it would be one of those two. Also note, I use ext3 on that partition. All my others are reiserfs but everything done was on ext3. Other info that may matter: [ebuild R ] app-cdr/cdrtools-2.01.01_alpha41 USE="unicode" 0 kB [ebuild R ] app-cdr/cdrdao-1.2.2-r2 USE="encode -debug -gcdmaster -pccts" 0 kB [ebuild R ] app-cdr/k3b-1.0.5-r1 USE="alsa arts dvdr dvdread encode hal mp3 vorbis -css -debug -emovix -ffmpeg -flac -musepack -musicbrainz -sndfile -vcd -xinerama" LINGUAS="-af -ar -bg -br -bs -ca -cs -cy -da -de -el -en_GB -es -et -eu -fa -fi -fr -ga -gl -he -hi -hu -is -it -ja -ka -lt -mk -ms -nb -nds -nl -nn -pa -pl -pt -pt_BR -ru -rw -se -sk -sr [EMAIL PROTECTED] -sv -ta -tr -uk -uz -zh_CN -zh_TW" 0 kB [EMAIL PROTECTED] / # uname -r 2.6.23-gentoo-r8 [EMAIL PROTECTED] / # That is the latest kernel that I can get to work correctly. 2.6.24-r8 makes KDE slow as leap year for some reason. No clue, just went back to what worked for me. Thoughts? Opinions? Suggestions? Thanks Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
