Re: Backup thousands of files 1 at a time with growisofs?
Hi, it looks i will have to forward the core of your problem to Andy, since it is growisofs which triggers the failure. Still absolutely unclear, what or who is to blame, though. I try it now for some times to make a sdvdbackup ./ without any I got this : ... /dev/hdc: updating RMA /dev/hdc: closing disc ## part 2 of 4 (26 items4250m bytes) So the first DVD was ok ? Did it verify with sdvdbacckup_verify ? beginning to create DVD ... Locking of /usr/src/scdbackup-0.8/cdrecorder refreshed To Andy: This locking has nothing to do with the recording process itself. It is an internal, voluntary feature of my software. growisofs is not affected in any way. /usr/src/scdbackup-0.8/growisofs_wrapper Here runs : growisofs -use-the-force-luke \ -dvd-compat \ -Z /dev/hdc=/proc/self/fd/0 getting its stdin from stdout of mkisofs WARNING: /dev/hdc already carries isofs! About to execute 'builtin_dd if=/proc/self/fd/0 of=/dev/hdc obs=32k seek=0' /dev/hdc: FEATURE 21h is not on, engaging DAO... /dev/hdc: reserving 2171632 blocks :-[ RESERVE TRACK failed with SK=5h/ASC=24h/ACQ=00h]: Input/output error I looked up this code in http://fy.chalmers.se/~appro/linux/DVD+RW/keys.txt 5 24 00 INVALID FIELD IN CDB /dev/hdc: Current Write Speed is 4.1x1385KBps. :-[ [EMAIL PROTECTED] failed with SK=5h/ASC=21h/ACQ=02h]: Invalid argument 5 21 02 INVALID ADDRESS FOR WRITE :-( write failed: Invalid argument I feel unable to interpret these error conditions. These messages are mere consequence of the failure : /usr/src/scdbackup-0.8/raedchen: Broken pipe /usr/src/scdbackup-0.8/cd_backup_command: line 143: 12717 Broken pipe $mkisofs_prog $@ 12718 | $bdir/cd_backup_planer -filter_pipe $cdrecord_prog speed=$speed dev=$scsi_adr $cdrecord_opt - $make_checksum_option $listname $recname To avoid the impression this would run cdrecord* : The wrapper is running growisofs and ignores the cdrecord specific arguments given by its boss who knows nothing about the wrapper's real content. Any idea what's wrong with non first DVD ? I can assure you that scdbackup does treat the first media exactly like the second and any further one. If there really is a correlation between being follow-up media and failing, then it must be something about the recorder, the media or (very remote) growisofs. Since the problem seems to vanish before you make the next #1 try, it could be a thermo problem or the need for a kind of reset which then happens automatically after a timeout. Maybe it would be helpful to have all messages of growisofs from the first successful disk. They should have started with : WARNING: /dev/hdc already carries isofs!. or at least About to execute 'builtin_dd ... I would rather suspect that the writer has a problem with consequtive writes of media. You have excluded any connection to particular DVD disks, i presume. Tell us your recorder type and the media you use. growisofs -version Proposal for a test: Wait the same time as you waited between two successful #1 burns. Then run the old scdbackup job for the second media : sdvdbackup -resume 2 (Do no new runs with sdvdbackup and a file address in the meantime) Vice versa : Can you burn with the example from man growisofs a tree of about 4 GB to more than one DVD in a row ? Like : growisofs -Z /dev/hdc -R -J /some/files For overwriting ISO images use option : -use-the-force-luke Interesting to see wether this might trigger the problem : -dvd-compat I am clueless, i have to confess. Have a nice day :) Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
On Wed, Jul 07, 2004 at 08:50:49AM +0200, [EMAIL PROTECTED] wrote: http://scdbackup.webframe.org/ http://scdbackup.webframe.org/examples.html Hello, I try it now for some times to make a sdvdbackup ./ without any I got this : 99.55% done, estimate finish Sat Aug 28 19:07:26 2004 99.78% done, estimate finish Sat Aug 28 19:07:26 2004 4451794944/4454047744 (99.9%) @3.9x, remaining 0:00 Total translation table size: 0 Total rockridge attributes bytes: 90932 Total directory bytes: 219136 Path table size(bytes): 1648 Max brk space used c6000 2174828 extents written (4247 MB) builtin_dd: 2174992*2KB out @ average 3.7x1385KBps /dev/hdc: flushing cache /dev/hdc: updating RMA /dev/hdc: closing disc ## part 2 of 4 (26 items4250m bytes) insert DVD and press Enter beginning to create DVD ... Locking of /usr/src/scdbackup-0.8/cdrecorder refreshed mkisofs: The option '-L' is reserved by POSIX.1-2001. mkisofs: The option '-L' means 'follow all symbolic links'. mkisofs: Mkisofs-2.02 will introduce POSIX semantics for '-L'. mkisofs: Use -allow-leading-dots in future to get old mkisofs behavior. Warning: creating filesystem that does not conform to ISO-9660. /usr/src/scdbackup-0.8/growisofs_wrapper WARNING: /dev/hdc already carries isofs! About to execute 'builtin_dd if=/proc/self/fd/0 of=/dev/hdc obs=32k seek=0' /dev/hdc: FEATURE 21h is not on, engaging DAO... /dev/hdc: reserving 2171632 blocks :-[ RESERVE TRACK failed with SK=5h/ASC=24h/ACQ=00h]: Input/output error /dev/hdc: Current Write Speed is 4.1x1385KBps. :-[ [EMAIL PROTECTED] failed with SK=5h/ASC=21h/ACQ=02h]: Invalid argument :-( write failed: Invalid argument /usr/src/scdbackup-0.8/raedchen: Broken pipe /usr/src/scdbackup-0.8/cd_backup_command: line 143: 12717 Broken pipe $mkisofs_prog $@ 12718 | $bdir/cd_backup_planer -filter_pipe $cdrecord_prog speed=$speed dev=$scsi_adr $cdrecord_opt - $make_checksum_option $listname $recname ## part 3 of 4 (127 items4246m bytes) insert DVD and press Enter beginning to create DVD ... Locking of /usr/src/scdbackup-0.8/cdrecorder refreshed mkisofs: The option '-L' is reserved by POSIX.1-2001. mkisofs: The option '-L' means 'follow all symbolic links'. mkisofs: Mkisofs-2.02 will introduce POSIX semantics for '-L'. mkisofs: Use -allow-leading-dots in future to get old mkisofs behavior. Warning: creating filesystem that does not conform to ISO-9660. /usr/src/scdbackup-0.8/growisofs_wrapper WARNING: /dev/hdc already carries isofs! About to execute 'builtin_dd if=/proc/self/fd/0 of=/dev/hdc obs=32k seek=0' /dev/hdc: FEATURE 21h is not on, engaging DAO... /dev/hdc: reserving 2152064 blocks :-[ RESERVE TRACK failed with SK=5h/ASC=24h/ACQ=00h]: Input/output error /dev/hdc: Current Write Speed is 4.1x1385KBps. :-[ [EMAIL PROTECTED] failed with SK=5h/ASC=21h/ACQ=02h]: Invalid argument :-( write failed: Invalid argument /usr/src/scdbackup-0.8/raedchen: Broken pipe /usr/src/scdbackup-0.8/cd_backup_command: line 143: 13479 Broken pipe $mkisofs_prog $@ 13480 | $bdir/cd_backup_planer -filter_pipe $cdrecord_prog speed=$speed dev=$scsi_adr $cdrecord_opt - $make_checksum_option $listname $recname ## part 4 of 4 (21 items3677m bytes) insert DVD and press Enter Any idea what's wrong with non first DVD ? Thank you very much, -- Grégoire Favre http://magma.epfl.ch/greg ICQ:16624071 mailto:[EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
SORRY for the post, the second DVD I tried was a bad medianot recognised by my writer!!! It works perfectly and it's THE tool I needed :-) -- Grégoire Favre http://magma.epfl.ch/greg ICQ:16624071 mailto:[EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
Dear Nick, mkisofs: Error: '2000_02_28/vh40.png' and '2000_02_26/vh40.png' have the same Rock Ridge name 'vh40.png'. As the message says, you suffer from name collisions. Aha, it seems that I should use the --graft-points Yes. That is a way to solve collisions. mkisofs ... -graft-points /target_address=/source_address \ /target_dir/=/source_dir ... I am watching your efforts since yesterday and now i believe that you found enough reasons to give scdbackup a try : - Multi volume, - independent ISO file systems on each media, single files accessible on Win, Mac too (alternatively you can get afio archives) - collison test and automatic remedy (via graft points) - (manual grafting too) Be invited to have a look at : http://scdbackup.webfram.org http://scdbackup.webfram.org/examples.html Post eventual support requests here or send them directly to [EMAIL PROTECTED] . Have a nice day :) Thomas PS : shunt seems not to create openly accessible ISO filesystem structures but to part the data stream of an archiver into digestible pieces. Those pieces seem to be stored in an ISO filesystem. After mounting such a media, you will see the pieces of the archiver's output, not the single files like they are on your disk. Quote from http://www.serice.net/shunt/ Backup utilities that are capable of handling multi-volume archives have to directly control the opening, reading or writing, and closing of the underlying device. Apparently, this is not supported by the current generation of device drivers for CDs and DVDs. That means it is a kind of userland substitute for Andy Polyakov's DVD kernel patch. (Is it still valid, that patch ?) Nevertheless it does not write the raw data stream to the media as a tape device would do. scdbackup_afio is nearer to the tape model. With all advantages and disadvantages. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
I should learn to write my own URLs correctly : http://scdbackup.webframe.org/ http://scdbackup.webframe.org/examples.html One may find it by google as well. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
Volker Kuhlmann wrote: I would particularly like to have my music uncompressed, untarred so I can just put the dvd into a machine and play them. This would require each disk to have an in itself complete iso fs with a subset of the files. Each disk may be slightly underfull. http://www.serice.net/shunt/ Brilliant piece of software by the looks of it. Thanks for the URL! Check whether it creates disks with a complete filesystem, or whether it creates one huge filesystem with a piece of the filesystem on each disk. In the latter case individual disks wouldn't be accessible. Alternatively, you could hack up some script which reads through your music directory, sizing up as many files as fit on one disk, burn that disk, and then proceed with the next file in the directory. I have a generalized perl script which reads a file in size-name format, and generates output file(s) of items which will fit on a single media. The size of a media and the per-item overhead are command line parameters. This isn't limited to files, I use du -S to generate backups with all of a single directory on a single media. It also tries to equalize the content on each media, so you don't get a bunch of full media ending with one having virtually nothing. That's just my preference, it could be optional (and may be, I don't have the code in front of me). I originally wrote it in awk a few decades ago when I run a UNIX BBS and backed up my 20MB hard drive to 400k floppies. World changed, problems unchanged, backup media is still too small. In either case you have no control over which file goes on which disk, i.e. it'll be somehwat random. Yes, that's a problem. And the processing time needed to get a really perfect packing can be great if the data just fit on N media (or just don't quite fit). -- E. Robert Bogusta It seemed like a good idea at the time -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
On 6 Jul, Nick Urbanik wrote: Dear Folks, I just bought a DVD writer on Sunday and am ignorant about DVDs; please excuse me. Can I back up thousands of small files, one at a time to a DVD, with growisofs? Can is do something simple like this, and get a reliable result? What are the limitations of this if it is possible? The kind of thing I wonder is: can I do something like this: growisofs -Z /dev/dvd -J -R file1 for f in otherfiles...;do growisofs -M /dev/dvd -J -R $f done where otherfiles... expands to many tens of thousands of files. Why don't you use 'tar' especially with compression like tar czf- otherfiles | growisofs ... This compresses the tar archive and otherfiles may me millions. I have'nt worked with growisofs, yet, since I am waiting for affordable double layer writers and media. Have a look at http://www.serice.net/shunt/ and see the flyisofs / growisofs example there. -- Helmut Jarausch Lehrstuhl fuer Numerische Mathematik RWTH - Aachen University D 52056 Aachen, Germany -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Backup thousands of files 1 at a time with growisofs?
Dear Folks, On Tue, Jul 06, 2004 at 10:57:34AM +0200, Helmut Jarausch wrote: On 6 Jul, Nick Urbanik wrote: Dear Folks, I just bought a DVD writer on Sunday and am ignorant about DVDs; please excuse me. Can I back up thousands of small files, one at a time to a DVD, with growisofs? Can is do something simple like this, and get a reliable result? What are the limitations of this if it is possible? The kind of thing I wonder is: can I do something like this: growisofs -Z /dev/dvd -J -R file1 for f in otherfiles...;do growisofs -M /dev/dvd -J -R $f done where otherfiles... expands to many tens of thousands of files. Why don't you use 'tar' especially with compression like tar czf- otherfiles | growisofs ... I would particularly like to have my music uncompressed, untarred so I can just put the dvd into a machine and play them. I would also like to be able to visually browse the photos on the dvd, since I have a web site with many gigabytes of family photos, and it would be nice to be able to give copies of the photos to my family more easily. This compresses the tar archive and otherfiles may me millions. I have'nt worked with growisofs, yet, since I am waiting for affordable double layer writers and media. Have a look at http://www.serice.net/shunt/ and see the flyisofs / growisofs example there. That is wonderful! It looks like exactly the right thing! How Unixy that is; it allows me to put it into a pipeline, simplifying everything! I look forward to running this tomorrow if I can make time. Thank you again, that looks like a perfect solution. -- Nick Urbanik RHCE nicku(at)vtc.edu.hk Proud member of the Dept. of Information Communications Technology, Home of Visual Paradigm: Jolt Productivity Award winner, programmed by our own graduates! Tel: (852) 2436 8576 Fax: (852) 2436 8526 GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 pgpC3we0mQLjL.pgp Description: PGP signature
Re: Backup thousands of files 1 at a time with growisofs?
Dear Folks, On Wed, Jul 07, 2004 at 09:19:16AM +1200, Volker Kuhlmann wrote: I would particularly like to have my music uncompressed, untarred so I can just put the dvd into a machine and play them. This would require each disk to have an in itself complete iso fs with a subset of the files. Each disk may be slightly underfull. http://www.serice.net/shunt/ Brilliant piece of software by the looks of it. Thanks for the URL! Check whether it creates disks with a complete filesystem, or whether it creates one huge filesystem with a piece of the filesystem on each disk. In the latter case individual disks wouldn't be accessible. Yes, I will try it out later for putting tarballs onto DVD. I don't see how I can use it to put individual files, unless I can somehow hook things together with xargs. It looks like a wonderful idea that will have many other applications besides creating backups. Alternatively, you could hack up some script which reads through your music directory, sizing up as many files as fit on one disk, burn that disk, and then proceed with the next file in the directory. In either case you have no control over which file goes on which disk, i.e. it'll be somehwat random. Won't it be possible for mkisofs to keep the directories in order, if my (yet to be written) Perl program provides the paths in a particular order? I am currently thinking of writing a program that calls growisofs once for each diskful, after generating a list of paths, the contents of which can fit into one DVD. And yes, as I've just discovered, using --graft-points seems to be required here. -- Nick Urbanik RHCE nicku(at)vtc.edu.hk Proud member of the Dept. of Information Communications Technology, Home of Visual Paradigm: Jolt Productivity Award winner, programmed by our own graduates! Tel: (852) 2436 8576 Fax: (852) 2436 8526 GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 pgpgbPVCZWfNG.pgp Description: PGP signature
