Re: Sense fetching [Was: cdrtools /devel ...]
If FreeBSD writes any warning for SCSI commands send by cdrtools, then this is definitely a kernel bug that needs to be fixed. SCSI is a protocol that lives from aparently failing SCSI commands. These warnings are related to "high level interaction" between cdrecord and the drive. Only cdrecord is able to decide whether a SCSI command that apparently failed is worth for being reported or not. It therefore is a bug if the kernel prints related messages. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Sense fetching [Was: cdrtools /devel ...]
Alexander Motin wrote: > > What is the requested size with the various HBAs in earlier kernels? > > For HBAs with automatic sense fetching -- as passed in sence_len request > field. In case of libscg it was SSD_FULL_SIZE before and I've set it to > be real value now. Returned sense_resid should be real in both cases, > respecting submitted sense_len. > > For HBAs without sense fetching, CAM was always requesting SSD_FULL_SIZE > and returned fake zero sense_resid, like if it just completely fulfilled > sense_len from request. Now it should start honor sense_len on request > and return real sense_resid on reply, relative to sense_len. Then your patch to libscg seems to be OK and without risk. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Sense fetching [Was: cdrtools /devel ...]
Alexander Motin wrote: > > Compare the number of sense bytes I like to request (18) with the number > > previous FreeBSD versions did actually request. It is obvious that in case > > there is a resid reported onm an old kernel, libscg (with your chanhge) > > would > > believe that probably less that the absolute minumum of sense data is > > available. > > Kernel code I have changed affects only controllers without automatic > sense fetching. For such controllers CAM previously always reported zero > sense_resid, independently of requested size (which actually was also > ignored in this case). So previously whatever you asked for 18 or 32 > bytes - sense_resid was always zero, like if you got full 18 or 32 > bytes. That was wrong. Now, in the same situation, if device wants to > return let's say 20 bytes -- you will get 0 and 12 bytes of sense_resid > respectively, meaning 18 and 20 bytes of received sense. > > AFAIK sense_resid always measured from the requested sense_len. In my > patch for libscg you may see that I have changed both request and > response paths. Resulted sense_count should be absolutely the same. What is the requested size with the various HBAs in earlier kernels? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Sense fetching [Was: cdrtools /devel ...]
Alexander Motin wrote: > > The question still remains whether the previous implementation did return > > resid > >> 0 in some cases. In this case, I would need to implement both variants in > >> the > > libscg adaption layer and I would need to know whether I am running on an > > old > > version or on a new version kernel. Do you know of a simple method to > > implement > > this distinction? > > Yes, some drivers were permanently reporting zero resid, while others > (mostly real SCSI) were reporting proper values. Now it is the same, > just more cases should work properly. > > Why do you want/need to distinguish them? You should already handle > non-zero resid properly. Zero resid may be wrong in some cases, but at > first I don't see fatal problem from it and at second I don't see what > can you do about it. > > If I am missing something - sorry, explain it to me please. Compare the number of sense bytes I like to request (18) with the number previous FreeBSD versions did actually request. It is obvious that in case there is a resid reported onm an old kernel, libscg (with your chanhge) would believe that probably less that the absolute minumum of sense data is available. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Sense fetching [Was: cdrtools /devel ...]
Alexander Motin wrote: > >>> Given the fact that many drives will probably only return 18 bytes of > >>> sense > >>> data, this will happen every time libscg is told to fetch more sense than > >>> the > >>> drive is willing to return. > >>> > >>> Is there a way to distinct an old kernel from a new one? > >> I don't see the problem. Previous kernel in most cases reported > >> sesnse_resid == 0, lying that there is more sense data then really is. > >> New one should report real (often positive) value. In both cases > >> sesnse_resid value measured from the value submitted to the kernel. > > > > Did the old kernel return a zero sense_resid for any implemented SCSI > > transport? Libscg is a generic SCSI transport library and cdrecord is just > > one > > user of this lib. > > Not sure I understand your question. Zero sesnse_resid is absolutely > normal situation if device gave same amount of sense as application > requested. As I can see, many of SCSI controllers report sesnse_resid > properly. I may assume that some, like atapicd don't -- in that case > you'll also see 0 there. FreeBSD-CAM did try to fetcth more than 18 bytes of sense data and it may be that some drives did return only 18 bytes. In this case, I would asume that there is a resid > 0. > > Do you know the CAM behavior for other SCSI transports? > > I don't have real SCSI CD to test, but a as I can see, most of SCSI > controllers return sense data automatically. Sense fetching changes > should not affect/break anything there. The question still remains whether the previous implementation did return resid > 0 in some cases. In this case, I would need to implement both variants in the libscg adaption layer and I would need to know whether I am running on an old version or on a new version kernel. Do you know of a simple method to implement this distinction? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Sense fetching [Was: cdrtools /devel ...]
Alexander Motin wrote: > > Your patch to libscg looks definitely OK if we only look at the new > > corrected > > kernel driver behavior. > > > > There is a problem: > > > > In case that there is a sense data residual > 0, libscg will asume that > > there > > is less sense data that really present in case that a "new" libscg is > > runnung > > on an old kernel. > > > > Given the fact that many drives will probably only return 18 bytes of sense > > data, this will happen every time libscg is told to fetch more sense than > > the > > drive is willing to return. > > > > Is there a way to distinct an old kernel from a new one? > > I don't see the problem. Previous kernel in most cases reported > sesnse_resid == 0, lying that there is more sense data then really is. > New one should report real (often positive) value. In both cases > sesnse_resid value measured from the value submitted to the kernel. Did the old kernel return a zero sense_resid for any implemented SCSI transport? Libscg is a generic SCSI transport library and cdrecord is just one user of this lib. > > I see no advantage in removing the call to fillbytes(). > > scgcheck tests how much sense data received by counting 0x00 and 0xff > bytes. Zero-filling response buffer breaks that check. Though I have no > idea if other crdtools' applications depend on these zeros. There could > be some internal inconsistency. O, I need to check other low level transport adaptation layers and think about this statement. > > Did you test a modified libscg on an unmodified kernel? > > Unmodified kernel by default doesn't return any sense data at all for > new CAM-based ATA -- this changes should be invariant. New scgcheck runs > same bad as old. It just can't become worse. :) > > Legacy atapicam wrapper ignores sense_len on input and doesn't fill > sense_resig on output -- I haven't tested, but it also should be invariant. I am not only talking about ATAPI but abut SCSI in general. Do you know the CAM behavior for other SCSI transports? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Sense fetching [Was: cdrtools /devel ...]
Marius Strobl wrote: > On Fri, Nov 05, 2010 at 08:50:49PM +0200, Alexander Motin wrote: > > Hi. > > > > I've reviewed tests that scgcheck does to SCSI subsystem. It shown > > combination of several issues in both CAM, ahci(4) and cdrtools itself. > > Several small patches allow us to pass most of that tests: > > http://people.freebsd.org/~mav/sense/ > > > > ahci_resid.patch: Add support for reporting residual length on data > > underrun. SCSI commands often returns results shorter then expected. > > Returned value allows application to know/check how much data it really > > has. It is also important for sense fetching, as ATAPI and USB devices > > return sense as data in response to REQUEST_SENSE command. > > > > sense_resid.patch: When manually requesting sense data (ATAPI or USB), > > request only as much data as user requested (not the fixed structure > > size), and return respective sense residual length. Your patch to libscg looks definitely OK if we only look at the new corrected kernel driver behavior. There is a problem: In case that there is a sense data residual > 0, libscg will asume that there is less sense data that really present in case that a "new" libscg is runnung on an old kernel. Given the fact that many drives will probably only return 18 bytes of sense data, this will happen every time libscg is told to fetch more sense than the drive is willing to return. Is there a way to distinct an old kernel from a new one? > > pass_autosence.patch: Unless CAM_DIS_AUTOSENSE is set, always fetch > > sense if not done by SIM, independently of CAM_PASS_ERR_RECOVER. As soon > > as device freeze released before returning to user-level, user-level > > application by definition can't reliably fetch sense data if some other > > application (like hald) tries to access device same time. This is what we need! > > cdrtools.patch: Make libscg (part of cdrtools) on FreeBSD to submit > > wanted sense length to CAM and do not clear sense return buffer. It is > > mostly cosmetics, important probably only for scgcheck. I see no advantage in removing the call to fillbytes(). > Please don't commit this to the port directly but let it loop back > via upstream (CC'ed) instead, otherwise we would need to obey the > following, which is undesirable, especially if these really are > mostly cosmetic issues: > /* > * Warning: you may change this source, but if you do that > * you need to change the _scg_version and _scg_auth* string below. > * You may not return "schily" for an SCG_AUTHOR request anymore. > * Choose your name instead of "schily" and make clear that the version > * string is related to a modified source. > */ > > > > > Testers and reviewers welcome. I am especially interested in opinion > > about pass_autosence.patch -- may be we should lower sense fetching even > > deeper, to make it work for all cam_periph_runccb() consumers. Did you test a modified libscg on an unmodified kernel? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: cdrtools /devel and wodim broken
>Ready to start test for failing command? Enter to continue: >**> Testing for failed SCSI command. >scgcheck: Input/output error. inquiry: scsi sendcmd: retryable error >CDB: 12 00 00 FF 24 00 >status: 0x0 (GOOD STATUS) >cmd finished after 0.013s timeout 40s >--> SCSI Transport return != SCG_NO_ERROR (1) >--> SCSI status byte set to 0 (0x0) >--> SCSI failed command test FAILED You seem to have a general problem with correct error reporting in SCSI with your kernel SCSI transport. Please try to contact the maintainer of this driver. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: cdrtools /devel and wodim broken
Hi, your problem: >Track 01:0 of 2858 MB written.cdrecord: Input/output error. write_g1: >scsi sendcmd: retryable error >CDB: 2A 00 00 00 00 00 00 00 20 00 >status: 0x2 (CHECK CONDITION) >Sense Bytes: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >00 00 00 00 00 00 00 00 00 00 00 >Sense Key: 0x [], Segment 0 >Sense Code: 0x00 Qual 0x00 (no additional sense information) Fru 0x0 >Sense flags: Blk 0 (not valid)=20 >cmd finished after 23.946s timeout 200s is caused by a bug in a driver in the kernel. Status byte == 2 (Check Condition) together with empty sense data and unset sense key is impossible. Cdrecord can only behave in a useful way if SCSI error messages are correctly returned by the kernel. I recommend you to run "scgcheck" in order to find whether the driver yiu are using always behaves incorectly or whether this may be a result os a situation the autor of the driver did not expect. If you don't know how to run scgcheck, you may start with "scgcheck -auto". Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: cdrtools /devel and wodim broken
You wrote: ># cdrecord(wodim) dev=1,0,0 -v -data *.iso >Track 01:0 of 2858 MB written.Errno: 5 (Input/output error), write_g1 >scsi sendcmd: retryable error >CDB: 2A 00 00 00 00 00 00 00 1F 00 >status: 0x2 (CHECK CONDITION) >Sense Bytes: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >00 00 00 00 00 00 00 00 00 00 00 >Sense Key: 0x [], Segment 0 >Sense Code: 0x00 Qual 0x00 (no additional sense information) Fru 0x0 >Sense flags: Blk 0 (not valid) >cmd finished after 22.883s timeout 40s >write track data: error after 0 bytes >wodim: A write error occured. >wodim: Please properly read the error message above. >Writing time: 32.681s >Average write speed 77.7x. wodim is a dead end from a 6 year old version of cdrecord with the DVD support ripped off and replaced by something broken. But there have been even more bugs added to wodim and it is not under "development" since May 2007. If cdrecord gives the same message, I encourage you to make a kernel bug report. This message is a hint to a serious kernel problem. A sense key value -1 cannot happen, so you need to find out why the SCSI command has not been transported correctly by the kernel. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Burning DVD with files>4GB from console
>My backup script split filesystem dumps to files with size of 4,37 GB (4 >588 544 kB). It's just an optimal size to fill out DVDs. At this moment >I have to burn them from windows via smb-link becuase I didn't manage to >do this task from FreeBSD console due to 2GB/4GB filesize restrictions >(growisofs). I'm using freeware CDBurnerXP to burn my backups What "limitations" are you thinking of? Growisofs is based on mkisofs which is part of cdrtools. Mkisofs supports large files since a long time (~ 8 years). >I never had a problem writing huge files. But I did have a problem >subsequently reading it with Windows XP. Maybe the file size >restriction is a Windows thing. >From the information I have, MS-WIN handles large files correctly. What problems do you have? >Not too say you're .iso images can't be >2GB/4GB, but I'm pretty sure >the ISO9660 standard is limited to a 2GB maximum file size (for files >within the .iso). You must use UDF to burn files of greater size. >mkisofs(8) seems to support this, if only in alpha/hybrid stage: The ISO-9660 standard is limited to 8 TB for the max. filesystem size as well as for the size of a single file. Mkisofs supports single files up to 4 GB since ~ Y2000 and it suppports files up to 8 TB since 2 years. >growisofs will write 3 files to the DVD just as easily as one. The magic >of growisofs is that it invokes mkisofs on the fly. And also that >cdrecord had obnoxious (and broken) licensing in years past when I last >tried it. Cdrecord alwas had and still has a free license. There is nothing broken with cdrtools/cdrecord. There is however a big social problem with the anti-social habbit and propaganda from Debian. Fortunately, cdrtools is not related to Debian, so where is your problem? Growisofs depends on mkisofs and mkisofs is part of cdrtools, so you need cdrtools anyway. >1) I expect the problem is with mkisofs version 2.01 that is used in >FreeBSD (at least in 7.0), but I haven't attempted to confirm this. >Growisofs is supposed to handle large files, and so are recent >versions of mkisofs. The author of mkisofs considers 2.01 to be >obsolete. Perhaps installing a newer mkisofs is the solution (e.g. >sysutils/cdrtools-devel appears to be fairly recent)? mkisofs-2.01 is more than 4 years old. It is not just obsolete but extremely outdated. Since mkisofs-2.01 30-50% of the code has been replaced or added. Cdrtools is only a few weeks from the next major release and it would be silly not to use a recent version. Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
