Hi Randy,
> > Jianfei Wang wrote: >> Hi Randy, >> >> This project focuses on negotiating and setting the TLR control bit >> through the driver interaction. The TLR operation is done at the >> firmware level, not the driver level. So question 1, 2, 4 and 5 are >> not in the scope of this project. > > When MPxIO was implemented it was tested with multi port SAS drives. > >> >> For question 3, SAS1 doesn't support TLR when MPXIO is enabled. This >> is specified in document 07-027r2 (SAS-2 Enabling and disabling >> Transport Layer Retries) on T10. The SAS1 TLR mode sense/mode select >> has been proved to be impractical when MPXIO is enabled, and the >> reason is ' If the mode page is shared, then it's possible that >> multiple initiators don?t agree on the setting interfere with each >> other. This is less of a problem if the mode page is per I_T nexus, >> but such implementations are rare.' The protocol author found mode >> sense/mode select is a bad support way, so SAS1 TLR support is >> abandoned when MPXIO is enabled. SAS2 inquiries the VPD page to >> support TLR, it would support MPXIO. > > Tape drives are explicitly one user at at time so doing mode sense > mode select on open (reservation) and on detected resets should not be > an issue. > > Can the target driver determine if the drive supports TLR by reading > the mode sense page? All of this will work if the target driver can > detect if TLR is supported without the scsi_getcap() and if the HBA > does not attempt to recover a single command forever. The target driver could not determine the I_T_L support TLR, it should need initiator support TLR. Whether a I-T-L supports TLR not only need target device confirm that it has such capability by Mode Page or Inquiry VPD, but also initiator(HBA) confirm it's capability. That's why we introduce the scsi_getcap/scsi_setcap for HBA TLR capability. I do understand tape is able to support MPxIO, our concern is not whether we can detect Mode page information change after target reset. The problem is MPxIO introduced multiple paths to single target, but no one can guarantee all initiators associated to the paths are TLR-capable. SAS1.x TLR has some defects in protocol perspective, it request initiator enable/disable TLR feature by setting Mode page of the target. Take a example, if a tape connects to two HBA cards, one supports TLR but another doesn't support the feature, it is too hard to manage TLR since it's very hard disable/enable TLR support by mode sense/select frequently across different paths during transport. It's not worth to introduce complex logic in the solaris for the defect protocol implementation since the SAS2.0 correct the defects. So we decide to provide minimum support for SAS1.x tape only when MPxIO is disabled. The SAS2 TLR doesn't have this problem. It uses the inquiryVPD get TLR property, the capability of the LUN is fixed. Enabling the TLR only relies on initiator and we don't need enable/disable any mode page of target per different paths. Best Regards Jianfei Wang > >> >> Best Regards >> Jianfei Wang >> >> >> Frank Che wrote: >>> Some questions from Randy on this project: >>> >>> 1) How will this both work with command recovery that is in Nevada >>> and doesn't exsist in previous versions of Solaris where it is not? >>> >>> 2) This looks like it expects the tape driver to be the only source >>> of resets. With MPxIO enabled scsi_vhci can do resets when failing >>> over paths that would not have originated from the target driver. >>> Some HBA's also initiate resets on their own. >>> >>> 3) If MPxIO is enabled, the scsi_ifgetcap(9F) would not get the >>> capability from the HBA and therefor would not do the mode >>> sense/mode select correctly. See bug 6755363 as this would also >>> effect SCSI_CAP_TRAN_LAYER_RETRIES. >>> >>> 4) How will this work with ComStar where the transport may not be as >>> obvious to the target driver. >>> >>> 5) Does TLR give up? If a connection fails in a MPxIO invironment >>> waiting for a connection to reconnect would prevent failing over. >>> Also in a non-MPxIO environment not waiting could cause a failure >>> that could otherwise could be recovered when reconnected. >>> >>> Frank. >>> >>> Frank Che wrote: >>>> I'm sponsoring this case for Jianfei Wang. The timer is set for >>>> 09/02/2009. >>>> >>>> -Frank. >>>> >>>> 1. Introduction >>>> 1.1. Project/Component Working Name: >>>> Transport Layer Retries (TLR) Support >>>> >>>> 1.2. Name of Document Author/Supplier: >>>> Author: Jianfei Wang >>>> >>>> 1.3. Date of This Document: >>>> 08/24/2009 >>>> >>>> 4. Technical Description: >>>> >>>> 4.1. Summary >>>> >>>> This project is to provide TLR (Transport Layer Retries) support >>>> in Solaris so that tape drive devices could be better supported. >>>> >>>> Requested release binding is Patch/Micro. >>>> >>>> 4.2. Details >>>> >>>> To support Transport Layer Retries(TLR), a new SCSI transport >>>> capability, SCSI_CAP_TRAN_LAYER_RETRIES, will be added. >>>> HBA drivers need TLR support should be updated to know this >>>> capability. >>>> The storage target driver should also be updated to coordinate >>>> the TLR support. >>>> >>>> When a new tape drive device is detected, the target driver checks >>>> the HBA's capability of TLR support with the scsi_ifgetcap(9F) >>>> interface >>>> in the attach(9E) entry point. >>>> >>>> The target driver reads Vital Product Data (VPD) page 0x90 to >>>> get the target device type. If VPD page 0x90 exists, the target device >>>> type is SAS2; otherwise the target device type is SAS1. >>>> >>>> If the HBA supports TLR and the target device is a SAS1 device, >>>> then the >>>> target driver uses SCSI command MODE_SENSE and MODE_SELECT to set the >>>> mode page (Protocol-Specific Logical Unit mode page)'s TLR flag. If >>>> set >>>> successfully, Initiator_Target_Lun nexus support TLR property. Because >>>> the mode page is not persistent, the TLR flag bit will be lost when >>>> the >>>> target device do a hard reset. So the target driver needs to set >>>> the mode >>>> page's TLR flag each time the target device completes hard reset. >>>> >>>> If the HBA supports TLR and the target device is SAS2 device, then the >>>> target driver issues InquiryVPD to get the capability of the tape >>>> device >>>> to check whether the target device is able to support TLR. If both the >>>> HBA driver and the tape device support TLR, the target driver will >>>> send a >>>> commands packet with a special pkg_flag, FLAG_TLR, to notify >>>> the HBA driver to set TLR in control bit before transportation. If the >>>> Initiator_Target_Lun nexus does not support TLR, commands from the >>>> target >>>> driver to the HBA driver would fail and the pkt_reason will be set to >>>> CMD_TLR_OFF. When the target driver get this reason, it will turn off >>>> the TLR control. >>>> >>>> 4.3. Exported Interfaces >>>> >>>> Interface Commitment Comments >>>> ============= ========== ===================== >>>> SCSI_CAP_TRAN_LAYER_RETRIES Committed SCSI TLR capability >>>> tran-layer-retries Committed SCSI_CAP_ASCII content >>>> FLAG_TLR Committed pkg_flag indicating TLR is supported >>>> CMD_TLR_OFF Committed pkt_reason indicating TLR is not supported >>>> >>>> 4.4. Bug/RFE Number(s) >>>> 6647764 - Solaris storage driver Transport Layer Retries (TLR) support >>>> >>>> 4.5. References >>>> Serial Attached SCSI - 1.1 (SAS-1.1) >>>> Serial Attached SCSI - 2 (SAS-2) >>>> SCSI Primary Commands - 4 (SPC-4) >>>> http://www.t10.org/ >>>> >>>> 4.6. Manpage changes >>>> scsi_hba_lookup_capstr.diff >>>> ============================ >>>> 156a157 >>>> > SCSI_CAP_TRAN_LAYER_RETRIES >>>> 157a159,161 >>>> > "tran-layer-retries" >>>> >>>> scsi_ifgetcap.diff >>>> =========================== >>>> 126a127,128 >>>> > tran-layer-retries Transport Layer retries is support by >>>> > the host adapter: 0 disables, 1 enables. >>>> >>>> scsi_pkt.diff >>>> =========================== >>>> 226a227,230 >>>> > FLAG_TLR Run command with Transport >>>> > Layer Retries support >>>> > >>>> 323a328,330 >>>> > CMD_TLR_OFF Transport Layer Retries turn off >>>> > >>>> >>>> 6. Resources and Schedule: >>>> 6.4. Steering Committee requested information >>>> 6.4.1. Consolidation C-team Name: >>>> ON >>>> 6.5. ARC review type: Fast Track >>>> 6.6. ARC Exposure: Open >>>> >>>> >>>> >>
