From: Joe Landman <[EMAIL PROTECTED]> Subject: [Stgt-devel] Update (Re: open iSCSI over iSER target RPM ...) Date: Thu, 07 Feb 2008 11:05:03 -0500
> Update: > > [EMAIL PROTECTED] etc]# dd if=/dev/zero of=/big/local.file bs=256k > count=100000 > 100000+0 records in > 100000+0 records out > 26214400000 bytes (26 GB) copied, 58.7484 seconds, 446 MB/s > > Better. I rebuilt OFED 1.2.5.5. Are there specific recommended tuning > guides for iSER? Backing store in this case are real disks, and we can > sink/source >750 MB/s on them, so I am not worried about disk IO > bottlenecks, more worried about bad config of iSCSI/iSER. > > BTW: the 2TB LUN limit I asked about is still here in this code. Same > machines (initiator and target) used for SRP reported correct LUN sizes. > Here we are using the -868 open-iscsi initiator, and the tgt RPM > announced. I would like to dig into this. > > This is what I am getting in dmesg for this iSER target: > > iscsi: registered transport (tcp) > iscsi: registered transport (iser) > iser: iser_connect:connecting to: 10.2.1.2, port 0xbc0c > iser: iser_cma_handler:event 0 conn ffff81024b9f69c0 id ffff810209748c00 > iser: iser_cma_handler:event 2 conn ffff81024b9f69c0 id ffff810209748c00 > iser: iser_create_ib_conn_res:setting conn ffff81024b9f69c0 cma_id > ffff810209748c00: fmr_pool ffff81024bfb32c0 qp ffff8101cb16d600 > iser: iser_cma_handler:event 9 conn ffff81024b9f69c0 id ffff810209748c00 > iser: iscsi_iser_ep_poll:ib conn ffff81024b9f69c0 rc = 1 > scsi13 : iSCSI Initiator over iSER, v.0.1 > iser: iscsi_iser_conn_bind:binding iscsi conn ffff81021b65fa90 to > iser_conn ffff81024b9f69c0 > Vendor: IET Model: Controller Rev: 0001 > Type: RAID ANSI SCSI revision: 05 > scsi 13:0:0:0: Attached scsi generic sg2 type 12 > Vendor: IET Model: VIRTUAL-DISK Rev: 0001 > Type: Direct-Access ANSI SCSI revision: 05 > sdc : very big device. try to use READ CAPACITY(16). > sdc : READ CAPACITY(16) failed. > sdc : status=1, message=00, host=0, driver=08 > sdc : use 0xffffffff as device size > SCSI device sdc: 4294967296 512-byte hdwr sectors (2199023 MB) > sdc: Write Protect is off > sdc: Mode Sense: 79 00 00 08 > SCSI device sdc: drive cache: write back > sdc : very big device. try to use READ CAPACITY(16). > sdc : READ CAPACITY(16) failed. > sdc : status=1, message=00, host=0, driver=08 > sdc : use 0xffffffff as device size > SCSI device sdc: 4294967296 512-byte hdwr sectors (2199023 MB) > sdc: Write Protect is off > sdc: Mode Sense: 79 00 00 08 > SCSI device sdc: drive cache: write back > sdc: unknown partition table > sd 13:0:0:1: Attached scsi disk sdc > sd 13:0:0:1: Attached scsi generic sg3 type 0 > > > and this is what we get in SRP > > scsi6 : SRP.T10:0008F104039862A4 > Vendor: SCST_BIO Model: vdisk0 Rev: 096 > Type: Direct-Access ANSI SCSI revision: 04 > sdc : very big device. try to use READ CAPACITY(16). > SCSI device sdc: 12693355130 512-byte hdwr sectors (6498998 MB) > sdc: Write Protect is off > sdc: Mode Sense: 6b 00 10 08 > SCSI device sdc: drive cache: write back w/ FUA > > > This looks suspiciously like a 2^32 limit somewhere. Can you try the latest git tree (65a3f8b0c14305aaee5bcaade569b40882e8dd88)? It works for me: scsi3 : iSCSI Initiator over TCP/IP scsi 3:0:0:0: RAID IET Controller 0001 PQ: 0 ANSI: 5 scsi 3:0:0:1: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5 sd 3:0:0:1: [sdb] Very big device. Trying to use READ CAPACITY(16). sd 3:0:0:1: [sdb] 12884901888 512-byte hardware sectors (6597070 MB) sd 3:0:0:1: [sdb] Write Protect is off sd 3:0:0:1: [sdb] Mode Sense: 79 00 00 08 sd 3:0:0:1: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Somehow I forgot to add READ_CAPACITY_16 support. I would appreciate it if you could read and write on over 2TB position properly. > Our exported device is > > [EMAIL PROTECTED] ~]# parted /dev/sdb print > > Model: Areca jrvs1 (scsi) > Disk /dev/sdb: 6500GB > Sector size (logical/physical): 512B/512B > Partition Table: loop > > Number Start End Size File system Flags > 1 0.00kB 6500GB 6500GB xfs > > > and this is what tgtadm reports > > [EMAIL PROTECTED] ~]# tgtadm --lld iscsi --op show --mode target > Target 1: iqn.2001-04.com.jr1-jackrabbit.small > System information: > Driver: iscsi > Status: running > I_T nexus information: > I_T nexus: 4 > Initiator: iqn.1996-04.voltaire.com:01:dfa8888a3fd > Connection: 0 > RDMA IP Address: 10.2.1.1 > LUN information: > LUN: 0 > Type: controller > SCSI ID: deadbeaf1:0 > SCSI SN: beaf10 > Size: 0 > Online: No > Poweron/Reset: Yes > Removable media: No > Backing store: No backing store > LUN: 1 > Type: disk > SCSI ID: deadbeaf1:1 > SCSI SN: beaf11 > Size: 5T > Online: Yes > Poweron/Reset: No > Removable media: No > Backing store: /dev/sdb > Account information: > ACL information: > 10.2.1.1 > > So it looks like the LUN 1 is approximately correct (5T ???) on the > target, and incorrect when the initiator asks for it. I changed tgt to show the capacity like Linux does: Target 1: iqn.2007-03:marks-vtl-tgt:tulip System information: Driver: iscsi Status: running I_T nexus information: I_T nexus: 1 Initiator: iqn.2005-03.org.open-iscsi:d38a581f3318 Connection: 0 IP Address: 192.168.11.15 LUN information: LUN: 0 Type: controller SCSI ID: deadbeaf1:0 SCSI SN: beaf10 Size: 0 MB Online: Yes Poweron/Reset: Yes Removable media: No Backing store: No backing store LUN: 1 Type: disk SCSI ID: deadbeaf1:1 SCSI SN: beaf11 Size: 6597070 MB Online: Yes Poweron/Reset: No Removable media: No Backing store: /dev/sde Account information: ACL information: ALL > Please note that I have successfully used the full 6+TB as an iSCSI > target using the SCST-iscsi code, so I do know that the initiator works > correctly. > > Is there a source RPM/tree for this target? I guess that RedHat, SUSE, and OFED have tgt RPMs now so I think that you can find something. _______________________________________________ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg