Few iSCSI questions
1. You say that performance figures are this.. single iSCSI session: * 450MB/s Read and 450 MB/s Write for 64KB block * 510 MB/s Read and 550 MB/s Write for 256KB block * 65,000 IOPS - 1K, 58,000 IOPS - 2K, 50,000 IOPS with 4KB Read two iSCSI sessions: * 550 MB/s Read and 810 MB/s Write for 256KB block * 75,000 IOPS for 1K block How can we measure figures on our machine? "Iometer" , "disktest" are few tools available for that. Can u suggest any good one? --- 2. When Initiator can get access to disk from target,What all applications user can have?Means besides Storage,what is the use of this Disk? 3. If one initiator is logged in to target "iqn. 2008-06.com.companyname:BLOCKIO_TARGET" which is 1 GB. It thus have access to it,it partitions it using "fdisk".Creates file system using "mkfs".mounts it and writes 300 MB data on it. If second initiator machine logs in to the same target,it can view all the 300 MB data written by 1st initiator if 2nd initiator doesn't format that disk after login. 2nd initiator only mounts the logged in disk and thus had access to all the 300 MB data. Is it ethical? Should we allow same target to be shared by multiple initiators at the same time? What will be the pros and cons? I think,we need to handle synchronization issues there.Data written by 1st initiator shouldn't be deleted by others and all.. Or Data from 1st initiator should not be visible/accessible to others. What approach looks more ethical and practical? --- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Few iSCSI questions
My thoughts inline:
On Tue, Jun 17, 2008 at 5:40 PM, HIMANSHU <[EMAIL PROTECTED]> wrote:
>
> 1.
> You say that performance figures are this..
>
> single iSCSI session:
>
>* 450MB/s Read and 450 MB/s Write for 64KB block
>* 510 MB/s Read and 550 MB/s Write for 256KB block
>* 65,000 IOPS - 1K, 58,000 IOPS - 2K, 50,000 IOPS with 4KB Read
>
> two iSCSI sessions:
>
>* 550 MB/s Read and 810 MB/s Write for 256KB block
>* 75,000 IOPS for 1K block
>
> How can we measure figures on our machine?
> "Iometer" , "disktest" are few tools available for that.
> Can u suggest any good one?
>
>
> ---
> 2.
> When Initiator can get access to disk from target,What all
> applications user can have?Means besides Storage,what is the use of
> this Disk?
[Dheeraj] Its just like a disk. Apart from storage, a disk can also be used
as a lock, albeit via a filesystem using file locks.
Similarly using scsi reservations, I believe it could be used as a lock.
>
>
>
> 3.
> If one initiator is logged in to target "iqn.
> 2008-06.com.companyname:BLOCKIO_TARGET" which is 1 GB.
>
> It thus have access to it,it partitions it using "fdisk".Creates file
> system using "mkfs".mounts it and writes 300 MB data on it.
>
> If second initiator machine logs in to the same target,it can view all
> the 300 MB data written by 1st initiator if 2nd initiator doesn't
> format that disk after login. 2nd initiator only mounts the logged in
> disk and thus had access to all the 300 MB data. Is it ethical?
>
> Should we allow same target to be shared by multiple initiators at the
> same time?
>
> What will be the pros and cons?
{DHeeraj]:Unless there are more than one writers, the data can be accessed
without inconsistency.
However in case the LU is to be shared among hosts, they must form a
group(aka cluster) and reserve the LU before accessing it.
SCSI provides persistent reserve and release commands to achieve this.
>
>
> I think,we need to handle synchronization issues there.Data written by
> 1st initiator shouldn't be deleted by others and all..
>
Or Data from 1st initiator should not be visible/accessible to others.
>
> What approach looks more ethical and practical?
>
> ---
>
> >
>
--
Simplicity of character is the natural result of profound thought.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To post to this group, send email to open-iscsi@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/open-iscsi
-~--~~~~--~~--~--~---
Re: Removing individual LUNs
On Mon, Jun 16, 2008 at 06:03:25PM -0700, Alex M. wrote: > > On Jun 16, 4:29 pm, Mike Christie <[EMAIL PROTECTED]> wrote: > > Do: > > > > iscsiadm -m session -r $SID --rescan > > Doing a rescan doesn't appear to pick up the new size of the devices. > Having manually resized the filesystem on the target end, I now get > errors on the initiator since the filesystem extents are outside of > the apparent device extents. That is not a iSCSI issue. It is the SCSI subsystem in the kernel. When you give the rescan command iscsiadm passes in '- - -' to the /sys/class/scsi_host/hostX/rescan which kicks off a kernel thread that re-scans for the LUNs. The logic in the kernel is that if it finds that a Scsi_Host with a LUN exists already it returns that one and doesn't do a full re-discover on it. So you have to delete the Scsi_Host from userland and then follow it with a rescan. But the reason this is done (and that the kernel doesn't do this automaticlly) is that this operation is very dangerous. If you don't unmount any of the filesystems on those block devices before you delete them, you will incur data corruption. > > > The above commands will rescan existing devices and add new ones. It > > will not remove old ones. > > As you said, rescans don't pick up deleted LUN's, so my only option > would be to delete the LUN on the target and then re-create it under a > _different_ LUN and then do a rescan. This is obviously not an ideal > solution as it's going to result in a large number of "zombie" LUNs on > the initiator as time goes on. You can delete the old block devices that have changed and do a SCSI rescan: You have to delete the old devices (well, this script deletes _ALL_ so that might not be that good): for disk in `find /sys/class/iscsi_session/session*/device/target*/*:*:*:*/delete ` do echo 1 > $disk done And then do the rescan: #iscsiadm -m session --rescan Unfortunatly this will delete _all_ of your block devices (sdc, sde, sdd, etc) that were provided. But it will pick up your new one after the rescan. You can fiddle with the for loop above and run 'sg_caps' on the block device, see if it has changed compared to /sys/block/sdX/device/size and if so delete it: scsi_sz=`sg_readcap /dev/sdko | grep address | sed s'/.*blocks=//'` kernel_sz=`cat /sys/block/sdko/device/block\:sdko/size` if (($scsi_sz==$kernel_sz)); then echo "Good." else echo "Do the deletion of the block device." echo 1 > /sys/block/sdko/device/delete fi Oh, please be carefull. These scripts will delete your block devices, so if you delete your root one or any other one that is used - you are screwed. Make sure you first unmount any of the filesystems on those block devices and sync it so that there are no outstanding I/Os. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Few iSCSI questions
On Tue, Jun 17, 2008 at 05:10:10AM -0700, HIMANSHU wrote: > > 1. > You say that performance figures are this.. > > single iSCSI session: > > * 450MB/s Read and 450 MB/s Write for 64KB block > * 510 MB/s Read and 550 MB/s Write for 256KB block > * 65,000 IOPS - 1K, 58,000 IOPS - 2K, 50,000 IOPS with 4KB Read > > two iSCSI sessions: > > * 550 MB/s Read and 810 MB/s Write for 256KB block > * 75,000 IOPS for 1K block > > How can we measure figures on our machine? > "Iometer" , "disktest" are few tools available for that. > Can u suggest any good one? I've used 'iozone' but you can also just do variations of 'dd' with different block sizes. And play around with changing the MTU. > > --- > 2. > When Initiator can get access to disk from target,What all > applications user can have?Means besides Storage,what is the use of > this Disk? None. It is just as a normal disk. > > 3. > If one initiator is logged in to target "iqn. > 2008-06.com.companyname:BLOCKIO_TARGET" which is 1 GB. > > It thus have access to it,it partitions it using "fdisk".Creates file > system using "mkfs".mounts it and writes 300 MB data on it. > > If second initiator machine logs in to the same target,it can view all > the 300 MB data written by 1st initiator if 2nd initiator doesn't > format that disk after login. 2nd initiator only mounts the logged in > disk and thus had access to all the 300 MB data. Is it ethical? You can do it but without using a proper filesystem that can deal with multiple writers/readers it will be hosed. Look in using 'gfs2' filesystem (or GFS from Red Hat, or GPFS from IBM) along with the cluster componenents to synchronize the disk usage. You might be tempted in thinking that you can mount from one initiator the disk as read-only with 'ext3' or 'ext3' - but from what I've heard that still leads to errors. > > Should we allow same target to be shared by multiple initiators at the > same time? If you want to use Xen and transition between nodes then yes you want to use it. Also for clustering software this can be used - where there is a shared storage. > > What will be the pros and cons? Look up on Google 'cluster software pros' to find that out. > > I think,we need to handle synchronization issues there.Data written by > 1st initiator shouldn't be deleted by others and all.. > > Or Data from 1st initiator should not be visible/accessible to others. > > What approach looks more ethical and practical? Use cluster software. > --- > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Disconnected iSCSI and umount problems
I posted a little while back about this, but I still seem to be having trouble with this issue. Originally I tried to setup my iSCSI connection so that it had a 24 day timeout period and the no-op timers would be disabled. However this timeout led to a variety of issues including causing umount, reboot, and other commands to hang. So in the end the long timeout proved to be too much trouble so I moved back to the 120s timeout with noop timers enabled. However even this is causing me trouble. Currently I am using my iSCSI device to store video which means I am sending a large amount of data over the network into my device at a pretty high rate. In my tests if I cut the connection sometimes things will work out fine. The connection gets cut and after 120s I get a whole slew of iSCSI "queuing" errors and such and finally the iSCSI device gets remounted as read only. Once the error messages stop if I stop all of my video archiving, reconnect the iSCSI device, logout, umount the iSCSI, remount the iSCSI, log back in, and restart my video archiver everything will work fine. However in other cases when i cut the connection the iSCSI debugs won't be as numerous and it goes to read only mode almost immediately. When I try the above steps to recover my system hangs like before on the umount. I used KDB to get a dump of what is going during the umount and will add it to this message. It appears that the umount process has context switched out waiting for the io to complete. The io to be completed are 'sync'ing of the buffers which never happens or completed and does not wake up umount. I've tried numerous things to get around this umount issue including a variety of umount flags, the remount command, long delays in my code and the kernel code. But nothing has worked up to this point. I'm currently working on version 2.6.16 of the kernel with open iSCSI version 2.0-865.9. I am going to try the latest and greatest to see if that helps at all. I'm convinced that the current problem has something to do with a quick change to 'ro' mode vs a slower change to 'ro' mode after timeout. If you guys have any advice or insight I'd appreciate the help. I posted the debug in the files section. The filename is umount_hang.rtf. I've bolded the area where the umount gets called. Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
