Re: What happens if an iSCSI target resets.
Mike Christie wrote: On 03/25/2010 03:50 PM, netz-haut - stephan seitz wrote: -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of Alvin Starr Sent: Friday, March 19, 2010 3:13 PM To: open-iscsi@googlegroups.com Subject: What happens if an iSCSI target resets. I am thinking of a 2 node cluster that sync the disks via DRBD. I have seen some articles about using multipath and 2 independent targets but have also seen generic problem descriptions of using multipath in the face of failures. I was thinking of using a floating IP address for the target and using the clustering/HA glue to make it move between servers. When you say floating IP's does that mean they will not be static? That might not work very well with open-iscsi. If we lose the connection or run the error recovery and have to relogin we will try to use the IP that we started with. Some targets like from Equallogic will have a static IP that the initiator logs into, then the target will tell us about a different IP to use. We then login there and do IO. If we get disconnected though, we go back the first IP and it redirects us again. I am thinking more that if server A has the ip address 1.2.3.4 and server B has 1.2.3.5 if server A breaks server B takes over 1.2.3.4 along with 1.2.3.5. Another scenario would be that either server A or B has 1.2.3.4 and if running server dies the other server takes over the address. -- Alvin Starr || voice: (416)585-9971x690 Interlink Connectivity|| fax: (416)585-9974 al...@iplink.net || -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: iSCSI target recommendation
It doesn't seem like this is a true SAN though.. Any other suggestions? On Mar 26, 12:54 pm, heini66 wrote: > 2010/3/26 An Oneironaut > > > Hey all. Could anyone suggest a good NAS that has about 2 to 6TB of > > storage which is under 4k? its hard to find out whether these people > > have tested with open-iscsi or not. So I was hoping some of you out > > there who had used a storage device within this range would have some > > opinions. Please tell me if you have any suggestions. > > > Thanks, > > you might want to have a look ahttp://www.drobo.com/products/drobopro/ > drive without hdd's and 19" mounting option 999€. > one of the nicest options in my oppionion is the possibillity to put in only > 2 2tb hdd's and virtual format it to 16tb. if it runs out of space with > first attached hdd's, you'll get an email or smtp notice, that you have to > add hdd's. this can be done without shuting the drobo down or taking it > offline. an no!!! special tray's or mounts for the additonal hdds's are > needed. just open one of the empty slot's and plug hdd into it. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: iSCSI target recommendation
2010/3/26 An Oneironaut > Hey all. Could anyone suggest a good NAS that has about 2 to 6TB of > storage which is under 4k? its hard to find out whether these people > have tested with open-iscsi or not. So I was hoping some of you out > there who had used a storage device within this range would have some > opinions. Please tell me if you have any suggestions. > > Thanks, > > you might want to have a look a http://www.drobo.com/products/drobopro/ drive without hdd's and 19" mounting option 999€. one of the nicest options in my oppionion is the possibillity to put in only 2 2tb hdd's and virtual format it to 16tb. if it runs out of space with first attached hdd's, you'll get an email or smtp notice, that you have to add hdd's. this can be done without shuting the drobo down or taking it offline. an no!!! special tray's or mounts for the additonal hdds's are needed. just open one of the empty slot's and plug hdd into it. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: What happens if an iSCSI target resets.
On 03/26/2010 01:42 PM, Dheeraj Sangamkar wrote: You may want to check out 'target moved temporarily' in the following document: http://ietfreport.isoc.org/all-ids/draft-gilligan-iscsi-fault-tolerance-00.txt Neat. Thanks. In the open-iscsi.git code (working on making a release here http://www.kernel.org/pub/linux/kernel/people/mnc/open-iscsi/releases/open-iscsi-2.0-872-rc1.tar.gz) we sort of do this. Instead of running the iscsiadm discovery command "iscsiadm -m discovery " you can set discovery.daemon.sendtargets.addresses = $IP (for isns replace sendtargets with isns) When iscsid starts up it will do discovery, and try to login to the portals found. It will then do discovery every poll_interval seconds (or for isns it will react to isns SCNs), and log in/out of portals that are found or no longer found. To implement what is in that spec we could just add some code to fire an event to iscsid to force it to do discovery right away instead of poll_interval secs when it cannot log into the portal. Dheeraj On Sat, Mar 27, 2010 at 12:08 AM, Mike Christie wrote: On 03/25/2010 03:50 PM, netz-haut - stephan seitz wrote: -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of Alvin Starr Sent: Friday, March 19, 2010 3:13 PM To: open-iscsi@googlegroups.com Subject: What happens if an iSCSI target resets. I am thinking of a 2 node cluster that sync the disks via DRBD. I have seen some articles about using multipath and 2 independent targets but have also seen generic problem descriptions of using multipath in the face of failures. I was thinking of using a floating IP address for the target and using the clustering/HA glue to make it move between servers. When you say floating IP's does that mean they will not be static? That might not work very well with open-iscsi. If we lose the connection or run the error recovery and have to relogin we will try to use the IP that we started with. Some targets like from Equallogic will have a static IP that the initiator logs into, then the target will tell us about a different IP to use. We then login there and do IO. If we get disconnected though, we go back the first IP and it redirects us again. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
iSCSI target recommendation
Hey all. Could anyone suggest a good NAS that has about 2 to 6TB of storage which is under 4k? its hard to find out whether these people have tested with open-iscsi or not. So I was hoping some of you out there who had used a storage device within this range would have some opinions. Please tell me if you have any suggestions. Thanks, JD -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: What happens if an iSCSI target resets.
You may want to check out 'target moved temporarily' in the following document: http://ietfreport.isoc.org/all-ids/draft-gilligan-iscsi-fault-tolerance-00.txt Dheeraj On Sat, Mar 27, 2010 at 12:08 AM, Mike Christie wrote: > On 03/25/2010 03:50 PM, netz-haut - stephan seitz wrote: >>> >>> -Original Message- >>> From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] >>> On Behalf Of Alvin Starr >>> Sent: Friday, March 19, 2010 3:13 PM >>> To: open-iscsi@googlegroups.com >>> Subject: What happens if an iSCSI target resets. >>> >>> >>> I am thinking of a 2 node cluster that sync the disks via DRBD. >>> >>> I have seen some articles about using multipath and 2 independent >>> targets but have also seen generic problem descriptions of using >>> multipath in the face of failures. >>> >>> I was thinking of using a floating IP address for the target and using >>> the clustering/HA glue to make it move between servers. > > When you say floating IP's does that mean they will not be static? That > might not work very well with open-iscsi. If we lose the connection or run > the error recovery and have to relogin we will try to use the IP that we > started with. > > Some targets like from Equallogic will have a static IP that the initiator > logs into, then the target will tell us about a different IP to use. We then > login there and do IO. If we get disconnected though, we go back the first > IP and it redirects us again. > > -- > You received this message because you are subscribed to the Google Groups > "open-iscsi" group. > To post to this group, send email to open-is...@googlegroups.com. > To unsubscribe from this group, send email to > open-iscsi+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/open-iscsi?hl=en. > > -- Simplicity of character is the natural result of profound thought. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Building iscsid with iscsi modules built into the kernel
On 03/26/2010 01:34 PM, Yangkook Kim wrote:
Hi, all.
I want to know how to compile iscsid with iscsi modules already built
into the kernel.
I built a kernel with the following options selected in .config so that
iscsi_tcp, libiscsi and scsi_transport_iscsi are built into the kernel.
CONFIG_ISCSI_TCP=y
I compiled iscsid shipped with open-iscsi-2.0-871 and run it.
Then, I hit the following error and iscsid won't come up.
Mar 23 04:58:03 ubuntu iscsid: Missing or Invalid version from
/sys/module/scsi_transport_iscsi/version. Make sure a up to date
scsi_transport_iscsi module is loaded and a up todate version of
iscsid is running. Exiting...
I understand that the error is telling that it can't read version of
scsi_transport_iscsi from /sys/module directory.
But, since my scsi_transport_iscsi is built into the kernel,
there is no /sys/module/scsi_transport_iscsi direcotry.
So, I removed the piece of codes in iscsid.c to avoid checking
/sys/module/scsi_transport_iscsi/version to run iscsid.
if (iscsi_sysfs_check_class_version()) {
log_close(log_pid);
exit(1);
}
However, I don't think removing the codes is not a correct way
to build iscsid with iscsi modules already built into the kernel.
Can anybody tell the correct way to make iscsid when iscsi modules
are built into the kernel?
There is not way. Some people have worked on patches so that those
module sysfs files are exported even when the module is built into the
kernel, but I guess they have not been merged.
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: What happens if an iSCSI target resets.
On 03/25/2010 03:50 PM, netz-haut - stephan seitz wrote: -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of Alvin Starr Sent: Friday, March 19, 2010 3:13 PM To: open-iscsi@googlegroups.com Subject: What happens if an iSCSI target resets. I am thinking of a 2 node cluster that sync the disks via DRBD. I have seen some articles about using multipath and 2 independent targets but have also seen generic problem descriptions of using multipath in the face of failures. I was thinking of using a floating IP address for the target and using the clustering/HA glue to make it move between servers. When you say floating IP's does that mean they will not be static? That might not work very well with open-iscsi. If we lose the connection or run the error recovery and have to relogin we will try to use the IP that we started with. Some targets like from Equallogic will have a static IP that the initiator logs into, then the target will tell us about a different IP to use. We then login there and do IO. If we get disconnected though, we go back the first IP and it redirects us again. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Building iscsid with iscsi modules built into the kernel
Hi, all.
I want to know how to compile iscsid with iscsi modules already built
into the kernel.
I built a kernel with the following options selected in .config so that
iscsi_tcp, libiscsi and scsi_transport_iscsi are built into the kernel.
CONFIG_ISCSI_TCP=y
I compiled iscsid shipped with open-iscsi-2.0-871 and run it.
Then, I hit the following error and iscsid won't come up.
Mar 23 04:58:03 ubuntu iscsid: Missing or Invalid version from
/sys/module/scsi_transport_iscsi/version. Make sure a up to date
scsi_transport_iscsi module is loaded and a up todate version of
iscsid is running. Exiting...
I understand that the error is telling that it can't read version of
scsi_transport_iscsi from /sys/module directory.
But, since my scsi_transport_iscsi is built into the kernel,
there is no /sys/module/scsi_transport_iscsi direcotry.
So, I removed the piece of codes in iscsid.c to avoid checking
/sys/module/scsi_transport_iscsi/version to run iscsid.
if (iscsi_sysfs_check_class_version()) {
log_close(log_pid);
exit(1);
}
However, I don't think removing the codes is not a correct way
to build iscsid with iscsi modules already built into the kernel.
Can anybody tell the correct way to make iscsid when iscsi modules
are built into the kernel?
Thanks,
Kim
#
Here is my system information.
bash-3.2$ /sbin/iscsid -v
iscsid version 2.0-871
bash-3.2$ uname -a
Linux ubuntu 2.6.28.11-kgdb #1 SMP Sat Mar 20 13:57:02 PDT 2010 i686 GNU/Linux
bash-3.2$ cat open-iscsi-2.0-871/usr/Makefile
# This Makefile will work only with GNU make.
OSNAME=$(shell uname -s)
# allow users to override these
# eg to compile for a kernel that you aren't currently running
KERNELRELEASE ?= $(shell uname -r)
KSRC ?= /lib/modules/$(KERNELRELEASE)/build
KSUBLEVEL=$(shell cat $(KSRC)/Makefile | awk -F= '/^SUBLEVEL =/ {print $$2}' | \
sed 's/^[ \t]*//;s/[ \t]*$$//')
ifeq ($(OSNAME),Linux)
ifeq ($(KSUBLEVEL),11)
IPC_CFLAGS=-DNETLINK_ISCSI=12 -D_GNU_SOURCE
else
ifeq ($(KSUBLEVEL),12)
IPC_CFLAGS=-DNETLINK_ISCSI=12 -D_GNU_SOURCE
else
IPC_CFLAGS=-DNETLINK_ISCSI=8 -D_GNU_SOURCE
endif
endif
IPC_OBJ=netlink.o
else
ifeq ($(OSNAME),FreeBSD)
IPC_CFLAGS=
IPC_OBJ=ioctl.o
endif
endif
OPTFLAGS ?= -O2 -g
WARNFLAGS ?= -Wall -Wstrict-prototypes
CFLAGS += $(OPTFLAGS) $(WARNFLAGS) -I../include -I. -D$(OSNAME) $(IPC_CFLAGS)
PROGRAMS = iscsid iscsiadm iscsistart
# libc compat files
SYSDEPS_SRCS = $(wildcard ../utils/sysdeps/*.o)
# sources shared between iscsid, iscsiadm and iscsistart
ISCSI_LIB_SRCS = util.o io.o auth.o login.o log.o md5.o sha1.o iface.o
idbm.o sysfs.o host.o session_info.o iscsi_sysfs.o $(SYSDEPS_SRCS)
# sources shared between iscsid and iscsiadm
COMMON_SRCS = $(ISCSI_LIB_SRCS)
# core initiator files
INITIATOR_SRCS = initiator.o scsi.o actor.o event_poll.o mgmt_ipc.o isns.o \
cxgb3i.o transport.o
# fw boot files
FW_BOOT_SRCS = $(wildcard ../utils/fwparam_ibft/*.o)
all: $(PROGRAMS)
iscsid: $(COMMON_SRCS) $(IPC_OBJ) $(INITIATOR_SRCS) iscsid.o
$(CC) $(CFLAGS) $^ -o $@
iscsiadm: $(COMMON_SRCS) $(FW_BOOT_SRCS) strings.o discovery.o iscsiadm.o
$(CC) $(CFLAGS) $^ -o $@
iscsistart: $(IPC_OBJ) $(ISCSI_LIB_SRCS) $(INITIATOR_SRCS) $(FW_BOOT_SRCS) \
iscsistart.o statics.o
$(CC) $(CFLAGS) -static $^ -o $@
clean:
rm -f *.o $(PROGRAMS) .depend $(LIBSYS)
depend:
gcc $(CFLAGS) -M `ls *.c` > .depend
-include .depend
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
RE: What happens if an iSCSI target resets.
We're running a 2 node DRBD active/passive target. During pre-production tests I've found, that kicking the primary immediately off by heartbeat/sys_req or STONITH works much better than shutdown. I've configured dm-multipath on top of bonding devices (mode 0) utilizing two switches. The bonding eats some performance, but target availability is 100%. The heartbeat is configured to 1. move the target ip's to the other node 2. sleep 1 second 3. move a drbd device with only configurations for iscsitarget, etc.. 4. another 1 second delay 5. move (say: primary stop and secondary start) lvm VG (iscsitarget uses this VG on top of a drbd) 6. move the big data drbd 7. move iscsitarget > -Original Message- > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] > On Behalf Of Alvin Starr > Sent: Friday, March 19, 2010 3:13 PM > To: open-iscsi@googlegroups.com > Subject: What happens if an iSCSI target resets. > > > I am thinking of a 2 node cluster that sync the disks via DRBD. > > I have seen some articles about using multipath and 2 independent > targets but have also seen generic problem descriptions of using > multipath in the face of failures. > > I was thinking of using a floating IP address for the target and using > the clustering/HA glue to make it move between servers. > > This would be the equivalent of resetting the target software. > > Does anyone have a handle on how the initiators will deal with this. > Ideally in theory and practice. > -- > > Alvin Starr || voice: (416)585-9971x690 > Interlink Connectivity|| fax: (416)585-9974 > al...@iplink.net || > > -- > You received this message because you are subscribed to the Google > Groups "open-iscsi" group. > To post to this group, send email to open-is...@googlegroups.com. > To unsubscribe from this group, send email to open- > iscsi+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/open-iscsi?hl=en. > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: [PATCH 1/2] minor manpage updates
On 2/26/10, Ulrich Windl wrote: > On 26 Feb 2010 at 1:07, Ritesh Raj Sarraf wrote: > >> >> I: open-iscsi: hyphen-used-as-minus-sign > > Just in case somebody wonders: On an ASCII terminal, hyphen and minus > are the same character, but when typesetting manual pages (i.e. > "troff") the minus is somewhat longer than the hyphen, and it may even > be taken from a different font (e.g. "Symbol"). > > (In typesetting there are even spaces of different width) > > Regards, > Ulrich > > -- > You received this message because you are subscribed to the Google Groups > "open-iscsi" group. > To post to this group, send email to open-is...@googlegroups.com. > To unsubscribe from this group, send email to > open-iscsi+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/open-iscsi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
RE: Reboot hangs on failing multipath devices
This is a reported bug of the device-mapper on debian. There's a patch at debians bugtracker available, but as far as I remember, it has been refused by upstream developers. We're also running open-iscsi/dm-multipath/lvm/clvm stack on virtualization Hosts. Due to this behavior one big point is to never ever let multipath loose all pathes. Try to add features"1 queue_if_no_path" to your related multipath.conf device section. Regards, Stephan > -Original Message- > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] > On Behalf Of James Hammer > Sent: Tuesday, March 23, 2010 4:13 PM > To: open-iscsi@googlegroups.com > Subject: Re: Reboot hangs on failing multipath devices > > Mike Christie wrote: > > On 03/22/2010 03:38 PM, James Hammer wrote: > >> Every time I reboot my server it hangs on the multipath devices. > >> > >> The server is Debian based. I've had this problem with all kernels > I've > >> tried (2.6.18, 2.6.24, 2.6.32). In /etc/multipath.conf, > no_path_retry is > >> set to queue > >> > >> Here are snippets from the reboot log: > >> > >> > >> Stopping multipath daemon: multipathd. > >> ... > >> Shutting down LVM Volume Groupsdevice-mapper: multipath: Failing > path > >> 8:64. > >> device-mapper: uevent: dm_send_uevents: kobject_uevent_env failed > >> device-mapper: multipath: Failing path 8:48. > >> device-mapper: uevent: dm_send_uevents: kobject_uevent_env failed > >> device-mapper: multipath: Failing path 8:80. mult > > > > Are there file systems mounted on the multipath device? > > > > As far as I can tell, there are *no* file systems mounted on the > multipath device. This multipath device is used by a virtual machine. > The virtual machine is turned off at that point. The 'mount' command > on > the physical host does not list the multipath device as being mounted. > > This is what I have found...I ran the whole shutdown sequence manually, > i.e. running each script in /etc/rc0.d manually in order (with > *no_path_retry* set to *queue*). Between each shutdown script, I ran > '*multipath -f mpath5*' to try and remove the multipath device manually. > Each time I got this result: > > mpath5: map in use > > All the way down until I got to the last 3 scripts: > > S50lvm2 -> ../init.d/lvm2 > S60umountroot -> ../init.d/umountroot > S90halt -> ../init.d/halt > > > > When that lvm2 script gets run to shutdown lvm2, I again get the > "multipath: Failing path" results: > > Shutting down LVM Volume Groupsdevice-mapper: multipath: Failing path > 8:48. > device-mapper: uevent: dm_send_uevents: kobject_uevent_env failed > device-mapper: multipath: Failing path 8:80. > device-mapper: uevent: dm_send_uevents: kobject_uevent_env failed > device-mapper: multipath: Failing path 8:64. > device-mapper: uevent: dm_send_uevents: kobject_uevent_env failed > > That hangs indefinitely. > > Now, if I do the same thing with *no_path_retry* set to *fail* the > sequence goes similarly, except that when I run */etc/init.d/lvm2 stop* > I get the same as above followed by a few of these lines: > > /dev/dm-9: read failed after 0 of 2048 at 0: Input/output error > end_request: I/O error, dev dm-9, sector 20971776 > > Then the script finishes and the reboot can proceed. > > So the key seems to be the *no_path_retry* setting. > > From my tests, things seem to go so much better if *no_path_retry* is > set to *queue* and the connection to the iSCSI server is interrupted. > > So, is it possible to get those paths to "fail" with *no_path_retry* > set > to *queue* so the reboot can continue? > > Thanks! > > -- James > > -- > You received this message because you are subscribed to the Google > Groups "open-iscsi" group. > To post to this group, send email to open-is...@googlegroups.com. > To unsubscribe from this group, send email to open- > iscsi+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/open-iscsi?hl=en. > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
