Roshan, As far as I know, there is no problem at all with using SAN storage with ZFS and it does look like you were having an underlying problem with either powerpath or the array.
The best practices guide on opensolaris does recommend replicated pools even if your backend storage is redundant. There are at least 2 good reasons for that. ZFS needs a replica for the self healing feature to work. Also there is no fsck like tool for ZFS so it is a good idea to make sure self healing can work. I think first I would track down the cause of the messages just prior to the zfs write error because even with replicated pools if several devices error at once then the pool could be lost. Regards, Vic On 6/19/07, Roshan Perera <[EMAIL PROTECTED]> wrote:
Victror, Thanks for your comments but I believe it contradict what ZFS information given below and now Bruce's mail. After some digging around I found that the messages file has thrown out some powerpath errors to one of the devices that may have caused the proble. attached below the errors. But the question still remains is ZFS only happy with JBOD disks and not SAN storage with hardware raid. Thanks Roshan Jun 4 16:30:09 su621dwdb ltid[23093]: [ID 815759 daemon.error] Cannot start rdevmi pr ocess for remote shared drive operations on host su621dh01, cannot connect to vmd Jun 4 16:30:12 su621dwdb emcp: [ID 801593 kern.notice] Info: Assigned volume Symm 000 290100491 vol 0ffe to Jun 4 16:30:12 su621dwdb last message repeated 1 time Jun 4 16:30:12 su621dwdb emcp: [ID 801593 kern.notice] Info: Assigned volume Symm 000 290100491 vol 0fee to Jun 4 16:30:12 su621dwdb unix: [ID 836849 kern.notice] Jun 4 16:30:12 su621dwdb ^Mpanic[cpu550]/thread=2a101dd9cc0: Jun 4 16:30:12 su621dwdb unix: [ID 809409 kern.notice] ZFS: I/O failure (write on <un known> off 0: zio 600574e7500 [L0 unallocated] 4000L/400P DVA[0]=<5:55c00:400> DVA[1]= <6:2b800:400> fletcher4 lzjb BE contiguous birth=107027 fill=0 cksum=673200f97f:34804a 0e20dc:102879bdcf1d13:3ce1b8dac7357de): error 5 Jun 4 16:30:12 su621dwdb unix: [ID 100000 kern.notice] Jun 4 16:30:12 su621dwdb genunix: [ID 723222 kern.notice] 000002a101dd9740 zfs:zio_do ne+284 (600574e7500, 0, a8, 708fdca0, 0, 6000f26cdc0) Jun 4 16:30:12 su621dwdb genunix: [ID 179002 kern.notice] %l0-3: 0000060015beaf00 0 0000000708fdc00 0000000000000005 0000000000000005 > We have the same problem and I have just moved back to UFS because of > this issue. According to the engineer at Sun that i spoke with, he > implied that there is an RFE out internally that is to address > this problem. > > The issue is this: > > When configuring a zpool with 1 vdev in it and zfs times out a write > operation to the pool/filesystem for whatever reason, possibly > just a > hold back or retyrable error, the zfs module will cause a system panic > because it thinks there are no other mirror's in the pool to write to > and forces a kernel panic. > > The way around this is to configure the zpools with mirror's which > negates the use of a hardware raid array, and sends twice the > amount of > data down to the RAID cache that is actually required (because of the > mirroring at the ZFS layer). In our case it was a little old Sun > StorEdge 3511 FC SATA Array, but the principle applies to any RAID > arraythat is not configured as a JBOD. > > > > Victor Engle wrote: > > Roshan, > > > > Could you provide more detail please. The host and zfs should be > > unaware of any EMC array side replication so this sounds more > like an > > EMC misconfiguration than a ZFS problem. Did you look in the > messages> file to see if anything happened to the devices that > were in your > > zpools? If so then that wouldn't be a zfs error. If your EMC devices > > fall offline because of something happening on the array or fabric > > then zfs is not to blame. The same thing would have happened > with any > > other filesystem built on those devices. > > > > What kind of pools were in use, raidz, mirror or simple stripe? > > > > Regards, > > Vic > > > > > > > > > > On 6/19/07, Roshan Perera <[EMAIL PROTECTED]> wrote: > >> Hi All, > >> > >> We have come across a problem at a client where ZFS brought the > system>> down with a write error on a EMC device due to mirroring > done at the > >> EMC level and not ZFS, Client is total EMC committed and not > too happy > >> to use the ZFS for oring/RAID-Z. I have seen the notes below > about the > >> ZFS and SAN attached devices and understand the ZFS behaviour. > >> > >> Can someone help me with the following Questions: > >> > >> Is this the way ZFS will work in the future ? > >> is there going to be any compromise to allow SAN Raid and ZFS > to do > >> the rest. > >> If so when and if possible details of it ? > >> > >> > >> Many Thanks > >> > >> Rgds > >> > >> Roshan > >> > >> ZFS work with SAN-attached devices? > >> > > >> > Yes, ZFS works with either direct-attached devices or SAN- > attached>> > devices. However, if your storage pool contains no > mirror or RAID-Z > >> > top-level devices, ZFS can only report checksum errors but cannot > >> > correct them. If your storage pool consists of mirror or RAID-Z > >> > devices built using storage from SAN-attached devices, ZFS > can report > >> > and correct checksum errors. > >> > > >> > This says that if we are not using ZFS raid or mirror then the > >> > expected event would be for ZFS to report but not fix the > error. In > >> > our case the system kernel panicked, which is something > different. Is > >> > the FAQ wrong or is there a bug in ZFS? > >> > >> _______________________________________________ > >> zfs-discuss mailing list > >> zfs-discuss@opensolaris.org > >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >> > >
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
