Hello again,
I swapped out the PSU and replaced the cables and ran scrubs almost every day
(after hours) with no reported faults. I also upgraded to SNV_130 thanks to
Brock & changed cables and PSU after the suggestion from Richard. I owe you two
both beers!
We thought our troubles were resolved but I'm noticing alot of the messages
above from my /var/adm/messages and I'm starting to worry. I tailed the log
whilst we streamed some MPEG2 captures (it was about 12Gb) and the log went
crazy!
Jan 9 23:52:04 razor nwamd[34]: [ID 244715 daemon.error] scf_handle_destroy()
failed: repository server unavailable
Jan 9 23:52:04 razor smbd[13585]: [ID 354691 daemon.error] smb_nicmon_daemon:
failed to refresh SMF instance svc:/network/smb/server:default
Jan 9 23:52:04 razor last message repeated 11 times
Jan 9 23:52:04 razor nwamd[34]: [ID 244715 daemon.error] scf_handle_destroy()
failed: repository server unavailable
Jan 9 23:52:04 razor smbd[13585]: [ID 354691 daemon.error] smb_nicmon_daemon:
failed to refresh SMF instance svc:/network/smb/server:default
Jan 9 23:52:05 razor last message repeated 4 times
Jan 9 23:52:05 razor nwamd[34]: [ID 244715 daemon.error] scf_handle_destroy()
failed: repository server unavailable
Jan 9 23:52:05 razor smbd[13585]: [ID 354691 daemon.error] smb_nicmon_daemon:
failed to refresh SMF instance svc:/network/smb/server:default
Any ideas why this may be happenning? I'm really starting to worry, is it a ZFS
issue or SMB again?
Cheers,
Emily
> On Dec 31, 2009, at 11:38 PM, Emily Grettel wrote:
>
> > Hi Richard,
> >
> > This is my zpool status -v
> >
> > pool: tank
> > state: ONLINE
> > status: One or more devices has experienced an unrecoverable error.
> > An
> > attempt was made to correct the error. Applications are
> > unaffected.
> > action: Determine if the device needs to be replaced, and clear the
> > errors
> > using 'zpool clear' or replace the device with 'zpool
> > replace'.
> > see: http://www.sun.com/msg/ZFS-8000-9P
> > scrub: scrub completed after 5h15m with 0 errors on Fri Jan 1
> > 17:39:57 2010
> > config:
> > NAME STATE READ WRITE CKSUM
> > tank ONLINE 0 0 0
> > raidz1-0 ONLINE 0 0 0
> > c7t1d0 ONLINE 0 0 2 51.5K repaired
> > c7t4d0 ONLINE 0 0 2 52K repaired
> > c0t1d0 ONLINE 0 0 3 77.5K repaired
> > c7t5d0 ONLINE 0 0 0
> > c7t3d0 ONLINE 0 0 1 26K repaired
> > c7t2d0 ONLINE 0 0 0
> > errors: No known data errors
> >
> > I might swap the SATA cables to some better quality ones that are
> > shielded I think (ACRyan has some) and see if its that.
> >
> > Cheers,
> > Em
> >
> > > From: richard.ell...@gmail.com
> > > To: emilygrettelis...@hotmail.com
> > > Subject: Re: [zfs-discuss] ZFS extremely slow performance
> > > Date: Thu, 31 Dec 2009 19:58:24 -0800
> > >
> > > hmmm... might be something other than the disk, like cables or
> > > vibration.
> > > Let's see what happens after the scrub completes.
> > > -- richard
> > >
> > > On Dec 31, 2009, at 5:34 PM, Emily Grettel wrote:
> > >
> > > > Hello!
> > > >
> > > >
> > > > > This could be a broken disk, or it could be some other
> > > > > hardware/software/firmware issue. Check the errors on the
> > > > > device with
> > > > > iostat -En
> > > >
> > > > Heres the output:
> > > >
> > > > c7t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD10EADS-00L Revision: 1A01 Serial No:
> > > > Size: 1000.20GB <1000204886016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 4 Predictive Failure Analysis: 0
> > > > c7t2d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD10EADS-00P Revision: 0A01 Serial No:
> > > > Size: 1000.20GB <1000204886016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 4 Predictive Failure Analysis: 0
> > > > c7t3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD10EADS-00P Revision: 0A01 Serial No:
> > > > Size: 1000.20GB <1000204886016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 4 Predictive Failure Analysis: 0
> > > > c7t4d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD10EADS-00P Revision: 0A01 Serial No:
> > > > Size: 1000.20GB <1000204886016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 4 Predictive Failure Analysis: 0
> > > > c7t5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD10EADS-00P Revision: 0A01 Serial No:
> > > > Size: 1000.20GB <1000204886016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 4 Predictive Failure Analysis: 0
> > > > c7t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD740GD-00FL Revision: 8F33 Serial No:
> > > > Size: 74.36GB <74355769344 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 6 Predictive Failure Analysis: 0
> > > > c0t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD10EADS-00P Revision: 0A01 Serial No:
> > > > Size: 1000.20GB <1000204886016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 5 Predictive Failure Analysis: 0
> > > > c3t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD7500AAKS-0 Revision: 4G30 Serial No:
> > > > Size: 750.16GB <750156374016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 5 Predictive Failure Analysis: 0
> > > > c3t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> > > > Vendor: ATA Product: WDC WD7500AAKS-0 Revision: 4G30 Serial No:
> > > > Size: 750.16GB <750156374016 bytes>
> > > > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> > > > Illegal Request: 5 Predictive Failure Analysis: 0
> > > >
> > > > > You should also check the fma logs:
> > > > > fmadm faulty
> > > >
> > > > Empty
> > > >
> > > > > fmdump -eV
> > > > >
> > > >
> > > > This turned out to be huge. But they're mostly something like
> > this:
> > > >
> > > >
> > > > Nov 13 2009 10:15:41.883716494 ereport.fs.zfs.checksum
> > > > nvlist version: 0
> > > > class = ereport.fs.zfs.checksum
> > > > ena = 0x7cfde552fd100401
> > > > detector = (embedded nvlist)
> > > > nvlist version: 0
> > > > version = 0x0
> > > > scheme = zfs
> > > > pool = 0xda1d003c03abad23
> > > > vdev = 0x4389ee65271b9187
> > > > (end detector)
> > > > pool = tank
> > > > pool_guid = 0xda1d003c03abad23
> > > > pool_context = 0
> > > > pool_failmode = wait
> > > > vdev_guid = 0x4389ee65271b9187
> > > > vdev_type = replacing
> > > > parent_guid = 0x79c2f2cf0b81ae5a
> > > > parent_type = raidz
> > > > zio_err = 0
> > > > zio_offset = 0xae9b3fa00
> > > > zio_size = 0x6600
> > > > zio_objset = 0x24
> > > > zio_object = 0x1b2
> > > > zio_level = 0
> > > > zio_blkid = 0x635
> > > > __ttl = 0x1
> > > > __tod = 0x4afc971d 0x34ac718e
> > > >
> > > >
> > > > Thanks for helping and telling me about those commands :-)
> > > >
> > > > The scrub I started last night is still running, it usually takes
> > > > about 8 hours. Will post the results.
> > > >
> > > > - Em
> > > >
> > > >
> > > > > From: richard.ell...@gmail.com
> > > > > To: mi...@task.gda.pl
> > > > > Date: Thu, 31 Dec 2009 08:37:03 -0800
> > > > > CC: zfs-discuss@opensolaris.org
> > > > > Subject: Re: [zfs-discuss] ZFS extremely slow performance
> > > > >
> > > > > On Dec 31, 2009, at 2:49 AM, Robert Milkowski wrote:
> > > > >
> > > > > >
> > > > > > judging by a *very* quick glance it looks like you have an
> > issue
> > > > > > with c3t0d0 device which is responding very slowly.
> > > > >
> > > > > Yes, there is an I/O stuck on the device which is not getting
> > > > serviced.
> > > > > See below...
> > > > >
> > > > > >
> > > > > > --
> > > > > > Robert Milkowski
> > > > > > http://milek.blogspot.com
> > > > > >
> > > > > >
> > > >
> > > > Australia's #1 job site If It Exists, You'll Find it on SEEK
> > >
> >
> > Check out Domain Radar NOW! A world FIRST in property search has
> > arrived!
>
Browse profiles for FREE View photos of singles in your area!
_________________________________________________________________
If It Exists, You'll Find it on SEEK Australia's #1 job site
http://clk.atdmt.com/NMN/go/157639755/direct/01/_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
