On 11 Apr 2013, at 0:29 , Peter Wood <peterwood...@gmail.com> wrote:
> On Wed, Apr 10, 2013 at 7:35 AM, Paul van der Zwan > <pa...@vanderzwan.org>wrote: > >> >> On 9 Apr 2013, at 3:13 , Peter Wood <peterwood...@gmail.com> wrote: >> >>> I've asked the ZFS discussion list for help on this but now I have more >>> information and it looks like a bug in the drivers or something. >>> >>> I have number of Dell PE R710 and PE 2950 servers running OpenSolaris, OI >>> 151a and OI 151a.7. All these systems are used as storage servers, clean >> OS >>> install, no extra services running. The systems are NFS exporting a lot >> of >>> ZFS datasets that are mounted on about ten CentOS-5.9 systems. >>> >>> The above setup has been working for 2+ years with no problem. >>> >>> Recently we bought two Supermicro systems: >>> Supermicro X9DRH-iF >>> Xeon E5-2620 @ 2.0 GHz 6-Core >>> 128GB RAM >>> LSI SAS9211-8i HBA >>> 32x 3TB Hitachi HUS723030ALS640, SAS, 7.2K >>> >>> I installed OI151.a.7 on them and started migrating data from the old >> Dell >>> servers (zfs send/receive). >>> >>> Things have been working great for about two months until I migrated one >>> particular directory to one of the new Supermicro systems and after about >>> two days the system crashed. No network connectivity, black console, no >>> response to keyboard keys, no activity lights (no error lights either) on >>> the chassis. The only way out is to hit the reset button. Nothing in the >>> logs as far as I can tell. Log entries just stop when the system crashes. >>> >>> In the following two months I did a lot of testing and a lot of trips to >>> the colo in the middle of the night and the observation is that >> regardless >>> of the OS everything works on the Dell servers. As soon as I move that >>> directory to any of the Supermicro servers with OI151.a.7 it will crash >>> them within 2 hours up to 5 days. >>> >>> The Supermicro servers can be idle, exporting nothing, or can be >> exporting >>> 15+ other directories with high IOPS and working for months with no >>> problems but as soon as I have them export that directory they'll crash >> in >>> 5 days the most. >>> >>> There is only one difference between that directory an all others >> exported >>> directories. One of the client systems that mounts it and writes to it is >>> an old Debian 5.0 system. No idea why that would crash a Supermicro >> system >>> but not a Dell system. >>> >>> We worked directly with LSI developers and upgraded the firmware to some >>> unpublished, prerelease development version to no avail. We disabled all >>> power saving features and CPU C states in the BIOS and nothing changed. >>> >>> Any idea? >> >> I had a similar kind of problem where a VirtualBox Freebsd 9.1 VM could >> hang the server. >> It had /usr/src and /usr/obj NFS mounted from the OI a7 box it was running >> on. >> The are separate NFS shared datasets in on of my 3 pools. >> >> When I ran a make buildworld in that VM it consistently locked up the OI >> host, no console access, >> no network access ( not even ping ). >> As a test I switched to NFSv4 instead of NFSv3 and I have not seen a hang >> since. >> So it looked like a heavy NFSv3 load was the issue. >> >> Paul >> >> > Make sense. I haven't tried that. > > If I'm correct ZFS on OI supports NFSv2,3 and 4. > > By switching to NFSv4 you mean that on your client machine (the FreeBSD VM) > you setup the NFS client to use NFSv4 protocol. Do I understand this > correctly? Or, did you do something on the OI server to accept only NFSv4 > connections? > > Could you please give more information. I haven't changed the server but only the mount options on on the client. It's /etc/fstab now has: 192.168.178.24:/data/ports /usr/ports nfs rw,nfsv4 - - 192.168.178.24:/data/src /usr/src nfs rw,nfsv4 - - 192.168.178.24:/data/obj /usr/obj nfs rw,nfsv4 - - A make buildworld does seem to take quite a bit longer than when I was using nfsv3 so it might just be a case of a lighter load. I have no hard data but it feels like it takes twice as long. Paul _______________________________________________ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss