FWIW, I had the same problem. I installed into VMware Workstation 6.5.1 using osol-0811.iso. First I booted into the LiveCD, once it was booted I used the Install Solaris option. I never received any notice to configure the network interface during the load...or I didn't see it. In any case, I ended up with an 'Automatic' DHCP configuration and that's where the problems began. I launched the Network Administration applet and it asked if I wanted to convert to manual. I said yes and configured the interface. It never started instead saying it was disconnected. SO I tried to relaunch the configuration to activate the interface, the applet then would not launch at all. I exited and rebooted the instance and went back in. Again it came up as disconnected and I still could not launch the admin applet, eventually the system became unstable so I switched the interface back to automatic and rebooted it again. This time when it came up it said it was doing the automatic configuration, which since I have no DHCP server is meaningless so I again went into the ADmin Network applet and configured the interface. Still, the interface was disconnected so I rebooted again as the system appeared to be getting more unstable by the minute. This time on reboot things were different. The network icon was circulating green arrows. I thought success, so I launched Firefox. No joy, can't browse. So went to a terminal and did some digging. ping a host on the lan...works!, run nslookup and query dns for google.com....works!....WTF?! so edited, /etc/nsswitch. All is well.
No idea what happened, but apparently there is something in common. The term 'unstable' in this instance means, the mouse is not moving, or moving and hanging, system response is terribly slow. -- This message posted from opensolaris.org