I am trying to setup a 3 node cluster using LDOM 1.1 guest domains and Sun
Cluster 3.2 Update 2 software on a T2000. My guest domains are ldg1, ldg2, ldg2
(each guest domain gets 4GB RAM, 8VCPU and is backed by a 16GB ZVOL virtual
disk backend created from a separate zpool, i.e., not the root pool).
In the primary domain I have allocated services as follows:
# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vcc0 primary 5000-5100
VSW
NAME LDOM MAC NET-DEV DEVICE
DEFAULT-VLAN-ID PVID VID MODE
primary-vsw0 primary 00:14:4f:2a:b4:5a e1000g0 switch at 0
1 1
primary-vsw1 primary 00:14:4f:2a:b4:5b e1000g1 switch at 1
1 1
primary-vsw2 primary 00:14:4f:fa:bd:ce switch at 2
1 1 sc
primary-vsw3 primary 00:14:4f:fb:c8:2d switch at 3
1 1 sc
VDS
NAME LDOM VOLUME OPTIONS MPGROUP
DEVICE
primary-vds0 primary vol1
/dev/zvol/dsk/lpool/ldoms/ldg1
vol2
/dev/zvol/dsk/lpool/ldoms/ldg2
vol3
/dev/zvol/dsk/lpool/ldoms/ldg3
For each of the three guest domains, I assign one vnet from each virtual switch.
for ldg1
NETWORK
NAME SERVICE DEVICE MAC
MODE PVID VID
vnet1 primary-vsw0 at primary network at 0
00:14:4f:fa:ed:30 1
vnet2 primary-vsw1 at primary network at 1
00:14:4f:f9:be:70 1
vnet3 primary-vsw2 at primary network at 2
00:14:4f:f8:80:bc 1
vnet4 primary-vsw3 at primary network at 3
00:14:4f:f8:2e:7d 1
for ldg2
NETWORK
NAME SERVICE DEVICE MAC
MODE PVID VID
vnet1 primary-vsw0 at primary network at 0
00:14:4f:f9:ba:45 1
vnet2 primary-vsw1 at primary network at 1
00:14:4f:fa:42:a9 1
vnet3 primary-vsw2 at primary network at 2
00:14:4f:fb:6c:3a 1
vnet4 primary-vsw3 at primary network at 3
00:14:4f:f8:31:bd 1
for ldg3
NETWORK
NAME SERVICE DEVICE MAC
MODE PVID VID
vnet1 primary-vsw0 at primary network at 0
00:14:4f:fa:9c:c5 1
vnet2 primary-vsw1 at primary network at 1
00:14:4f:f9:1e:7a 1
vnet3 primary-vsw2 at primary network at 2
00:14:4f:fb:34:00 1
vnet4 primary-vsw3 at primary network at 3
00:14:4f:f8:38:98 1
The intention is to use vnet1 & 2 as a IPMP group for the public network and
vnet3 and vnet4 for the cluster interconnect.
After installing the Solaris 10/08 OS, all latest recommened patches, Sun
Cluster 3.2 I go about building the cluster and map all guest doman vnet 3
interfaces to switch1 and all vnet4 interfaces to switch2 during the cluster
configuration prompts. The cluster check completes without error and the node
reboot starts. ldg1 is rebooted up to the login prompt and next is ldg2 which
never gets beyond the following error message (see below). ldg3 is never
rebooted because the cluster hangs on ldg2.
The error message from ldg2:
Boot device: disk File and args:
SunOS Release 5.10 Version Generic_138888-03 64-bit
Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hostname: ldg2
Configuring devices.
/usr/cluster/bin/scdidadm: Could not load DID instance list.
/usr/cluster/bin/scdidadm: Cannot open /etc/cluster/ccr/global/did_instances.
Booting as part of a cluster
NOTICE: CMM: Node ldg1 (nodeid = 1) with votecount = 1 added.
NOTICE: CMM: Node ldg2 (nodeid = 2) with votecount = 0 added.
NOTICE: clcomm: Adapter vnet2 constructed
NOTICE: clcomm: Adapter vnet3 constructed
NOTICE: CMM: Node ldg2: attempting to join cluster.
NOTICE: CMM: Cluster doesn't have operational quorum yet; waiting for quorum.
NOTICE: clcomm: Path ldg2:vnet2 - ldg1:vnet2 errors during initiation
NOTICE: clcomm: Path ldg2:vnet3 - ldg1:vnet3 errors during initiation
WARNING: Path ldg2:vnet2 - ldg1:vnet2 initiation encountered errors, errno =
62. Remote node may be down or unreachable through this path.
WARNING: Path ldg2:vnet3 - ldg1:vnet3 initiation encountered errors, errno =
62. Remote node may be down or unreachable through this path.
Any suggestions on how to resolve this issue ? Am I doing something incorrect
in the configuration/setup of the cluster ?
PS. I did check that the vnets are visible in each domain using ifconfig -a
plumb and cross-matched the MAC addresses back to what is assigned in the
control domain.
--
This message posted from opensolaris.org
