Can u check output of "xcatprobe switch_macmap eth-c33-1" and "xcatprobe
switch_macmap eth-d33-1" ?
switch eth-c33-1 port 1 should have node hypnonode001 and switch eth-d33-1
port 1 should have node hypnonode029
or you can add -V option to that command, it will show snmp config:
# xcatprobe switch_macmap xxxxxx -V
<INFO>xxxxx: Attempting to refresh switch information...
<INFO>xxxxx: Generate SNMP session with parameter:
'UseNumeric' => '1'
'SecName' => 'xcatadmin'
'AuthPass' => 'xcatadminPassw0rd!'
'Version' => '3'
'PrivProto' => 'DES'
'DestHost' => 'xx.xx.xx.xx'
'SecLevel' => 'authPriv'
'AuthProto' => 'MD5'
'PrivPass' => 'xcatadminPassw0rd!'
Thanks,
Casandra
...................................................................
Casandra Hong Qiu
Phone: (845) 433-9291, t/l 293-9291
Office: Building 8, 3-B-04
[email protected]
From: Thomas Indelli <[email protected]>
To: xCAT Users Mailing list <[email protected]>
Date: 07/20/2017 12:58 PM
Subject: [xcat-user] Switch-based discovery mis-assigning nodes
Hello,
I’ve been working to set up a small cluster with xCAT using switch-based
discovery, and everything was going perfectly with the first switch, but
now I’m running into issues where nodes on a second switch are clobbering
the previous ones. I can’t seem to track down where this is happening.
Setup:
Nodes hypnonode[001-028] connected to switch eth-c33-1, ports 1-28
Nodes hypnonode[029-042] connected to switch eth-d33-1, ports 1-14
Running xCAT 2.13.4
# lsxcatd -v
Version 2.13.4 (git commit 6ee3741498768994e4bb10d2a77c9699bcabde90, built
Tue May 16 10:03:13 EDT 2017)
The nodes have the correct switch/switchport definitions:
# lsdef hypnonode[001,029] -i switch,switchport
Object name: hypnonode001
switch=eth-c33-1
switchport=1
Object name: hypnonode029
switch=eth-d33-1
switchport=1
SNMP is configured properly and the settings are correct in the switches
table:
# snmpget -v3 -a sha -u XXXXX -A XXXXX -l authNoPriv eth-c33-1
SNMPv2-MIB::sysName.0
SNMPv2-MIB::sysName.0 = STRING: eth-c33-1
# snmpget -v3 -a sha -u XXXXX -A XXXXX -l authNoPriv eth-d33-1
SNMPv2-MIB::sysName.0
SNMPv2-MIB::sysName.0 = STRING: eth-d33-1
# tabdump switches
#switch,snmpversion,username,password,privacy,auth,linkports,sshusername,sshpassword,protocol,switchtype,comments,disable
"eth-c33-1","3","XXXXX","XXXXX",,"sha",,,,,,,
"eth-d33-1","3","XXXXX","XXXXX",,"sha",,,,,,,
Also, nodes 001-028 show up properly in the output of nodediscoverls, e.g.:
UUID NODE METHOD
MTM SERIAL
4C4C4544-0035-3710-8037-B8C04F424B31 hypnonode023 switch
DELL: 8577BK1
4C4C4544-0051-3510-8058-C7C04F4B4D31 hypnonode001 switch
DELL: GQ5XKM1
[…]
However, when I boot up a node connected to eth-d33-1, it gets assigned to
be hypnonode001. I’ve noticed that this even happens irrespective of the
switch port, but rather in the order the machines are booted. I did
confirm that Sequential Discovery is *not* running:
# nodediscoverstatus
Sequential Discovery is stopped.
Profile Discovery is stopped.
Relevant bits from computes.log and cluster.log:
Jul 20 11:43:51 10.137.5.227 (none) rsyslogd: [origin software="rsyslogd"
swVersion="5.8.10" x-pid="1430" x-info="http://www.rsyslog.com";] start
Jul 20 11:43:51 10.137.5.227 (none) xcat.genesis.doxcat: Getting initial
certificate --> 10.137.4.8:3001
Jul 20 11:43:51 10.137.5.227 (none) dhclient[468]: XMT: Solicit on eth0,
interval 7810ms.
Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: Running
getdestiny --> 10.137.4.8:3001
Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: Received
destiny=discover
Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: The
destiny=discover, destiny parameters=discover
Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: Running
dodiscovery...
Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.dodiscovery: Beginning
node discovery process...
Jul 20 11:43:59 10.137.5.227 (none) dhclient[468]: XMT: Solicit on eth0,
interval 16300ms.
Jul 20 11:44:02 10.137.5.227 (none) xcat.genesis.dodiscovery: No link
detected on eth1#012 eth2#012 eth3 after 10 seconds
Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Waiting for
nics to get addresses
Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Network
configuration complete, commencing transmit of discovery packets
Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Beginning
echo infomation to discovery packet file...
Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Discovery
packet file is ready.
Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Sending the
discovery packet to xCAT (10.137.4.8:3001)...
Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Sleeping 5
seconds...
Jul 20 11:44:03 hypnotoad-mgt01 xcat[13593]: xcatd: Processing discovery
request from 10.137.5.227
Jul 20 11:44:03 hypnotoad-mgt01 xcat[13593]: xcat.discovery.aaadiscovery:
(00:21:9b:a2:57:b1) Got a discovery request, attempting to discover the
node...
Jul 20 11:44:03 hypnotoad-mgt01 xcat[13593]: xcat.discovery.blade:
(00:21:9b:a2:57:b1) Warning: Could not find any nodes using blade-based
discovery
Jul 20 11:44:04 hypnotoad-mgt01 xcat[13593]: xcat.discovery.switch:
(00:21:9b:a2:57:b1) Found node: hypnonode001
Jul 20 11:44:06 hypnotoad-mgt01 xcat[13593]: xcat.discovery.nodediscover:
hypnonode001 has been discovered
Jul 20 11:44:06 hypnotoad-mgt01 xcat[13593]: xcat.discovery.zzzdiscovery:
(00:21:9b:a2:57:b1) Successfully discovered the node using switch discovery
method.
Jul 20 11:44:08 10.137.5.227 (none) xcat.genesis.dodiscovery: Restart
network interfaces...
Jul 20 11:44:08 10.137.5.227 (none) dhclient[2709]: DHCPRELEASE on eth0 to
10.137.4.8 port 67 (xid=0x62d52f2b)
Jul 20 11:44:13 hypnonode001 rsyslogd: [origin software="rsyslogd"
swVersion="5.8.10" x-pid="2784" x-info="http://www.rsyslog.com";] start
Jul 20 11:44:13 hypnonode001 xcat.genesis.doxcat: Beginning doxcat
process...
Jul 20 11:44:13 hypnonode001 xcat.genesis.doxcat: Waiting for device with
address 00:21:9b:a2:57:b1 to appear..
What I *don’t* understand, though, is that I took a look at the discopacket
file, and it clearly lists the correct switch there (abbreviated output):
<xcatrequest>
<uuid>4C4C4544-0051-4410-805A-C7C04F4B4D31</uuid>
<hwaddr>00:21:9B:A2:57:B1</hwaddr>
<switchname>eth-d33-1</switchname>
<switchaddr>10.137.5.248</switchaddr>
<switchport>1</switchport>
</xcatrequest>
At this point, nodediscoverls shows two both nodes as being hypnonode001:
# nodediscoverls | grep hypnonode001
4C4C4544-0051-3510-8058-C7C04F4B4D31 hypnonode001 switch
DELL: GQ5XKM1
4C4C4544-0051-4410-805A-C7C04F4B4D31 hypnonode001 switch
DELL:makena-2-3.localGQDZKM1
I’ve run out of ideas to check. Please let me know if you need any other
information to help track down why this is happening and/or how to fix it.
Thanks,
Tom
--
Tom Indelli
[email protected]
Systems Administrator
The University of Chicago
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
