Casandra, Thanks for your quick reply. At first everything looked right… Switch Port MAC address Node ------------------------------------------------------------------ eth-c33-1 1 00:21:9b:a2:c2:1a hypnonode001 eth-d33-1 1 00:21:9b:a2:57:b1 hypnonode029
But upon further inspection, I think I see what the problem is: eth-c33-1 Trk1 00:23:47:54:0b:00 hypnonode001 eth-c33-1 Trk1 00:06:6a:ff:bd:fa hypnonode001 eth-c33-1 Trk1 00:18:73:da:97:03 hypnonode001 eth-c33-1 Trk1 78:2b:cb:4a:49:7d hypnonode001 eth-c33-1 Trk1 00:21:9b:a2:57:b1 hypnonode001 eth-c33-1 Trk1 00:18:73:da:97:04 hypnonode001 eth-c33-1 Trk1 00:00:0c:9f:f3:59 hypnonode001 eth-c33-1 Trk1 64:a0:e7:43:d4:41 hypnonode001 eth-c33-1 Trk1 00:05:ad:09:26:1a hypnonode001 eth-c33-1 Trk1 c0:91:34:cb:30:40 hypnonode001 eth-c33-1 Trk1 08:2e:5f:ed:ec:40 hypnonode001 eth-d33-1 Trk1 00:06:6a:ff:bd:fa hypnonode029 eth-d33-1 Trk1 00:00:0c:9f:f3:59 hypnonode029 eth-d33-1 Trk1 78:2b:cb:4a:49:7d hypnonode029 eth-d33-1 Trk1 00:23:47:54:0b:00 hypnonode029 eth-d33-1 Trk1 00:21:9b:a5:74:d1 hypnonode029 eth-d33-1 Trk1 00:18:73:da:97:14 hypnonode029 eth-d33-1 Trk1 00:18:73:da:97:13 hypnonode029 eth-d33-1 Trk1 08:2e:5f:ed:ec:40 hypnonode029 eth-d33-1 Trk1 64:a0:e7:43:d4:41 hypnonode029 eth-d33-1 Trk1 00:05:ad:09:26:1a hypnonode029 eth-d33-1 Trk1 00:23:47:4d:25:80 hypnonode029 The Trk1 interfaces on each of those switches is the uplink. I looked at the MacMap.pm module and found some exclusions (e.g. for vlan interfaces and PortChannels). I’m using HP switches (they’re what I inherited with the system) and they use “trunk” to mean what “PortChannel” means in other, more sensible switches. I added an exclusion to MacMap.pm for interfaces matching Trk and “xcatprobe switch_macmap” correctly ignores those interfaces now. I also created a PR (#3518) on the gitlab repo to integrate the fix into the main repo. Thanks again, Tom -- Thomas Indelli On July 20, 2017 at 12:30:05 PM, Casandra H Qiu ([email protected]<mailto:[email protected]>) wrote: Can u check output of "xcatprobe switch_macmap eth-c33-1" and "xcatprobe switch_macmap eth-d33-1" ? switch eth-c33-1 port 1 should have node hypnonode001 and switch eth-d33-1 port 1 should have node hypnonode029 or you can add -V option to that command, it will show snmp config: # xcatprobe switch_macmap xxxxxx -V <INFO>xxxxx: Attempting to refresh switch information... <INFO>xxxxx: Generate SNMP session with parameter: 'UseNumeric' => '1' 'SecName' => 'xcatadmin' 'AuthPass' => 'xcatadminPassw0rd!' 'Version' => '3' 'PrivProto' => 'DES' 'DestHost' => 'xx.xx.xx.xx' 'SecLevel' => 'authPriv' 'AuthProto' => 'MD5' 'PrivPass' => 'xcatadminPassw0rd!' Thanks, Casandra ................................................................... Casandra Hong Qiu Phone: (845) 433-9291, t/l 293-9291 Office: Building 8, 3-B-04 [email protected] [Inactive hide details for Thomas Indelli ---07/20/2017 12:58:28 PM---Hello, I’ve been working to set up a small cluster with x] Thomas Indelli ---07/20/2017 12:58:28 PM---Hello, I’ve been working to set up a small cluster with xCAT using switch-based discovery, and every From: Thomas Indelli <[email protected]> To: xCAT Users Mailing list <[email protected]> Date: 07/20/2017 12:58 PM Subject: [xcat-user] Switch-based discovery mis-assigning nodes ________________________________ Hello, I’ve been working to set up a small cluster with xCAT using switch-based discovery, and everything was going perfectly with the first switch, but now I’m running into issues where nodes on a second switch are clobbering the previous ones. I can’t seem to track down where this is happening. Setup: Nodes hypnonode[001-028] connected to switch eth-c33-1, ports 1-28 Nodes hypnonode[029-042] connected to switch eth-d33-1, ports 1-14 Running xCAT 2.13.4 # lsxcatd -v Version 2.13.4 (git commit 6ee3741498768994e4bb10d2a77c9699bcabde90, built Tue May 16 10:03:13 EDT 2017) The nodes have the correct switch/switchport definitions: # lsdef hypnonode[001,029] -i switch,switchport Object name: hypnonode001 switch=eth-c33-1 switchport=1 Object name: hypnonode029 switch=eth-d33-1 switchport=1 SNMP is configured properly and the settings are correct in the switches table: # snmpget -v3 -a sha -u XXXXX -A XXXXX -l authNoPriv eth-c33-1 SNMPv2-MIB::sysName.0 SNMPv2-MIB::sysName.0 = STRING: eth-c33-1 # snmpget -v3 -a sha -u XXXXX -A XXXXX -l authNoPriv eth-d33-1 SNMPv2-MIB::sysName.0 SNMPv2-MIB::sysName.0 = STRING: eth-d33-1 # tabdump switches #switch,snmpversion,username,password,privacy,auth,linkports,sshusername,sshpassword,protocol,switchtype,comments,disable "eth-c33-1","3","XXXXX","XXXXX",,"sha",,,,,,, "eth-d33-1","3","XXXXX","XXXXX",,"sha",,,,,,, Also, nodes 001-028 show up properly in the output of nodediscoverls, e.g.: UUID NODE METHOD MTM SERIAL 4C4C4544-0035-3710-8037-B8C04F424B31 hypnonode023 switch DELL: 8577BK1 4C4C4544-0051-3510-8058-C7C04F4B4D31 hypnonode001 switch DELL: GQ5XKM1 […] However, when I boot up a node connected to eth-d33-1, it gets assigned to be hypnonode001. I’ve noticed that this even happens irrespective of the switch port, but rather in the order the machines are booted. I did confirm that Sequential Discovery is *not* running: # nodediscoverstatus Sequential Discovery is stopped. Profile Discovery is stopped. Relevant bits from computes.log and cluster.log: Jul 20 11:43:51 10.137.5.227 (none) rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1430" x-info="http://www.rsyslog.com";] start Jul 20 11:43:51 10.137.5.227 (none) xcat.genesis.doxcat: Getting initial certificate --> 10.137.4.8:3001 Jul 20 11:43:51 10.137.5.227 (none) dhclient[468]: XMT: Solicit on eth0, interval 7810ms. Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: Running getdestiny --> 10.137.4.8:3001 Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: Received destiny=discover Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: The destiny=discover, destiny parameters=discover Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.doxcat: Running dodiscovery... Jul 20 11:43:52 10.137.5.227 (none) xcat.genesis.dodiscovery: Beginning node discovery process... Jul 20 11:43:59 10.137.5.227 (none) dhclient[468]: XMT: Solicit on eth0, interval 16300ms. Jul 20 11:44:02 10.137.5.227 (none) xcat.genesis.dodiscovery: No link detected on eth1#012 eth2#012 eth3 after 10 seconds Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Waiting for nics to get addresses Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Network configuration complete, commencing transmit of discovery packets Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Beginning echo infomation to discovery packet file... Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Discovery packet file is ready. Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Sending the discovery packet to xCAT (10.137.4.8:3001)... Jul 20 11:44:03 10.137.5.227 (none) xcat.genesis.dodiscovery: Sleeping 5 seconds... Jul 20 11:44:03 hypnotoad-mgt01 xcat[13593]: xcatd: Processing discovery request from 10.137.5.227 Jul 20 11:44:03 hypnotoad-mgt01 xcat[13593]: xcat.discovery.aaadiscovery: (00:21:9b:a2:57:b1) Got a discovery request, attempting to discover the node... Jul 20 11:44:03 hypnotoad-mgt01 xcat[13593]: xcat.discovery.blade: (00:21:9b:a2:57:b1) Warning: Could not find any nodes using blade-based discovery Jul 20 11:44:04 hypnotoad-mgt01 xcat[13593]: xcat.discovery.switch: (00:21:9b:a2:57:b1) Found node: hypnonode001 Jul 20 11:44:06 hypnotoad-mgt01 xcat[13593]: xcat.discovery.nodediscover: hypnonode001 has been discovered Jul 20 11:44:06 hypnotoad-mgt01 xcat[13593]: xcat.discovery.zzzdiscovery: (00:21:9b:a2:57:b1) Successfully discovered the node using switch discovery method. Jul 20 11:44:08 10.137.5.227 (none) xcat.genesis.dodiscovery: Restart network interfaces... Jul 20 11:44:08 10.137.5.227 (none) dhclient[2709]: DHCPRELEASE on eth0 to 10.137.4.8 port 67 (xid=0x62d52f2b) Jul 20 11:44:13 hypnonode001 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="2784" x-info="http://www.rsyslog.com";] start Jul 20 11:44:13 hypnonode001 xcat.genesis.doxcat: Beginning doxcat process... Jul 20 11:44:13 hypnonode001 xcat.genesis.doxcat: Waiting for device with address 00:21:9b:a2:57:b1 to appear.. What I *don’t* understand, though, is that I took a look at the discopacket file, and it clearly lists the correct switch there (abbreviated output): <xcatrequest> <uuid>4C4C4544-0051-4410-805A-C7C04F4B4D31</uuid> <hwaddr>00:21:9B:A2:57:B1</hwaddr> <switchname>eth-d33-1</switchname> <switchaddr>10.137.5.248</switchaddr> <switchport>1</switchport> </xcatrequest> At this point, nodediscoverls shows two both nodes as being hypnonode001: # nodediscoverls | grep hypnonode001 4C4C4544-0051-3510-8058-C7C04F4B4D31 hypnonode001 switch DELL: GQ5XKM1 4C4C4544-0051-4410-805A-C7C04F4B4D31 hypnonode001 switch DELL:makena-2-3.localGQDZKM1 I’ve run out of ideas to check. Please let me know if you need any other information to help track down why this is happening and/or how to fix it. Thanks, Tom -- Tom Indelli [email protected]<mailto:[email protected]> Systems Administrator The University of Chicago------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
1__=8FBB0BF0DFCD38928f9e8a93df938690918c8FB@
Description: 1__=8FBB0BF0DFCD38928f9e8a93df938690918c8FB@
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
