Problem solved. I first had tried a reload on that module and it failed diags, and went to a powered off state. Powered down the switch, reseated the module causing the problem, restarted the switch and all is well. Thanks everyone for all the suggestions.
-----Original Message----- From: Glen Johnson [mailto:gjohn...@vhcc.edu] Sent: Tuesday, July 31, 2012 9:16 AM To: NT System Admin Issues Subject: RE: Calling the cisco experts. Took a while to check but no, the vlan mismatch errors only show on 6 switches all attached to module 4 in the core. Module 4 is an 8 port fiber card. 2 ports on this module are connected to switches that don't show the errors. Switches attached to modules 1, 3, 5 don't show those errors. Total of 22 switches without errors. Here is the status from module 4s ports. 4/1 Errors ---> cisco 3560 4/2 No errors --> cisco 2960s 4/3 Errors --> cisco 3560 4/4 No errors --> 3560 4/5 Errors --> 2960s 4/6 Errors --> 2960s 4/7 Errors --> 2960s 4/8 Errors --> 2960s It also doesn't seem to be remote switch model specific. Here is the configuration from module 4 #module 4 : 8-port 1000BaseX Ethernet set trunk 4/1 nonegotiate dot1q 1-4094 set trunk 4/2 nonegotiate dot1q 1-4094 set trunk 4/3 nonegotiate dot1q 1-4094 set trunk 4/4 nonegotiate dot1q 1-4094 set trunk 4/5 nonegotiate dot1q 1-4094 set trunk 4/6 nonegotiate dot1q 1-4094 set trunk 4/7 nonegotiate dot1q 1-4094 set trunk 4/8 nonegotiate dot1q 1-4094 set port qos 4/1-8 trust trust-cos set port qos 4/1-8 vlan-based set port lacp-channel 4/7-8 mode off #End Ports 7 and 8 were originally configured as an LACP channel to one building but when the errors started the remotes switches erro-disabled their ports. I had to disable the channel as that building was down when I came in Monday morning. I am thinking that a core switch reboot may fix this strangeness and I hope to test tonight. I may also clear the config from module 4 and recreate it as it is pretty simple. -----Original Message----- From: Eldridge, D K, [mailto:d...@parkviewmc.com] Sent: Monday, July 30, 2012 2:59 PM To: NT System Admin Issues Subject: RE: Calling the cisco experts. This doesn't help now but this is where Kiwi Kat tools really shines. I have it grabbing 160 running switch configs 5 times a day 7 days a week. You could compare them side by side to see differences. Great tool and not very expensive. If you do a term monitor on the remote switches, are they all showing vlan mismatches? -----Original Message----- From: Glen Johnson [mailto:gjohn...@vhcc.edu] Sent: Monday, July 30, 2012 10:29 AM To: NT System Admin Issues Subject: Calling the cisco experts. Got a really strange one here. About 1:30pm on Saturday, July 28 our log server starts showing lots of VLAN mismatch errors. I'm looking around, but not finding any changes. The only two people that have access to the infrastructure were not working on Saturday. Our config is a Cisco 6506 running hybrid catos/ios as core and several cisco 2960 and other switches branched out from it. All the errors are coming from various closet switches. What is even stranger. If I telnet to one of the closet switches, issue a sho cdp neighbor command, I see all the devices I expect to see, ip phones and such, but I also see every port on the core switch listed as a neighbor, hence the vlan mismatches. Nothing has changed, that I can detect. Nothing in the config for the core switch looks amiss, but I'm stumped. I plan to reboot the core tonight, but hoping for some suggestions. ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin This e-mail contains the thoughts and opinions of the sender and does not represent official Parkview Medical Center policy. This communication is intended only for the recipient(s) named above, may be confidential and/or legally privileged: and, must be treated as such in accordance with state and federal laws. If you are not the intended recipient, you are hereby notified that any use of this communication, or any of its contents, is prohibited. If you have received this communication in error, please return to sender and delete the message from your computer system. Parkview Medical Center<http://www.parkviewmc.com> ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin