Been staring at an issue in Spectrum for about 3 days now and think I
finally have it sorted out.
About 2 weeks ago the Network team at my site started to configure V2
Routers and Switches to V3 and enabling Traps to Spectrum. I suddenly
got about 90 Orange "SNMP NOT RESPONDING" alarms and bunches of
Pingables in my "New Devices" container. (I have Trap Based Discovery
enabled)
So I got the V3 credentials from the network team, configured a V3
Profile and started to modify device-by-device. (you have to destroy and
remodel each one in Spectrum, as its the only method for converting)
I soon experienced an inability to discover some devices with no
apparent pattern. The error from "Model By IP" was "device is ICMP
pingable but SNMP access failed. (the canned message)
Shortly after modeling the dozen or so devices I COULD model under V3 I
started getting the "Duplicate SNMPv3 Engine ID" alarms by the hundreds.
I called support and there is a post H06 patch (H0621) that I was given.
Didn't fix the issue.
By now I had a pattern of what devices were failing, and it appeared to
track pairs of devices. (Distro Rtr 1A models fine, Distro Rtr 1B does
not) I logged into my TACACS account on the devices and issued a:
show snmp engineID
Bingo the ID's were identical. Looks like the Network Team copied the
V3 configs from one device to another and didn't make the engineID's unique.
Spectrum looks at the engineID when Modeling By IP and compares to its
existing ones, if its a duplicate, it silently fails. (an intuitive
message would be nice) The nice thing is Spectrum is respecting the
RFC's and Solarwinds (in use and working fine on ALL duplicate engineID
V3 devices by Networks) is not.
MIBTOOLS also fails in much the silent unhelpful way which is
unfortunate. Because if you do NOT have a TACACS login, you'd have no
idea how to tell the engineID's are duplicates unless your network guys
are nicer than mine.
Oh and the pingables in New Devices? Tracked those down to redundant
IP's on the devices that were NOT modeled with success. I believe that's
what H0621 deals with... V3 traps coming from the device confuse
Spectrum and it can't conclusively determine the source device. I'm also
puzzled since there are MULTIPLE Pingable devices from the SAME chassis
in "New Devices" so either Spectrum confused the source addr on
successive traps from the device or the device sent traps from different
interfaces/ip's. I think the former is true in this case.
Of course then Spectrum upon receipt of the V3 trap tried to discover
the device (I think V3 trap based discovery is supported in 9.2.1) and
couldn't/wouldn't since there was a duplicate engineID in the DB already.
Hope this helps somebody out there, cause I was spinning my wheels for
more than a few hours.
Now to get the Network Team to update all the duplicate engineIDs...
easier said than done...
<BIG SIGH>
-Rob
---
To unsubscribe from spectrum, send email to [email protected] with the body:
unsubscribe spectrum [email protected]
