Although the implementation is Cisco-specific, this feels more appropriate for NANOG.
We've started rolling out a state-wide monitoring system based on Cisco's "IP SLA" feature set. Out of 5 sites deployed so far (different locations, different providers), we are consistently seeing one-way latency mirror the opposite direction. As source-destination latency goes up, destination-source latency goes down and vice versa. Myself and the monitoring team have ripped apart the OIDs, IP SLA configuration, and monitoring system. We've also built an ad-hoc system to compare the results. It's still consistent behavior. It's not a true mirror; there is definitely variation between the data collection, but at the 10,000 foot level, there is an obvious and consistent mirror to the data. The network topology is independant service providers all providing backhaul to a local ethernet exchange. Has anybody seen this type of behavior? We are solidly convinced that we are using the proper OIDs and making the proper transformations of the data. The two remaining causes appear to be either "natural behavior of the links" and/or "artifact in the IP SLA mechanism". Any ideas? Thanks!
