On 22/04/2021 02:24, Job Snijders via NANOG wrote:
On Wed, Apr 21, 2021 at 09:22:57PM +0000, Jakob Heitz (jheitz) wrote:
I'd like to get some data on what actually happened in the real cases
and analyze it.
[snip]
TCP zero window is possible, but many other things could
cause it too.
Indeed. There could be a number of reasons that caused it.
Switchings away from TCP win=0 towards "Zombie Routes":
*RIGHT NOW* (at the moment of writing), there are a number of zombie
route visible in the IPv6 Default-Free Zone:
One example is
http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2a0b:6b86:d15::/48
2a0b:6b86:d15::/48 via:
BGP.as_path: 204092 57199 35280 6939 42615 42615 212232
BGP.as_path: 208627 207910 57199 35280 6939 42615 42615 212232
BGP.as_path: 208627 207910 57199 35280 6939 42615 42615 212232
(first announced April 15th, last withdrawn April 15th, 2021)
Another one is http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2a0b:6b86:d24::/48
2a0b:6b86:d24::/48 via:
BGP.as_path: 201701 9002 6939 42615 212232
BGP.as_path: 34927 9002 6939 42615 212232
BGP.as_path: 207960 34927 9002 6939 42615 212232
BGP.as_path: 44103 50673 9002 6939 42615 212232
BGP.as_path: 208627 207910 34927 9002 6939 42615 212232
BGP.as_path: 3280 34927 9002 6939 42615 212232
BGP.as_path: 206628 34927 9002 6939 42615 212232
BGP.as_path: 208627 207910 34927 9002 6939 42615 212232
(first announced March 24th, last withdrawn March 24th, 2021)
Just now, I literally rebooted the BGP speaker behind lg.ring.nlnog.net
to make ensure that those routes are not stuck in the BGP looking glass
itself.
2a0b:6b86:d24::/48 was first announced on March 24th, 2021, and
withdrawn at the end of March 24th, 2021 by the originator, and now
almost a month later, this prefix still is visible in the default-free
zone despite WITHDRAW messages having been sent and the AS 212232
operator confirming they are not announcing that IP prefix anywhere.
I checked the AS 6939 Looking glass, but the d24::/48 route is not
visible in the http://lg.he.net/ web interface. This leads me to believe
the the route got stuck somewhere along way in either of 201701, 204092,
206628, 207910, 207960, 208627, 3280, 34927, 35280, 44103, 50673, 57199,
and/or 9002.
This implies indeed might be multiple reasons a BGP route gets stuck
('stuck' as in - a WITHDRAW was not generated, or ignored). Perhaps on
any one of these edges there is a very high Out Queue for one reason or
another:
34927 9002
206628 34927
44103 50673
207960 34927
3280 34927
9002 6939
201701 9002
208627 207910
I'm not sure all the these sightings of stuck routes can be pinpointed
to one specific BGP vendor (or one bug).
I would guess that all the stuck route sightings manifest from one
undiscovered TCP library bug that some BGP vendors are all commonly using.
-Hank
Kind regards,
Job