Yes, I did do that so I can filter AS-PATHS, but too many. I just activated again. For not spamming the list too much added just some of the output,, the full output is here: https://pastie.dev/CdnSr8.yaml
[29.06.2023, 8:53:34,297 PM] Jun 29 20:53:34.351 BGP: Incoming TCP connection. peer 10.11.1.15 OKed [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: Rcv incoming TCP connection UP. handle a001143a:1b7fadf4, key 0 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 Connection Collision, connection_up=0 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 Accept incoming TCP connection from peer, local IP 10.11.1.4 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 TCP Connection opened [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 sending MultiProtocol cap, afi/safi=1/1, length 4 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 sending 4-octet ASN cap, asn=56430, length 4 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 fbit is 0, for AFI/SAFI 1/1 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 sending Graceful Restart cap, rbit 0, time 120, length 6 [29.06.2023, 8:53:34,299 PM] Jun 29 20:53:34.352 BGP: 10.11.1.15 sending OPEN, My asn=56430 holdTime=90 route_refresh=1 cooperative= 1, restart 1/0 [29.06.2023, 8:53:34,300 PM] Jun 29 20:53:34.355 BGP: 10.11.1.15 rcv OPEN w/Option parameter length 20, My asn 56430, hold_time 180 [29.06.2023, 8:53:34,300 PM] Jun 29 20:53:34.355 BGP: 10.11.1.15 rcv OPEN w/Option parameter length 20 [29.06.2023, 8:53:34,300 PM] Jun 29 20:53:34.355 BGP: 10.11.1.15 rcv capability 2, len 0 [29.06.2023, 8:53:34,300 PM] Jun 29 20:53:34.355 BGP: 10.11.1.15 rcv 4-octet ASN capability 65, len 4, asn=56430, [29.06.2023, 8:53:34,300 PM] Jun 29 20:53:34.355 BGP: 10.11.1.15 rcv MP_EXT capability 1, len 4, afi/safi=1/1 [29.06.2023, 8:53:34,300 PM] Jun 29 20:53:34.355 BGP: 10.11.1.15 rcv Graceful Restart capability 64, len 2, rbit 0, time 0 [29.06.2023, 8:53:34,301 PM] Jun 29 20:53:34.357 BGP: 10.11.1.15 Peer went to ESTABLISHED state [29.06.2023, 8:53:35,495 PM] Jun 29 20:53:35.549 BGP: 10.11.1.15 received invalid AGGREGATOR attribute flag (0xd0) [29.06.2023, 8:53:35,495 PM] Jun 29 20:53:35.549 BGP: 10.11.1.15 received invalid AGGREGATOR attribute flag (0xd0) [29.06.2023, 8:53:35,495 PM] Jun 29 20:53:35.549 BGP: 10.11.1.15 sending NOTIFICATION 3/4 (Attribute Flags Error) [29.06.2023, 8:53:35,495 PM] Jun 29 20:53:35.549 BGP: 10.11.1.15 reset due to BGP notification sent [29.06.2023, 8:53:35,496 PM] Jun 29 20:53:35.549 BGP: 10.11.1.15 Closing TCP connection 0x00000002 [29.06.2023, 8:53:35,496 PM] Jun 29 20:53:35.550 BGP: 10.11.1.15 BGP connection closed [29.06.2023, 8:53:35,496 PM] Jun 29 20:53:35.550 BGP: 10.11.1.15 Peer went to IDLE state (Attribute Flags Error) [29.06.2023, 8:53:35,496 PM] Jun 29 20:53:35.550 BGP: 10.11.1.15 Peer already in IDLE state, stays in IDLE state. [29.06.2023, 8:53:35,496 PM] Jun 29 20:53:35.550 BGP: Attribute Error: BGP: 10.11.1.15 rcv UPDATE w/attr: Origin=IGP AS_PATH= AS_SEQ(2) 8708 5606 44418 44418 44418 NextHop=10.11.1.15 LOCAL_PREF=100 ATOMIC_AGGREGATE COMMUNITY=8708:100 [29.06.2023, 8:53:35,544 PM] Jun 29 20:53:35.600 BGP: 10.11.1.15 RIB_out peer reset #RIB_out 0 (safi 0) > On 29 Jun 2023, at 19:01, Jörg Kost <j...@ip-clear.de> wrote: > > Have you ever debugged the CER and looked at the BGP capability exchange > section and updates? > > (Caution: CPU spikes may occur when running many sessions) > > debug ip bgp neighbor $X > debug ip bgp general > debug ip bgp events > debug ip bgp updates > debug destination ssh/telnet (some other session window) > -> no debug all > > > On 29 Jun 2023, at 16:48, Bogdan Rotariu wrote: > >> Thanks again Jörg for your interest in this issue! >> >> Will answer both questions here, yes, thats me too, I’ve done days of >> testing as we ordered a bunch of Mikrotik’s and the Mikrotik support keeps >> quoting me from RFC’s and I almost got convinced that the CER’s are the >> problem. >> Operating at least 160 BGP sessions on the CER’s and we did not had any >> issue before (except memory from time to time). I believe that the issue is >> related just with RouterOS7, tested from at least 7.3 to 7.10. >> >> We have multiple POP’s and each pop has at least one CER2024 due to lack of >> space and costs of electricity this device is awesome! In some cases we need >> some locations we need a bunch more of 10G ports and the CCR2216 specs are >> insane. >> >> I have used Mikrotik before but not that much, did not wanted to learn their >> CLI but in the end it is not that bad. >> >> So, regarding our setups, each location has at least 1 CER2024 with 1 >> Upstream and some peerings (IX, or other local ISP’s) and each location is >> connected to at least location via OSPF and iBGP (all CER’s support MPLS but >> we did not use it). We share all the upstream prefixes between the locations. >> >> In one location where we have 2 CER2024 we want to replace 1 CER2024 with a >> CCR2216 and this would be a simple task, but during the testing I’ve found >> out that it is not that simple due to this issue. >> >> Fort testing I have moved 1 peering and 1 internal link from another >> location to the Mikrotik, did the eBGP with the peering, did OSPF and iBGP >> with our location, till now everything seems to be OK. Sending all the >> prefixes received from upstream by location on the CER2024 to Mikrotik >> works, but when sending the prefixes from that only peering to the CER2024, >> the session closes with "Error: Invalid AGGREGATOR attribute length 8”. >> >> Testing some more so I added the CCR2216 in the middle of two CER’s instead >> of directly connected to the peers: >> >> CER2024 (full table) -> CCR2216 (full table) -> CER2024 - the session closes >> with Error: Invalid AGGREGATOR attribute length 8 >> >> I’ve been packet sniffing for some days and see whats going on and I am >> unable to find out really whats the problem, so creating a setup in GNS3 >> where I tried to simulate some routers that aggregate prefixes and announce >> them, unfortunately I was unable to replicate the issue in GNS3, most likely >> is that I cannot feed the full global table (or I do not know how) >> >> Moving along I have created some FRR/Quagga machines and send to them the >> prefixes from CCR2216, all good, no errors, sending the prefixes that >> quagga/frr received from CCR2216 to one CER2024, everything works. I have >> tried with or without as4 capability, there is no difference. >> >> Doing some more tests, I have created test iBGP sessions and sent prefixes >> to a Huawei Router, to a Cisco 7600 and a ASR 1001, all of them handled the >> prefixes received from the Mikrotik router. >> The only equipment except the CER’s that we tested that had issues with the >> BGP session is a stack of Dell 4032F switches that (I think) do not support >> as4 and their error was: >> >> <187> Jun 18 23:58:11 core-stack-2 BGP[BGP Protocol]: bgpattr.c(1628) 955741 >> %% ERR [VRF ""] Received UPDATE from peer x invalid AGGREGATOR attribute. >> Aggregator AS is 65052. Aggregator ID is 0.0.0.0. Resetting peer. >> <187> Jun 19 00:00:16 core-stack-2 BGP[BGP Protocol]: bgpattr.c(1628) 955842 >> %% ERR [VRF ""] Received UPDATE from peer x invalid AGGREGATOR attribute. >> Aggregator AS is 25773. Aggregator ID is 0.0.0.0. Resetting peer. >> >> Did not test Mikrotik CHR in a VM, will do that test today but I think the >> outcome is the same. >> >> As for testing needs, a upstream or a full table bgp peer to the >> RouterOS7(CHR VM/HW one) and a Netiron iBGP peer. >> _______________________________________________ foundry-nsp mailing list foundry-nsp@puck.nether.net http://puck.nether.net/mailman/listinfo/foundry-nsp
