I have been seeing a lot more Mikrotik gear in major colo facilities. There's at least a few racks with them out of every row.
----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Paul Stewart" <p...@paulstewart.org> To: af@afmug.com Sent: Saturday, September 3, 2016 6:12:20 AM Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness Hey Faisal …. Lots of really great questions! ;) Route reflectors in BGP configuration are optional but at some point it’ll make complete sense depending on the size of the network. I wouldn’t say one is more complicated than the other …. But after stepping back from this a bit and thinking about it – OSPF is going to seem easier/simpler to set up though…. Would agree with that for sure. As you mentioned, a lot of the responses were Cisco/Juniper related vs Microtik and perhaps it is related to network size, traffic levels, budgets, services and other factors …. In the world I live in, I see almost zero Microtik’s where a lot of folks on this list are surrounded in them. I would believe that many folks on the list primary business is WISP and then there are some folks (myself included) where WISP is a small part (but important part) of the business. It’s two different ways of skinning the cat but dependent on what you want to accomplish, what network size you are working with, MPLS capabilities, if you need full Internet tables in parts of the network for downstream customers… basically, in my opinion, as your network grows and the services/requirements change then you may find moving from the “OSPF model” to the “BGP model” necessary … and you may not. J Thanks, Paul From: Af [mailto:af-boun...@afmug.com] On Behalf Of Faisal Imtiaz Sent: September 2, 2016 9:07 PM To: af@afmug.com Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness Thanks Paul, So we have established that we can do this via ibgp/route reflectors/ and communities/filters to manage and control the route distribution, and we rely on loopback ip's to be known by all routers using OSFP (or some other underlying dynamic protocol) We started this conversation by a claim that doing so via ibgp is simpler or better.... After flushing out all the details, it is obvious that a) it is a bit of an intricate process, filters and communities have to be planned. b) it requires an in-dept knowledge of BGP, or at least a fair amount of comfort level. c) it requires to follow best practices.. d) and the configuration requires at least two Route Reflectors and at least two sessions per router (one to each of the RR). and we have not gotten into traffic engineering .... (influencing the path over one link vs the other). now if we compare this to an OSPF Setup... a) planning and setting up the areas is good to have, most tend not to pay attention to it. b) dealing OSPF properly does require a bit of in depth knowledge, winging it does not (OSFP is much more forgiving ?) c) for things to work well, it is recommended to follow best practices d) configuration tends to be simpler So, would it be better for someone managing lots of routers to spend a bit of time learning OSPF intricacies and follow the best practices vs trading this for an ibgp configuration ? I would also like to point out that the folks who responded to the ibgp setup, appear to be using Juniper and or Cisco routers.. I cannot help in making the observation that folks who are deploying Juniper or Cisco or even Brocade, tend to do a lot more on their routers (thus have a fewer boxes ) , while those of us who are deploying Mikrotik Routers, tend to deploy more in quantity of these (distributing the functions, rather than trying to do everything on one or two boxes). Going back to my original question... is this just two different ways of skinning the cat, and the choice of one vs the other is simply a matter of personal choice... or is one method truly better than the other method ? if yes, can someone please share as to why ? Thanks :) Faisal Imtiaz Snappy Internet & Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net ----- Original Message ----- From: "Paul Stewart" < p...@paulstewart.org > To: af@afmug.com Sent: Friday, September 2, 2016 8:13:52 PM Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness <blockquote> Been on holidays so apologies for posting on what might be older messages… Communities is one way to do it (filter that only accepts certain communities). In the Juniper world you can limit it simply by which “family” you accept on BGP neighbor as well. Some folks also separate their “Internet routes” from their “internal routes” into separate routing tables all together Paul From: Af [ mailto:af-boun...@afmug.com ] On Behalf Of Bruce Robertson Sent: August 30, 2016 7:23 PM To: af@afmug.com Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness Communities. Lemme know if you need more detail on that. I'm a little pressed for time right now. On 08/30/2016 03:23 PM, Faisal Imtiaz wrote: <blockquote> I have a follow up question in regards to this... How do you prevent having ebgp routes being sent to your smaller routers which are doing ibgp with the Route Reflectors ? Are you using filters ? or some there method ? Thanks. Faisal Imtiaz Snappy Internet & Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net <blockquote> From: "Jesse DuPont" <jesse.dup...@celeritycorp.net> To: af@afmug.com Sent: Friday, August 26, 2016 11:36:42 PM Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness </blockquote> <blockquote> For me, it was a bit of an experiment, but I have ended up liking it. Yes, it does add some overhead, but I didn't have to add routers to be the route reflectors - I just chose two routers which provided good geographic redundancy balanced with being as well-connected as possible to the rest of the routers and checked the "route reflect to peers" box. Route reflecting is really no more intensive than just BGP peering; probably most already know this, but the only different between a route reflector and a non-route reflector is that at route reflector is allowed to break the iBGP rule of not disseminating routes learned from one peer to another peer. One of the things I really like about using BGP for access prefixes is that I don't have to mess with filters or using non-backbone areas and area-ranges to summarize pools used for things like PPPoE. It's nice that more recent versions of MikroTik automate adding the U route of a summarized area-range after the first connected route shows up, but with BGP, I simply add the prefix to Networks and it's done. Another advantage, albeit a "band-aid" one is that if I'm having some link quality issue that is ultimately causing OSPF to lose adjacency (packet loss causing dropped Hello's, for example, or some jackass carrier providing a circuit that upgrades their platform and they don't read the release notes and multicast gets dropped...), I can deploy a small handful of static routes to improve stability slightly until I can resolve the issue (just a small time saver). Obviously, none of this functionality REQUIRES the use of BGP and it can all be done using OSPF. Indeed, while I'm using OSPF + iBGP in my WISP, the telco I'm also the network architect/engineer at uses only OSPF as the IGP and we have thousands of internal OSPF routes and dozens of routers in the backbone area (along with others in non-backbone areas) and it's extremely stable. I think its easy to misinterpret problems which manifest themselves as OSPF issues, but are really just OSPF reacting to some other condition; the canary in the coal mine, if you will. <rant> If you're having issues with OSPF losing adjacencies or changing from full to down or full to init, you've got some problem with the link. Period. OSPF is not the problem. OSPF has been stable in MikroTiks since 3.x.</rant> Jesse DuPont Network Architect email: jesse.dup...@celeritycorp.net Celerity Networks LLC Celerity Broadband LLC Like us! facebook.com / celeritynetworksllc Like us! facebook.com /celeritybroadband On 8/26/16 1:16 PM, Faisal Imtiaz wrote: <blockquote> So just for the sake of a technical discussion... In your opinion, what is the merit of such a config (osfp + ibgp) ? It can be argued that such a config, a) Still depends on OSPF functioning. b) Layer an additional dynamic protocol on top of it (ibgp) c) Requires additional Routers (route reflectors). If the merit of such an approach is to manage manage OSFP behavior in a more granular fashion, Why not use the those features as they are available in OSPF / Best Practices... (OSFP best practices, suggest that, don't advertise connected or static routes, setup all interfaces as passive, and control prefix advertisements via the network section of OSPF). OSPF also tends to be the most common denominator (protocol) across different mfg. Bgp being the 2nd. Regards Faisal Imtiaz Snappy Internet & Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net <blockquote> From: "Jesse DuPont" <jesse.dup...@celeritycorp.net> To: af@afmug.com Sent: Friday, August 26, 2016 12:03:58 AM Subject: Re: [AFMUG] Mikrotik OSPF weirdness </blockquote> <blockquote> Right, PTP and loopback prefixes are distributed with OSPF (and possibly management subnets for radios) and "access" network prefixes (customer-facing) are distributed via iBGP. I have two of my routers configured as BGP route reflectors and all other routers peer with only these two; this solves the full mesh and provides redundancy. Jesse DuPont Network Architect email: jesse.dup...@celeritycorp.net Celerity Networks LLC Celerity Broadband LLC Like us! facebook.com / celeritynetworksllc Like us! facebook.com /celeritybroadband On 8/25/16 8:40 PM, David Milholen wrote: <blockquote> He may have meant only have the ptp and loopback addresses listed in networks On 8/25/2016 9:31 PM, Mike Hammett wrote: <blockquote> I've heard this concept a few times now. I'm not sure how only using OSPF for the loopbacks works. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Bruce Robertson" <br...@pooh.com> To: af@afmug.com Sent: Thursday, August 25, 2016 6:28:43 PM Subject: Re: [AFMUG] Mikrotik OSPF weirdness I've said it before, and been argued with... this is one of many reasons why you use iBGP to distribute {customer, dynamic pool, server subnets, anything} routes, and use OSPF *only* to distribute router loopback addresses.� All your weird OSPF problems will go away.� My apologies if I'm misunderstanding the problem, but my point still stands. On 08/25/2016 10:22 AM, Robert Haas wrote: <blockquote> Alright, this problem has raised it head again on my network since I started to renumber some PPPoE pools. Customer gets a new IP address via PPPoE x.x.x.208/32 (from x.x.x.192/27 pool). Customer can�t surf and I can�t ping them from my office: � [office] � [Bernie Router] � [Braggcity Router] � [Ross Router] � [Hayti Router] � [customer] � A traceroute from my office dies @ the Bernie router but I am not getting any type of ICMP response from the Bernie router ie no ICMP Host Unreachable/Dest unreachable etc � just blackholes after my office router. A traceroute from the Customer to the office again dies at the Bernie router with no type of response. � Checking the routing table on the Bernie router shows a valid route pointing to the Braggcity router. It is also in the OSPF LSA�s. -- Another customer gets x.x.x.207/32 and has no issue at all. � -- Force the original customer to a new ip address of x.x.x.205/32 and the service starts working again. � -- � Now � even though there is no valid route to x.x.x.208/32 in the routing table � traffic destined to the x.x.x.208/32 IP is still getting blackholed.. I should be getting a Destination host unreachable from the Bernie router. � This is correct the correct response .206 is not being used and there is no route to it: C:\Users\netadmin>ping x.x.x.206 � Pinging x.x.x.206 with 32 bytes of data: Reply from y.y.y.1: Destination host unreachable. Reply from y.y.y.1: Destination host unreachable. � Ping statistics for x.x.x.206: ��� Packets: Sent = 2, Received = 2, Lost = 0 (0% loss), � C:\Users\netadmin>tracert 74.91.65.206 � Tracing route to host-x.x.x.206.bpsnetworks.com [x.x.x.206] over a maximum of 30 hops: � � 1���� 6 ms���� 6 ms���� 7 ms� z.z.z.z � 2���� 6 ms���� 6 ms���� 6 ms� y.bpsnetworks.com [y.y.y.1] � 3� y.bpsnetworks.com [y.y.y.1] �reports: Destination host unreachable. � Trace complete. � This is what I see to x.x.x.208 even though it is not being used and there is no route to it. C:\Users\netadmin>ping x.x.x.208 � Pinging x.x.x.208 with 32 bytes of data: Request timed out. Request timed out. � Ping statistics for x.x.x.208: ��� Packets: Sent = 2, Received = 0, Lost = 2 (100% loss), � C:\Users\netadmin>tracert x.x.x.208 � Tracing route to host-x.x.x.208.bpsnetworks.com [x.x.x.208] over a maximum of 30 hops: � � 1���� 6 ms���� 6 ms���� 6 ms� z.z.z.z � 2���� *������� *������� *���� Request timed out. � 3���� *������� *���� ^C � -- � I�ve verified there is no firewall that would affect the traffic � I even put an accept rule in the forward chain for both the source and destination of x.x.x.208 and neither increment at all. So the traffic is not even making out of the routing flow and into the firewall.. � Any pointers are where to start troubleshooting next? </blockquote> </blockquote> -- </blockquote> </blockquote> </blockquote> </blockquote> !DSPAM:2,57c60796289379943469318! </blockquote> </blockquote>
