Thanks for sharing... I now understand what you are suggesting.
I am going to point out a few things, for myself as well as others who have
been following this thread.
1) one can tag communities to routes pretty easily in the Mikrotik ROS via the
filter chains. you can tag them in in-coming filter chain as well as the
out-going filter chain as needed.
2) one can also build filter chains via a ref to another chain.. (i.e. define a
chain e.g. SD-11280 consisting of the prefix blocks, and then using this
chain=SD-11280 in your filters so that one does not need to keep relisting
prefixes).
3) There are some very important best practices you have listed and it should
be important to point out that the BGP configuration you have listed uses
update-source = loopback, that is an integral part of the design.
I will share ... I learned BGP before I learned OSFP, which is rather un-usual,
as such my comfort level is higher with BGP than OSFP. But I can also say that
what you are suggesting, I will call it a 'tip of the iceberg' ibgp setup, can
be rather challenging to understand and deploy (cause it needs one to have a
pretty decent knowledge of BGP). Once deployed, I am pretty sure it is fairly
stable and easy to maintain for the trained network admin who has a fair amount
of comfort level with BGP.
However, if the best practices related to OSPF are followed, and a little bit
of in-depth understanding beyond area 0, would also provide a fairly stable and
manageable system. I am no expert in OSFP, but I can tell it is a bit 'easier'
and less forgiving than BGP to setup, especially when you have large amount of
routers involved.
I do thank you for sharing, this type of exchange tends to be of great learning
experience for me. We are using eBGP and iBGP/Route Refelctor along with OSPF
in our network. For the moment we are using BGP for handling the full routing
tables (IP Transit, Peering and IP Transit customers), and using OSFP for our
internal 'core'.
I will be giving more thoughts to your suggestion and comparing it to the OSFP
solution with the best practices, I have a feeling that the BGP setup you are
suggesting can be fairly tedious to setup and time consuming when activating a
new router.
Regards.
Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232
Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net
> From: "Bruce Robertson" <br...@pooh.com>
> To: af@afmug.com
> Sent: Wednesday, August 31, 2016 8:36:52 PM
> Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
>> one would have to use filters to make sure the Ebgp does not get injected
>> into
> > the smaller routers.
> Yes, and the best way to do that IMO is with communities. If iBGP only
> distributes routes tagged with a community (I used <ASN:42>, 42 being a
> reference to Douglas Adams) then external routes (which are not so tagged)
> never make it into iBGP.
> How do you tag those iBGP routes? Well, I don't think I ever mentioned this
> extremely important point - you should never ever ever ever use any kind of
> "redistribute" option. Don't redistribute your connected, OSPF, static,
> anything prefixes into iBGP, or any other routing protocol. Instead, mention
> each prefix you want in iBGP individually using the "add network" statement or
> equivalent, then give it a community somehow. This was slightly cumbersome in
> Cisco, requiring two different router config sections per prefix. With a quick
> glance at the RouterOS manual, it doesn't look any more straightforward there.
> So perhaps there's any easy way in RouterOS to filter out the prefixes learned
> from eBGP peers so that they aren't distributed to iBGP.
> Oooh!!!! I just found an old Cisco config. Here's an example. In the example,
> nnnn is your AS number, x.x.x.x is a route reflector, and y.y.y.y is a
> customer
> prefix. All of this can be done on the Mikrotik as well. It looks painful and
> cumbersome, and it is, but that's why you have automated tools to do it for
> you! The first two sections are one per route reflector, the next two are one
> per router, and the last two duplicate for each customer. We created a
> route-map for each customer only so that we could add a human readable
> association to the network statement. You could just as easily use a generic
> route-map.
> router bgp nnnn
> neighbor x.x.x.x description iBGP session with one-of-your-route-reflectors
> neighbor x.x.x.x remote-as nnnn
> neighbor x.x.x.x update-source Loopback0
> !
> address-family ipv4
> neighbor x.x.x.x next-hop-self
> neighbor x.x.x.x send-community
> neighbor x.x.x.x route-map ibgp-match-community out
> !
> route-map ibgp-match-community permit 10
> match community 42 70 80 90 100 !
> ip community-list 42 permit nnnn:42
> ip community-list 70 permit nnnn:70
> ip community-list 80 permit nnnn:80
> ip community-list 90 permit nnnn:90
> ip community-list 100 permit nnnn:100
> !
> ! The remaining statements are per customer.
> !
> network y.y.y.y mask 255.255.255.252 route-map ibgp-cust-OurInternalCustomerID
> (separate route-map for documentation purposes only)
> !
> route-map ibgp-cust-OurInternalCustomerID permit 10
> set community nnnn:42
> On 08/31/2016 04:05 PM, Faisal Imtiaz wrote:
>> Thanks Bruce,
>> one would have to use filters to make sure the Ebgp does not get injected
>> into
>> the smaller routers.
>> Anyone with more examples ? Would be appreciated.
>> Thanks.
>> Faisal Imtiaz
>> Snappy Internet & Telecom
>> 7266 SW 48 Street
>> Miami, FL 33155
>> Tel: 305 663 5518 x 232
>> Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net
>>> From: "Bruce Robertson" <br...@pooh.com>
>>> To: af@afmug.com
>>> Sent: Wednesday, August 31 , 2016 1:59:23 PM
>>> Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
>>> Okay, had a moment to remember how I've done this. On Cisco, I've used
>>> communities. You tag every prefix you put into iBGP with a community, and
>>> then
>>> only distribute tagged routes within the iBGP mesh. Hopefully you won't ask
>>> me
>>> for Cusco examples... It's been 3+ years and I don't have access to those
>>> routers anymore. Barry Sherwood, if you're reading this, maybe you can
>>> contribute some sample configs.
>>> I peeked at the most recent Mikrotik network I set up (still over a year
>>> ago)
>>> and it appears that that network is not yet doing eBGP to anyone, so I
>>> didn't
>>> set up that part. But I would imagine that I would either do the same trick
>>> with communities, or would use filters on the eBGP routers. Should be pretty
>>> easy to set up, since the gateway routers are only doing iBGP with the route
>>> reflectors, so there's only one choke point. This is also an MPLS network,
>>> so
>>> the BGP configuration is really pretty trivial. Most routers aren't even
>>> running a BGP instance.
>>> On 08/30/2016 03:23 PM, Faisal Imtiaz wrote:
>>>> I have a follow up question in regards to this...
>>>> How do you prevent having ebgp routes being sent to your smaller routers
>>>> which
>>>> are doing ibgp with the Route Reflectors ?
>>>> Are you using filters ? or some there method ?
>>>> Thanks.
>>>> Faisal Imtiaz
>>>> Snappy Internet & Telecom
>>>> 7266 SW 48 Street
>>>> Miami, FL 33155
>>>> Tel: 305 663 5518 x 232
>>>> Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net
>>>>> From: "Jesse DuPont" <jesse.dup...@celeritycorp.net>
>>>>> To: af@afmug.com
>>>>> Sent: Friday, August 26 , 2016 11:36:42 PM
>>>>> Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
>>>>> For me, it was a bit of an experiment, but I have ended up liking it.
>>>>> Yes, it
>>>>> does add some overhead, but I didn't have to add routers to be the route
>>>>> reflectors - I just chose two routers which provided good geographic
>>>>> redundancy
>>>>> balanced with being as well-connected as possible to the rest of the
>>>>> routers
>>>>> and checked the "route reflect to peers" box. Route reflecting is really
>>>>> no
>>>>> more intensive than just BGP peering; probably most already know this,
>>>>> but the
>>>>> only different between a route reflector and a non-route reflector is
>>>>> that at
>>>>> route reflector is allowed to break the iBGP rule of not disseminating
>>>>> routes
>>>>> learned from one peer to another peer.
>>>>> One of the things I really like about using BGP for access prefixes is
>>>>> that I
>>>>> don't have to mess with filters or using non-backbone areas and
>>>>> area-ranges to
>>>>> summarize pools used for things like PPPoE. It's nice that more recent
>>>>> versions
>>>>> of MikroTik automate adding the U route of a summarized area-range after
>>>>> the
>>>>> first connected route shows up, but with BGP, I simply add the prefix to
>>>>> Networks and it's done.
>>>>> Another advantage, albeit a "band-aid" one is that if I'm having some link
>>>>> quality issue that is ultimately causing OSPF to lose adjacency (packet
>>>>> loss
>>>>> causing dropped Hello's, for example, or some jackass carrier providing a
>>>>> circuit that upgrades their platform and they don't read the release
>>>>> notes and
>>>>> multicast gets dropped...), I can deploy a small handful of static routes
>>>>> to
>>>>> improve stability slightly until I can resolve the issue (just a small
>>>>> time
>>>>> saver).
>>>>> Obviously, none of this functionality REQUIRES the use of BGP and it can
>>>>> all be
>>>>> done using OSPF. Indeed, while I'm using OSPF + iBGP in my WISP, the
>>>>> telco I'm
>>>>> also the network architect/engineer at uses only OSPF as the IGP and we
>>>>> have
>>>>> thousands of internal OSPF routes and dozens of routers in the backbone
>>>>> area
>>>>> (along with others in non-backbone areas) and it's extremely stable. I
>>>>> think
>>>>> its easy to misinterpret problems which manifest themselves as OSPF
>>>>> issues, but
>>>>> are really just OSPF reacting to some other condition; the canary in the
>>>>> coal
>>>>> mine, if you will.
>>>>> <rant> If you're having issues with OSPF losing adjacencies or changing
>>>>> from
>>>>> full to down or full to init, you've got some problem with the link.
>>>>> Period.
>>>>> OSPF is not the problem. OSPF has been stable in MikroTiks since
>>>>> 3.x.</rant>
>>>>> Jesse DuPont
>>>>> Network Architect
>>>>> email: jesse.dup...@celeritycorp.net
>>>>> Celerity Networks LLC
>>>>> Celerity Broadband LLC
>>>>> Like us! facebook.com / celeritynetworksllc
>>>>> Like us! facebook.com /celeritybroadband
>>>>> On 8/26/16 1:16 PM, Faisal Imtiaz wrote:
>>>>>> So just for the sake of a technical discussion...
>>>>>> In your opinion, what is the merit of such a config (osfp + ibgp) ?
>>>>>> It can be argued that such a config,
>>>>>> a) Still depends on OSPF functioning.
>>>>>> b) Layer an additional dynamic protocol on top of it (ibgp)
>>>>>> c) Requires additional Routers (route reflectors).
>>>>>> If the merit of such an approach is to manage manage OSFP behavior in a
>>>>>> more
>>>>>> granular fashion, Why not use the those features as they are available
>>>>>> in OSPF
>>>>>> / Best Practices...
>>>>>> (OSFP best practices, suggest that, don't advertise connected or static
>>>>>> routes,
>>>>>> setup all interfaces as passive, and control prefix advertisements via
>>>>>> the
>>>>>> network section of OSPF).
>>>>>> OSPF also tends to be the most common denominator (protocol) across
>>>>>> different
>>>>>> mfg. Bgp being the 2nd.
>>>>>> Regards
>>>>>> Faisal Imtiaz
>>>>>> Snappy Internet & Telecom
>>>>>> 7266 SW 48 Street
>>>>>> Miami, FL 33155
>>>>>> Tel: 305 663 5518 x 232
>>>>>> Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net
>>>>>>> From: "Jesse DuPont" <jesse.dup...@celeritycorp.net>
>>>>>>> To: af@afmug.com
>>>>>>> Sent: Friday, August 26 , 2016 12:03:58 AM
>>>>>>> Subject: Re: [AFMUG] Mikrotik OSPF weirdness
>>>>>>> Right, PTP and loopback prefixes are distributed with OSPF (and possibly
>>>>>>> management subnets for radios) and "access" network prefixes
>>>>>>> (customer-facing)
>>>>>>> are distributed via iBGP.
>>>>>>> I have two of my routers configured as BGP route reflectors and all
>>>>>>> other
>>>>>>> routers peer with only these two; this solves the full mesh and provides
>>>>>>> redundancy.
>>>>>>> Jesse DuPont
>>>>>>> Network Architect
>>>>>>> email: jesse.dup...@celeritycorp.net
>>>>>>> Celerity Networks LLC
>>>>>>> Celerity Broadband LLC
>>>>>>> Like us! facebook.com / celeritynetworksllc
>>>>>>> Like us! facebook.com /celeritybroadband
>>>>>>> On 8/25/16 8:40 PM, David Milholen wrote:
>>>>>>>> He may have meant only have the ptp and loopback addresses listed in
>>>>>>>> networks
>>>>>>>> On 8/25/2016 9:31 PM, Mike Hammett wrote:
>>>>>>>>> I've heard this concept a few times now. I'm not sure how only using
>>>>>>>>> OSPF for
>>>>>>>>> the loopbacks works.
>>>>>>>>> -----
>>>>>>>>> Mike Hammett
>>>>>>>>> Intelligent Computing Solutions
>>>>>>>>> Midwest Internet Exchange
>>>>>>>>> The Brothers WISP
>>>>>>>>> From: "Bruce Robertson" <br...@pooh.com>
>>>>>>>>> To: af@afmug.com
>>>>>>>>> Sent: Thursday, August 25 , 2016 6:28:43 PM
>>>>>>>>> Subject: Re: [AFMUG] Mikrotik OSPF weirdness
>>>>>>>>> I've said it before, and been argued with... this is one of many
>>>>>>>>> reasons why you
>>>>>>>>> use iBGP to distribute {customer, dynamic pool, server subnets,
>>>>>>>>> anything}
>>>>>>>>> routes, and use OSPF *only* to distribute router loopback
>>>>>>>>> addresses.� All
>>>>>>>>> your weird OSPF problems will go away.� My apologies if I'm
>>>>>>>>> misunderstanding
>>>>>>>>> the problem, but my point still stands.
>>>>>>>>> On 08/25/2016 10:22 AM, Robert Haas wrote:
>>>>>>>>>> Alright, this problem has raised it head again on my network since I
>>>>>>>>>> started to
>>>>>>>>>> renumber some PPPoE pools.
>>>>>>>>>> Customer gets a new IP address via PPPoE x.x.x.208/32 (from
>>>>>>>>>> x.x.x.192/27 pool).
>>>>>>>>>> Customer can�t surf and I can�t ping them from my office:
>>>>>>>>>> �
>>>>>>>>>> [office] � [Bernie Router] � [Braggcity Router] � [Ross
>>>>>>>>>> Router] � [Hayti
>>>>>>>>>> Router] � [customer]
>>>>>>>>>> �
>>>>>>>>>> A traceroute from my office dies @ the Bernie router but I am not
>>>>>>>>>> getting any
>>>>>>>>>> type of ICMP response from the Bernie router ie no ICMP Host
>>>>>>>>>> Unreachable/Dest
>>>>>>>>>> unreachable etc � just blackholes after my office router.
>>>>>>>>>> A traceroute from the Customer to the office again dies at the
>>>>>>>>>> Bernie router
>>>>>>>>>> with no type of response.
>>>>>>>>>> �
>>>>>>>>>> Checking the routing table on the Bernie router shows a valid route
>>>>>>>>>> pointing to
>>>>>>>>>> the Braggcity router. It is also in the OSPF LSA�s.
>>>>>>>>>> --
>>>>>>>>>> Another customer gets x.x.x.207/32 and has no issue at all.
>>>>>>>>>> �
>>>>>>>>>> --
>>>>>>>>>> Force the original customer to a new ip address of x.x.x.205/32 and
>>>>>>>>>> the service
>>>>>>>>>> starts working again.
>>>>>>>>>> �
>>>>>>>>>> --
>>>>>>>>>> �
>>>>>>>>>> Now � even though there is no valid route to x.x.x.208/32 in the
>>>>>>>>>> routing table
>>>>>>>>>> � traffic destined to the x.x.x.208/32 IP is still getting
>>>>>>>>>> blackholed.. I
>>>>>>>>>> should be getting a Destination host unreachable from the Bernie
>>>>>>>>>> router.
>>>>>>>>>> �
>>>>>>>>>> This is correct the correct response .206 is not being used and
>>>>>>>>>> there is no
>>>>>>>>>> route to it:
>>>>>>>>>> C:\Users\netadmin>ping x.x.x.206
>>>>>>>>>> �
>>>>>>>>>> Pinging x.x.x.206 with 32 bytes of data:
>>>>>>>>>> Reply from y.y.y.1: Destination host unreachable.
>>>>>>>>>> Reply from y.y.y.1: Destination host unreachable.
>>>>>>>>>> �
>>>>>>>>>> Ping statistics for x.x.x.206:
>>>>>>>>>> ��� Packets: Sent = 2, Received = 2, Lost = 0 (0% loss),
>>>>>>>>>> �
>>>>>>>>>> C:\Users\netadmin>tracert 74.91.65.206
>>>>>>>>>> �
>>>>>>>>>> Tracing route to host-x.x.x.206.bpsnetworks.com [x.x.x.206]
>>>>>>>>>> over a maximum of 30 hops:
>>>>>>>>>> �
>>>>>>>>>> � 1���� 6 ms���� 6 ms���� 7 ms� z.z.z.z
>>>>>>>>>> � 2���� 6 ms���� 6 ms���� 6 ms�
>>>>>>>>>> y.bpsnetworks.com
>>>>>>>>>> [y.y.y.1]
>>>>>>>>>> � 3� y.bpsnetworks.com [y.y.y.1] �reports: Destination host
>>>>>>>>>> unreachable.
>>>>>>>>>> �
>>>>>>>>>> Trace complete.
>>>>>>>>>> �
>>>>>>>>>> This is what I see to x.x.x.208 even though it is not being used and
>>>>>>>>>> there is no
>>>>>>>>>> route to it.
>>>>>>>>>> C:\Users\netadmin>ping x.x.x.208
>>>>>>>>>> �
>>>>>>>>>> Pinging x.x.x.208 with 32 bytes of data:
>>>>>>>>>> Request timed out.
>>>>>>>>>> Request timed out.
>>>>>>>>>> �
>>>>>>>>>> Ping statistics for x.x.x.208:
>>>>>>>>>> ��� Packets: Sent = 2, Received = 0, Lost = 2 (100% loss),
>>>>>>>>>> �
>>>>>>>>>> C:\Users\netadmin>tracert x.x.x.208
>>>>>>>>>> �
>>>>>>>>>> Tracing route to host-x.x.x.208.bpsnetworks.com [x.x.x.208]
>>>>>>>>>> over a maximum of 30 hops:
>>>>>>>>>> �
>>>>>>>>>> � 1���� 6 ms���� 6 ms���� 6 ms� z.z.z.z
>>>>>>>>>> � 2���� *������� *�������
>>>>>>>>>> *����
>>>>>>>>>> Request timed out.
>>>>>>>>>> � 3���� *������� *���� ^C
>>>>>>>>>> �
>>>>>>>>>> --
>>>>>>>>>> �
>>>>>>>>>> I�ve verified there is no firewall that would affect the traffic
>>>>>>>>>> � I even
>>>>>>>>>> put an accept rule in the forward chain for both the source and
>>>>>>>>>> destination of
>>>>>>>>>> x.x.x.208 and neither increment at all. So the traffic is not even
>>>>>>>>>> making out
>>>>>>>>>> of the routing flow and into the firewall..
>>>>>>>>>> �
>>>>>>>>>> Any pointers are where to start troubleshooting next?
>>>>>>>> --
>> !DSPAM:2,57c762d716631835116384!
