Re: [AusNOG] Assistance with Cisco vPC configuration on 4 x Cisco Nexus 3000 switches

[Ahad Aboss]

Spot on, Jason. Since each SW pair is treated as a single switch
(logically) you can run a single port channel between them.

Your suggested setup is also ideal for data center interconnects and
multi-layer vPC environment.


Ahad

On Mon, Nov 26, 2018 at 2:21 PM Jason Leschnik  wrote:

> Oh and obviously the normal  😁
>
> feature lacp ; feature vpc ; feature interface-vlan
>
> On Mon, 26 Nov 2018 at 14:19, Jason Leschnik  wrote:
>
>> Hi all,
>>
>> With vPC on both pairs, all 4 links between the pairs can be aggregated
>> into a single port-channel like the one pictured below. Practically you're
>> going to setup two different vPC domains (vpc domain 1, vpc domain 2 for
>> example) on the two pairs (11,12) and (13,14). With a port-channel
>> configuration like the following example:
>>
>> Assuming e1/1-2 are the downstream links (between pairs) and e1/10-11 are
>> the links between vPC peer switches
>>
>> *# Setup vPC*
>> sw11# vpc domain 11 ; peer-keepalive destination  ; int
>> e1/10-11 ; channel-group 1 mode active ; int po1 ; vpc peer-link ;
>> peer-switch ; peer-gateway
>> sw12# vcp domain 11 ; peer-keepalive destination  ; int
>> e1/10-11 ; channel-group 1 mode active ; int po1 ; vpc peer-link ;
>> peer-switch ; peer-gateway
>>
>> sw13# vpc domain 13 ; peer-keepalive destination  ; int
>> e1/10-11 ; channel-group 1 mode active ; int po1 ; vpc peer-link ;
>> peer-switch ; peer-gateway
>> sw14# vpc domain 13 ; peer-keepalive destination  ; int
>> e1/10-11 ; channel-group 1 mode active ; int po1 ; vpc peer-link ;
>> peer-switch ; peer-gateway
>>
>> ! Peer-switch will allow both vPC peers to propagate the same STP BPDU
>> information by sharing a vMAC
>> ! Peer-gateway allows the partner to route on behalf of the other, this
>> gets around some non-RFC behavior seen in certain NAS vendors products when
>> using FHRP
>> ! It's perfectly fine to use mgmt0 interfaces for the peer-keepalive,
>> it's a stream of UDP packets to src/dst port 3200. This link can go down
>> with no ill effect on vPC
>> ! Note conventions for numbering of port-channels and domains is just for
>> example, use whatever numbering system works for you
>>
>> *# Setup the port-channels*
>> sw11# int e1/1-2 ; channel-group 1314 mode active ; int po1314 ; vpc
>> sw12# int e1/1-2 ; channel-group 1314 mode active ; int po1314 ; vpc
>>
>> sw13# int e1/1-2 ; channel-group 1112 mode active ; int po1112 ; vpc
>> sw14# int e1/1-2 ; channel-group 1112 mode active ; int po1112 ; vpc
>>
>> [image: vpc.png]
>>
>> Regards,
>> Jason.
>>
>> On Mon, 26 Nov 2018 at 11:10, Ahad Aboss 
>> wrote:
>>
>>> Radeck,
>>>
>>> To avoid blocked ports, you will need to configure your upstream links
>>> as follows;
>>>
>>> Built a port-channel from SW13 to SW11/12.
>>> Build a port-channel from SW14 to SW11/12.
>>>
>>> Physically, SW 11 and 12 looks as 2 x Switches but logically, SW 13 and
>>> 14 treats SW11/12 as a single switch.
>>>
>>> I suggest using at least 2 x 10GE ports between SW13 & 14 as SFP's do
>>> fail from time to time.
>>> You can use MGM for peer link keep alive or a dedicated link.
>>>
>>> You can increase your port-channel capacity as your traffic grows.
>>>
>>> Ahad
>>>
>>> On Mon, Nov 26, 2018 at 10:16 AM Radek Tkaczyk 
>>> wrote:
>>>
>>>> Hi Guys,
>>>>
>>>>
>>>>
>>>> So from the feedback so far is that we should also link SW-13 and SW-14
>>>> directly – updated as below.
>>>>
>>>>
>>>>
>>>> The primary purpose for this design is to ensure redundancy across
>>>> switches, and to be able to provide approx. 50 x 10Gbps ports, and 50 x 1
>>>> Gbps ports – this also leaves heaps of room for growth as well.
>>>>
>>>>
>>>>
>>>> Still looking for someone to go over the config with, so if you are
>>>> interested (paid gig) please let me know.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> Radek
>>>>
>>>>
>>>>
>>>> *From:* AusNOG [mailto:ausnog-boun...@lists.ausnog.net] *On Behalf Of 
>>>> *Radek
>>>> Tkaczyk
>>>> *Sent:* Sunday, 25 November 2018 4:34 PM
>>>

Re: [AusNOG] Assistance with Cisco vPC configuration on 4 x Cisco Nexus 3000 switches

[Ahad Aboss]

Radeck,

To avoid blocked ports, you will need to configure your upstream links as
follows;

Built a port-channel from SW13 to SW11/12.
Build a port-channel from SW14 to SW11/12.

Physically, SW 11 and 12 looks as 2 x Switches but logically, SW 13 and 14
treats SW11/12 as a single switch.

I suggest using at least 2 x 10GE ports between SW13 & 14 as SFP's do fail
from time to time.
You can use MGM for peer link keep alive or a dedicated link.

You can increase your port-channel capacity as your traffic grows.

Ahad

On Mon, Nov 26, 2018 at 10:16 AM Radek Tkaczyk  wrote:

> Hi Guys,
>
>
>
> So from the feedback so far is that we should also link SW-13 and SW-14
> directly – updated as below.
>
>
>
> The primary purpose for this design is to ensure redundancy across
> switches, and to be able to provide approx. 50 x 10Gbps ports, and 50 x 1
> Gbps ports – this also leaves heaps of room for growth as well.
>
>
>
> Still looking for someone to go over the config with, so if you are
> interested (paid gig) please let me know.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Radek
>
>
>
> *From:* AusNOG [mailto:ausnog-boun...@lists.ausnog.net] *On Behalf Of *Radek
> Tkaczyk
> *Sent:* Sunday, 25 November 2018 4:34 PM
> *To:* Jacob Taylor 
> *Cc:*  
> *Subject:* Re: [AusNOG] Assistance with Cisco vPC configuration on 4 x
> Cisco Nexus 3000 switches
>
>
>
> Hi Jake,
>
>
>
> That’s something that I wanted to check if it was needed/recommended.
>
>
>
> Can certainly put it in if it will help achieve better performance and
> redundancy
>
> Thanks
>
>
>
> Radek
>
>
> On 25 Nov 2018, at 4:26 pm, Jacob Taylor  wrote:
>
> Hi Radek,
>
>
>
> Not personally familiar with vPC, more so Arista MLAG and Juniper MC-AE.
>
>
>
> In the diagram there isn’t a peer link between 13 & 14 - is that a mistake
> in the diagram or the actual design?
>
>
>
> If you intend to build 2x20G bonds to two standalone nexus switches,
> that’ll work fine.
>
>
>
> If you are trying to achieve a 40G bowtie between the two pairs I’m fairly
> certain that won’t work (unless Cisco has some special black magic to
> transport signalling/MAC synchronisation over the bond itself).
>
>
>
> Cheers,
>
> Jake
>
>
>
>
>
>
> On 25 Nov 2018, at 15:04, Radek Tkaczyk  wrote:
>
> Hi Guys,
>
>
>
> I have a need to configure vPC on 4 x Cisco Nexus 3000 switches at one of
> our data centres – a design that we will replicate to other data centres as
> well.
>
>
>
> I think I have the config down pat, but I’d like someone with another pair
> of eyes to go over it with me to ensure it’s 100% correct.
>
>
>
> Is there anyone who can give me a hand here with this configuration, happy
> to pay for someone’s time to go over it to ensure we are doing this
> correctly.
>
>
>
> The Physical setup that I’m looking for:
>
>
>
>
>
> 
>
>
>
> Thanks
>
>
>
> Radek
>
> ___
> AusNOG mailing list
> AusNOG@lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
> ___
> AusNOG mailing list
> AusNOG@lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
___
AusNOG mailing list
AusNOG@lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog

Re: [AusNOG] Equipment upgrade path

[Ahad Aboss]

Hi Paul,

Hope you are well.

Based on the information you’ve shared below are some suggestions that does
not take high availability into account.

Given that you are running a small ISP, my suggestion is to continue using
the ASR1K for Telstra & AAPT ethernet services until you get close to
maxing out the backplane and separate the L2TP or IPOE sessions into a
separate PE. You can always upgrade the PE to ASR1004/1006/1013 as your
customer base grows.

The ASR1K enables you to shape Telstra / AAPT customer circuits at the
headend or per vlan sub-interfaces. It also comes with a lot of features
that a Nexus 9K in L3 setup cannot support/perform as well as the ASR1K.

I am assuming your aggregate traffic handled by ASR1001 for Telstra / AAPT
is less than 4-5Gbps though most ISPs oversubscribe these services 4:1 at
the aggregation point/headend.

Given the above, a key starting point is to separate your residential (DSL)
and corporate (ethernet / fibre) customers into at least 2 pairs of PE
routers – this is a good practice from the HA and operations point of view.

The Nexus 9K is a nice ToR & leaf switch but in a server facing environment
where its often used as a L3 gateway. It supports QoS, BGP and even NAT
with limitations 😊

I’ll be more than happy to answer any specific questions in terms of the
design or implementation for these services as I’ve deployed it in
small-scale and large-scale ISP environments.

Have a great weekend.

Ahad

On Mon, Nov 19, 2018 at 10:47 AM paul hollanton 
wrote:

> Good morning list,
>
> I hope you all have had a good weekend.
>
> I’m returning to the ISP industry after a longer than expected stint in
> the corporate space and was hoping to get some pointers on some
> infrastructure upgrade options which I’m having to consider.
>
>
>
> I work for a small-ish ISP that offers some (but not a lot) DSL/NBN
> services and a bunch of  TLS such as Telstra’s Ethernet Access and AAPT
> e-lan etc. with the odd mpls layer3 vpn too.
>
>
>
> We’ve been using Cisco ASR1001 routers for L2TP (DSL/NBN) termination as
> well as sub-interfaces for the TLS services with the headend trunks from
> the suppliers terminated on a switch that’s providing a layer2 only
> function.
>
>
>
> Rather than upgrading and continuing to terminate all TLS services on the
> ASR, I thinking of purchasing a layer 3 switch such as the Cisco Nexus
> 9236C or similar and terminating the TLS services on this as well as the
> supplier trunks – the 100Gb port functionality should allow us to have the
> device(s) in operation for some time before needing to upgrade.
>
>
>
> The documentation on the units state that they support mpls and BGP which
> is nice, but if anything too heavy is required for customers with special
> requirements , perhaps we’d leave that to the ASR – which will also
> continue to perform any L2TP and NAT requirements.  To be honest, none of
> the documentation on the Cisco layer 3 switches suggest they are suited to
> what I have in mind, which brings me to my main question...
>
>
> Is whether the introduction of a layer3 switch for this function is a good
> idea, or should we continue to use ASR’s for the job?  My other concern
> is will the Nexus be able (or is suitable) to do the traffic shaping that
> is required for the Telstra Ethernet Access services (which is important
> that it’s done exactly right) and other QoS functions such as voice
> prioritisation.
>
>
>
> If there’s a better design or more suitable equipment I should consider,
> please let me know.  I’d prefer to stay with Cisco as the vendor, primarily
> as the migration path will (should) be simpler and I have reasonably good
> experience with them over the years.
>
>
>
> Thanks,
>
> Paul
>
>
> 
>  Virus-free.
> www.avg.com
> 
> <#m_1899469324380628143_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> ___
> AusNOG mailing list
> AusNOG@lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
___
AusNOG mailing list
AusNOG@lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog

Re: [AusNOG] Contention, congestion, and link capacity planning

[Ahad Aboss]

Hi Paul,

When it comes to backhaul capacity planning for business grade customers,
there is no one size fit all formula. It all comes down to the type of
customers you have and the frequency of internet usage during business hrs
and after hrs.

Generally, the peak usage for business customer is between 8am – 6pm, MON -
FRI. You’ll need to have enough backhaul capacity at the head-end to cater
for occasional burst during peak hrs though not all customers download at
full speed all at once.This is just for safety measures.

For Ethernet or midband Ethernet services through Optus, TPG/AAPT or
Telstra, you could get away with 3:1 contention on the backhaul but I
strongly recommend that you don’t risk this contention ratio if you have
less than 100 customers per state.

Let’s say you have 100 customers, a combination of 50 x 10Mbps and 50 x
20Mbps, you can safely use 1Gbps backhaul per state through a single
provider (AAPT/Telstra or Optus) and as you add more customers, you can
closely watch the average usage across the customer base and increase the
bandwidth as required.

Just to be clear, the 3:1 contention is for best effort Ethernet services
ONLY which is mostly used by SMEs, if you are providing guaranteed
bandwidth 1:1, you will have to honour the contention all the way to your
POP and internet.

If you are building up your customer base slowly, be prepared for very slim
or no margins at all as you still need to pay for the access links to
customers, trunk port (head end or backhaul), IP transit, rack space, power
and cross connect fees.

In these circumstances, it’s best to resell these services through a
reliable ISP until your Ethernet customer base is sizeable to justify the
head end built.

For residential grade broadband factor in 50% traffic growth every year, a
blessing that all ISPs have to deal with while trying to maintain their
profit margin. :)

Based on industry contacts, the current average usage per SIO (IN AUS) for
NBN and DSL are as follows;

NBN: 1.3Mbps

xDSL: 850Kbps

Netflix and HD/UHD video streaming is changing the peak average rapidly.

I hope this information helps.

Cheers,

Ahad

On Mon, Sep 18, 2017 at 12:41 PM, paul+aus...@oxygennetworks.com.au <
paul+aus...@oxygennetworks.com.au> wrote:

> Hi All, I was hoping to gain some thoughts from the list around contention
> and backhaul link capacity planning.
>
>
>
> We are working on some new site plans and have plenty of existing sites to
> draw usage statistics from when it comes to capacity planning, typically
> all of our backhaul links are running pretty low contention as all of our
> customers are business customers, but I am wondering if anybody has any
> formulas they have used successfully in the past.
>
>
>
> Being that we only provide business Ethernet connections planning is
> usually pretty straight forward, but in modelling some expansion plans I
> want to try and actually wrap something around the planning process for
> backhaul capacity.
>
>
>
> For example, 1 x 50M customer will clearly need 50M of backhaul from the
> POP they connect to, but what about 2, or 4, or 10 ?
>
> You could easily surmise that 2 x 50M customers don’t need 100M of
> backhaul unless they are very heavy users, so let’s say they may need 75M,
> but this requirement for backhaul is realistically a sliding scale as the
> customers and bandwidth requirements grow the backhaul is not necessarily
> going to need to grow at the same rate.
>
>
>
> I have worked this stuff out for some time now manually and had good
> results, our customers are happy, but I was hoping there would be some sort
> of calculation or formula that I could apply to some modelling figures
> which would give me a pretty close indication of requirements.
>
>
>
> Thanks in advance
>
>
>
> Regards
>
> Paul
>
> ___
> AusNOG mailing list
> AusNOG@lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
___
AusNOG mailing list
AusNOG@lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog

Re: [AusNOG] Routing issues with Google

[Ahad Aboss]

Thanks Phillip and everyone who responded off list. The issue was resolved.

Cheers
Ahad

On Thu, Aug 3, 2017 at 6:55 PM, Phillip Grasso 
wrote:

> n...@google.com bosshat
>
> On Aug 2, 2017 4:16 PM, "Ahad Aboss"  wrote:
>
>> Hi,
>>
>> Can someone from Google networks contact me off list regarding a strange
>> routing issue?
>>
>> Thanks
>> Ahad
>>
>> ___
>> AusNOG mailing list
>> AusNOG@lists.ausnog.net
>> http://lists.ausnog.net/mailman/listinfo/ausnog
>>
>>
___
AusNOG mailing list
AusNOG@lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog

[AusNOG] Routing issues with Google

[Ahad Aboss]

Hi,

Can someone from Google networks contact me off list regarding a strange
routing issue?

Thanks
Ahad
___
AusNOG mailing list
AusNOG@lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog