[j-nsp] NFV
Looking for real word experiences virtualizing router and firewall services with rates above 1Gbps on x86 platforms. Most testing I have been involved with virtualizing routers and firewalls, performance drops dramatically above 1Gbps. Connections per second are critical for a firewall in particular, can a virtual firewall handle high connections per second as appliances? Anyone experience good results at 10GigE with a virtual firewall? Where do you draw the line for router based virtualization? Mike ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 20:39, Gert Doering wrote: > Among routers (full flexibility, large tables, buffers, ...) it's a > fairly good bargain - compare this to ASR9001 or ASR9901 from the Cisco > camp. This. Mark. signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 20:37, Jason Lixfeld wrote: > Is the MX204 not a prohibitively expensive 10G port? When you consider that you can: - Drive a Metro-E ring at 100Gbps with no special Transport gear. - Hang an ASR920 or two off of those 10Gbps ports to manage low-speed customers. - Place your "serious" premium 10Gbps customers into one of them at low volume with decent margin. It makes plenty sense if you want to maintain an intelligent Access network. If your design is to have a dumb Layer 2 Access/Metro and haul everything into a central "IPGW", ah well... > If you were considering the NCS540, then you’re OK with BCM there? If I'm honest, IOS XR was the first thing that put us off. We didn't make it far enough to even worry about the chipset :-). > If so, something like the ACX5448, except far more in-line with the NCS540 > price per port. The ACX5000 was a box we looked at back in 2015 and just moved on. Nothing has changed there. > NCS540 pricing/10G port is much more realistic to what a 10G port should > cost these days vs. MX204 or ACX5448.. Agree, but with us, it's not all just feeds and speeds. A box is more than the sum of its ports. I'm happy to take a reasonably less dense box if I am making money on it, than a dense, cheaper box where I'm still making money but also staying up at 3AM every night. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Mon, Sep 23, 2019 at 02:37:08PM -0400, Jason Lixfeld wrote: > > Juniper have dropped the ball here for years. Until the MX204. However, > > the MX204 is good if you run 10Gbps customers in the Metro. Otherwise, > > for now, nothing beats the ASR920, IMHO. > > Is the MX204 not a prohibitively expensive 10G port? Among routers (full flexibility, large tables, buffers, ...) it's a fairly good bargain - compare this to ASR9001 or ASR9901 from the Cisco camp. gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> On Sep 23, 2019, at 2:17 PM, Mark Tinka wrote: > > On 23/Sep/19 14:07, Jason Lixfeld wrote: >> What are your other requirements? Who/what else are you looking at? > > We were the first ISP in the world to run IP/MPLS all the way into the > Access back in 2009 - TIME dotCom, Malaysia - on the Cisco ME3600X. > > I haven't operated that network since 2012, but it keeps growing from > what I know, now having migrated to the Cisco ASR920. > > We looked at other vendors, and the only one that came close to Cisco's > competition was Brocade with their CES/CER2000 NetIron. But Cisco's > overall price, design and feature set on the ME3600X was just too good. > > Juniper have dropped the ball here for years. Until the MX204. However, > the MX204 is good if you run 10Gbps customers in the Metro. Otherwise, > for now, nothing beats the ASR920, IMHO. Is the MX204 not a prohibitively expensive 10G port? > So in short, we want a full-blown router in the Access, but designed > cheaply enough for Metro applications. If you were considering the NCS540, then you’re OK with BCM there? If so, something like the ACX5448, except far more in-line with the NCS540 price per port. NCS540 pricing/10G port is much more realistic to what a 10G port should cost these days vs. MX204 or ACX5448. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 14:07, Jason Lixfeld wrote: > What are your other requirements? Who/what else are you looking at? We were the first ISP in the world to run IP/MPLS all the way into the Access back in 2009 - TIME dotCom, Malaysia - on the Cisco ME3600X. I haven't operated that network since 2012, but it keeps growing from what I know, now having migrated to the Cisco ASR920. We looked at other vendors, and the only one that came close to Cisco's competition was Brocade with their CES/CER2000 NetIron. But Cisco's overall price, design and feature set on the ME3600X was just too good. Juniper have dropped the ball here for years. Until the MX204. However, the MX204 is good if you run 10Gbps customers in the Metro. Otherwise, for now, nothing beats the ASR920, IMHO. So in short, we want a full-blown router in the Access, but designed cheaply enough for Metro applications. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
--- Begin Message --- Nikolas, I have been running into "committed config doesn't match operational reality" issues with JunOS since at least 16.1. I've seen this under protocol bgp, firewall filters, etc. My issues appear apply-group related. Are your affected BGP policies achieved via apply-group inheritance? Do you use " system commit delta-export " or " system commit persist-groups-inheritance" ? I was a bit skeptical but JTAC pointed me to PR1357802. Public notes were not an exact match of my symptoms by any means (I was not experiencing RPD crashes), but the JTAC engineer implied that there were some more holistic apply-group fixes as part of this release. Smoke might have been blown but we were targeting 18.X releases in our lab so it was easy to test. I haven't been able to reliability reproduce the problem in either release (16.x or 18.x) but at the same time I haven't had it recur in our 18.x lab so... -Michael > -Original Message- > From: juniper-nsp On Behalf Of > Nikolas Geyer > Sent: Sunday, September 22, 2019 6:27 PM > To: juniper-nsp@puck.nether.net > Subject: Re: [j-nsp] Suggestions for Edge/Peering Router.. > > I have to play devils advocate around “Right this inconsistency between > configured and operational state in my opinion is THE biggest problem of XR” > > Have had this problem occur multiple times in Junos, as recently as Junos 17, > where what was being advertised did not reflect configured policy. Worse > still, the only recovery was complete deletion of the policy (and any peers > using it) then re-adding it. > > So it’s definitely not a XR only problem. > > Sent from my iPhone > > On Sep 19, 2019, at 8:11 AM, "adamv0...@netconsultings.com" > wrote: > > >> From: Saku Ytti > >> Sent: Thursday, September 19, 2019 12:33 PM > >> > >>> On Thu, 19 Sep 2019 at 14:22, wrote: > >>> > >>> Just a few examples when you change export policy it resets the peer > >>> or the cockup with RR clearing all sessions or the fact BGP is part of > >>> very complex RDP monolith -to me that's not really "carrier grade" > >>> implementation > >> > >> This happens when export policy breaks update-group. It may sometimes > be > >> difficult for operator to understand if it will do that or not, so it's > >> fair > concern. > >> Perhaps system should not clear, but tell manual clear is needed for policy > >> change to take effect. > >> > > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in > Junos. > > > >> If monolith is good or bad, I'm not sure. If you thread you have high > >> performance with some risk. If you have process separation you have IPC > >> problem, and you have low performance and many will solve this by > >> duplicating state. Junos is moving towards multi process model with Junos > >> Evolved, if this will be positive or negative direction remains to be seen. > >> > > I like where XR and Junos Evolved is heading, > > In future I'd like to have the option to install only stuff I need on a > > particular > type of node/deployment and not worry about the rest all the way to being > able to mix and match protocols of different vendors. > > Although cRPD is also interesting development pathway, but again cBGP > would be even better :) > > > >> Operationally speaking, BGP in JunOS for us works great, on IOS-XR right > now > >> we have sessions where policy isn't what is configured and there is no > way to > >> verify which one, and we've propagated leaks because acting > configuration > >> isn't the one we've configured. We've not had similar problems in JunOS. > This > >> is anecdote, not data. > >> > > Right this inconsistency between configured and operational state in my > opinion is THE biggest problem of XR, I'm afraid it has to be something > fundamental since they haven't been able to consistently address these > inconsistencies across the board for years now (or ASR9k HW? Not sure if > these types of issues can be experienced on other platforms). > > Though usually it's CP state does not trickle down to DP > correctly/completely where what you described seems to be CP only. > > > > adam > > > > ___ > > juniper-nsp mailing list juniper-nsp@puck.nether.net > > https://puck.nether.net/mailman/listinfo/juniper-nsp > ___ > juniper-nsp mailing list juniper-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp --- End Message --- ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> On Sep 23, 2019, at 5:11 AM, Mark Tinka wrote: > > This is the major driving reason behind us avoiding the NCS540 > for the Metro. What are your other requirements? Who/what else are you looking at? ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 13:14, Tarko Tikan wrote: > > > What is the motivation to run jericho in your L2-only setup (instead > trident)? Only buffer space? We were chasing the actual switch, as it met all of what we needed, including the larger buffer space. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
hey, 7280R, Jericho. What is the motivation to run jericho in your L2-only setup (instead trident)? Only buffer space? -- tarko ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 10:58, Karl Gerhard wrote: > > The big companies have fancy and expensive labs and employees that spend > weeks testing new releases. However, we're a small hosting provider running a > bunch of MX480ies and other Juniper stuff. I need routers that I can upgrade > without fearing that my network will explode. Can't have that with Juniper. If you've been doing this for as long as many of us on here have, you'll note that what you're experiencing with Juniper is not unique to them. I've been running IOS since 10. I've been running Junos since 7. I've been running IOS XR since 3. I've been running FreeBSD since 4. I've been running SuSE Linux since 5. I've been running Windows since 3. There is always a risk with doing upgrades, both with basic and exotic features. That's why testing as much and for as long as you can before the activity is not to be underestimated. And even then, you'll still get caught out. This is not about to change, but will get slightly better. Saku and others will tell you that QA for Juniper got really bad between 10 and 15, and only started to get better again around 16 onward. But also, they moved away from a single-OS marketing line to forking for various platforms. This is to be expected, and don't expect Arista to avoid this as they grow. > > If I were to rebuild our network again, I'd take a very good look at > Arista/ANET. As Saku already mentioned, they're the ones that have the best > practices in developing software. They might not have all the bells and > whistles that Juniper have, but at least I might get more sleep and peace of > mind when upgrading those than I got with my Juniper gear. See my previous post about our ongoing Arista woes (and this is only in a Layer 2 application). Point is, you can't escape it, whichever vendor you go for, especially when all the new ones on the block are running merchant silicon and all the pleasure & joy that brings. For me, the biggest thing I enjoy about Junos vs. IOS XR is the upgrade process. This is the major driving reason behind us avoiding the NCS540 for the Metro. Too many devices to upgrade the XR way. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, I'd like to point out one more thing because I feel that this point hasn't been stressed enough: Upgrading Junos might be more time consuming than many people expect it to be. The reason for this is that quite often, things that previously worked in Junos will break in a new release. This affects very, very basic things too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN handoff or any MPLS at all, you might get bitten by this at some point. One example: After an upgrade we noticed that local-preference just stopped working. After investigating this, we realized that traffic for some subnets was being diverted, while for others it was still following the old path. Reason for this was that we had BGP PIC enabled. We had to downgrade to another version because BGP PIC is important to us. Another, more nasty example of very basic things failing after an upgrade is this: We had packet loss on an IBGP link due to a dirty fiber. No problem, we run LAGs everywhere, so we just disabled the link and send a technician to the datacenter to clean the fiber. However, at the moment when we disabled the dirty fiber some customers went completely offline. At this point a minor incident had turned into a major one because customers were feeling the impact and we started receiving calls from them. However, the routers did not show any errors on either end. LACP was still working fine on the three other links, BGP was still fine, lots of traffic was being forwarded. But we were dropping packets somewhere, some customers were offline and network engineers couldn't tell their boss what was wrong. Not a very pleasant situation to be in. In the end, we found out that the last upgrade introduced a new bug: If you disable an interface that is part of a LAG, Junos would still hash traffic on to that interface, so with a LAG consisting of 4 interfaces where you disabled one, a quater of the traffic would just be blackholed. Be ready to put up with shit like this if you buy Juniper. The officially recommended Juniper releases won't save you either: They too contain newly introduced bugs that break things that previously worked flawlessly. This is the main problem I have with Juniper: If you upgrade, you might spend days debugging stuff that used to work flawlessly for years. Even the most basic things like LAGs aren't safe. You might think that you found a version that works for you, but then, weeks later, you find something that got broken with the upgrade and then you need to schedule a new upgrade/downgrade. The big companies have fancy and expensive labs and employees that spend weeks testing new releases. However, we're a small hosting provider running a bunch of MX480ies and other Juniper stuff. I need routers that I can upgrade without fearing that my network will explode. Can't have that with Juniper. If I were to rebuild our network again, I'd take a very good look at Arista/ANET. As Saku already mentioned, they're the ones that have the best practices in developing software. They might not have all the bells and whistles that Juniper have, but at least I might get more sleep and peace of mind when upgrading those than I got with my Juniper gear. Regards Karl On 19.09.19 09:09, Gert Doering wrote: > Hi, > > On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote: >> MX104's are the dual brain unit of the 204. Though a 204 has 40/100G >> capabilities. If I read your original request correctly about ip >> routing. Not sure the 104/204 is grunty enough to deal with multiple >> internet tables. Thats a demanding task these days best left to the >> larger chassis. > You can't really compare MX104 to MX204. > > The MX104 is ppc based and *slow*, and should have never ever shipped. > > The MX204 is a really really nice box, with a fast intel RE and 40/100G > ports (though some - documented - restrictions on how they can be combined), > and from a RE/BGP point of view, en par with the larger MXes. > > And given the price point of the MX204, if the amount of interfaces is > sufficient, just get two of them :-) > > gert > > ___ > juniper-nsp mailing list juniper-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 09:44, Gert Doering wrote: > Ewww... thanks for the heads up. I have one of those "incoming" as > new "full table, many features, external links" L3 device, and that > one will definitely need QoS + netflow/ipfix + L3... > > Will test very thoroughly :-) I'll also let you know once we are done lab'ing Arista's solution to all of this. Of course, if you are doing IP/MPLS on this box, you will have way more considerations about this than we will. Mark. signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Mon, Sep 23, 2019 at 09:28:38AM +0200, Mark Tinka wrote: > > Which box was this? Trident or Jericho? > 7280R, Jericho. Ewww... thanks for the heads up. I have one of those "incoming" as new "full table, many features, external links" L3 device, and that one will definitely need QoS + netflow/ipfix + L3... Will test very thoroughly :-) gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 09:23, Gert Doering wrote: > > Which box was this? Trident or Jericho? 7280R, Jericho. Mark. signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Mon, Sep 23, 2019 at 09:15:48AM +0200, Mark Tinka wrote: > We hit an issue where policing did not work, despite being activated. We > then realized we had to explicitly enable "l2 qos" for our TCAM profile. > This is traffic-affecting. You then verify by bumping the hardware ACL > counters. > > Now, where this gets hairy, is when you update the TCAM profile to > enable policing, all TCAM resources are then exhausted because the > default slicing of the TCAM in EOS shares it across various protocols > and features, including ACL's, MPLS, IPv6, non-IP, PBR, pcap, e.t.c. So > in order to support policing, you have to give up some of these features Which box was this? Trident or Jericho? gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 19/Sep/19 00:52, Jason Lixfeld wrote: > FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair > of these for EVPN-MPLS and they’re slick, and from what I understand, they > have the FIB scale to be able to act as a border router. It’s a very > IOS-like CLI (but so many things about the CLI are so much more refined than > IOS) so it may be more familiar, unless you’re Cisco experience is limited to > IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including > licensing. > > It’s a BCM Jericho based pizza box, so that’s redundant powered, but not > “redundant” in so far as there are no redundant supervisor/management cards. > But, for the number of times I’ve had that kind of failure on any of my boxes > that have had redundant cards, I don’t think it’s worth the cost or the rack > space, especially if it’s just a border router where you’ve probably got a > bunch of other border router that can accommodate a crash or a reboot or > whatever. We recently migrated our Juniper Ethernet switches over to Arista (Layer 2-only aggregation). We hit an issue where policing did not work, despite being activated. We then realized we had to explicitly enable "l2 qos" for our TCAM profile. This is traffic-affecting. You then verify by bumping the hardware ACL counters. Now, where this gets hairy, is when you update the TCAM profile to enable policing, all TCAM resources are then exhausted because the default slicing of the TCAM in EOS shares it across various protocols and features, including ACL's, MPLS, IPv6, non-IP, PBR, pcap, e.t.c. So in order to support policing, you have to give up some of these features Luckily for us, this is a pure Layer 2-only device, so we don't need a bunch of these things; just ACL's to protect terminal sessions. We are now going to test this again and hope nothing else breaks. Definitely not something we were expecting, and a bit of a surprise for the Arista TAC too. Takes me back several steps with merchant silicon... ah well. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
