Re: [j-nsp] Suggestions for Edge/Peering Router..
Hello Kamal thanks for sharing your valuable insight. Very interesting to see what happens when one tries to upgrade hundreds of Juniper devices. Upon reading your mail one could get the impression that the upgrade woes and all the bugs+regressions are a structural problem related to how Junos is being developed. Two more things: *The two bugs I mentioned in my last mail affected Junos 17.4 and Junos 18.1. I wish QA was bad just for Junos 14/15, but no, the problems seem to persist. *We've solved the problem you described Kamal by never upgrading any of our access switches (EX4300 and EX4200) at all. We install a tried and tested firmware before deploying them to prod and then we're done, hopefully forever. After lots of experimentation, we use our EX4200 and EX4300 only for switching, me0 is the only interface with an IP address. However, I do understand that this only feasible if you run a very uniform environment as we do. Regards Karl On 01.10.19 00:40, Kamal Dissanayaka wrote: > Hi, > > > Up grade issues ! > > This is very valid point, we are totally fed up with, > > Upgrade is a horrible > > We got round 2 and half years to upgrade around 200 Ex4200 devices. > Initially every one in three devices failed to boot after the upgrade. > Juniper keep on preaching us go to the site and format install. > They don’t seems understand how impractical to upgrade large number of > devices on console. > > Finally long chase up they agreed they have issue and provided working long > process which took lots of time to script. Still more than 5% failed > causing incidents, > > Worse thing is after 6 months all these hard works. We found some nasty > bugs, now they recommend new version. > > > Two months back i started up grading around 120 numbers of ACx2100 devices. > Process is very complicated. Firstly we found Re filters are broken in > interim Version loosing remote access. Some has to go to site and remove RE > filters. > That is fixed by removing RE filters before the upgrade then we found ssh > folders are deleted during the request system software add process this > cause access issues when upgrade fails due to validation errors. There are > were many incompatible configs between versions. > It took two solid months and many sleepless nights to get smooth running > process, JTAC response to these issues are not very helpfull some times. > there response is always workarounds not to the root course. > > On MX upgrades we had very horrible stories too. Large number of DPC cards > didnt boot up after the upgrades, They are like price of a luxury car when > we bought them. I cant imagine why they are so delicate. Our upgrades put > on hold few times due large number of failures. We still have one MX960 > with old version, no one likes to touch because it had many DPC cards on it. > > I agree that Junos is very mature OS but these issues causing the network > hard to maintain. > > Thank you > > Kamal > > On Monday, September 23, 2019, Karl Gerhard wrote: > >> Hi, >> >> I'd like to point out one more thing because I feel that this point hasn't >> been stressed enough: >> Upgrading Junos might be more time consuming than many people expect it to >> be. >> >> The reason for this is that quite often, things that previously worked in >> Junos will break in a new release. This affects very, very basic things >> too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN >> handoff or any MPLS at all, you might get bitten by this at some point. >> >> One example: >> After an upgrade we noticed that local-preference just stopped working. >> After investigating this, we realized that traffic for some subnets was >> being diverted, while for others it was still following the old path. >> Reason for this was that we had BGP PIC enabled. We had to downgrade to >> another version because BGP PIC is important to us. >> >> Another, more nasty example of very basic things failing after an upgrade >> is this: >> We had packet loss on an IBGP link due to a dirty fiber. No problem, we >> run LAGs everywhere, so we just disabled the link and send a technician to >> the datacenter to clean the fiber. However, at the moment when we disabled >> the dirty fiber some customers went completely offline. At this point a >> minor incident had turned into a major one because customers were feeling >> the impact and we started receiving calls from them. However, the routers >> did not show any errors on either end. LACP was still working fine on the >> three other links, BGP was still fine, lots of traffic was being forwarded. >> But we were dropping packets somewhere, some customers were offline and >> network engineers couldn't tell their boss what was wrong. Not a very >> pleasant situation to be in. >> In the end, we found out that the last upgrade introduced a new bug: If >> you disable an interface that is part of a LAG, Junos would still hash >> traffic on to that interface, so with a LAG consisting of 4
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, Up grade issues ! This is very valid point, we are totally fed up with, Upgrade is a horrible We got round 2 and half years to upgrade around 200 Ex4200 devices. Initially every one in three devices failed to boot after the upgrade. Juniper keep on preaching us go to the site and format install. They don’t seems understand how impractical to upgrade large number of devices on console. Finally long chase up they agreed they have issue and provided working long process which took lots of time to script. Still more than 5% failed causing incidents, Worse thing is after 6 months all these hard works. We found some nasty bugs, now they recommend new version. Two months back i started up grading around 120 numbers of ACx2100 devices. Process is very complicated. Firstly we found Re filters are broken in interim Version loosing remote access. Some has to go to site and remove RE filters. That is fixed by removing RE filters before the upgrade then we found ssh folders are deleted during the request system software add process this cause access issues when upgrade fails due to validation errors. There are were many incompatible configs between versions. It took two solid months and many sleepless nights to get smooth running process, JTAC response to these issues are not very helpfull some times. there response is always workarounds not to the root course. On MX upgrades we had very horrible stories too. Large number of DPC cards didnt boot up after the upgrades, They are like price of a luxury car when we bought them. I cant imagine why they are so delicate. Our upgrades put on hold few times due large number of failures. We still have one MX960 with old version, no one likes to touch because it had many DPC cards on it. I agree that Junos is very mature OS but these issues causing the network hard to maintain. Thank you Kamal On Monday, September 23, 2019, Karl Gerhard wrote: > Hi, > > I'd like to point out one more thing because I feel that this point hasn't > been stressed enough: > Upgrading Junos might be more time consuming than many people expect it to > be. > > The reason for this is that quite often, things that previously worked in > Junos will break in a new release. This affects very, very basic things > too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN > handoff or any MPLS at all, you might get bitten by this at some point. > > One example: > After an upgrade we noticed that local-preference just stopped working. > After investigating this, we realized that traffic for some subnets was > being diverted, while for others it was still following the old path. > Reason for this was that we had BGP PIC enabled. We had to downgrade to > another version because BGP PIC is important to us. > > Another, more nasty example of very basic things failing after an upgrade > is this: > We had packet loss on an IBGP link due to a dirty fiber. No problem, we > run LAGs everywhere, so we just disabled the link and send a technician to > the datacenter to clean the fiber. However, at the moment when we disabled > the dirty fiber some customers went completely offline. At this point a > minor incident had turned into a major one because customers were feeling > the impact and we started receiving calls from them. However, the routers > did not show any errors on either end. LACP was still working fine on the > three other links, BGP was still fine, lots of traffic was being forwarded. > But we were dropping packets somewhere, some customers were offline and > network engineers couldn't tell their boss what was wrong. Not a very > pleasant situation to be in. > In the end, we found out that the last upgrade introduced a new bug: If > you disable an interface that is part of a LAG, Junos would still hash > traffic on to that interface, so with a LAG consisting of 4 interfaces > where you disabled one, a quater of the traffic would just be blackholed. > > Be ready to put up with shit like this if you buy Juniper. The officially > recommended Juniper releases won't save you either: They too contain newly > introduced bugs that break things that previously worked flawlessly. This > is the main problem I have with Juniper: If you upgrade, you might spend > days debugging stuff that used to work flawlessly for years. Even the most > basic things like LAGs aren't safe. You might think that you found a > version that works for you, but then, weeks later, you find something that > got broken with the upgrade and then you need to schedule a new > upgrade/downgrade. > > The big companies have fancy and expensive labs and employees that spend > weeks testing new releases. However, we're a small hosting provider running > a bunch of MX480ies and other Juniper stuff. I need routers that I can > upgrade without fearing that my network will explode. Can't have that with > Juniper. > > If I were to rebuild our network again, I'd take a very good look at > Arista/ANET. As Saku already
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 20:39, Gert Doering wrote: > Among routers (full flexibility, large tables, buffers, ...) it's a > fairly good bargain - compare this to ASR9001 or ASR9901 from the Cisco > camp. This. Mark. signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 20:37, Jason Lixfeld wrote: > Is the MX204 not a prohibitively expensive 10G port? When you consider that you can: - Drive a Metro-E ring at 100Gbps with no special Transport gear. - Hang an ASR920 or two off of those 10Gbps ports to manage low-speed customers. - Place your "serious" premium 10Gbps customers into one of them at low volume with decent margin. It makes plenty sense if you want to maintain an intelligent Access network. If your design is to have a dumb Layer 2 Access/Metro and haul everything into a central "IPGW", ah well... > If you were considering the NCS540, then you’re OK with BCM there? If I'm honest, IOS XR was the first thing that put us off. We didn't make it far enough to even worry about the chipset :-). > If so, something like the ACX5448, except far more in-line with the NCS540 > price per port. The ACX5000 was a box we looked at back in 2015 and just moved on. Nothing has changed there. > NCS540 pricing/10G port is much more realistic to what a 10G port should > cost these days vs. MX204 or ACX5448.. Agree, but with us, it's not all just feeds and speeds. A box is more than the sum of its ports. I'm happy to take a reasonably less dense box if I am making money on it, than a dense, cheaper box where I'm still making money but also staying up at 3AM every night. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Mon, Sep 23, 2019 at 02:37:08PM -0400, Jason Lixfeld wrote: > > Juniper have dropped the ball here for years. Until the MX204. However, > > the MX204 is good if you run 10Gbps customers in the Metro. Otherwise, > > for now, nothing beats the ASR920, IMHO. > > Is the MX204 not a prohibitively expensive 10G port? Among routers (full flexibility, large tables, buffers, ...) it's a fairly good bargain - compare this to ASR9001 or ASR9901 from the Cisco camp. gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> On Sep 23, 2019, at 2:17 PM, Mark Tinka wrote: > > On 23/Sep/19 14:07, Jason Lixfeld wrote: >> What are your other requirements? Who/what else are you looking at? > > We were the first ISP in the world to run IP/MPLS all the way into the > Access back in 2009 - TIME dotCom, Malaysia - on the Cisco ME3600X. > > I haven't operated that network since 2012, but it keeps growing from > what I know, now having migrated to the Cisco ASR920. > > We looked at other vendors, and the only one that came close to Cisco's > competition was Brocade with their CES/CER2000 NetIron. But Cisco's > overall price, design and feature set on the ME3600X was just too good. > > Juniper have dropped the ball here for years. Until the MX204. However, > the MX204 is good if you run 10Gbps customers in the Metro. Otherwise, > for now, nothing beats the ASR920, IMHO. Is the MX204 not a prohibitively expensive 10G port? > So in short, we want a full-blown router in the Access, but designed > cheaply enough for Metro applications. If you were considering the NCS540, then you’re OK with BCM there? If so, something like the ACX5448, except far more in-line with the NCS540 price per port. NCS540 pricing/10G port is much more realistic to what a 10G port should cost these days vs. MX204 or ACX5448. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 14:07, Jason Lixfeld wrote: > What are your other requirements? Who/what else are you looking at? We were the first ISP in the world to run IP/MPLS all the way into the Access back in 2009 - TIME dotCom, Malaysia - on the Cisco ME3600X. I haven't operated that network since 2012, but it keeps growing from what I know, now having migrated to the Cisco ASR920. We looked at other vendors, and the only one that came close to Cisco's competition was Brocade with their CES/CER2000 NetIron. But Cisco's overall price, design and feature set on the ME3600X was just too good. Juniper have dropped the ball here for years. Until the MX204. However, the MX204 is good if you run 10Gbps customers in the Metro. Otherwise, for now, nothing beats the ASR920, IMHO. So in short, we want a full-blown router in the Access, but designed cheaply enough for Metro applications. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
--- Begin Message --- Nikolas, I have been running into "committed config doesn't match operational reality" issues with JunOS since at least 16.1. I've seen this under protocol bgp, firewall filters, etc. My issues appear apply-group related. Are your affected BGP policies achieved via apply-group inheritance? Do you use " system commit delta-export " or " system commit persist-groups-inheritance" ? I was a bit skeptical but JTAC pointed me to PR1357802. Public notes were not an exact match of my symptoms by any means (I was not experiencing RPD crashes), but the JTAC engineer implied that there were some more holistic apply-group fixes as part of this release. Smoke might have been blown but we were targeting 18.X releases in our lab so it was easy to test. I haven't been able to reliability reproduce the problem in either release (16.x or 18.x) but at the same time I haven't had it recur in our 18.x lab so... -Michael > -Original Message- > From: juniper-nsp On Behalf Of > Nikolas Geyer > Sent: Sunday, September 22, 2019 6:27 PM > To: juniper-nsp@puck.nether.net > Subject: Re: [j-nsp] Suggestions for Edge/Peering Router.. > > I have to play devils advocate around “Right this inconsistency between > configured and operational state in my opinion is THE biggest problem of XR” > > Have had this problem occur multiple times in Junos, as recently as Junos 17, > where what was being advertised did not reflect configured policy. Worse > still, the only recovery was complete deletion of the policy (and any peers > using it) then re-adding it. > > So it’s definitely not a XR only problem. > > Sent from my iPhone > > On Sep 19, 2019, at 8:11 AM, "adamv0...@netconsultings.com" > wrote: > > >> From: Saku Ytti > >> Sent: Thursday, September 19, 2019 12:33 PM > >> > >>> On Thu, 19 Sep 2019 at 14:22, wrote: > >>> > >>> Just a few examples when you change export policy it resets the peer > >>> or the cockup with RR clearing all sessions or the fact BGP is part of > >>> very complex RDP monolith -to me that's not really "carrier grade" > >>> implementation > >> > >> This happens when export policy breaks update-group. It may sometimes > be > >> difficult for operator to understand if it will do that or not, so it's > >> fair > concern. > >> Perhaps system should not clear, but tell manual clear is needed for policy > >> change to take effect. > >> > > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in > Junos. > > > >> If monolith is good or bad, I'm not sure. If you thread you have high > >> performance with some risk. If you have process separation you have IPC > >> problem, and you have low performance and many will solve this by > >> duplicating state. Junos is moving towards multi process model with Junos > >> Evolved, if this will be positive or negative direction remains to be seen. > >> > > I like where XR and Junos Evolved is heading, > > In future I'd like to have the option to install only stuff I need on a > > particular > type of node/deployment and not worry about the rest all the way to being > able to mix and match protocols of different vendors. > > Although cRPD is also interesting development pathway, but again cBGP > would be even better :) > > > >> Operationally speaking, BGP in JunOS for us works great, on IOS-XR right > now > >> we have sessions where policy isn't what is configured and there is no > way to > >> verify which one, and we've propagated leaks because acting > configuration > >> isn't the one we've configured. We've not had similar problems in JunOS. > This > >> is anecdote, not data. > >> > > Right this inconsistency between configured and operational state in my > opinion is THE biggest problem of XR, I'm afraid it has to be something > fundamental since they haven't been able to consistently address these > inconsistencies across the board for years now (or ASR9k HW? Not sure if > these types of issues can be experienced on other platforms). > > Though usually it's CP state does not trickle down to DP > correctly/completely where what you described seems to be CP only. > > > > adam > > > > ___ > > juniper-nsp mailing list juniper-nsp@puck.nether.net > > https://puck.nether.net/mailman/listinfo/juniper-nsp > ___ > juniper-nsp mailing list juniper-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp --- End Message --- ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> On Sep 23, 2019, at 5:11 AM, Mark Tinka wrote: > > This is the major driving reason behind us avoiding the NCS540 > for the Metro. What are your other requirements? Who/what else are you looking at? ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 13:14, Tarko Tikan wrote: > > > What is the motivation to run jericho in your L2-only setup (instead > trident)? Only buffer space? We were chasing the actual switch, as it met all of what we needed, including the larger buffer space. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
hey, 7280R, Jericho. What is the motivation to run jericho in your L2-only setup (instead trident)? Only buffer space? -- tarko ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 10:58, Karl Gerhard wrote: > > The big companies have fancy and expensive labs and employees that spend > weeks testing new releases. However, we're a small hosting provider running a > bunch of MX480ies and other Juniper stuff. I need routers that I can upgrade > without fearing that my network will explode. Can't have that with Juniper. If you've been doing this for as long as many of us on here have, you'll note that what you're experiencing with Juniper is not unique to them. I've been running IOS since 10. I've been running Junos since 7. I've been running IOS XR since 3. I've been running FreeBSD since 4. I've been running SuSE Linux since 5. I've been running Windows since 3. There is always a risk with doing upgrades, both with basic and exotic features. That's why testing as much and for as long as you can before the activity is not to be underestimated. And even then, you'll still get caught out. This is not about to change, but will get slightly better. Saku and others will tell you that QA for Juniper got really bad between 10 and 15, and only started to get better again around 16 onward. But also, they moved away from a single-OS marketing line to forking for various platforms. This is to be expected, and don't expect Arista to avoid this as they grow. > > If I were to rebuild our network again, I'd take a very good look at > Arista/ANET. As Saku already mentioned, they're the ones that have the best > practices in developing software. They might not have all the bells and > whistles that Juniper have, but at least I might get more sleep and peace of > mind when upgrading those than I got with my Juniper gear. See my previous post about our ongoing Arista woes (and this is only in a Layer 2 application). Point is, you can't escape it, whichever vendor you go for, especially when all the new ones on the block are running merchant silicon and all the pleasure & joy that brings. For me, the biggest thing I enjoy about Junos vs. IOS XR is the upgrade process. This is the major driving reason behind us avoiding the NCS540 for the Metro. Too many devices to upgrade the XR way. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, I'd like to point out one more thing because I feel that this point hasn't been stressed enough: Upgrading Junos might be more time consuming than many people expect it to be. The reason for this is that quite often, things that previously worked in Junos will break in a new release. This affects very, very basic things too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN handoff or any MPLS at all, you might get bitten by this at some point. One example: After an upgrade we noticed that local-preference just stopped working. After investigating this, we realized that traffic for some subnets was being diverted, while for others it was still following the old path. Reason for this was that we had BGP PIC enabled. We had to downgrade to another version because BGP PIC is important to us. Another, more nasty example of very basic things failing after an upgrade is this: We had packet loss on an IBGP link due to a dirty fiber. No problem, we run LAGs everywhere, so we just disabled the link and send a technician to the datacenter to clean the fiber. However, at the moment when we disabled the dirty fiber some customers went completely offline. At this point a minor incident had turned into a major one because customers were feeling the impact and we started receiving calls from them. However, the routers did not show any errors on either end. LACP was still working fine on the three other links, BGP was still fine, lots of traffic was being forwarded. But we were dropping packets somewhere, some customers were offline and network engineers couldn't tell their boss what was wrong. Not a very pleasant situation to be in. In the end, we found out that the last upgrade introduced a new bug: If you disable an interface that is part of a LAG, Junos would still hash traffic on to that interface, so with a LAG consisting of 4 interfaces where you disabled one, a quater of the traffic would just be blackholed. Be ready to put up with shit like this if you buy Juniper. The officially recommended Juniper releases won't save you either: They too contain newly introduced bugs that break things that previously worked flawlessly. This is the main problem I have with Juniper: If you upgrade, you might spend days debugging stuff that used to work flawlessly for years. Even the most basic things like LAGs aren't safe. You might think that you found a version that works for you, but then, weeks later, you find something that got broken with the upgrade and then you need to schedule a new upgrade/downgrade. The big companies have fancy and expensive labs and employees that spend weeks testing new releases. However, we're a small hosting provider running a bunch of MX480ies and other Juniper stuff. I need routers that I can upgrade without fearing that my network will explode. Can't have that with Juniper. If I were to rebuild our network again, I'd take a very good look at Arista/ANET. As Saku already mentioned, they're the ones that have the best practices in developing software. They might not have all the bells and whistles that Juniper have, but at least I might get more sleep and peace of mind when upgrading those than I got with my Juniper gear. Regards Karl On 19.09.19 09:09, Gert Doering wrote: > Hi, > > On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote: >> MX104's are the dual brain unit of the 204. Though a 204 has 40/100G >> capabilities. If I read your original request correctly about ip >> routing. Not sure the 104/204 is grunty enough to deal with multiple >> internet tables. Thats a demanding task these days best left to the >> larger chassis. > You can't really compare MX104 to MX204. > > The MX104 is ppc based and *slow*, and should have never ever shipped. > > The MX204 is a really really nice box, with a fast intel RE and 40/100G > ports (though some - documented - restrictions on how they can be combined), > and from a RE/BGP point of view, en par with the larger MXes. > > And given the price point of the MX204, if the amount of interfaces is > sufficient, just get two of them :-) > > gert > > ___ > juniper-nsp mailing list juniper-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 09:44, Gert Doering wrote: > Ewww... thanks for the heads up. I have one of those "incoming" as > new "full table, many features, external links" L3 device, and that > one will definitely need QoS + netflow/ipfix + L3... > > Will test very thoroughly :-) I'll also let you know once we are done lab'ing Arista's solution to all of this. Of course, if you are doing IP/MPLS on this box, you will have way more considerations about this than we will. Mark. signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Mon, Sep 23, 2019 at 09:28:38AM +0200, Mark Tinka wrote: > > Which box was this? Trident or Jericho? > 7280R, Jericho. Ewww... thanks for the heads up. I have one of those "incoming" as new "full table, many features, external links" L3 device, and that one will definitely need QoS + netflow/ipfix + L3... Will test very thoroughly :-) gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 23/Sep/19 09:23, Gert Doering wrote: > > Which box was this? Trident or Jericho? 7280R, Jericho. Mark. signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Mon, Sep 23, 2019 at 09:15:48AM +0200, Mark Tinka wrote: > We hit an issue where policing did not work, despite being activated. We > then realized we had to explicitly enable "l2 qos" for our TCAM profile. > This is traffic-affecting. You then verify by bumping the hardware ACL > counters. > > Now, where this gets hairy, is when you update the TCAM profile to > enable policing, all TCAM resources are then exhausted because the > default slicing of the TCAM in EOS shares it across various protocols > and features, including ACL's, MPLS, IPv6, non-IP, PBR, pcap, e.t.c. So > in order to support policing, you have to give up some of these features Which box was this? Trident or Jericho? gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 19/Sep/19 00:52, Jason Lixfeld wrote: > FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair > of these for EVPN-MPLS and they’re slick, and from what I understand, they > have the FIB scale to be able to act as a border router. It’s a very > IOS-like CLI (but so many things about the CLI are so much more refined than > IOS) so it may be more familiar, unless you’re Cisco experience is limited to > IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including > licensing. > > It’s a BCM Jericho based pizza box, so that’s redundant powered, but not > “redundant” in so far as there are no redundant supervisor/management cards. > But, for the number of times I’ve had that kind of failure on any of my boxes > that have had redundant cards, I don’t think it’s worth the cost or the rack > space, especially if it’s just a border router where you’ve probably got a > bunch of other border router that can accommodate a crash or a reboot or > whatever. We recently migrated our Juniper Ethernet switches over to Arista (Layer 2-only aggregation). We hit an issue where policing did not work, despite being activated. We then realized we had to explicitly enable "l2 qos" for our TCAM profile. This is traffic-affecting. You then verify by bumping the hardware ACL counters. Now, where this gets hairy, is when you update the TCAM profile to enable policing, all TCAM resources are then exhausted because the default slicing of the TCAM in EOS shares it across various protocols and features, including ACL's, MPLS, IPv6, non-IP, PBR, pcap, e.t.c. So in order to support policing, you have to give up some of these features Luckily for us, this is a pure Layer 2-only device, so we don't need a bunch of these things; just ACL's to protect terminal sessions. We are now going to test this again and hope nothing else breaks. Definitely not something we were expecting, and a bit of a surprise for the Arista TAC too. Takes me back several steps with merchant silicon... ah well. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
I have to play devils advocate around “Right this inconsistency between configured and operational state in my opinion is THE biggest problem of XR” Have had this problem occur multiple times in Junos, as recently as Junos 17, where what was being advertised did not reflect configured policy. Worse still, the only recovery was complete deletion of the policy (and any peers using it) then re-adding it. So it’s definitely not a XR only problem. Sent from my iPhone On Sep 19, 2019, at 8:11 AM, "adamv0...@netconsultings.com" wrote: >> From: Saku Ytti >> Sent: Thursday, September 19, 2019 12:33 PM >> >>> On Thu, 19 Sep 2019 at 14:22, wrote: >>> >>> Just a few examples when you change export policy it resets the peer >>> or the cockup with RR clearing all sessions or the fact BGP is part of >>> very complex RDP monolith -to me that's not really "carrier grade" >>> implementation >> >> This happens when export policy breaks update-group. It may sometimes be >> difficult for operator to understand if it will do that or not, so it's fair >> concern. >> Perhaps system should not clear, but tell manual clear is needed for policy >> change to take effect. >> > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in > Junos. > >> If monolith is good or bad, I'm not sure. If you thread you have high >> performance with some risk. If you have process separation you have IPC >> problem, and you have low performance and many will solve this by >> duplicating state. Junos is moving towards multi process model with Junos >> Evolved, if this will be positive or negative direction remains to be seen. >> > I like where XR and Junos Evolved is heading, > In future I'd like to have the option to install only stuff I need on a > particular type of node/deployment and not worry about the rest all the way > to being able to mix and match protocols of different vendors. > Although cRPD is also interesting development pathway, but again cBGP would > be even better :) > >> Operationally speaking, BGP in JunOS for us works great, on IOS-XR right now >> we have sessions where policy isn't what is configured and there is no way to >> verify which one, and we've propagated leaks because acting configuration >> isn't the one we've configured. We've not had similar problems in JunOS. This >> is anecdote, not data. >> > Right this inconsistency between configured and operational state in my > opinion is THE biggest problem of XR, I'm afraid it has to be something > fundamental since they haven't been able to consistently address these > inconsistencies across the board for years now (or ASR9k HW? Not sure if > these types of issues can be experienced on other platforms). > Though usually it's CP state does not trickle down to DP correctly/completely > where what you described seems to be CP only. > > adam > > ___ > juniper-nsp mailing list juniper-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On 9/19/19, 1:05 AM, "juniper-nsp on behalf of Phil Reilly" wrote: MX104's are the dual brain unit of the 204. Though a 204 has 40/100G capabilities. If I read your original request correctly about ip routing. Not sure the 104/204 is grunty enough to deal with multiple internet tables. Thats a demanding task these days best left to the larger chassis. The MX204 should work fine with multiple full table peers. The MX104 is the dual-RE version of the MX80 PPC-powered router. The MX204 is essentially an MPC7E, has a 64-bit Junos that runs the OS in a VM and has 16GB of RAM. MX204 is pretty powerful for what it is, which is a 1U router with 100G/40G/10G/1G capabilities, but you cannot use all ports simultaneously if you enable 100G or 40G. It's a bit convoluted, but it's explained here: https://www.juniper.net/documentation/en_US/junos/topics/concept/port-speed-capability-mx204router.html They offer a port-checker tool to verify configurations here: https://apps.juniper.net/home/port-checker/index.html -evt ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Thu, Sep 19, 2019 at 06:42:58PM +0300, Saku Ytti: > Quite, so I guess to be eligible for update group junos has two > constraints and ios has one, junos constraints are same egress-policy > and same peer-group, ios constraints same egress-policy. I can > understand thne rationale to both, junos rationale makes sense in a > context where you want to force update-group separation when you know > the two sets are not guaranteed to have the same policy every time. And, why wouldn't you want that control? ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On Thu, 19 Sep 2019 at 18:36, wrote: > In cisco these should be under common update group This is a fair point. I've not tried to create unique peer-groups with non-unique policies so this has escaped me. > What is dynamic is that the peer 192.0.2.41 will be kicked out of group 11 if > configured with different export policy > And reset in that process unfortunately Quite, so I guess to be eligible for update group junos has two constraints and ios has one, junos constraints are same egress-policy and same peer-group, ios constraints same egress-policy. I can understand thne rationale to both, junos rationale makes sense in a context where you want to force update-group separation when you know the two sets are not guaranteed to have the same policy every time. Of course the ideal solution would be to never reset and use as few update groups as theoretically possible. But for me as an operator, both behaviour are good enough and don't really bite operationally, as I can design configuration around it. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> From: Saku Ytti > Sent: Thursday, September 19, 2019 2:42 PM > > On Thu, 19 Sep 2019 at 16:29, wrote: > > > The dynamic in Cisco implementation means that peers are automatically > placed to update groups based on commonalities in export policies, > regardless of the configuration. > > In juniper case you can actually have two peers with exact same export > policies be part of different update groups just because they have been > configured that way -which is inefficient, but who cares throw more CPU at it > and you're done -real operational problem is the meaningless peer reset. > > I'm not sure I've seen this, but I'm ready to accept it to be true, but seems > like something that could get PR and get fixed. Do you have a particular > example where neighbours which will export the same set of routes and > have same behaviour get separate update-groups? > +group test-group1 { +type external; +family inet { +unicast; +} +peer-as 65000; +neighbor 192.0.2.10; +} +group test-group2 { +type external; +family inet { +unicast; +} +peer-as 65000; +neighbor 192.0.2.20; +} +group test-group3 { +type internal; +family inet { +unicast; +} +neighbor 192.0.2.30; +} +group test-group4 { +type internal; +family inet { +unicast; +} +neighbor 192.0.2.40 +neighbor 192.0.2.41; +} Group Type: External Local AS: 31655 Name: test-group1 Index: 8 Flags: <> Holdtime: 0 Total peers: 1Established: 0 192.0.2.10 Group Type: External Local AS: 31655 Name: test-group2 Index: 9 Flags: <> Holdtime: 0 Total peers: 1Established: 0 192.0.2.20 Group Type: InternalAS: 31655 Local AS: 31655 Name: test-group3 Index: 10 Flags: <> Holdtime: 0 Total peers: 1Established: 0 192.0.2.30 Group Type: InternalAS: 31655 Local AS: 31655 Name: test-group4 Index: 11 Flags: <> Holdtime: 0 Total peers: 2Established: 0 192.0.2.40 192.0.2.41 In cisco these should be under common update group What is dynamic is that the peer 192.0.2.41 will be kicked out of group 11 if configured with different export policy And reset in that process unfortunately set protocols bgp group test-group4 neighbor 192.0.2.41 export INTERNET_EX Group Type: InternalAS: 31655 Local AS: 31655 Name: test-group4 Index: 12 Flags: <> Export: [ INTERNET_EX ] Holdtime: 0 Total peers: 1Established: 0 192.0.2.41 > > True but the argument can go both ways -in case of common memory > space there's no protection so one needs to worry about leaking into > different parts of memory. > > Yes, the whole house of cards effect, which is very relevant in multitenant > time-shared systems like Linux server, just because sshd crashes doesn't > mean httpd should suffer. But I'm not sure if it's so important in router. > > > Hmm, but we wouldn’t second guess this multi-process approach for > desktop/mobile operating systems right? > > They are much less controlled system, you install nillywilly random > applications to your phone, and it's not good if phone crashes just because > some background app I already forgot crashed. And servers have multiple > tenants. I think in most cases routers have very different profile, vendor > knows exactly what is running there and it's exactly one use-case. > Note I'm not saying multiprocess is worse than monolithic, I'm just saying, > it's > not obvious to me multiprocess is the superior in this use-case. Certainly C > has exuberated the problem as it makes it too easy to write memory unsafe > code, but considering how long lived NOS are, I think every NOS should be > written in its own DSL which compiles to C++ or Rust. Then you have more > choices where given problem is best solved. > > Like forwarding code, should we write it in C or P4 which compiles to > whatever language the NPU can consume? Less capable developers will be > able to write good code in P4 and can communicate to the smaller set of > developers who write the P4 compiler about missing features. And if the P4 > compiler developer notices some nasty hacks the P4 coders need to write to > solve some problem they can move the solution to the compiler and have > the P4 coders write something less hacky. > I think DSL as an abstraction is justified in any complex long lived project. > Your point on all or nothing and single use-case nature of NOSes is certainly interesting. I'll
Re: [j-nsp] Suggestions for Edge/Peering Router..
> > > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups > in Junos. > > They are dynamic, but once you make export change which affects subset > of members in peer-group, that member gets reset while placed to new > update-group. And that is how dynamic update groups works by design. They automatically group peers based on their export policy configuration. To reset or not reset the peers really depends on the nature of the policy change. In some cases clear soft out or in will do just fine without bringing down the session. Of course if you enable soft reconfig in and policy is just a new import filter no reset of the peer is required if OS still does it they can fix it easily. R. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On Thu, 19 Sep 2019 at 16:29, wrote: > The dynamic in Cisco implementation means that peers are automatically placed > to update groups based on commonalities in export policies, regardless of the > configuration. > In juniper case you can actually have two peers with exact same export > policies be part of different update groups just because they have been > configured that way -which is inefficient, but who cares throw more CPU at it > and you're done -real operational problem is the meaningless peer reset. I'm not sure I've seen this, but I'm ready to accept it to be true, but seems like something that could get PR and get fixed. Do you have a particular example where neighbours which will export the same set of routes and have same behaviour get separate update-groups? > True but the argument can go both ways -in case of common memory space > there's no protection so one needs to worry about leaking into different > parts of memory. Yes, the whole house of cards effect, which is very relevant in multitenant time-shared systems like Linux server, just because sshd crashes doesn't mean httpd should suffer. But I'm not sure if it's so important in router. > Hmm, but we wouldn’t second guess this multi-process approach for > desktop/mobile operating systems right? They are much less controlled system, you install nillywilly random applications to your phone, and it's not good if phone crashes just because some background app I already forgot crashed. And servers have multiple tenants. I think in most cases routers have very different profile, vendor knows exactly what is running there and it's exactly one use-case. Note I'm not saying multiprocess is worse than monolithic, I'm just saying, it's not obvious to me multiprocess is the superior in this use-case. Certainly C has exuberated the problem as it makes it too easy to write memory unsafe code, but considering how long lived NOS are, I think every NOS should be written in its own DSL which compiles to C++ or Rust. Then you have more choices where given problem is best solved. Like forwarding code, should we write it in C or P4 which compiles to whatever language the NPU can consume? Less capable developers will be able to write good code in P4 and can communicate to the smaller set of developers who write the P4 compiler about missing features. And if the P4 compiler developer notices some nasty hacks the P4 coders need to write to solve some problem they can move the solution to the compiler and have the P4 coders write something less hacky. I think DSL as an abstraction is justified in any complex long lived project. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> From: Saku Ytti > Sent: Thursday, September 19, 2019 1:32 PM > > On Thu, 19 Sep 2019 at 15:11, wrote: > > > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in > Junos. > > They are dynamic, but once you make export change which affects subset of > members in peer-group, that member gets reset while placed to new > update-group. > The dynamic in Cisco implementation means that peers are automatically placed to update groups based on commonalities in export policies, regardless of the configuration. In juniper case you can actually have two peers with exact same export policies be part of different update groups just because they have been configured that way -which is inefficient, but who cares throw more CPU at it and you're done -real operational problem is the meaningless peer reset. > > Right this inconsistency between configured and operational state in my > opinion is THE biggest problem of XR, I'm afraid it has to be something > fundamental since they haven't been able to consistently address these > inconsistencies across the board for years now (or ASR9k HW? Not sure if > these types of issues can be experienced on other platforms). > > It is fundamental with non-monolithic design, because processA has no > access to the memory of processB you need IPC, which for many things is too > slow, so you replicate state and sync the state, which increases complexity > and adds bugs. True but the argument can go both ways -in case of common memory space there's no protection so one needs to worry about leaking into different parts of memory. > The argument against monolithic design is that single component failure will > bring the whole house of cards down. I think this is debatable, if it's BGP or > ISIS or LDP or oror which fails, my whole system is dead anyhow, router isn't > multiservice multitenant in most cases, all pieces are needed and any piece > failure is total failure. > Touche, you're right in routers for control plane functions is mostly all or nothing. Management plane is different, but that one is separate in both Junos and XR. Then there's the flexibility aspect, for example being able to run multiple completely separate instances of BGP would be useful in some cases. > Hopefully Juniper avoids the pitfalls with Evolved, I'm not sure how. > It seems like a complex problem to me. Having own DSL which compiles with > many safety guarantees, to tune of rust, perhaps monolithic design is what > you want, perhaps using the kernel facilities for multiprocess isn't the right > solution, I don't think it's axiomatically true multiprocess is right > solution. > Hmm, but we wouldn’t second guess this multi-process approach for desktop/mobile operating systems right? adam ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On Thu, 19 Sep 2019 at 15:11, wrote: > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in > Junos. They are dynamic, but once you make export change which affects subset of members in peer-group, that member gets reset while placed to new update-group. > Right this inconsistency between configured and operational state in my > opinion is THE biggest problem of XR, I'm afraid it has to be something > fundamental since they haven't been able to consistently address these > inconsistencies across the board for years now (or ASR9k HW? Not sure if > these types of issues can be experienced on other platforms). It is fundamental with non-monolithic design, because processA has no access to the memory of processB you need IPC, which for many things is too slow, so you replicate state and sync the state, which increases complexity and adds bugs. The argument against monolithic design is that single component failure will bring the whole house of cards down. I think this is debatable, if it's BGP or ISIS or LDP or oror which fails, my whole system is dead anyhow, router isn't multiservice multitenant in most cases, all pieces are needed and any piece failure is total failure. Hopefully Juniper avoids the pitfalls with Evolved, I'm not sure how. It seems like a complex problem to me. Having own DSL which compiles with many safety guarantees, to tune of rust, perhaps monolithic design is what you want, perhaps using the kernel facilities for multiprocess isn't the right solution, I don't think it's axiomatically true multiprocess is right solution. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> From: Saku Ytti > Sent: Thursday, September 19, 2019 12:33 PM > > On Thu, 19 Sep 2019 at 14:22, wrote: > > > Just a few examples when you change export policy it resets the peer > > or the cockup with RR clearing all sessions or the fact BGP is part of > > very complex RDP monolith -to me that's not really "carrier grade" > > implementation > > This happens when export policy breaks update-group. It may sometimes be > difficult for operator to understand if it will do that or not, so it's fair > concern. > Perhaps system should not clear, but tell manual clear is needed for policy > change to take effect. > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in Junos. > If monolith is good or bad, I'm not sure. If you thread you have high > performance with some risk. If you have process separation you have IPC > problem, and you have low performance and many will solve this by > duplicating state. Junos is moving towards multi process model with Junos > Evolved, if this will be positive or negative direction remains to be seen. > I like where XR and Junos Evolved is heading, In future I'd like to have the option to install only stuff I need on a particular type of node/deployment and not worry about the rest all the way to being able to mix and match protocols of different vendors. Although cRPD is also interesting development pathway, but again cBGP would be even better :) > Operationally speaking, BGP in JunOS for us works great, on IOS-XR right now > we have sessions where policy isn't what is configured and there is no way to > verify which one, and we've propagated leaks because acting configuration > isn't the one we've configured. We've not had similar problems in JunOS. This > is anecdote, not data. > Right this inconsistency between configured and operational state in my opinion is THE biggest problem of XR, I'm afraid it has to be something fundamental since they haven't been able to consistently address these inconsistencies across the board for years now (or ASR9k HW? Not sure if these types of issues can be experienced on other platforms). Though usually it's CP state does not trickle down to DP correctly/completely where what you described seems to be CP only. adam ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On Thu, 19 Sep 2019 at 14:22, wrote: > Just a few examples when you change export policy it resets the peer or the > cockup with RR clearing all sessions or the fact BGP is part of very complex > RDP monolith -to me that's not really "carrier grade" implementation This happens when export policy breaks update-group. It may sometimes be difficult for operator to understand if it will do that or not, so it's fair concern. Perhaps system should not clear, but tell manual clear is needed for policy change to take effect. If monolith is good or bad, I'm not sure. If you thread you have high performance with some risk. If you have process separation you have IPC problem, and you have low performance and many will solve this by duplicating state. Junos is moving towards multi process model with Junos Evolved, if this will be positive or negative direction remains to be seen. Operationally speaking, BGP in JunOS for us works great, on IOS-XR right now we have sessions where policy isn't what is configured and there is no way to verify which one, and we've propagated leaks because acting configuration isn't the one we've configured. We've not had similar problems in JunOS. This is anecdote, not data. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
> Phil Reilly > Sent: Thursday, September 19, 2019 6:05 AM > > the BGP functionality and extensions are well > developed in JUNOS. Ha... :) Just a few examples when you change export policy it resets the peer or the cockup with RR clearing all sessions or the fact BGP is part of very complex RDP monolith -to me that's not really "carrier grade" implementation adam ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
g...@greenie.muc.de (Gert Doering) wrote: > And given the price point of the MX204, if the amount of interfaces is > sufficient, just get two of them :-) Alternatively, just add QFXn as satellites. El El ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
On Thu, 19 Sep 2019 at 01:52, Jason Lixfeld wrote: > Don’t get me wrong, Cisco definitely owns their share of BS, but they seem to > be predictable in how they are going to screw you. Juniper will screw you, > but how they are going to screw you seems to be predictably unpredictable. I have to say this is entirely subjective and my subjective experience is rather different. But it is to be expected, if we would have very homogenous view on which vendor is superior, competing vendors would disappear. My subjective views where JNPR shines compared to market a) model driven configuration since day1, this is a big thing, much bigger than YANG or Netconf or whatever other automation hype is going on. Junos doesn't actively resist automation. b) Trio is the only platform I know how to protect control-plane so that I don't know how to break it c) very debuggable (CSCO is good too here, this is more complaint to Other Options). d) I will echo your sentiment about complexity and agree to it, junos gives a lot of power to the operator, where CSCO might have some specific feature they support, have a name for, and have single line config, JNPR doesn't have anything, because the feature has always been possible due to the inherent flexibility and expressiveness of the system. Some particular examples, BGP GTSM, VRF source selection, egress-acl-at-ingress, there are plenty of examples in this, but can be very involved to get right. I don't have any dislike for CSCO, I'm sure you can support your business case with both vendors, or Nokia or Huawei. Subjectively for SP role, I think Nokia and Huawei have best port/chassis options. I think Juniper has best NPU and best NOS on the market. For SP role particularly, from the big names, today I think CSCO has least attractive portfolio, but this is normal timing related thing, at some point any vendor is behind, once they release some new generation stuff they'll move ahead and start regressing. I believe ANET has best practices in developing software and considering how young they are on SP market, it's quite impressive where they are. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote: > MX104's are the dual brain unit of the 204. Though a 204 has 40/100G > capabilities. If I read your original request correctly about ip > routing. Not sure the 104/204 is grunty enough to deal with multiple > internet tables. Thats a demanding task these days best left to the > larger chassis. You can't really compare MX104 to MX204. The MX104 is ppc based and *slow*, and should have never ever shipped. The MX204 is a really really nice box, with a fast intel RE and 40/100G ports (though some - documented - restrictions on how they can be combined), and from a RE/BGP point of view, en par with the larger MXes. And given the price point of the MX204, if the amount of interfaces is sufficient, just get two of them :-) gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
However, based on feedback received from people who have been running Juniper for a long time, and reading what folks have been saying here and elsewhere, we feel that it’s just too risky to put this stuff anywhere else. We have been using Juniper since 2003 and that is definitely _not_ my experience. We have used them extensively as P, PE, and Internet Edges in a tier 1 regional carrier. (We also have used ASRs in the same capabilities). IMO both do Internet Edge well. But I and most of whom I know would take an MX960 over an ASR9k any day, the BGP functionality and extensions are well developed in JUNOS. Also note that Junipers main philosophy and focus is based on MPLS so thats why its dominant in carriers for that function. Cisco do EVPN/vxlan well. They do CE well, (though I'd argue the SRX is a good candidate for this too). But they just cant compete against Juniper for SP stuff IMO. Same for Arista. That are good in the DC but they don't have much carrier market share I believe. Anyway you will note my bias leans towards Juniper in the IP routing world. I find the juniper licensing a bit more upfront and honest too. Though with the higher speed capabilities that's eroding somewhat. MX104's are the dual brain unit of the 204. Though a 204 has 40/100G capabilities. If I read your original request correctly about ip routing. Not sure the 104/204 is grunty enough to deal with multiple internet tables. Thats a demanding task these days best left to the larger chassis. On 19/09/2019 10:52 AM, Jason Lixfeld wrote: Hi, On Sep 18, 2019, at 5:15 PM, Howard Leadmon wrote: I am looking to replace an older Cisco I have sitting down in Equinix, and have l have had a few tell me that I should look at the Juniper routers as well. Diving into Juniper/JunOS isn’t for the faint of heart. It’s a completely different animal. JunOS is a slick, powerful OS, no doubt, but there’s so much nuanced configuration to hardware compatibility, and more feature-x-stops-working-after-software-upgrade headaches than I could have ever imagined from a vendor. Their policy and filter language is very comprehensive, but very complicated and takes a long time to get familiar with and understand. For the past year I’ve been testing Juniper/JunOS for the first time, and you know, I’ve got to tell you, you might want to stick to Cisco if that’s what you're comfortable with. Testing has been long and frustrating, and of the 4 platform that I was looking at, all for various tasks, I decided that the only role I’m comfortable having played by Juniper is an MX204 running as a BFD/ISIS/LDP/MPLS enabled peering/transit border router. So far, it seems to do that *really* well. However, based on feedback received from people who have been running Juniper for a long time, and reading what folks have been saying here and elsewhere, we feel that it’s just too risky to put this stuff anywhere else. Don’t get me wrong, Cisco definitely owns their share of BS, but they seem to be predictable in how they are going to screw you. Juniper will screw you, but how they are going to screw you seems to be predictably unpredictable. FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair of these for EVPN-MPLS and they’re slick, and from what I understand, they have the FIB scale to be able to act as a border router. It’s a very IOS-like CLI (but so many things about the CLI are so much more refined than IOS) so it may be more familiar, unless you’re Cisco experience is limited to IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including licensing. It’s a BCM Jericho based pizza box, so that’s redundant powered, but not “redundant” in so far as there are no redundant supervisor/management cards. But, for the number of times I’ve had that kind of failure on any of my boxes that have had redundant cards, I don’t think it’s worth the cost or the rack space, especially if it’s just a border router where you’ve probably got a bunch of other border router that can accommodate a crash or a reboot or whatever. Hope that helps. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Suggestions for Edge/Peering Router..
Hi, > On Sep 18, 2019, at 5:15 PM, Howard Leadmon wrote: > > > I am looking to replace an older Cisco I have sitting down in Equinix, and > have l have had a few tell me that I should look at the Juniper routers as > well. Diving into Juniper/JunOS isn’t for the faint of heart. It’s a completely different animal. JunOS is a slick, powerful OS, no doubt, but there’s so much nuanced configuration to hardware compatibility, and more feature-x-stops-working-after-software-upgrade headaches than I could have ever imagined from a vendor. Their policy and filter language is very comprehensive, but very complicated and takes a long time to get familiar with and understand. For the past year I’ve been testing Juniper/JunOS for the first time, and you know, I’ve got to tell you, you might want to stick to Cisco if that’s what you're comfortable with. Testing has been long and frustrating, and of the 4 platform that I was looking at, all for various tasks, I decided that the only role I’m comfortable having played by Juniper is an MX204 running as a BFD/ISIS/LDP/MPLS enabled peering/transit border router. So far, it seems to do that *really* well. However, based on feedback received from people who have been running Juniper for a long time, and reading what folks have been saying here and elsewhere, we feel that it’s just too risky to put this stuff anywhere else. Don’t get me wrong, Cisco definitely owns their share of BS, but they seem to be predictable in how they are going to screw you. Juniper will screw you, but how they are going to screw you seems to be predictably unpredictable. FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair of these for EVPN-MPLS and they’re slick, and from what I understand, they have the FIB scale to be able to act as a border router. It’s a very IOS-like CLI (but so many things about the CLI are so much more refined than IOS) so it may be more familiar, unless you’re Cisco experience is limited to IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including licensing. It’s a BCM Jericho based pizza box, so that’s redundant powered, but not “redundant” in so far as there are no redundant supervisor/management cards. But, for the number of times I’ve had that kind of failure on any of my boxes that have had redundant cards, I don’t think it’s worth the cost or the rack space, especially if it’s just a border router where you’ve probably got a bunch of other border router that can accommodate a crash or a reboot or whatever. Hope that helps. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] Suggestions for Edge/Peering Router..
I am looking to replace an older Cisco I have sitting down in Equinix, and have l have had a few tell me that I should look at the Juniper routers as well. I need a router with at least eight 10GE ports (12 to 16 is desirable), so the option for more would be nice, plus I also need dozen or so GE ports. I also need it to be able to handle full routing, so where my current unit will top out at about a million routes (which I know is coming between IPv4 and IPv6), I want an option that can handle 2 to hopefully at least 4 million routes so we can continue out current transit and peering relationships. Another thing that is important is some redundancy, so having a unit with redundant processors and power would pretty much a much to keep everyone happy, as the current gear is redundant. I am new to the Juniper realm, so when looking at options like the MX240 and MX480, I see there are different midplanes, power supplies, and line cards for the chassis, so am looking to learn more about this possible choice. Also how is Juniper licensing handled, as honestly I would like to pick this stuff up in the used market as I know it will save a ton, but again don't want to back myself into the corner and end up with a pile of useless hardware. If any can share some info, on or off list on this topic, it would sure be appreciated before I pick a direction to go.. --- Howard Leadmon PBW Communications, LLC http://www.pbwcomm.com ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp