Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-24 20:54 -0400), Jeff Wheeler wrote: My view is that fxp0 is an out-of-band interface for manual intervention; not one that I ever use for SNMP. My view is that fxp0 is completely useless interface. It's not OOB, it's completely fate-sharing the freebsd/junos. In RS232 you can at least do 'panic-on-break', not having to rely freebsd/junos to work to kick the box. But RS232 sucks otherwise: * you can't send images over it (not supported in any modern device, and even when supported, it's faster to fly to router than upload hundreds of megs of data). * you can't multiplex over it (oh RS232 is taken by some level2 tech who has gone to lunch) * rs232 console servers cost more than switches * still fate-sharing CPU, memory etc, even with HW interrupted via break Of course correct solution would be, to connect fxp0 to separate HW, running its own miniOS, from which you could copy imagees, reload RE etc. And it dos not really affect BOM at all, I can't imagine why we still have ridiculously bad OOB capabilities. And no, you would not use this FXP0 for SNMP or Netflow or whatnot. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] J/SRX ICMP handling
Hi Dale just give set security flow allow-icmp-without-flow a try Regards Klaus — Sent from Mailbox for iPhone On Thu, Apr 25, 2013 at 7:35 AM, Dale Shaw dale.shaw+j-...@gmail.com wrote: Hi all, This post relates to a previous post of mine on asymmetrically routed UDP traffic: https://puck.nether.net/pipermail/juniper-nsp/2012-December/024878.html It seems as though a J/SRX in flow mode will drop ICMP packets such as unreachable and ttl-exceeded if, after consulting the session table, an entry corresponding to the header embedded in the ICMP packet is not found. In other words, I'm gonna drop any ICMP packets[1] I see if I didn't handle the associated conversation. Assume I send a UDP packet between hosts A and D and it's routed outbound via SRX B, and for whatever reason an ICMP unreachable or ttl-exceeded is generated (think traceroute). If that ICMP packet is sent towards host D not via SRX B but via SRX C, SRX C drops it: (src/dst IPs replaced with A and D) Jan 23 14:53:45 14:53:44.938394:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: st0.1033:D-A, icmp, (3/3) Jan 23 14:53:45 14:53:44.938424:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: find flow: table 0x63ce7688, hash 494060(0x7), sa D, da A, sp 33438, dp 47488, proto 17, tok 7 Jan 23 14:53:45 14:53:44.938483:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: packet dropped, no session found for embedded icmp pak Jan 23 14:53:45 14:53:44.938495:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: flow find session returns error. Seems like perfectly reasonable behaviour for a firewall, right? Right, except when it's not :-) Can this behaviour be modified without fully or selectively running in packet mode? I'm running JUNOS 10.4R11. Cheers, Dale [1] Well, any ICMP packets that include a copy of the original datagram's header: echo request/reply are forwarded (subject to being permitted by security policy, of course). ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
2013/4/25 Brandon Ross br...@pobox.com Many operators have backbone routers with 10's of 10GbE ports and maybe even a few 40 or 100GbE ports at this point, perhaps a variety of SONET still, etc. Are you suggesting that they should purchase a 10/100/1000 copper card just for management? Or are you suggesting that they should buy 10GbE switches for their out of band management network? No, I propose to not even bother with copper, if you prefer. Just use a VLAN or any other type of virtual circuit inside those TerabitEthernet / SONET / FrameRelay / whatever. And do you propose to use dedicated fibers just for management? Or, otherwise, how would you pass traffic to/from the copper switch, into which you plug the fxp0 of a remote router? I bet in 99% of cases, connectivity from NOC to the OOB switch shares fate with the connectivity to the router, whose fxp0 is plugged into this switch. Not even mentioning that this OOB switch is an additional point of failure with little chances to be backed up. So there is nothing in the fxp0's OOBness, what is worth its clumsiness. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
From: Saku Ytti s...@ytti.fi And no, you would not use this FXP0 for SNMP or Netflow or whatnot. -- ++ytti And why is that may I ask? Care to elaborate? Just curious - maybe You don't know how to cook it properly :-) I personally set up SNMPv1/v2/v3 over fxp0 enough times, including SNMPv3 with separate auth/enc keys for RE0 and RE1. Many thanks Alex ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] J/SRX ICMP handling
Hi Klaus, On Thu, Apr 25, 2013 at 4:44 PM, Klaus Groeger kla...@gmail.com wrote: set security flow allow-icmp-without-flow This doesn't seem to be a valid command - at least not on 10.4R11. I couldn't find a reference in the documentation either. The closest I can find is security idp sensor-configuration flow allow-icmp-without-flow, but I'm not using IDP. Cheers, Dale ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] J/SRX ICMP handling
Selective packet services is always an option assuming you're in a branch srx (650 and below). Basically just write a firewall filter that allows icmp with a then action of packet mode. Keeping track of icmp may not add any value but be aware with SPS you now lose NAT, screens and IDP (which you said you don't use already) but those may not be an issue in your environment. Hope this helps, Tim Eberhard On Apr 24, 2013, at 10:23 PM, Dale Shaw dale.shaw+j-...@gmail.com wrote: Hi all, This post relates to a previous post of mine on asymmetrically routed UDP traffic: https://puck.nether.net/pipermail/juniper-nsp/2012-December/024878.html It seems as though a J/SRX in flow mode will drop ICMP packets such as unreachable and ttl-exceeded if, after consulting the session table, an entry corresponding to the header embedded in the ICMP packet is not found. In other words, I'm gonna drop any ICMP packets[1] I see if I didn't handle the associated conversation. Assume I send a UDP packet between hosts A and D and it's routed outbound via SRX B, and for whatever reason an ICMP unreachable or ttl-exceeded is generated (think traceroute). If that ICMP packet is sent towards host D not via SRX B but via SRX C, SRX C drops it: (src/dst IPs replaced with A and D) Jan 23 14:53:45 14:53:44.938394:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: st0.1033:D-A, icmp, (3/3) Jan 23 14:53:45 14:53:44.938424:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: find flow: table 0x63ce7688, hash 494060(0x7), sa D, da A, sp 33438, dp 47488, proto 17, tok 7 Jan 23 14:53:45 14:53:44.938483:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: packet dropped, no session found for embedded icmp pak Jan 23 14:53:45 14:53:44.938495:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: flow find session returns error. Seems like perfectly reasonable behaviour for a firewall, right? Right, except when it's not :-) Can this behaviour be modified without fully or selectively running in packet mode? I'm running JUNOS 10.4R11. Cheers, Dale [1] Well, any ICMP packets that include a copy of the original datagram's header: echo request/reply are forwarded (subject to being permitted by security policy, of course). ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Saku Ytti wrote: On (2013-04-24 20:54 -0400), Jeff Wheeler wrote: My view is that fxp0 is an out-of-band interface for manual intervention; not one that I ever use for SNMP. My view is that fxp0 is completely useless interface. It's not OOB, it's completely fate-sharing the freebsd/junos. OOB in this context refers to having a different path through the network than the path used for production IP traffic. I see the value in having an Ethernet/IP interface that is not fate-sharing with the core OS as well, but that doesn't make fxp0 completely useless as previously stated. In RS232 you can at least do 'panic-on-break', not having to rely freebsd/junos to work to kick the box. But RS232 sucks otherwise: I'm not sure why we are suddenly debating the benefits and drawbacks of RS232. The two interface types are there for very different reasons. Of course correct solution would be, to connect fxp0 to separate HW, running its own miniOS, from which you could copy imagees, reload RE etc. That essentially what we are talking about. Connect fxp0 to a SEPARATE switch that is used for only out of band traffic. You then use this network to copy images, etc. What am I missing here? And no, you would not use this FXP0 for SNMP or Netflow or whatnot. Sure you can, I've done it, others here have said they've done it. Assuming the OOB network is well protected from outside traffic to avoid attacks and the like, why not? -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
From: Saku Ytti s...@ytti.fi There is nothing stopping vendors from implementing netflow and SNMP in HW, allowing instant refresh of octet counters. SNMPv3 would require encryption capabilities in HW making Your idea (a) potentially too expensive and (b) prone to export restrictions==must develop maintain 2 separate HW sets, same as for JUNOS software. Netflow often is already implemented in HW. Netflow does NOT require encryption as standard (SNMPv3 does). And as Jeff mentioned, you cannot do CoPP to protect your RE from being congested by fxp0 traffic. Something simple and easy mistake to do as L2 loop in FXP0 could be disaster, and no way to protect. (a) lo0.0 filter copy is applied to fxp0 as well (b) only if You build OOB network as flat L2 I would expect L2 BUM storms affecting fxp0. The providers I worked with build their OOB networks using same design principles as their production networks - never flat L2, routed hops, every site has at least 1 (often 2 or multi-staged) firewall(s) protecting the rest of the OOB domain from rogue elements. HTH Thanks Alex ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Pavel Lunin wrote: No, I propose to not even bother with copper, if you prefer. Just use a VLAN or any other type of virtual circuit inside those TerabitEthernet / SONET / FrameRelay / whatever. So you propose to do away with the out of band network entirely, and instead do all of your management inband. While that is a certainly a fine choice for some networks, others do not want to fate share their management capabilities with their production traffic. Is that unreasonable? And do you propose to use dedicated fibers just for management? I don't just propose it, I do it. Depends on the network of course, but if you'd like an example, the network we put together for Interop every year has what we call Access Ether. It is a true out of band management network. It's flat layer 2 (to keep it simple since it doesn't have to scale greater than a 100 devices or so), runs on completely independent switches from production traffic and runs on separate backhaul fiber (or copper in some cases). There are operators that run their networks in a similar way. Or, otherwise, how would you pass traffic to/from the copper switch, into which you plug the fxp0 of a remote router? I bet in 99% of cases, connectivity from NOC to the OOB switch shares fate with the connectivity to the router, whose fxp0 is plugged into this switch. At some level there's fate sharing, sure. If Darth Vader uses the Death Star against Earth, then I'll lose both my inband and out of band management capabilities. If, however, I have a failure of the data plane somewhere in my production network, at least I can still reach the processor and send it new code or whatever I need to do. Not even mentioning that this OOB switch is an additional point of failure with little chances to be backed up. That's only the case if you completely disable inband management capabilities. Some networks choose to do this, some do not. For those that do not (again see the Interop network) it is not an additional failure point at all, but actually redundancy. So there is nothing in the fxp0's OOBness, what is worth its clumsiness. Please don't get me wrong. I think the way fxp0 is implemented sucks big time. The fact that I have to use the default logical-system to make it useful and move all of my production traffic to others is super annoying. The lack of other features on that port makes it downright dangerous if you don't pay attention, but it is NOT useless. -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On 4/25/13 7:55 AM, Brandon Ross wrote: On Thu, 25 Apr 2013, Saku Ytti wrote: On (2013-04-24 20:54 -0400), Jeff Wheeler wrote: My view is that fxp0 is an out-of-band interface for manual intervention; not one that I ever use for SNMP. there are differing deployment models, our pop routers are polled by devices that are located on their oob network. they are also polled inband... I know personally that it would be convenient to put the management interfaces in another routing instance and have snmp work. the former works, the later doesn't. I belive that there have been feature requests about that for some time. My view is that fxp0 is completely useless interface. It's not OOB, it's completely fate-sharing the freebsd/junos. it's not part of the forwarding plane so it certainly is not in-band, what you connect it to of course is your business. we connect them to our oob network. OOB in this context refers to having a different path through the network than the path used for production IP traffic. I see the value in having an Ethernet/IP interface that is not fate-sharing with the core OS as well, but that doesn't make fxp0 completely useless as previously stated. In RS232 you can at least do 'panic-on-break', not having to rely freebsd/junos to work to kick the box. But RS232 sucks otherwise: I'm not sure why we are suddenly debating the benefits and drawbacks of RS232. The two interface types are there for very different reasons. Of course correct solution would be, to connect fxp0 to separate HW, running its own miniOS, from which you could copy imagees, reload RE etc. That essentially what we are talking about. Connect fxp0 to a SEPARATE switch that is used for only out of band traffic. You then use this network to copy images, etc. What am I missing here? a basband management controller... In DUAL RE deployments that's somewhat less acute becasue you can power cycle the SCB that the alternate RE is in. but having serial console on on ethernet for example would eliminate a terminal server potentiallly and that needs to happen eventually imho. And no, you would not use this FXP0 for SNMP or Netflow or whatnot. Sure you can, I've done it, others here have said they've done it. Assuming the OOB network is well protected from outside traffic to avoid attacks and the like, why not? inline flow export is generated in linecard asics so it's not really suitable for the oob port. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 16:04 +0100), Alex Arseniev wrote: SNMPv3 would require encryption capabilities in HW making Your idea (a) potentially too expensive and (b) prone to export restrictions==must develop maintain 2 separate HW sets, same as for JUNOS software. HW port can easily go through RE if needed. RE port can't be WH only ever. That is, if you are going to need _anything_ which can be done via HW, then the port cannot be in RE. (a) lo0.0 filter copy is applied to fxp0 as well It's not in HW. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 10:55 -0400), Brandon Ross wrote: I'm not sure why we are suddenly debating the benefits and drawbacks of RS232. The two interface types are there for very different reasons. Done right, you'd need one MGMT interface, and ethernet is obvious solution. That essentially what we are talking about. Connect fxp0 to a SEPARATE switch that is used for only out of band traffic. You then use this network to copy images, etc. What am I missing here? What are you winning by not doing this on-band in HW interface? Sure you can, I've done it, others here have said they've done it. Assuming the OOB network is well protected from outside traffic to avoid attacks and the like, why not? Additional ports, wiring. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 08:29 -0700), joel jaeggli wrote: It's not OOB, it's completely fate-sharing the freebsd/junos. it's not part of the forwarding plane so it certainly is not in-band, what you connect it to of course is your business. we connect them to our oob network. Yes it's not fate-sharing forwarding-plane, but it's fate-sharing the whole control-plane. You need ports, wiring to build fxp0 management network, which isn't even redundant, single port down and it's not reachable. Lot of cost+complexity for only benefit of being able to configure router when forwarding is broken but router not. power cycle the SCB that the alternate RE is in. but having serial console on on ethernet for example would eliminate a terminal server potentiallly and that needs to happen eventually imho. Sadly Cisco did CMP, but removed in Nexus7k RP2, citing thermal/pincount and lack of customer demand. People aren't asking for proper solution to this problem in RFQs. inline flow export is generated in linecard asics so it's not really suitable for the oob port. I think this is really my point, you need * fxp0 for ssh, snmp * inband for netflow, snmp (if HW) (redundant) * rs232 to attempt recovering box from control-plane software failure Why build fxp0, if you need inband for something anyhow? It costs money, adds complexity, and delivers no value if RS232 is also implemented with in-band. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
Well, I agree, if you are brave enough to run a real OOB management network, you have reasons to use fxp0 on the devices, that don't have 1G ports, though I believe it's at least not cheaper than buying 1GE ports just for management :) But in my experience real OOB mgt is a too rare case in real life of the ISP world. BTW, yes, there is much more sense in real OOB management in the access, but you first gave an example of an all 10/100GE core, which is a slightly different thing. And even in the access nothing really pushes you to use fxp0 for OOB mgt. However it doesn't really matter, whether and when it's common or not. If you know what and why you are doing, there is no problem. But most people, who I talk with about using fxp0, want to use it just because, with no good reason except it is specially developed by vendors, so I think, it's better to manage devices through it and they just don't really understand implications of bypassing data plane. BTW, I don't say it's useless. When you need to remotely upload software to a non-operationg box, this is an only option. But I'm sure it's better to not use it for every day routine management like SNMP, if only you have an option. 25.04.2013 19:07, Brandon Ross wrote: On Thu, 25 Apr 2013, Pavel Lunin wrote: No, I propose to not even bother with copper, if you prefer. Just use a VLAN or any other type of virtual circuit inside those TerabitEthernet / SONET / FrameRelay / whatever. So you propose to do away with the out of band network entirely, and instead do all of your management inband. While that is a certainly a fine choice for some networks, others do not want to fate share their management capabilities with their production traffic. Is that unreasonable? And do you propose to use dedicated fibers just for management? I don't just propose it, I do it. Depends on the network of course, but if you'd like an example, the network we put together for Interop every year has what we call Access Ether. It is a true out of band management network. It's flat layer 2 (to keep it simple since it doesn't have to scale greater than a 100 devices or so), runs on completely independent switches from production traffic and runs on separate backhaul fiber (or copper in some cases). There are operators that run their networks in a similar way. Or, otherwise, how would you pass traffic to/from the copper switch, into which you plug the fxp0 of a remote router? I bet in 99% of cases, connectivity from NOC to the OOB switch shares fate with the connectivity to the router, whose fxp0 is plugged into this switch. At some level there's fate sharing, sure. If Darth Vader uses the Death Star against Earth, then I'll lose both my inband and out of band management capabilities. If, however, I have a failure of the data plane somewhere in my production network, at least I can still reach the processor and send it new code or whatever I need to do. Not even mentioning that this OOB switch is an additional point of failure with little chances to be backed up. That's only the case if you completely disable inband management capabilities. Some networks choose to do this, some do not. For those that do not (again see the Interop network) it is not an additional failure point at all, but actually redundancy. So there is nothing in the fxp0's OOBness, what is worth its clumsiness. Please don't get me wrong. I think the way fxp0 is implemented sucks big time. The fact that I have to use the default logical-system to make it useful and move all of my production traffic to others is super annoying. The lack of other features on that port makes it downright dangerous if you don't pay attention, but it is NOT useless. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
25.04.2013 19:04, Alex Arseniev wrote: Netflow does NOT require encryption as standard (SNMPv3 does). Netflow or stateful log export is very often not supported on fxp0 and analogues. Even if it is, high rate of those logs can easily overwhelm RE or the link between RE and data plane. (a) lo0.0 filter copy is applied to fxp0 as well It's not in hardware. So, say, the new multistage DoS-protection feature of MX won't work. BTW, do policers work at all on fxp0? I think they should but it's a good example of a special need to care, spend time, etc. Moreover, it can be easily poorly documented or not documented at all. The providers I worked with build their OOB networks using same design principles as their production networks - never flat L2, routed hops, every site has at least 1 (often 2 or multi-staged) firewall(s) protecting the rest of the OOB domain from rogue elements. Even so. Why fxp0? Why not normal interface (given you have it)? Well, at the end it's not that important (though evident) why OOB mgt interfaces have their limitations, they just do. And while there are very few benefits (except some corner cases), there are lots of drawbacks, which, of course, can be worked around, but what for? ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Saku Ytti wrote: On (2013-04-25 10:55 -0400), Brandon Ross wrote: I'm not sure why we are suddenly debating the benefits and drawbacks of RS232. The two interface types are there for very different reasons. Done right, you'd need one MGMT interface, and ethernet is obvious solution. I guess we'll just have to agree to disagree then. One management interface causes a truck roll when that one management interface fails. They all fail in one way or another. Clearly this is acceptable risk to you and the networks that you build. As a well known network expert likes to say, I encourage my competition to build their networks this way. That essentially what we are talking about. Connect fxp0 to a SEPARATE switch that is used for only out of band traffic. You then use this network to copy images, etc. What am I missing here? What are you winning by not doing this on-band in HW interface? Once again, when I have a device without an interface card that can economically connect to an out of band switch (i.e. a core router with all 10GbE interfaces, a router with SONET line cards, etc.) Sure you can, I've done it, others here have said they've done it. Assuming the OOB network is well protected from outside traffic to avoid attacks and the like, why not? Additional ports, wiring. Your words were, And no, you would not use this FXP0 for SNMP or Netflow or whatnot. I am disagreeing with that statement, I WOULD, in the appropriate cases, use fxp0 for SNMP, and have. I did not say it did not come at extra cost or complexity. Like everything in networking, or business in general, doing something comes at the cost of something else. A good engineer is able to balance the costs and the benefits. For many networks, once again, I agree the that cost and complexity of the fxp0 port is beyond it's value. I've built several of those networks myself and have even been on your side of the argument. But to make sweeping generalizations that the port shouldn't ever be used or is useless, is not accurate. -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] QFX vs EX4550 as collapsed core
Hi, we're deploying to a new environment where there will be about 500 virtual servers hosted completely on Cisco UCS. The Core would mostly be hosting uplinks to the UCS Fabric Interconnects (End Host Mode), inter-vlan routing and links to service appliances (FW/LB) and the Internet edge routers. Nearly all of our traffic is North/South from server to LB to internet or server to LB to another server. The core would mostly be routing a few (dozens at most) routes so RIB/FIB size shouldn't be a great concern. Most links will be 10G, but there are a handful of 1G management links. We're considering either the QFX3500 ( x2) or the EX4450 (x2 as a VC) to fill this role (or potentially Cisco Nexus 6001) * are there any L3 benefits of one over the other? I haven't found clear numbers in the datasheets * Has anyone actually used MC-LAG on the QFX3500? Is it working well? any caveats? we've also considered collapsing the edge too, but the cost of say an MX-480 with similar port count is about twice that of an MX-80 + QFX/EX thanks! -andy ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
From: Saku Ytti s...@ytti.fi To: juniper-nsp@puck.nether.net Sent: Thursday, April 25, 2013 4:34 PM HW port can easily go through RE if needed. Unless there is single ASIC in the box, that would be statistical multiplexing. Unless You wish to maintain full potential perf.gain from generating SNMP in HW even in case through RE, that would require RE scalability performance == sum of SNMP performances of all ASICs in the box. Clearly won't happen. (a) lo0.0 filter copy is applied to fxp0 as well It's not in HW. I feel the need to return the favour here :-) SNMPv3 generated in ASIC and transiting via RE (for the purpose of being encrypted) is NOT in HW. It would be classified as HW-assisted. Thanks Alex ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Saku Ytti wrote: On (2013-04-25 08:29 -0700), joel jaeggli wrote: It's not OOB, it's completely fate-sharing the freebsd/junos. it's not part of the forwarding plane so it certainly is not in-band, what you connect it to of course is your business. we connect them to our oob network. Yes it's not fate-sharing forwarding-plane, but it's fate-sharing the whole control-plane. You need ports, wiring to build fxp0 management network, which isn't even redundant, single port down and it's not reachable. Which is MUCH better that not reachable, ever, at all. Lot of cost+complexity for only benefit of being able to configure router when forwarding is broken but router not. Which never happens, right? I guess I'm just the lucky one that gets routers that freak out due to a bug (not necessarily just Juniper, but in general) or attack or whatever and become unreachable except for out of band access. I'm also probably the only one that has worked on networks that had cascading routing protocol failure and needed some emergency reconfiguration (which could only be done from out of band). I'm sure Joel is the only one that's had this happen too. Right Joel? inline flow export is generated in linecard asics so it's not really suitable for the oob port. I think this is really my point, you need * fxp0 for ssh, snmp * inband for netflow, snmp (if HW) (redundant) * rs232 to attempt recovering box from control-plane software failure Why build fxp0, if you need inband for something anyhow? It costs money, adds complexity, and delivers no value if RS232 is also implemented with in-band. I think we've covered this multiple times now and you even covered it above a bit. ssh, snmp, software loads, etc. require the fxp0 port if/when you have no in-band access for wahtever reason, of which there could be many. -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Pavel Lunin wrote: Well, I agree, if you are brave enough to run a real OOB management network, you have reasons to use fxp0 on the devices, that don't have 1G ports, though I believe it's at least not cheaper than buying 1GE ports just for management :) I suppose that's a local calculation of all of the costs and complexity involved. But in my experience real OOB mgt is a too rare case in real life of the ISP world. We have very different experiences then. I'm not claiming it's a majority, but I will claim that many of the largest networks in the world do, indeed, have true OOB management networks. Enough that the business case for what is probably a fairly minimal cost for Juniper to keep the hardware in the box for fxp0 makes sense. BTW, yes, there is much more sense in real OOB management in the access, but you first gave an example of an all 10/100GE core, which is a slightly different thing. And even in the access nothing really pushes you to use fxp0 for OOB mgt. I see no difference in the purpose or usage of the port weather the box is access or core. If there's no economical ports in the box already, fxp0 makes sense. In many networks, consistency is more important than the cost of each deployment, so in those cases it may be cheaper overall for ALL Juniper devices to be managed via fxp0. If you know what and why you are doing, there is no problem. But most people, who I talk with about using fxp0, want to use it just because, with no good reason except it is specially developed by vendors, so I think, it's better to manage devices through it and they just don't really understand implications of bypassing data plane. Yup, there are many idiots out there that will do anything vendors say. There's even more that think they know what they are doing because they were able to pass the vendors trivia quiz. You can't fix stupid and taking away the tools that not-stupid need to do their job only results in boxes that not-stupid don't want to buy. So far, I'd say, Juniper caters to not-stupid. Stupid is just going to buy Cisco anyway because their fancy VP showed up and took the VPs out golfing. BTW, I don't say it's useless. When you need to remotely upload software to a non-operationg box, this is an only option. But I'm sure it's better to not use it for every day routine management like SNMP, if only you have an option. You did not. I've been partially responding to Saku who said, My view is that fxp0 is completely useless interface. My apologies if my comments implied that you made such a statement. -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
- Original Message - From: Saku Ytti s...@ytti.fi To: juniper-nsp@puck.nether.net Yes it's not fate-sharing forwarding-plane, but it's fate-sharing the whole control-plane. No, it is not. fxp0 is fully functional on backup RE (including Telnet/SSH/SNMP) - and backup RE by default does not run any control plane functions apart from monitoring master RE. I think this is really my point, you need * fxp0 for ssh, snmp * inband for netflow, snmp (if HW) (redundant) * rs232 to attempt recovering box from control-plane software failure Inband for high-perf Netflow - yes. Inband for SNMP - unless You want subsecond counter updates (for realtime billing maybe?) then no. And I already answered Your points regarding SNMP in HW my other email. Thanks Alex ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 12:56 -0400), Brandon Ross wrote: I guess I'm just the lucky one that gets routers that freak out due to a bug (not necessarily just Juniper, but in general) or attack or Yes it does happen. And yes it can break host OS completely, so that your fpx0 does not nothing. At least on RS232 you can send break. So I'm not saying you don't need OOB. I'm saying you need on-band + OOB, but the OOB needs to be robust, port which works as often as possible, which on-band (fate-sharing same memory, cpu, host OS) fxp0 is not. You don't need _three_ ports, on-band, oob and something-in-between. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
* Saku Ytti That essentially what we are talking about. Connect fxp0 to a SEPARATE switch that is used for only out of band traffic. You then use this network to copy images, etc. What am I missing here? What are you winning by not doing this on-band in HW interface? Cost. The fxp0 port is basically free. The ports on the line cards are anything but - there's not a chance in hell I could reserve one of the revenue ports on the line cards for emergency management access that I could use to get at the box, should for example my IGP melt down. Or procure a management switch with 10G ports I could connect the revenue-turned-mgmt port to, for that matter. Unmanaged 100M switches, is on the other hand practically free. Sure you can, I've done it, others here have said they've done it. Assuming the OOB network is well protected from outside traffic to avoid attacks and the like, why not? Additional ports, wiring. Use of the fxp0 port doesn't require any more ports/wiring than the CMP style approach you appear to be advocating? It would be better to have an embedded iLO/BMC in the chassis like you find on pretty much any x86 server these days - with internal serial console port, power control, etc. I do have x86 servers with iLOs connected to the same management switch I connect my fxp0s to. fxp0 isn't perfect, but IMHO it is better than nothing. Tore ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On 4/25/13 8:47 AM, Saku Ytti wrote: On (2013-04-25 08:29 -0700), joel jaeggli wrote: It's not OOB, it's completely fate-sharing the freebsd/junos. it's not part of the forwarding plane so it certainly is not in-band, what you connect it to of course is your business. we connect them to our oob network. Yes it's not fate-sharing forwarding-plane, but it's fate-sharing the whole control-plane. You need ports, wiring to build fxp0 management network, which isn't even redundant, single port down and it's not reachable. Lot of cost+complexity for only benefit of being able to configure router when forwarding is broken but router not. power cycle the SCB that the alternate RE is in. but having serial console on on ethernet for example would eliminate a terminal server potentiallly and that needs to happen eventually imho. Sadly Cisco did CMP, but removed in Nexus7k RP2, citing thermal/pincount and lack of customer demand. People aren't asking for proper solution to this problem in RFQs. inline flow export is generated in linecard asics so it's not really suitable for the oob port. I think this is really my point, you need * fxp0 for ssh, snmp * inband for netflow, snmp (if HW) (redundant) * rs232 to attempt recovering box from control-plane software failure Why build fxp0, if you need inband for something anyhow? It costs money, adds complexity, I doubt the impact on the BOM is signficant. the EM0/EM1 interfaces and the two ethernet switches embedded in the SCBs are a substantially more complex bit of RE supporting ethernet, then the third nic which is an intel 0x100f8086 sitting on one of the shared 32 pci busses and a port out the back. In the more embedded paltforms that's certainly just another ethernet embedded in the SOC. pciconf -lv on your RE can point out just how simple an embedded pc you're actually dealing with, there's not a lot of magic there. compared to what I'd rather have which would be a bmc or chassis management controller which actually probably is a significant integration issue particularly if you want access to both RE's and the SCBs and the linecards because that has to get physically connected via the midplane as the current REs are through the SCBs and delivers no value if RS232 is also implemented with in-band. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 10:21 -0700), joel jaeggli wrote: Why build fxp0, if you need inband for something anyhow? It costs money, adds complexity, I doubt the impact on the BOM is signficant. the EM0/EM1 interfaces and the two ethernet switches embedded in the SCBs are a I mean FXP0 MGMT NET costs me to build, and value is negligible to on-band MGMT, only difference is, fxp0 does not fail when forwarding-fails. But I still need RS232 as well, so I'm building 3 networks, on-band, RS232 and FXP0. BOM for proper OOB port in place of FXP, which connect to its own memory, cpu (think of vPro or CMP) is not significant. You can buy new vPro motherboard for like 50EUR. pciconf -lv on your RE can point out just how simple an embedded pc you're actually dealing with, there's not a lot of magic there. I actually talked to freescale about this, and they said they have something in works to this end, to deliver OOB ethernet. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 19:20 +0200), Tore Anderson wrote: Use of the fxp0 port doesn't require any more ports/wiring than the CMP style approach you appear to be advocating? My point is, with FXP0 I still need to build RS232 network as well. With CMP not. I fully accept you need on-band + OOB, but you don't need three methods. It would be better to have an embedded iLO/BMC in the chassis like you find on pretty much any x86 server these days - with internal serial This is what CMP is. Imagine offering FXP0 solution to server guys? Here is eth1 to your linux, it only works when linux works, have fun. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
- Original Message - From: Pavel Lunin plu...@senetsy.ru To: juniper-nsp@puck.nether.net Sent: Thursday, April 25, 2013 5:48 PM Subject: Re: [j-nsp] SNMP on logical-system fxp0 25.04.2013 19:04, Alex Arseniev wrote: Netflow does NOT require encryption as standard (SNMPv3 does). Netflow or stateful log export is very often not supported on fxp0 and analogues. Even if it is, high rate of those logs can easily overwhelm RE or the link between RE and data plane. (a) lo0.0 filter copy is applied to fxp0 as well It's not in hardware. Correct. Do you expect someone to attack fxp0 from within Your OOB network? Rogue NMS server perhaps? In that case You have OOB network design problems, see my point below wrt OOB design principles. The providers I worked with build their OOB networks using same design principles as their production networks - never flat L2, routed hops, every site has at least 1 (often 2 or multi-staged) firewall(s) protecting the rest of the OOB domain from rogue elements. Even so. Why fxp0? Why not normal interface (given you have it)? Because fxp0 is free in a sense that it is included in RE price? Well, at the end it's not that important (though evident) why OOB mgt interfaces have their limitations, they just do. It is clearly evident that for every vendor product which has management built-in interfaces on control modules, these built-in interfaces on control modules cannot deliver same features perf as revenue interfaces. Do You have expectations and/or experience/examples to the contrary? Thanks Alex ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
2013/4/25 Brandon Ross br...@pobox.com But in my experience real OOB mgt is a too rare case in real life of the ISP world. We have very different experiences then. I'm not claiming it's a majority, but I will claim that many of the largest networks in the world do, indeed, have true OOB management networks. Just let me clarify that with real OOB mgt network I mean dedicated media, switches and all or, more formally, independence of control traffic from forwarding plane of what it controls. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Pavel Lunin wrote: 2013/4/25 Brandon Ross br...@pobox.com But in my experience real OOB mgt is a too rare case in real life of the ISP world. We have very different experiences then. I'm not claiming it's a majority, but I will claim that many of the largest networks in the world do, indeed, have true OOB management networks. Just let me clarify that with real OOB mgt network I mean dedicated media, switches and all or, more formally, independence of control traffic from forwarding plane of what it controls. We are using the same definition informally I think. Formally, I don't know of any networks where the actual network control traffic (routing protocols) are on a completely independent, out of band network, nor would I recommend such a design for any use cases I can think of. If you change control traffic to management traffic in your statement, then I think we'd be in complete agreement. -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 17:53 +0100), Alex Arseniev wrote: I feel the need to return the favour here :-) SNMPv3 generated in ASIC and transiting via RE (for the purpose of being encrypted) is NOT in HW. It would be classified as HW-assisted. Quite. The main point is, in FXP you can never capitalize on any HW feature, as you always transit RE. In on-band/NPU ports you can avoid NPU completely when viable, and when not viable you can use RE. Only of course on top of on-band port, you need one port for disaster recovery, and for this role FXP is very ill-fitting. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] J/SRX ICMP handling
In the context of this conversation it is implicit that IPsec is used, with st0.x interfaces. They have nowhere to attach filters! To be able to use filters with st0.x interfaces, you have to wrap one more layer of interface. GRE is one obvious solution (can have attached filters), can probably use IP-IP instead, but I haven't thought that one through really. Adds complexity (GRE/IPsec must terminate at different points, different IP addresses, not as easy as with Cisco), but should work. /Per 25 apr 2013 kl. 16:50 skrev Tim Eberhard xmi...@gmail.com: Selective packet services is always an option assuming you're in a branch srx (650 and below). Basically just write a firewall filter that allows icmp with a then action of packet mode. Keeping track of icmp may not add any value but be aware with SPS you now lose NAT, screens and IDP (which you said you don't use already) but those may not be an issue in your environment. Hope this helps, Tim Eberhard On Apr 24, 2013, at 10:23 PM, Dale Shaw dale.shaw+j-...@gmail.com wrote: Hi all, This post relates to a previous post of mine on asymmetrically routed UDP traffic: https://puck.nether.net/pipermail/juniper-nsp/2012-December/024878.html It seems as though a J/SRX in flow mode will drop ICMP packets such as unreachable and ttl-exceeded if, after consulting the session table, an entry corresponding to the header embedded in the ICMP packet is not found. In other words, I'm gonna drop any ICMP packets[1] I see if I didn't handle the associated conversation. Assume I send a UDP packet between hosts A and D and it's routed outbound via SRX B, and for whatever reason an ICMP unreachable or ttl-exceeded is generated (think traceroute). If that ICMP packet is sent towards host D not via SRX B but via SRX C, SRX C drops it: (src/dst IPs replaced with A and D) Jan 23 14:53:45 14:53:44.938394:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: st0.1033:D-A, icmp, (3/3) Jan 23 14:53:45 14:53:44.938424:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: find flow: table 0x63ce7688, hash 494060(0x7), sa D, da A, sp 33438, dp 47488, proto 17, tok 7 Jan 23 14:53:45 14:53:44.938483:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: packet dropped, no session found for embedded icmp pak Jan 23 14:53:45 14:53:44.938495:CID-00:FPC-11:PIC-01:THREAD_ID-27:RT: flow find session returns error. Seems like perfectly reasonable behaviour for a firewall, right? Right, except when it's not :-) Can this behaviour be modified without fully or selectively running in packet mode? I'm running JUNOS 10.4R11. Cheers, Dale [1] Well, any ICMP packets that include a copy of the original datagram's header: echo request/reply are forwarded (subject to being permitted by security policy, of course). ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 13:03 -0400), Brandon Ross wrote: We have very different experiences then. I'm not claiming it's a majority, but I will claim that many of the largest networks in the world do, indeed, have true OOB management networks. Enough that I'm not saying don't build OOB, I'm saying, I don't want to build three networks to access the router. On-band is given. So will the second network be rs232 or fxp? Both of them are bad options, but RS232 is less bad, as you can reboot the box even if JunOS is wonky. What the 2nd option should be CMP/vPro type solution and everyone should start adding this as scoring item in their RFQ, so vendors will realize there is market demand for proper OOB port. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On (2013-04-25 18:07 +0100), Alex Arseniev wrote: No, it is not. fxp0 is fully functional on backup RE (including Telnet/SSH/SNMP) - and backup RE by default does not run any control plane functions apart from monitoring master RE. Not all devices have two RE. But this indeed is what we want, ethernet port which does not fate-share. Every gear to cheapest of the switches could have this with no significant BOM increase. If we start asking for it from vendors. The desirable solution is FXP0 which is connected to separate HW in the router, running its own miniOS (vPro/ilo/drac/cmp) -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
Saku Ytti writes: I'm not saying don't build OOB, I'm saying, I don't want to build three networks to access the router. On-band is given. So will the second network be rs232 or fxp? Both of them are bad options, but RS232 is less bad, as you can reboot the box even if JunOS is wonky. From what I've seen, the more typical setup is a terminal server serving console ports attached to the same network as the device management ports. So the terminal server gives console (and dial up) access as when Mayhem strikes, but mgmt network access for lesser/more common issues, as well as normal management. And when Mayhem strikes, you can use console access to get the mgmt port up to avoid 9600 baud terminal performance, especially for data transfers. So the second network is rs232 _and_ fxp. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
2013/4/25 Alex Arseniev alex.arsen...@gmail.com Correct. Do you expect someone to attack fxp0 from within Your OOB network? Rogue NMS server perhaps? In a big enough network — anything. Broken NMS (it turns out to happen more often than I could think), malware on management PC, misconfigured something (IP address of a syslog server), intentional hack, etc. Also, routing does not mean that you don't have broadcast domains and BUM storms are not possible. In that case You have OOB network design problems, see my point below wrt OOB design principles. Even if you have a firewall behind each fxp0 (and how do you manage that hell of firewalls, another OOB MGT network for OOB management devices? And we still consider price of a single port? :) — I bet you don't rate limit SNMP and even ICMP on the firewalls. Let's be honest, any big ISP have more than one mgt network and they rarely resemble the Eden. Just because ISPs merge and split, different BU manage different parts of the network, sometimes BU merge too, clever folks leave the company and stupid ones sometimes come, etc. This is why I'd prefer to have more tools to be sure. It is clearly evident that for every vendor product which has management built-in interfaces on control modules, these built-in interfaces on control modules cannot deliver same features perf as revenue interfaces. Do You have expectations and/or experience/examples to the contrary? Of course I don't :) This is the thesis I started with, so why should I expect the contrary? It becomes even worse, when it comes to multi-vendorness. Different equipment have different limitations for those ports. And all this makes the MGT network less and less flexible. I've once been involved in a project of a centralized monitoring system deployment for a big ISP. They had about 7 different routed OOB mgt networks (Core, Access, ATM, SDH, etc), I can't even say it was wrongly done. But just the need to provide connectivity to everything ended up with GRE over NAT over GRE over NAT salted with NAT and served through GRE sort of solutions (not everywhere, but partly). I won't say all or even most, but A LOT of troubles they had, came from the limitations of dedicated mgt interfaces. Of course, as a result, this whole interconnected network was not a thing, with which you could be sure of nothing bad happens ever. Despite some parts were originally well designed and run. Even so. Why fxp0? Why not normal interface (given you have it)? Because fxp0 is free in a sense that it is included in RE price? OK, I got it. The main reason is the physical port itself (I should've asked, what happened to VLANs on normal interfaces, but I won't :). Well, my point here is just that one must be very conscious and ask himself twice, whether he knows what he is doing, when choosing dedicated mgt interface for every-day management. P. S. I also have doubts that the economical assumption really holds (opex for administration also costs money) but let's leave this alone. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
- Original Message - From: Pavel Lunin To: Alex Arseniev Cc: juniper-nsp Sent: Thursday, April 25, 2013 9:56 PM Subject: Re: [j-nsp] SNMP on logical-system fxp0 In a big enough network — anything. Broken NMS (it turns out to happen more often than I could think), malware on management PC, misconfigured something (IP address of a syslog server), intentional hack, etc. Also, routing does not mean that you don't have broadcast domains and BUM storms are not possible. [AA] if You actually still dealing with such issues in Your customer networks, my condolences. Especially sad is the issue with management PC - do Your customers use commodity Windows PC with freeware Solarwinds version as NMS? Even if you have a firewall behind each fxp0 (and how do you manage that hell of firewalls, another OOB MGT network for OOB management devices? And we still consider price of a single port? :) — I bet you don't rate limit SNMP and even ICMP on the firewalls. [AA] The firewalls are usually clustered, so if one fails, the second one takes over. [AA] The providers I worked with usually know how many ports they need now and in the nearest future so the overall cost of adding single revenue port for OOB could reach thousands of $$$ if in order to add just 1 more port the whole new FPC/DPC/MPC need to be purchased. Let's be honest, any big ISP have more than one mgt network and they rarely resemble the Eden. Just because ISPs merge and split, different BU manage different parts of the network, sometimes BU merge too, clever folks leave the company and stupid ones sometimes come, etc. This is why I'd prefer to have more tools to be sure. [AA] Not sure if I follow, if BUs are administratively separate, how having a true OOB interface (i.e. CMP in Routing Engine) would make Your life easier? It becomes even worse, when it comes to multi-vendorness. Different equipment have different limitations for those ports. And all this makes the MGT network less and less flexible. I've once been involved in a project of a centralized monitoring system deployment for a big ISP. They had about 7 different routed OOB mgt networks (Core, Access, ATM, SDH, etc), I can't even say it was wrongly done. But just the need to provide connectivity to everything ended up with GRE over NAT over GRE over NAT salted with NAT and served through GRE sort of solutions (not everywhere, but partly). I won't say all or even most, but A LOT of troubles they had, came from the limitations of dedicated mgt interfaces. [AA] I also had to do a new OOB network design for another national ISP to replace similar mess of OOB networks because this national ISP clearly saw the value of unified OOB solution. Maybe Your ISP is not educated propely on the value of well-designed OOB network? ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] Fwd: bgp license mx480 MPC-3D-16XGE-SFPP
hi all i have a new MX480 with MPC-3D-16XGE-SFPP and I am trying to enable BGP but am not sure how? someone said I need a license is this true? you can email me: luklaupdates(AT, AT, FOR SPAM)gmail.comluklaupda...@gmail.com thanks ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Fwd: bgp license mx480 MPC-3D-16XGE-SFPP
No license needed. Just configure under protocols. Will O'Brien On Apr 25, 2013, at 5:17 PM, John pp luklaupda...@gmail.com wrote: hi all i have a new MX480 with MPC-3D-16XGE-SFPP and I am trying to enable BGP but am not sure how? someone said I need a license is this true? you can email me: luklaupdates(AT, AT, FOR SPAM)gmail.comluklaupda...@gmail.com thanks ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] SNMP on logical-system fxp0
On Thu, 25 Apr 2013, Saku Ytti wrote: On (2013-04-25 13:03 -0400), Brandon Ross wrote: We have very different experiences then. I'm not claiming it's a majority, but I will claim that many of the largest networks in the world do, indeed, have true OOB management networks. Enough that I'm not saying don't build OOB, I'm saying, I don't want to build three networks to access the router. On-band is given. So will the second network be rs232 or fxp? Both. Both of them are bad options, but RS232 is less bad, as you can reboot the box even if JunOS is wonky. Everything is a bad option. You work with what you can get economically. Your statement was, My view is that fxp0 is completely useless interface. I disagree with you. It is a useful interface for the reasons that I, and others, have stated. Is it perfect? Hell no. But that doesn't change that fact that it's useful. Either defend that statement or admit that it was overly broad. -- Brandon Ross Yahoo AIM: BrandonNRoss +1-404-635-6667ICQ: 2269442 Schedule a meeting: https://doodle.com/brossSkype: brandonross ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Fwd: bgp license mx480 MPC-3D-16XGE-SFPP
to be clear i need layer 3 features so i can do this under protocols right? On Thu, Apr 25, 2013 at 6:39 PM, OBrien, Will obri...@missouri.edu wrote: No license needed. Just configure under protocols. Will O'Brien On Apr 25, 2013, at 5:17 PM, John pp luklaupda...@gmail.com wrote: hi all i have a new MX480 with MPC-3D-16XGE-SFPP and I am trying to enable BGP but am not sure how? someone said I need a license is this true? you can email me: luklaupdates(AT, AT, FOR SPAM)gmail.comluklaupda...@gmail.com thanks ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Fwd: bgp license mx480 MPC-3D-16XGE-SFPP
My apologies, I didn't pay attention to the blade you referenced. That blade is only good for switching unless you have the appropriate license to enable layer 3 protocols. Here's the document the clarifies what you can do with that card. http://www.juniper.net/techpubs/en_US/junos10.3/topics/task/configuration/chassis-mx-series-ip-ethernet-mode-configuring.html On Apr 25, 2013, at 5:39 PM, OBrien, Will obri...@missouri.edu wrote: No license needed. Just configure under protocols. Will O'Brien On Apr 25, 2013, at 5:17 PM, John pp luklaupda...@gmail.com wrote: hi all i have a new MX480 with MPC-3D-16XGE-SFPP and I am trying to enable BGP but am not sure how? someone said I need a license is this true? you can email me: luklaupdates(AT, AT, FOR SPAM)gmail.comluklaupda...@gmail.com thanks ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
