Re: [j-nsp] MX960 JunOS recommendations
Hello! (If anyone interested) It was a PR463989 p.s. It took me almost month(!!) to extract _existing_ PR number from JTAC ! /angry Krzysztof Szarkowicz wrote: JNPR send notification because of hold timer expired (meaning no BGP messages are received from the neighbor) - this is correct behavior from BGP perspective. Do you have logs on CSCO side for the same event? I assume you will see retransmission of UPDATE message (not Keepalive message). This Update message is dropped somewhere on the path between CSCO and JNPR. And CSCO retrsmits this message. Since UPDATE message is sent within Keepalive timer, no Keepalives are sent. The most common cause of dropping is mismatch of MPLS MTU, or L2 device with misconfigured MTUs somewhere in between. You have to figure out (debugs, traceoptions, tcpdumps, whats ever) which device on the path is dropping. //Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Thursday, 12 November, 2009 9:07 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations First of all thanks to all who cares :) I'll reply one by one Derick Winkworth wrote: How about some debugs or traceoptions? traceoptions at last Jun says that box dosen't receive bgp notifications some times. haven't tried any more yet sth...@nethelp.no wrote: Make sure that your IP MTU is the same on both Cisco and Juniper sides. If you run IS-IS, make sure your CLNS MTU is the same on both Cisco and Juniper sides. IP mtu are the same, otherwise ospf do not come up People have been running interoperable Cisco and Juniper networks for many years. This is not rocket science. Yeah, we installed several Juns into our network several months ago and this is the only problem which we couldn't solve and rolled back to previous software (well i do not count some rpd crashes on box with aggregated interfaces which we can avoid for now. jtac evetually said that its PR439627. I can't read this hidden PR, but its supposed to be fixed in 10.x and 9.3Rnextrelease ) Krzysztof Szarkowicz wrote: With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP, since as per RFC4271, section 4: The maximum message size is 4096 octets. All implementations are required to support this maximum message size. So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP perspective. The only thing that comes in my mind, that there are some L2 switches in between and there is something wrong with MTU on those switches. Worth to check. There are no switches between them its 7301-geoptic-7606-tengig-t1600-tengig-mx960 Its lab setup. On the real network it was slightly different, but actually its the same from this problem point of view Could you paste from the log the Notification message generated when the BGP session is tear down? I didn't find any dependance from interfaces load or anything else. It can be 3-4 gig load (like it was on real network) or empty (like its in lab), bgp session may drop once per minute or stay up for 30 - 60 mins. Cisco can be either GSR or 7301, Juniper can be mx or T. There is nothing special in logs. Thats the one from mx960: Nov 12 06:18:31 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0 rcvcc: 0 TCP state: 4, snd_una: 307818660 snd_nxt: 307818660 snd_wnd: 16230 rcv_nxt: 614682635 rcv_adv: 614699019, hold timer 0 Nov 12 06:20:48 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0 rcvcc: 0 TCP state: 4, snd_una: 1301747029 snd_nxt: 1301747029 snd_wnd: 16211 rcv_nxt: 732160622 rcv_adv: 732177006, hold timer 0 Nov 12 06:22:53 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0 rcvcc: 0 TCP state: 4, snd_una: 2024212109 snd_nxt: 2024212109 snd_wnd: 16230 rcv_nxt: 3950965686 rcv_adv: 3950982070, hold timer 0 Nov 12 06:24:56 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0 rcvcc: 0 TCP state: 4, snd_una: 2363347692 snd_nxt: 2363347692 snd_wnd: 16230 rcv_nxt: 1449362513 rcv_adv: 1449378897, hold timer 0 Nov 12 06:59:09 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal
Re: [j-nsp] MX960 JunOS recommendations
buffer sndcc: 0 rcvcc: 0 TCP state: 4, snd_una: 3633708680 snd_nxt: 3633708680 snd_wnd: 16175 rcv_nxt: 1216109422 rcv_adv: 1216125806, hold timer 0 Nov 12 07:22:54 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0 rcvcc: 0 TCP state: 4, snd_una: 4034247055 snd_nxt: 4034247055 snd_wnd: 16211 rcv_nxt: 2010186633 rcv_adv: 2010203017, hold timer 0 Nov 12 07:25:00 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to 10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 38 rcvcc: 0 TCP state: 4, snd_una: 3122195868 snd_nxt: 3122195868 snd_wnd: 16268 rcv_nxt: 20860 rcv_adv: 210016244, hold timer 0 Thanks, Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 15:12 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations Uhm, i see your point here. We indeed have cisco - cisco - Jun - Jun setup My cisco interface mtu = ip mtu = mpls mtu =9000 But i raly doubt that bgp keepalive packet size can come close to that mtu. On Juniper i set interface mtu = cisco mtu +14 and it works fine! And! As you say, it reports different mpls mtu value: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 8988 Protocol multiservice, MTU: Unlimited As far as i understand default mpls mtu term (not sure that i _fully_ understand it though) it seems, Juniper supposes 3 labels maximum. I dont see any reasons for device to drop packets which has 1 or 2 labels and bigger than mpls mtu, but still ok from interface mtu point ov view. As per your logic, device should drop all traffic that match such criteria but it seems only bgp session keepalives and i didn't see any other problems But still, i made an experiment on Juniper and cisco which has bgp session between them. cisco: #sh mpls interfaces g 0/0 detail | i MTU MTU = 9000 #sh ip int g 0/0 | i MTU MTU is 9000 bytes #sh run int g 0/0 Building configuration... Current configuration : 212 bytes ! interface GigabitEthernet0/0 description --- to 7606-2 --- mtu 9000 ip address 10.3.13.2 255.255.255.0 load-interval 30 duplex full speed 1000 media-type gbic no negotiation auto tag-switching ip end If i set mtu 9000 under family mpls and commit it, it looks like this: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 9000 Flags: Is-Primary, User-MTU Protocol multiservice, MTU: Unlimited and problem still persists please let me know if you have any other ideas :) p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 'default' (=8988) on juniper Krzysztof Szarkowicz wrote: Let me guess. Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO? CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF). If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is sent. I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes. Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is terminated on CSCO on one side and JNPR on other side)? //Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 9:57 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations What did you mean by inappropriately configured ? There are the same mtu settings everywhere and traffic passes quite well. And ospf session goes up without problems. And how comes that inappropriately configured IP and MPLS MTU work well on 9.3R3.8 ? Krzysztof Szarkowicz wrote: It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes. //Krzysztof -Original Message- From: juniper-nsp-boun
Re: [j-nsp] MX960 JunOS recommendations
We have standardized on 4470 byte MTU. This means mtu 4470 on Cisco IOS physical Ethernet interfaces, and MTU 4484 on Juniper JunOS physical Ethernet interfaces. No need to explicitly set IP or CLNS MTU. Except when you enable VLANs on Ethernet, you need to crank it up to 4488.. A thing to keep in mind and/or to monitor using scripts. Absolutely. We use quite a bit of dual tagging on Ethernet, so then we need to crank it up to 4492. But all our backbone links are 4484 on the Juniper side. Steinar Haug, Nethelp consulting, sth...@nethelp.no ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
We have a case but i think they didn't make PR yet William Jackson wrote: Hi do you have a PR Number for this issue? -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: 11 November 2009 08:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
Hi do you have a PR Number for this issue? -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: 11 November 2009 08:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab Phil Shafer wrote: Derick Winkworth writes: 9.3r4 indeed. Perhaps even 9.4r4 when that comes out. FWIW: 9.5 has a number of scripting-related features, including interactivity and remote RPC access. We've been working to ensure that scripting PRs are backported to at least 9.5. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
On Thu, Nov 12, 2009 at 01:45:22PM +0100, Krzysztof Szarkowicz wrote: The most common cause of dropping is mismatch of MPLS MTU, or L2 device with misconfigured MTUs somewhere in between. Are you sur you mean MPLS MTU? There is only one BGP session between a set of peers, even if you have several address families. I always thought this used a regular L3 protocol like IP (could be exhanged with v6 in the future). Kari ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
Excerpts from sthaug's message of Thu Nov 12 00:12:16 -0800 2009: Absolutely. We use quite a bit of dual tagging on Ethernet, so then we need to crank it up to 4492. But all our backbone links are 4484 on the Juniper side. Is there a reason not to use 9000-bytes everywhere (accounting for bridges/interfaces that count/don't count ethernet headers)? --j ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
JUNOS uses native IPv4 packets to send BGP messages. IOS uses IPv4 encapsulated in MPLS when LDP without tricks is enabled to send BGP messages. That are defaults, which can be changed of course. //Krzysztof -Original Message- From: Kari Asheim [mailto:k...@mork.no] Sent: Thursday, 12 November, 2009 14:34 To: Krzysztof Szarkowicz Cc: 'Tima Maryin'; juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations On Thu, Nov 12, 2009 at 01:45:22PM +0100, Krzysztof Szarkowicz wrote: The most common cause of dropping is mismatch of MPLS MTU, or L2 device with misconfigured MTUs somewhere in between. Are you sur you mean MPLS MTU? There is only one BGP session between a set of peers, even if you have several address families. I always thought this used a regular L3 protocol like IP (could be exhanged with v6 in the future). Kari ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
You might have devices somewhere in your network that do not support a MTU of 9000. If that is the case you set the MTU to the highest supported MTU of any device to avoid problems. // Olof On Thu, Nov 12, 2009 at 8:12 PM, Krzysztof Szarkowicz kszarkow...@gmail.com wrote: JUNOS uses native IPv4 packets to send BGP messages. IOS uses IPv4 encapsulated in MPLS when LDP without tricks is enabled to send BGP messages. That are defaults, which can be changed of course. //Krzysztof -Original Message- From: Kari Asheim [mailto:k...@mork.no] Sent: Thursday, 12 November, 2009 14:34 To: Krzysztof Szarkowicz Cc: 'Tima Maryin'; juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations On Thu, Nov 12, 2009 at 01:45:22PM +0100, Krzysztof Szarkowicz wrote: The most common cause of dropping is mismatch of MPLS MTU, or L2 device with misconfigured MTUs somewhere in between. Are you sur you mean MPLS MTU? There is only one BGP session between a set of peers, even if you have several address families. I always thought this used a regular L3 protocol like IP (could be exhanged with v6 in the future). Kari ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
Thank you all for all the information, and thank you Richard for taking the time to elaborate. We're going to use 9.3R4.4 mainly because of the E-EOL (and that fact that it has all the features we need :) ). On Fri, Nov 6, 2009 at 5:43 AM, Richard A Steenbergen r...@e-gerbil.net wrote: On Thu, Nov 05, 2009 at 03:39:32PM +0200, Andrei Radu wrote: Hello everybody, We will be installing our first Juniper MX960 router in the network. We were a 100% Cisco shop and this is our first Juniper router. The MX will be a P/PE node in a pretty standard MPLS backbone network surrounded by Cisco 7600 routers, running OSPF, MPLS/LDP, BGP and PIM sparse-mode, no layer 2 or layer 3 VPNs whatsoever. Equal cost load-balancing over multiple 10GE links is a must. Does anyone have a recommendation for a stable JunOS version to run ? I'm currently looking at 9.5 or 9.6, I see that 10.0 came out on the 4th of November but from our experience with Cisco IOS releases we are reluctant to use the latest greatest release. For MX I'd say 9.3R4 for the conservative (we have loads of experience running this, 9.3R3+ was the first release without major showstopper bugs ON in well over a year before it, and 9.3R4 only made it better), 9.4R3 for the middle of the road (we've been running this on many routers for a few months now, only a relatively modest amount of grief and probably nothing you'd notice if you have to ask this question), 9.5R3 for the adventuresome (some major bugs in 9.5R1 but we're currently baking 9.5R3 on a couple production routers and haven't seen any issues thus far). 9.5 does have some scripting enhancements and optimizations which are noticable, so if you're big on those it might be worth a try. I don't currently have the testicular fortitude to try 9.6 outside of the lab (where all the real bugs are found :P), though I am looking forward to the ISSU support for RSVP. For the super conservative 9.2R4 is relatively light on the pfe and counter bugs (unlike earier revs of 9.2), but still has quite a few other unfixable issues until you go to 9.3+ (like netconf and commit scripts will not play together at all). I didn't used to put JUNOS in the you should really wait until R3 before you touch it category, but given recent history I don't think it's an irrational stance to take until they re-establish their track record of putting out non catastrophically broken code on a regular basis. -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) -- Andrei 2+2=5, for extremely large values of 2 ! ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab Phil Shafer wrote: Derick Winkworth writes: 9.3r4 indeed. Perhaps even 9.4r4 when that comes out. FWIW: 9.5 has a number of scripting-related features, including interactivity and remote RPC access. We've been working to ensure that scripting PRs are backported to at least 9.5. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes. //Krzysztof -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: Wednesday, 11 November, 2009 8:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab Phil Shafer wrote: Derick Winkworth writes: 9.3r4 indeed. Perhaps even 9.4r4 when that comes out. FWIW: 9.5 has a number of scripting-related features, including interactivity and remote RPC access. We've been working to ensure that scripting PRs are backported to at least 9.5. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
What did you mean by inappropriately configured ? There are the same mtu settings everywhere and traffic passes quite well. And ospf session goes up without problems. And how comes that inappropriately configured IP and MPLS MTU work well on 9.3R3.8 ? Krzysztof Szarkowicz wrote: It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes. //Krzysztof -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: Wednesday, 11 November, 2009 8:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab Phil Shafer wrote: Derick Winkworth writes: 9.3r4 indeed. Perhaps even 9.4r4 when that comes out. FWIW: 9.5 has a number of scripting-related features, including interactivity and remote RPC access. We've been working to ensure that scripting PRs are backported to at least 9.5. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
Let me guess. Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO? CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF). If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is sent. I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes. Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is terminated on CSCO on one side and JNPR on other side)? //Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 9:57 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations What did you mean by inappropriately configured ? There are the same mtu settings everywhere and traffic passes quite well. And ospf session goes up without problems. And how comes that inappropriately configured IP and MPLS MTU work well on 9.3R3.8 ? Krzysztof Szarkowicz wrote: It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes. //Krzysztof -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: Wednesday, 11 November, 2009 8:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab Phil Shafer wrote: Derick Winkworth writes: 9.3r4 indeed. Perhaps even 9.4r4 when that comes out. FWIW: 9.5 has a number of scripting-related features, including interactivity and remote RPC access. We've been working to ensure that scripting PRs are backported to at least 9.5. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
Uhm, i see your point here. We indeed have cisco - cisco - Jun - Jun setup My cisco interface mtu = ip mtu = mpls mtu =9000 But i raly doubt that bgp keepalive packet size can come close to that mtu. On Juniper i set interface mtu = cisco mtu +14 and it works fine! And! As you say, it reports different mpls mtu value: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 8988 Protocol multiservice, MTU: Unlimited As far as i understand default mpls mtu term (not sure that i _fully_ understand it though) it seems, Juniper supposes 3 labels maximum. I dont see any reasons for device to drop packets which has 1 or 2 labels and bigger than mpls mtu, but still ok from interface mtu point ov view. As per your logic, device should drop all traffic that match such criteria but it seems only bgp session keepalives and i didn't see any other problems But still, i made an experiment on Juniper and cisco which has bgp session between them. cisco: #sh mpls interfaces g 0/0 detail | i MTU MTU = 9000 #sh ip int g 0/0 | i MTU MTU is 9000 bytes #sh run int g 0/0 Building configuration... Current configuration : 212 bytes ! interface GigabitEthernet0/0 description --- to 7606-2 --- mtu 9000 ip address 10.3.13.2 255.255.255.0 load-interval 30 duplex full speed 1000 media-type gbic no negotiation auto tag-switching ip end If i set mtu 9000 under family mpls and commit it, it looks like this: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 9000 Flags: Is-Primary, User-MTU Protocol multiservice, MTU: Unlimited and problem still persists please let me know if you have any other ideas :) p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 'default' (=8988) on juniper Krzysztof Szarkowicz wrote: Let me guess. Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO? CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF). If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is sent. I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes. Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is terminated on CSCO on one side and JNPR on other side)? //Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 9:57 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations What did you mean by inappropriately configured ? There are the same mtu settings everywhere and traffic passes quite well. And ospf session goes up without problems. And how comes that inappropriately configured IP and MPLS MTU work well on 9.3R3.8 ? Krzysztof Szarkowicz wrote: It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes. //Krzysztof -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: Wednesday, 11 November, 2009 8:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
How about some debugs or traceoptions? From: Tima Maryin t...@transtelecom.net To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Sent: Wed, November 11, 2009 8:11:56 AM Subject: Re: [j-nsp] MX960 JunOS recommendations Uhm, i see your point here. We indeed have cisco - cisco - Jun - Jun setup My cisco interface mtu = ip mtu = mpls mtu =9000 But i raly doubt that bgp keepalive packet size can come close to that mtu. On Juniper i set interface mtu = cisco mtu +14 and it works fine! And! As you say, it reports different mpls mtu value: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 8988 Protocol multiservice, MTU: Unlimited As far as i understand default mpls mtu term (not sure that i _fully_ understand it though) it seems, Juniper supposes 3 labels maximum. I dont see any reasons for device to drop packets which has 1 or 2 labels and bigger than mpls mtu, but still ok from interface mtu point ov view. As per your logic, device should drop all traffic that match such criteria but it seems only bgp session keepalives and i didn't see any other problems But still, i made an experiment on Juniper and cisco which has bgp session between them. cisco: #sh mpls interfaces g 0/0 detail | i MTU MTU = 9000 #sh ip int g 0/0 | i MTU MTU is 9000 bytes #sh run int g 0/0 Building configuration... Current configuration : 212 bytes ! interface GigabitEthernet0/0 description --- to 7606-2 --- mtu 9000 ip address 10.3.13.2 255.255.255.0 load-interval 30 duplex full speed 1000 media-type gbic no negotiation auto tag-switching ip end If i set mtu 9000 under family mpls and commit it, it looks like this: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 9000 Flags: Is-Primary, User-MTU Protocol multiservice, MTU: Unlimited and problem still persists please let me know if you have any other ideas :) p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 'default' (=8988) on juniper Krzysztof Szarkowicz wrote: Let me guess. Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO? CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF). If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is sent. I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes. Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is terminated on CSCO on one side and JNPR on other side)? //Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 9:57 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations What did you mean by inappropriately configured ? There are the same mtu settings everywhere and traffic passes quite well. And ospf session goes up without problems. And how comes that inappropriately configured IP and MPLS MTU work well on 9.3R3.8 ? Krzysztof Szarkowicz wrote: It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes. //Krzysztof -Original Message- From: juniper-nsp-boun...@puck.nether.net [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of Tima Maryin Sent: Wednesday, 11 November, 2009 8:28 To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors rollback to 9.3r3 helps though JTAC still not confirmed it, but it easlily can be reprodused in lab ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper
Re: [j-nsp] MX960 JunOS recommendations
From: Tima Maryin t...@transtelecom.net Subject: Re: [j-nsp] MX960 JunOS recommendations Date: Wed, 11 Nov 2009 17:11:56 +0300 Uhm, i see your point here. We indeed have cisco - cisco - Jun - Jun setup My cisco interface mtu = ip mtu = mpls mtu =9000 But i raly doubt that bgp keepalive packet size can come close to that mtu. On Juniper i set interface mtu = cisco mtu +14 and it works fine! And! As you say, it reports different mpls mtu value: Make sure that your IP MTU is the same on both Cisco and Juniper sides. If you run IS-IS, make sure your CLNS MTU is the same on both Cisco and Juniper sides. People have been running interoperable Cisco and Juniper networks for many years. This is not rocket science. We have standardized on 4470 byte MTU. This means mtu 4470 on Cisco IOS physical Ethernet interfaces, and MTU 4484 on Juniper JunOS physical Ethernet interfaces. No need to explicitly set IP or CLNS MTU. (Also no need to specify anything at all for SDH/SONET interfaces...) Steinar Haug, Nethelp consulting, sth...@nethelp.no ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
I don't know if this will help because it has to deal with gigabit Ethernet interfaces but... If you set a Cisco device to use frame size of MTU 9000 it does not count the 18 bytes for TCP headers. However, Juniper does count the 18 bytes when you set the MTU. So if the Cisco interface is set to mtu jumbo 9000, the Juniper end must have an mtu of 9018. This only seems to apply for physical interfaces on Juniper. For logical interfaces on Juniper the MTU remains 9000. Jonathan From: kszarkow...@gmail.com To: t...@transtelecom.net Date: Wed, 11 Nov 2009 20:36:43 +0100 CC: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP, since as per RFC4271, section 4: The maximum message size is 4096 octets. All implementations are required to support this maximum message size. So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP perspective. The only thing that comes in my mind, that there are some L2 switches in between and there is something wrong with MTU on those switches. Worth to check. Could you paste from the log the Notification message generated when the BGP session is tear down? Thanks, Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 15:12 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations Uhm, i see your point here. We indeed have cisco - cisco - Jun - Jun setup My cisco interface mtu = ip mtu = mpls mtu =9000 But i raly doubt that bgp keepalive packet size can come close to that mtu. On Juniper i set interface mtu = cisco mtu +14 and it works fine! And! As you say, it reports different mpls mtu value: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 8988 Protocol multiservice, MTU: Unlimited As far as i understand default mpls mtu term (not sure that i _fully_ understand it though) it seems, Juniper supposes 3 labels maximum. I dont see any reasons for device to drop packets which has 1 or 2 labels and bigger than mpls mtu, but still ok from interface mtu point ov view. As per your logic, device should drop all traffic that match such criteria but it seems only bgp session keepalives and i didn't see any other problems But still, i made an experiment on Juniper and cisco which has bgp session between them. cisco: #sh mpls interfaces g 0/0 detail | i MTU MTU = 9000 #sh ip int g 0/0 | i MTU MTU is 9000 bytes #sh run int g 0/0 Building configuration... Current configuration : 212 bytes ! interface GigabitEthernet0/0 description --- to 7606-2 --- mtu 9000 ip address 10.3.13.2 255.255.255.0 load-interval 30 duplex full speed 1000 media-type gbic no negotiation auto tag-switching ip end If i set mtu 9000 under family mpls and commit it, it looks like this: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 9000 Flags: Is-Primary, User-MTU Protocol multiservice, MTU: Unlimited and problem still persists please let me know if you have any other ideas :) p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 'default' (=8988) on juniper Krzysztof Szarkowicz wrote: Let me guess. Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO? CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF). If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is sent. I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes. Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is terminated on CSCO on one side and JNPR on other side)? //Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 9:57
Re: [j-nsp] MX960 JunOS recommendations
I will assume that you mean ethernet headers instead of TCP headers ;) You should be fine with 9014 though as only the header is counted. The FCS shouldn't be counted in the total length at all. Scott Jonathan Call wrote: I don't know if this will help because it has to deal with gigabit Ethernet interfaces but... If you set a Cisco device to use frame size of MTU 9000 it does not count the 18 bytes for TCP headers. However, Juniper does count the 18 bytes when you set the MTU. So if the Cisco interface is set to mtu jumbo 9000, the Juniper end must have an mtu of 9018. This only seems to apply for physical interfaces on Juniper. For logical interfaces on Juniper the MTU remains 9000. Jonathan From: kszarkow...@gmail.com To: t...@transtelecom.net Date: Wed, 11 Nov 2009 20:36:43 +0100 CC: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP, since as per RFC4271, section 4: The maximum message size is 4096 octets. All implementations are required to support this maximum message size. So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP perspective. The only thing that comes in my mind, that there are some L2 switches in between and there is something wrong with MTU on those switches. Worth to check. Could you paste from the log the Notification message generated when the BGP session is tear down? Thanks, Krzysztof -Original Message- From: Tima Maryin [mailto:t...@transtelecom.net] Sent: Wednesday, 11 November, 2009 15:12 To: kszarkow...@gmail.com Cc: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX960 JunOS recommendations Uhm, i see your point here. We indeed have cisco - cisco - Jun - Jun setup My cisco interface mtu = ip mtu = mpls mtu =9000 But i raly doubt that bgp keepalive packet size can come close to that mtu. On Juniper i set interface mtu = cisco mtu +14 and it works fine! And! As you say, it reports different mpls mtu value: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 8988 Protocol multiservice, MTU: Unlimited As far as i understand default mpls mtu term (not sure that i _fully_ understand it though) it seems, Juniper supposes 3 labels maximum. I dont see any reasons for device to drop packets which has 1 or 2 labels and bigger than mpls mtu, but still ok from interface mtu point ov view. As per your logic, device should drop all traffic that match such criteria but it seems only bgp session keepalives and i didn't see any other problems But still, i made an experiment on Juniper and cisco which has bgp session between them. cisco: #sh mpls interfaces g 0/0 detail | i MTU MTU = 9000 #sh ip int g 0/0 | i MTU MTU is 9000 bytes #sh run int g 0/0 Building configuration... Current configuration : 212 bytes ! interface GigabitEthernet0/0 description --- to 7606-2 --- mtu 9000 ip address 10.3.13.2 255.255.255.0 load-interval 30 duplex full speed 1000 media-type gbic no negotiation auto tag-switching ip end If i set mtu 9000 under family mpls and commit it, it looks like this: show interfaces xe-1/0/0 | match MTU Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled, Protocol inet, MTU: 9000 Protocol mpls, MTU: 9000 Flags: Is-Primary, User-MTU Protocol multiservice, MTU: Unlimited and problem still persists please let me know if you have any other ideas :) p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 'default' (=8988) on juniper Krzysztof Szarkowicz wrote: Let me guess. Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO? CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF). If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is sent. I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes. Could you check with some show commands, what
Re: [j-nsp] MX960 JunOS recommendations
On Wed, 11 Nov 2009, sth...@nethelp.no wrote: We have standardized on 4470 byte MTU. This means mtu 4470 on Cisco IOS physical Ethernet interfaces, and MTU 4484 on Juniper JunOS physical Ethernet interfaces. No need to explicitly set IP or CLNS MTU. Except when you enable VLANs on Ethernet, you need to crank it up to 4488.. A thing to keep in mind and/or to monitor using scripts. -- Pekka Savola You each name yourselves king, yet the Netcore Oykingdom bleeds. Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
Derick Winkworth writes: 9.3r4 indeed. Perhaps even 9.4r4 when that comes out. FWIW: 9.5 has a number of scripting-related features, including interactivity and remote RPC access. We've been working to ensure that scripting PRs are backported to at least 9.5. Thanks, Phil ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
On Friday 06 November 2009 02:28:32 am Derick Winkworth wrote: There are some great new features in 10, if you haven't already read the release notes... I'm particularly loving the 'interface-range' feature, which should be great for the EX-series. Cheers, Mark. signature.asc Description: This is a digitally signed message part. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX960 JunOS recommendations
On Thu, Nov 05, 2009 at 03:39:32PM +0200, Andrei Radu wrote: Hello everybody, We will be installing our first Juniper MX960 router in the network. We were a 100% Cisco shop and this is our first Juniper router. The MX will be a P/PE node in a pretty standard MPLS backbone network surrounded by Cisco 7600 routers, running OSPF, MPLS/LDP, BGP and PIM sparse-mode, no layer 2 or layer 3 VPNs whatsoever. Equal cost load-balancing over multiple 10GE links is a must. Does anyone have a recommendation for a stable JunOS version to run ? I'm currently looking at 9.5 or 9.6, I see that 10.0 came out on the 4th of November but from our experience with Cisco IOS releases we are reluctant to use the latest greatest release. For MX I'd say 9.3R4 for the conservative (we have loads of experience running this, 9.3R3+ was the first release without major showstopper bugs ON in well over a year before it, and 9.3R4 only made it better), 9.4R3 for the middle of the road (we've been running this on many routers for a few months now, only a relatively modest amount of grief and probably nothing you'd notice if you have to ask this question), 9.5R3 for the adventuresome (some major bugs in 9.5R1 but we're currently baking 9.5R3 on a couple production routers and haven't seen any issues thus far). 9.5 does have some scripting enhancements and optimizations which are noticable, so if you're big on those it might be worth a try. I don't currently have the testicular fortitude to try 9.6 outside of the lab (where all the real bugs are found :P), though I am looking forward to the ISSU support for RSVP. For the super conservative 9.2R4 is relatively light on the pfe and counter bugs (unlike earier revs of 9.2), but still has quite a few other unfixable issues until you go to 9.3+ (like netconf and commit scripts will not play together at all). I didn't used to put JUNOS in the you should really wait until R3 before you touch it category, but given recent history I don't think it's an irrational stance to take until they re-establish their track record of putting out non catastrophically broken code on a regular basis. -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
