Re: [j-nsp] EX 8200 deployment
Hi Hoogen, I think this is just another story. SRX should have also had some more storage capacity to store IDP base and all the same things as Richard wrote about. But session logging can cause another problem — increased process switching or something like (if we talk about Branch SRX), because the session data is collected by fwdd_octeon (software forwarding and SPU daemon) when log processing an storing is done by control plane. Moreover control plane on branch SRX is run on a separate processor and is known to be ‘not that fast’ due to this. SRX 3/5k can not store session logs to flash drive at all since it can cause RE link overloading. You must use syslog exporting for this, which is done directly by SPU and does not consume RE resources. -- Regards, Pavel 2010/3/26 Hoogen hooge...@gmail.com Didn't word it right.. I meant shouldn't ... I was taking some example out from SRX.. where customers choose to log “session-init” and “session-close”, it could generates high rate of IO activity to /var/log/rtlogd. Though its not a problem logging all these; but on a compact flash when we have a life cycle of about 100k it might become an issue very soon. Do note that this may effect only event mode logs not the stream mode. -Hoogen ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Wed, Mar 24, 2010 at 10:45:07PM -0700, Hoogen wrote: I think flash isn't going to be considered... It has a finite erase/write cycles.. yeah but 8200 could have had more storage.. Erm... what do you think it uses currently, a 2GB hard drive? :) -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Flash gets a bad rap. I think most people have heard of supposed horror stories or they see the cycle limit and get wary. But I'm wondering... has anyone in this list actually had a personal flash horror story? I don't have one of my own, and I'm swimming in network devices (some quite old) that use them. Dan Farrell Applied Innovations Corp. da...@appliedi.net On Wed, Mar 24, 2010 at 10:45:07PM -0700, Hoogen wrote: I think flash isn't going to be considered... It has a finite erase/write cycles.. yeah but 8200 could have had more storage.. Erm... what do you think it uses currently, a 2GB hard drive? :) -- __ Information from ESET NOD32 Antivirus, version of virus signature database 4974 (20100325) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Excerpts from Dan Farrell's message of Thu Mar 25 09:13:59 -0700 2010:
Flash gets a bad rap. I think most people have heard of supposed horror
stories or they see the cycle limit and get wary.
But I'm wondering... has anyone in this list actually had a personal flash
horror story? I don't have one of my own, and I'm swimming in network devices
(some quite old) that use them.
I've definitely observed wearing out older multi-layer flash chips, but
every modern (in the last several years) flash device I've run across
implements some sort of damaged cell management on the chips controller.
If you're careful about how you access the device (mounting filesystems
with no atime, no heavy logging, etc.), I'm convinced that modern flash
works just fine in an embedded application like this.
That being said, I think Richard is right regarding adding expandable
flash.
Flash is so cheap and constantly developing, it seems like a no
brainer to just eat the cost of adding a small controller-on-a-chip,
some discrete components, and a CF slot to future-proof the storage.
In looking at the EX platforms though, this doesn't seem in line with
Juniper's design goals though (not that I actually know what they
planned). It seems like most of the hardware ('cept the EX-8200) comes
in a fixed configuration -- stuff that's just supposed to work, and
not to worry the manuf. with compatibility concerns.
If you're feeling gutsy and want to void any warranties, you might try
de-soldering and replacing the internal flash :)
Cheers,
jof
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On 10-03-25 9:51 AM, Jonathan Lassoff j...@thejof.com wrote:
Excerpts from Dan Farrell's message of Thu Mar 25 09:13:59 -0700 2010:
Flash gets a bad rap. I think most people have heard of supposed horror
stories or they see the cycle limit and get wary.
But I'm wondering... has anyone in this list actually had a personal flash
horror story? I don't have one of my own, and I'm swimming in network devices
(some quite old) that use them.
...
In looking at the EX platforms though, this doesn't seem in line with
Juniper's design goals though (not that I actually know what they
planned). It seems like most of the hardware ('cept the EX-8200) comes
in a fixed configuration -- stuff that's just supposed to work, and
not to worry the manuf. with compatibility concerns.
They do allow the mounting of a USB flash device. Of course, the usual
admonishment about using Juniper USB devices, but you can mount a fat32
formatted USB key:
* Enter the shell as root:
u...@switch start shell user root
Password:
r...@switch%
* Mount USB to /mnt
r...@switch% mount_msdosfs /dev/da1s1 /mnt
* Check the contents in USB disk
r...@switch% ls /mnt
juniper.conf.1.gz juniper.conf.3.gz rescue.conf.gz
juniper.conf.2.gz juniper.conf.gz
* Unmount usb disk and then pull it out.
u...@switch% umount /mnt
http://kb.juniper.net/KB12880
http://kb.juniper.net/KB12022
USB keys are cheap too - cheap enough to replace as part of yearly
maintenance.
As usual, YMMV - some USB keys are better than others, and I haven't tried
any larger than 4G.
If you're feeling gutsy and want to void any warranties, you might try
de-soldering and replacing the internal flash :)
Heh. Sounds like fun - maybe with a unit once it gets older/off
warranty/maintenance.
Take care
Brian Fitzgerald
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Didn't word it right.. I meant shouldn't ... I was taking some example out from SRX.. where customers choose to log “session-init” and “session-close”, it could generates high rate of IO activity to /var/log/rtlogd. Though its not a problem logging all these; but on a compact flash when we have a life cycle of about 100k it might become an issue very soon. Do note that this may effect only event mode logs not the stream mode. -Hoogen On Wed, Mar 24, 2010 at 11:54 PM, Richard A Steenbergen r...@e-gerbil.netwrote: On Wed, Mar 24, 2010 at 10:45:07PM -0700, Hoogen wrote: I think flash isn't going to be considered... It has a finite erase/write cycles.. yeah but 8200 could have had more storage.. Erm... what do you think it uses currently, a 2GB hard drive? :) -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Thu, Mar 25, 2010 at 09:13:59AM -0700, Dan Farrell wrote: Flash gets a bad rap. I think most people have heard of supposed horror stories or they see the cycle limit and get wary. But I'm wondering... has anyone in this list actually had a personal flash horror story? I don't have one of my own, and I'm swimming in network devices (some quite old) that use them. I've seen dozens of old RE-2.0s with CFs that have died over time, but I'm 99% certain there was no effort made to do wear leveling or bad block detection and avoidance on those things. :) The ironiy is that they were really put in as backup because nobody trusted spinning media in a router, go figure. -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Thu, Mar 25, 2010 at 09:51:20AM -0700, Jonathan Lassoff wrote:
In looking at the EX platforms though, this doesn't seem in line with
Juniper's design goals though (not that I actually know what they
planned). It seems like most of the hardware ('cept the EX-8200) comes
in a fixed configuration -- stuff that's just supposed to work, and
not to worry the manuf. with compatibility concerns.
If you're feeling gutsy and want to void any warranties, you might try
de-soldering and replacing the internal flash :)
8200 is fixed configuration too.
da0 at umass-sim0 bus 0 target 0 lun 0
da0: ST ST72682 2.10 Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 2000MB (4096000 512 byte sectors: 255H 63S/T 254C)
Same device as on EX3200/4200, just bigger.
Looks to be an embedded USB-Flash controller with non-modular flash.
http://www.st.com/stonline/books/pdf/docs/12029.pdf
I've taken a soldering iron to many a router and switch in my day to
correct design flaws, but I don't think that will work out here. Those
5mm USB flash units stuffed into the usb port on the front seem to be
the best option for making something that at least won't get bumped or
stolen in the datacenter, but I can't find them shipping yet either.
http://www.engadget.com/2009/06/24/buffalos-16gb-5mm-usb-thumbkey-its-really-small/
Then again, I have some promotional Cisco USB drives that might look
good sticking out of the box like a giant wart too. :)
--
Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
2010/3/22 Richard A Steenbergen r...@e-gerbil.net
But what happens when you do:
interface xe-1/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 1.2.3.4/24;
}
}
}
interface xe-2/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 2.3.4.5/24;
}
}
}
Commit check doesn't error on it at any rate, but does this share
packets within a vlan 101 space automatically, or not?
The outgoing iface for this piece of config will be chosen using the dst mac
address of the next hop, which is behind a particular interface. There is no
L2 switching here at all. I can confirm EX3/4200 allow to do so.
The main difference of EXs is that they only can do one lookup. E. g. either
L3 or L2, not both. So you can not mix families for units on the same
physical port.
However if you do inet on the port, you don't need to care if any other
ports do ether-switching for the same vlan or something. That said, it seems
like you don't need to be aware of all that VLAN significance cauchemar as
poor CCIE candidates :)
A good workaround (not counting doubled port consumption, delay and the
cable and SFPs cost) you can split the L2 and L3 lookups into two steps with
a hairpin as Alexandre proposed. It is also a way to convert input policiers
to output (thanks, Alexandre!).
But I am not sure about counters. No way if you want to count packets routed
through RVI on a VLAN sprung across multiple trunk ports.
Richard, one more thing. What do you do with the crash dumps untarzipping
them on the router/switch itself? I have never done anything with them but
sending to JTA. I believe it can have a lot of sense to pick them and
discover yourself (though I've never tried), but why on the switch itself?
Am I missing something important?
--
Kind regards,
Pavel
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Thu, Mar 25, 2010 at 12:31:15AM +0300, Pavel Lunin wrote: Richard, one more thing. What do you do with the crash dumps untarzipping them on the router/switch itself? I have never done anything with them but sending to JTA. I believe it can have a lot of sense to pick them and discover yourself (though I've never tried), but why on the switch itself? Am I missing something important? You can run gdb on the coredump files locally and get a pretty good idea of what blew up and where, which is often quite helpful in working around the original problem. Also, JTAC is far too often surprisingly bad at working with coredumps, and without the ability to independently verify things myself and tell them they were confused I've had some cases which would probably never have been solved. The story that was explained to me was that JTAC has some point and click tool that they load the core into, which parses it and searches their PR database to find matching backtraces. The problem is I'm convinced at this point nobody in JTAC actually knows what a backtrace is or how to read it, they just match it to whatever their tool tells them, and surprisingly often their tool is very very wrong. The other big problem of course is file size and compression. Apparently their tool only works with .zip files not .tgz files (which is a small bit of a problem, seeing as how the router only has gzip :P), so they have to uncompress it locally first before they can load it. I've had JTAC not know what a .tgz file was, I've had Advanced JTAC spend days trying to figure out why they couldn't get any data out of a coredump when the problem turned out to be their local filesystem quota wasn't big enough to work with a large core file, etc, etc. Even when things work right it seems to take them 12-72 hours to parse a coredump even on a p1 case, and a healthy percentage of the time their analysis is just flat out wrong. Without the ability to look at the dump yourself, you'd never know they were barking up the wrong tree. Because EX uses PowerPC, it isn't even particularly easy to find a FreeBSD ppc box where you can actually do any useful analysis of the coredumps. That assumes of course that you have working connectivity on the box in question and can quickly copy the sometimes very large files off, which due to the original problem that caused the crash is often times not the case. And where do they plan on writing a 2GB core dump when there is an EX kernel panic and you only have 600MB of free space on an empty box? You can bet there will be, I run into them at least 2 or 3 times a year on MX easily, it's just a fact of life. I mean seriously what does 32GB of flash cost, $100? Think about the amount of grief that will be caused by this in comparison, and tell me it was a smart move on their part. :) -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
I really appreciate all for their inputs. Thanks a lot.
Is there any caveat in RTG, Can we easily get rid of STP running?? do you
recommend it or not??
Is there any special socket required for powering this chassis up?? as we
need industrial sockets in case of Cisco 6500.
regards,
Muhammad Fahad Khan
JNCIP - M/T # 834
IT Specialist
Global Technology Services, IBM
fa...@pk.ibm.com
+92-321-2370510
+92-301-8247638
Skype: fahad-ibm
http://www.linkedin.com/in/muhammadfahadkhan
http://fahad-internetworker.blogspot.com
http://www.visualcv.com/g46ptnd
On Tue, Mar 23, 2010 at 4:16 AM, Richard A Steenbergen r...@e-gerbil.netwrote:
On Mon, Mar 22, 2010 at 11:35:19PM +0300, Alexandre Snarskii wrote:
EX-series (at least [34]200) has the same local vlan significance
principle that applies, for example, to OSM-equipped 6500/Sup2:
you can create chassis-wide vlan, and it will be used on all LAN
cards, but you still can reuse the same vlan id on OSM subinterface,
and the idea is actually stolen from some old recipe on how to run
6500/sup2 Vlan as a part of VRF.
In case of ex-series it's even better - there are no 'internal vlan'
allocation that happens in case of 65xx/76xx.
That is indeed a fair bit better than the 6500/7600 vlan model, I guess
EX's vlan support isn't quite as bad as I thought. I swear I tested this
on EX4200 when we first got one (2 years ago) and I have a very strong
memory of this behavior not working this way, but damned if I can find
the emails with the test results to prove it.
On 6500, when you do something like this:
interface TenGigabitEthernet1/1.101
encapsulation dot1Q 101
ip address 1.2.3.4 255.255.255.0
It simply creates vlan 101 as an internal vlan, which consumes vlan 101
across the entire chassis and blocks the creation of another vlan 101
anywhere else. Of course if you didn't do a subinterface but simply
slapped an IP directly on the physical interface, it would simply pick a
pseudo-random vlan ID to use, like so:
crisco6509#sh vlan internal usage
VLAN Usage
901 TenGigabitEthernet8/2.901
910 TenGigabitEthernet4/2.910
1606 TenGigabitEthernet8/2.1606
2201 TenGigabitEthernet8/2.2201
4032 TenGigabitEthernet3/4
4033 TenGigabitEthernet3/3
4034 TenGigabitEthernet3/2
4035 TenGigabitEthernet3/1
...
So... I'm wondering how much of this counter issue is really a hardware
limitation, and how much is just design limitation. For example, would
it be possible for Juniper to implement ethernet switching as
essentially a multi-port ccc, like so:
interfaces {
xe-1/0/0 {
vlan-tagging;
unit 101 {
family inet {
address 1.2.3.1/24;
}
}
unit 201 {
vlan-id 201;
family ethernet-switching;
}
}
xe-2/0/0 {
vlan-tagging;
unit 101 {
family inet {
address 2.3.4.1/24;
}
}
unit 201 {
vlan-id 201;
family ethernet-switching;
}
}
}
vlans {
blah {
interface {
xe-1/0/0.201;
xe-2/0/0.201;
}
}
}
To me this seems like a much more natural way of handling mixed L2 and
L3 configs on a single port anyways, and it could potentially let you
have everything on a port which could be properly counted. Extra bonus
points if there was a way to strip the vlan tag before putting it into
the multi-port ccc so you could do vlan translation, but I don't know
if that is possible in hardware (there is certainly no input-vlan-map to
pop the vlan like on MX/etc, but this will be a problem when they get
around to implementing mpls l2circuits).
The funny thing about the above configuration is that it doesn't seem to
be complaining about the lack of a vlan-id under vlan blah, only about
the mixing vlan-tagging and family ethernet-switching. :)
Now say I took the above scenario and made it:
vlans {
blah {
interface {
xe-1/0/0.201;
xe-2/0/0.201;
...
}
l3-interface vlan.201;
}
}
Today they don't have working counters on vlan.201, and Juniper claims
it is a hardware limitation they can't solve without some hackery like
firewall filter counters applied to each interface, but... If I can get
xe-1/0/0.101 counters today, could I also get xe-1/0/0.201 counters in
the above configuration? Possibly those could be added up internally to
make a virtual vlan.201 counter, but worst case I could definitely do
the additionl on my side post-SNMP collection.
--
Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Tore Anderson wrote: * Richard A Steenbergen Correct. I actually found some old gripes about this when I searched j-nsp after noticing the problem, but it is a big enough issue that I think it needs to be repeated again (and again and again, until it gets fixed :P). I'll be happy to join the choir on this one. I reported the problem back in March 2009, got PR 435648 opened, but haven't heard anything more since then. My workaround is to terminate the customer VLANs that needs counters for accounting purposes on the EX-es' upstream routers instead, which are MX-es with standard vlan-tagged family inet sub-interfaces. That works well enough but it's not as tidy as I would have preferred. Best regards, chiming in here - we need to do the same, and we also ended up terminating L3 on our MXes when we'd really want to do it on the 8200s, yet the stupid messed up RVI counters don't allow us to do so. Oh, there's more, btw. You can't do vlan members [ 2-2999 ] on a trunk port if you don't use ALL of those vlans. It won't commit, which means you have to use vlan members all, which in turn messes up your one-interface-RVIs (that you have to use because you need to trunk some vlans on that port in addition to the family inet RVI)... gah. Promising boxes, but they really messed up some points by looking to closely at how vendors B C are doing it. That being said, we do use them in the data center, but they'd better get around fixing these things. Kind regards, Felix -- Felix Schüren Head of Network --- Host Europe GmbH - http://www.hosteurope.de Welserstraße 14 - 51149 Köln - Germany Telefon: 0800 467 8387 - Fax: +49 180 5 66 3233 (*) HRB 28495 Amtsgericht Köln - USt-IdNr.: DE187370678 Geschäftsführer: Uwe Braun - Alex Collins - Mark Joseph - Patrick Pulvermüller (*) 0,14 EUR/Min. aus dem dt. Festnetz; maximal 0,42 EUR/Min. aus den dt. Mobilfunknetzen ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Mon, Mar 22, 2010 at 02:16:36PM -0500, Richard A Steenbergen wrote:
protocols {
connections {
interface-switch test {
interface xe-1/0/0.101;
interface xe-1/0/1.101;
}
}
}
Well for everyone woh asked, I tried the following on an EX8208 under
10.1R1, and it didn't work:
interfaces {
ae0 {
vlan-tagging;
mtu 9216;
...
unit 69 {
vlan-id 69;
family ccc;
}
}
ae2 {
vlan-tagging;
mtu 9216;
...
unit 69 {
vlan-id 69;
family ccc;
}
}
}
protocols {
connections {
interface-switch test {
interface ae0.69;
interface ae2.69;
}
}
}
Which of course errors because MPLS is still required to use CCC (who
knows why that has never been fixed :P):
r...@router# commit check
re1:
[edit protocols connections]
'interface-switch test'
CCC: Connection test error MPLS must be enabled to use CCC
error: configuration check-out failed
So turn on something in protocol MPLS to shut it up, which of course
turns on the log-filling no license nag (even though I do have an
advanced feature license, and the documentation says MPLS is included in
the AFL, go figure):
Mar 23 18:27:42.298 re1.router alarmd[734]: %DAEMON-4: Alarm cleared:
License color=YELLOW, class=CHASSIS, reason=Multi Protocol Label
Switching usage requires a license
And now the ccc looks like it should be up:
Connection/CircuitTypeSt Time last up # Up
trans
test if-sw Up Mar 23 18:18:55
1
ae0.69intf Up
ae2.69intf Up
# Paths
Time EventInterface/LabelRcv Xmt
Mar 23 18:18:56 CCC status update
Mar 23 18:18:55 Interface up ae2.69
Mar 23 18:18:55 Interface up ae0.69
Mar 23 18:18:55 CCC status update
Mar 23 18:18:55 Interface up ae2.69
Mar 23 18:18:55 Interface up ae0.69
But no packets are passed through it, and the interface counters for
both sides show 0 packets received. Of course the correct way to
configure this on every other platform would be vlan-ccc rather than
just ccc, but on EX there is no vlan-ccc option and the ccc config
commits without error.
--
Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Mon, Mar 22, 2010 at 05:31:38PM +0300, Alexandre Snarskii wrote:
I suppose you can use good old hairpin cable trick to have both
egress policers (converted to ingress ones on switched side of
hairpin) and counters on Vlan's (actually on subinterfaces on routed
side). Not checked with ex-82xx, but it works for ex-[34]200.
I'm trying to picture the exact configuration you're talking about, but
I'm not sure I get it. If you hairpined a trunk port, wouldn't you still
have to configure the layer 2 vlans on the other side to do anything
with them, and wouldn't they then be the same vlans as the originals?
I was waiting on some more lab EX's to show up before I started playing
with this further, but I suppose I might as well ask here and see if
someone else wants to test it... What happens when you configure the
same vlan-id under two different interfaces? For example, we know that
the counters for multiple subinterfaces work correctly like this:
interface xe-1/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 1.2.3.4/24;
}
}
}
But what happens when you do:
interface xe-1/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 1.2.3.4/24;
}
}
}
interface xe-2/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 2.3.4.5/24;
}
}
}
Commit check doesn't error on it at any rate, but does this share
packets within a vlan 101 space automatically, or not? Or were you
saying that when you do a subinterface style it doesn't actually use the
vlan chassis-wide like it would if you did this subinterface style
config on a 6509 for example, and you were proposing this:
interface xe-1/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 1.2.3.4/24;
}
}
}
interface xe-2/0/0 {
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members VLAN101;
}
}
}
}
vlans {
VLAN101 {
vlan-id 101;
}
}
With a hairpin between xe-1/0/0 and xe-2/0/0, and then you could use
VLAN101 in whatever other configuration you wanted while still using
xe-1/0/0.101 for the counting?
And if the above is true, can someone test as an alternative to family
ethernet-switching:
interface xe-1/0/0 {
vlan-tagging;
unit 101 {
family ccc;
}
}
interface xe-1/0/1 {
vlan-tagging;
unit 101 {
family ccc;
}
}
protocols {
connections {
interface-switch test {
interface xe-1/0/0.101;
interface xe-1/0/1.101;
}
}
}
--
Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Mon, Mar 22, 2010 at 02:16:36PM -0500, Richard A Steenbergen wrote:
On Mon, Mar 22, 2010 at 05:31:38PM +0300, Alexandre Snarskii wrote:
I suppose you can use good old hairpin cable trick to have both
egress policers (converted to ingress ones on switched side of
hairpin) and counters on Vlan's (actually on subinterfaces on routed
side). Not checked with ex-82xx, but it works for ex-[34]200.
I'm trying to picture the exact configuration you're talking about, but
I'm not sure I get it. If you hairpined a trunk port, wouldn't you still
have to configure the layer 2 vlans on the other side to do anything
with them, and wouldn't they then be the same vlans as the originals?
EX-series (at least [34]200) has the same local vlan significance
principle that applies, for example, to OSM-equipped 6500/Sup2:
you can create chassis-wide vlan, and it will be used on all LAN
cards, but you still can reuse the same vlan id on OSM subinterface,
and the idea is actually stolen from some old recipe on how to run
6500/sup2 Vlan as a part of VRF.
In case of ex-series it's even better - there are no 'internal vlan'
allocation that happens in case of 65xx/76xx.
Or were you saying that when you do a subinterface style it doesn't
actually use the vlan chassis-wide like it would if you did this
subinterface style config on a 6509 for example, and you were
proposing this:
Yes, you got the idea.
interface xe-1/0/0 {
vlan-tagging;
unit 101 {
vlan-id 101;
family inet {
address 1.2.3.4/24;
}
}
}
interface xe-2/0/0 {
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members VLAN101;
}
}
}
}
vlans {
VLAN101 {
vlan-id 101;
}
}
With a hairpin between xe-1/0/0 and xe-2/0/0, and then you could use
VLAN101 in whatever other configuration you wanted while still using
xe-1/0/0.101 for the counting?
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Mon, Mar 22, 2010 at 11:35:19PM +0300, Alexandre Snarskii wrote:
EX-series (at least [34]200) has the same local vlan significance
principle that applies, for example, to OSM-equipped 6500/Sup2:
you can create chassis-wide vlan, and it will be used on all LAN
cards, but you still can reuse the same vlan id on OSM subinterface,
and the idea is actually stolen from some old recipe on how to run
6500/sup2 Vlan as a part of VRF.
In case of ex-series it's even better - there are no 'internal vlan'
allocation that happens in case of 65xx/76xx.
That is indeed a fair bit better than the 6500/7600 vlan model, I guess
EX's vlan support isn't quite as bad as I thought. I swear I tested this
on EX4200 when we first got one (2 years ago) and I have a very strong
memory of this behavior not working this way, but damned if I can find
the emails with the test results to prove it.
On 6500, when you do something like this:
interface TenGigabitEthernet1/1.101
encapsulation dot1Q 101
ip address 1.2.3.4 255.255.255.0
It simply creates vlan 101 as an internal vlan, which consumes vlan 101
across the entire chassis and blocks the creation of another vlan 101
anywhere else. Of course if you didn't do a subinterface but simply
slapped an IP directly on the physical interface, it would simply pick a
pseudo-random vlan ID to use, like so:
crisco6509#sh vlan internal usage
VLAN Usage
901 TenGigabitEthernet8/2.901
910 TenGigabitEthernet4/2.910
1606 TenGigabitEthernet8/2.1606
2201 TenGigabitEthernet8/2.2201
4032 TenGigabitEthernet3/4
4033 TenGigabitEthernet3/3
4034 TenGigabitEthernet3/2
4035 TenGigabitEthernet3/1
...
So... I'm wondering how much of this counter issue is really a hardware
limitation, and how much is just design limitation. For example, would
it be possible for Juniper to implement ethernet switching as
essentially a multi-port ccc, like so:
interfaces {
xe-1/0/0 {
vlan-tagging;
unit 101 {
family inet {
address 1.2.3.1/24;
}
}
unit 201 {
vlan-id 201;
family ethernet-switching;
}
}
xe-2/0/0 {
vlan-tagging;
unit 101 {
family inet {
address 2.3.4.1/24;
}
}
unit 201 {
vlan-id 201;
family ethernet-switching;
}
}
}
vlans {
blah {
interface {
xe-1/0/0.201;
xe-2/0/0.201;
}
}
}
To me this seems like a much more natural way of handling mixed L2 and
L3 configs on a single port anyways, and it could potentially let you
have everything on a port which could be properly counted. Extra bonus
points if there was a way to strip the vlan tag before putting it into
the multi-port ccc so you could do vlan translation, but I don't know
if that is possible in hardware (there is certainly no input-vlan-map to
pop the vlan like on MX/etc, but this will be a problem when they get
around to implementing mpls l2circuits).
The funny thing about the above configuration is that it doesn't seem to
be complaining about the lack of a vlan-id under vlan blah, only about
the mixing vlan-tagging and family ethernet-switching. :)
Now say I took the above scenario and made it:
vlans {
blah {
interface {
xe-1/0/0.201;
xe-2/0/0.201;
...
}
l3-interface vlan.201;
}
}
Today they don't have working counters on vlan.201, and Juniper claims
it is a hardware limitation they can't solve without some hackery like
firewall filter counters applied to each interface, but... If I can get
xe-1/0/0.101 counters today, could I also get xe-1/0/0.201 counters in
the above configuration? Possibly those could be added up internally to
make a virtual vlan.201 counter, but worst case I could definitely do
the additionl on my side post-SNMP collection.
--
Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On 21/03/10 02:03, Richard A Steenbergen wrote: We just deployed our first EX8208 a few days ago, running 10.1R1. Gotchas so far: * Obviously this is a very different architecture from Juniper's normal boxes, so be prepared for vlan space being shared across the entire box, not a per-interface basis. So far, apart from the MX I'm not aware of any Juniper gear that does switching with multiple VLAN spaces. * In a move straight out of Foundry's playbook of how to fail at making a useable product, EX has no packet counters (cli or snmp) available for L3 vlan interfaces. It DOES have working counters if you do traditional Juniper subinterface style vlans (interface blah, vlan-tagging, unit 123 vlan-id 123), but it does NOT work if you have to do RVI style (vlan blah l3-interface vlan.123 and then put vlan blah in an ethernet switching interface). Subinterface style is my preference anyways, so as long as you only ever use vlans on point-to-point links this isn't a problem, but the instant you need to put a VLAN on more than one port you no longer get packet counters. Thank you for doing the testing on this, I was assuming this was a bug as I'd thought they couldn't be *that* stupid. To make things worse counters for vlan.XXX traffic are also only the traffic destined *to* the interface, not counting traffic routed *through*. * Related to the issue above, you can't mix subinterface style and RVI style vlans on the same trunk port. The instant you need to do anything more than classic subinterface style vlans, you have to convert everything on the trunk to vlan/rvi style. For example, where I might otherwise be able to get away with doing interface xe-1/0/0 unit 123 vlan-id 123 family inet blah, if I want to trunk a layer 2 vlan on that same interface I now have to convert unit 123 to RVI style. One possible workaround I have yet to test is doing a CCC instead of a vlan, to keep the subinterface style. This would only work with 2-port member vlans though, and I have yet to test the implications for mixing tagged and untagged ports on EX, so this may not actually work for anyone at all. Either way please post. * Firewall filters are still a bit of a mess. You can't count or log anything, you can't use policers on either control plane or egress filters (heck you can't even commit a firewall filter with a policer in it if applied as an output filter), you can't match frags, etc, etc. Lack of outbound policers also makes it fairly useless in many roles where enforcing max bandwidth on a WAN link is required (At least here in Australia carriers complain if you actually dump 100Mbit of traffic on a 100Mbit point-to-point link). * I don't know who thought 2GB of storage on an RE was sufficient, but it isn't. The best idea I've come up with so far is to grab some small USB flash devices like http://www.geckoandfly.com/tag/small-usb/ and deploy them on every RE so you have a little bit of working space. I've only just upgraded a bunch of stuff *to* 2GB, and don't have any real space issues. I would very much appreciate if Juniper would just give us two, externally accessible CF slots for storage and have that be it. Other than that we haven't found any fundamental flaws in the box yet (though that may change by the time MPLS features get implemented :P). Plenty of bugs to be sure, DOM isn't working right on any of our interfaces, pfe statistics don't work right, monitor interface on vlans isn't displaying correctly, prior to 10.1 the FPCs crashed if you tried to speak BGP flowspec to the box, etc, but we have cases open on all of the above. IMHO it definitely has the potential to be a very good box in the long term, but whoever didn't think to put vlan counters into the hardware really screwed the pooch something fierce. :) So the EX (4200) bits from my personal list: * EX4200 - bootp relay doesn't work when configured inside a routing-instance, works when configured at top to use an instance * The commands in http://kb.juniper.net/index?page=contentid=KB13206cat=JUNOS_EXactp=LIST don't exist in 9.5 I'm mostly on 10.0R2.10, but all our EX's are still 9.5. -- Julien Goodwin Studio442 Blue Sky Solutioneering signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Hi, ..straying a bit off topic.. On Sun, Mar 21, 2010 at 6:08 PM, Julien Goodwin jgood...@studio442.com.au wrote: So the EX (4200) bits from my personal list: * EX4200 - bootp relay doesn't work when configured inside a routing-instance, works when configured at top to use an instance Got bitten by this one a couple of weekends ago, during a big roll-out. Very counter-intuitive 'fix'. We had: set routing-instances blah-vr forwarding-options helpers bootp interface vlan.600 server 172.13.40.9 We actually needed: set forwarding-options helpers bootp interface vlan.600 server 172.13.40.9 routing-instance blah-vr cheers, Dale ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Sun, Mar 21, 2010 at 06:08:33PM +1100, Julien Goodwin wrote: * Obviously this is a very different architecture from Juniper's normal boxes, so be prepared for vlan space being shared across the entire box, not a per-interface basis. So far, apart from the MX I'm not aware of any Juniper gear that does switching with multiple VLAN spaces. That's not exactly correct. For all intents and purposes MX is a much cheaper ethernet-optimized T640, with an extra layer of stuff added to let it do ethernet switching between some interfaces. Under that layer of stuff though, it is still the same basic architecture as a T-series, in which every interface has its own totally unique vlan space. On a M/T series you didn't have the ethernet switching component, but that has nothing to do with the vlan id space. This is completely and totally different from the architecture of a layer 3 switch (like a 6509, Nexus, Foundry, Force10, etc) which is at its core a layer 2 ethernet switch, with some stuff glued on to make it do routing. In a layer 3 switch the vlan space is shared globally across the box just like in a layer 2 switch, and when you want to route on an interface what you're actually doing is secretly stealing one of those 4096 box-wide vlans, using some hacks to do routing on it, and applying it as an access mode vlan to a single interface. The ironic part of the whole thing is that, as far as I understand at any rate, EX isn't actually a layer 3 switch architecture like the rest of those boxes. In Juniper's rush to get into the enterprise switch market, it seems like they decided to copy the other enterprise-marketed switches out there, bad designs and all. At the end of the day it doesn't really matter, most of us are comparing this to similar boxes in the market segment which have the same limitations (Nexus 7k, etc), it's just something people coming from other Juniper products need to know. Thank you for doing the testing on this, I was assuming this was a bug as I'd thought they couldn't be *that* stupid. To make things worse counters for vlan.XXX traffic are also only the traffic destined *to* the interface, not counting traffic routed *through*. Correct. I actually found some old gripes about this when I searched j-nsp after noticing the problem, but it is a big enough issue that I think it needs to be repeated again (and again and again, until it gets fixed :P). At one point I work working on an sflow to snmp emulator for some of my poor Foundry-using friends who can't bill customers landing on vlans, may end up having to dust that off as a workaround for the EX design flaw. Lack of outbound policers also makes it fairly useless in many roles where enforcing max bandwidth on a WAN link is required (At least here in Australia carriers complain if you actually dump 100Mbit of traffic on a 100Mbit point-to-point link). I believe this is just a current limitation of the software, not a flaw in the design of the hardware, so it should be coming soon. Again, just something people need to be aware of, since as you say it can be a big problem if you need those features. :) I've only just upgraded a bunch of stuff *to* 2GB, and don't have any real space issues. I would very much appreciate if Juniper would just give us two, externally accessible CF slots for storage and have that be it. I'm talking about the EX8200 here, which has 2GB of DRAM, not a stackable box. When the thing crashes, where do you plan to write the kernel panic file? I wasn't even able to work with my 512mb rpd coredump, not enough free space to uncompress the tarball. Maybe you aren't a power user and you don't notice the issue right now, but trust me this is a setup for a lot of problems in the future. So the EX (4200) bits from my personal list: * EX4200 - bootp relay doesn't work when configured inside a routing-instance, works when configured at top to use an instance * The commands in http://kb.juniper.net/index?page=contentid=KB13206cat=JUNOS_EXactp=LIST don't exist in 9.5 I'm mostly on 10.0R2.10, but all our EX's are still 9.5. I'm more interested in the 8200 than the 4200, so we haven't done that much interesting testing on the 4200, but what I can tell (for the sake of the mailing list archives) you is that the 3200/4200 and 8200 are two different revisions of the same family of ASICs, with different feature supported by the hardware, and different levels of support in software for those two different revisions of hardware. What 3200/4200 does is not necessarily the same as what 8200 does. -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
* Richard A Steenbergen Correct. I actually found some old gripes about this when I searched j-nsp after noticing the problem, but it is a big enough issue that I think it needs to be repeated again (and again and again, until it gets fixed :P). I'll be happy to join the choir on this one. I reported the problem back in March 2009, got PR 435648 opened, but haven't heard anything more since then. My workaround is to terminate the customer VLANs that needs counters for accounting purposes on the EX-es' upstream routers instead, which are MX-es with standard vlan-tagged family inet sub-interfaces. That works well enough but it's not as tidy as I would have preferred. Best regards, -- Tore Anderson Redpill Linpro AS - http://www.redpill-linpro.com/ Tel: +47 21 54 41 27 ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
So the EX (4200) bits from my personal list: * EX4200 - bootp relay doesn't work when configured inside a routing-instance, works when configured at top to use an instance Got bitten by this one a couple of weekends ago, during a big roll-out. Very counter-intuitive 'fix'. We had: set routing-instances blah-vr forwarding-options helpers bootp interface vlan.600 server 172.13.40.9 We actually needed: set forwarding-options helpers bootp interface vlan.600 server 172.13.40.9 routing-instance blah-vr This is *not* specific to EX, the same applies to the M series. However, the Extended DHCP Relay functionality (forwarding-options dhcp-relay) needs to be configured under the routing-instance. Isn't consistency wonderful? Steinar Haug, Nethelp consulting, sth...@nethelp.no ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Everyone, you do realize that the EX switches were designed to get Juniper into the enterprise switching market (although poorly). This is what they are doing, Don't expect features of the MX to show up. As of the current software offering the EX 8200 isn't even up to par with a 6500 feature wise yet. While its not the best at everything, the 6500 is a swiss army knife. Its going to take quite a bit of time to be at the same level. Then also take the fact that the 6500 is dead as a strategic platform. The Nexus 7k is the new boy on the block. Juniper needs to push features and hardware faster to keep up, yet sadly I don't think they're doing it. --Original Message-- From: Stefan Fouant Sender: juniper-nsp-boun...@puck.nether.net To: mti...@globaltransit.net To: juniper-nsp@puck.nether.net Cc: Richard A Steenbergen ReplyTo: sfou...@shortestpathfirst.net Subject: Re: [j-nsp] EX 8200 deployment Sent: Mar 20, 2010 11:27 PM Chassis-wide VLAN space? Great!... Juniper just managed to build a Cisco 6509 ;) Stefan Fouant Sent from my Verizon Wireless BlackBerry -Original Message- From: Mark Tinka mti...@globaltransit.net Date: Sun, 21 Mar 2010 03:23:41 To: juniper-nsp@puck.nether.net Cc: Richard A Steenbergenr...@e-gerbil.net Subject: Re: [j-nsp] EX 8200 deployment ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp Sent via BlackBerry by ATT ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Yes, you are correct. The Cat6500 has some new hardware coming to it, however it's still going to be the Cat6500 at the end of the day. Still using IOS, still integrating with older line cards, etc.. The Cat6500 has served my company for years, we have thousands in production ranging from Sup1a to the Sup720 series, CatOS to IOS, etc.. While it has served us, along with bringing many headaches, it's time to move on to the new age. You are right though, currently we're only replacing out Cat6500's in the data centers as we just cannot get the reliability, 10GigE density and support for CEE/DCB (future 7K cards) that the N7K has/will bring.. The fact that we can do ISSU is huge for us.. I believe the EX8200 is on track to have ISSU support in 2011. However in the corp environment, we're still using Cat6500's. I honestly would like to see us start to use stackables. The access tier is a commodity anymore, there is no reason to have a high dollar box. We completed a RFP late last year and we reviewed Brocade(Foundry), Cisco and Juniper's latest offering. We found that the EX8200 didn't have some critical features that our 6500's have, so it was almost an immediate no. I wish somehow Juniper would realize this and put more resources into the EX series. It has TONS of potential, just little pushing on the product side.. Cisco is going to release the 2nd gen of Nexus 5K later this year. Cisco is also releasing a whole new line of modules for the 7K, new fabric extenders, etc.. That is just for the Nexus series. As you stated they're bringing new hardware to the Cat6500 as well. Juniper doesn't have any new modules for the EX8200 series, I've heard rumor of a high density 10Gig module, but it seems to be vaporware. No plans (as of now) to support CEE/DCB. The EX4500 platform is barely in beta stage.. The list goes on, they're moving too slow! Very FRUSTRATING! Currently we only use Juniper products for SSLVPN, Internet edge and some other minor roles, I'd like to see them have more of a play in our environment. How can I bring another switch vendor into my environment when the new box doesn't do what I have now and will be outdated in a short time frame. It's a very hard thing to justify. On Sun, Mar 21, 2010 at 10:15 AM, Mark Tinka mti...@globaltransit.netwrote: On Sunday 21 March 2010 08:52:54 pm chrisccnpsp...@gmail.com wrote: Then also take the fact that the 6500 is dead as a strategic platform. The Nexus 7k is the new boy on the block. You've probably heard that the 6500 will be getting a new switch fabric/supervisor module which should resolve a lot of the hardware limitations seen in the current EARL (for all intents and purposes, it should be the same EARL being used on the Nexus 7000 series platforms). It probably makes more sense for anyone looking at a new 6500 to consider the Nexus 7000 for a number of reasons, least of which isn't the potential for future 40Gbps and 100Gbps Ethernet support. But Cisco will still get customers for the new fabric, especially existing 6500 folk. Cheers, Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Sunday 21 March 2010 08:52:54 pm chrisccnpsp...@gmail.com wrote: Then also take the fact that the 6500 is dead as a strategic platform. The Nexus 7k is the new boy on the block. You've probably heard that the 6500 will be getting a new switch fabric/supervisor module which should resolve a lot of the hardware limitations seen in the current EARL (for all intents and purposes, it should be the same EARL being used on the Nexus 7000 series platforms). It probably makes more sense for anyone looking at a new 6500 to consider the Nexus 7000 for a number of reasons, least of which isn't the potential for future 40Gbps and 100Gbps Ethernet support. But Cisco will still get customers for the new fabric, especially existing 6500 folk. Cheers, Mark. signature.asc Description: This is a digitally signed message part. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] EX 8200 deployment
Hi Folks, Please share your experiences regarding the deployment of EX 8200, I need to deploy two chassis in VRRP. Please let shed some light on the following point - Any trick in power/power requirements?? - stability - best design( like Virtual routers are needed or not) - possible caveats - Best junos version Add any trick or issue which you have found out? waiting for ur inputs thanks and best regards, Muhammad Fahad Khan JNCIP - M/T # 834 IT Specialist Global Technology Services, IBM fa...@pk.ibm.com +92-321-2370510 +92-301-8247638 Skype: fahad-ibm http://www.linkedin.com/in/muhammadfahadkhan http://fahad-internetworker.blogspot.com http://www.visualcv.com/g46ptnd ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Richard, I agree with you with the EX platform.. It's really buggy. I personally think that Juniper is moving too slow bringing new features and bug fixes to the platform.. We're deploying the new Cisco Nexus platforms in our data centers at this point. Cisco is moving at light speed with these platforms, while Juniper is crawling to bring their aging boxes into the lime light left by the Cisco Cat6500 days. On another note. I know you're upset about the limit on the routing that the EX series can do, but a better question is why are you using this box for that high end routing solution? In my opinion, the EX's 8200's On Sat, Mar 20, 2010 at 11:03 AM, Richard A Steenbergen r...@e-gerbil.netwrote: On Sat, Mar 20, 2010 at 06:50:58PM +0500, Fahad Khan wrote: Hi Folks, Please share your experiences regarding the deployment of EX 8200, I need to deploy two chassis in VRRP. Please let shed some light on the following point - Any trick in power/power requirements?? - stability - best design( like Virtual routers are needed or not) - possible caveats - Best junos version Add any trick or issue which you have found out? We just deployed our first EX8208 a few days ago, running 10.1R1. Gotchas so far: * Obviously this is a very different architecture from Juniper's normal boxes, so be prepared for vlan space being shared across the entire box, not a per-interface basis. * In a move straight out of Foundry's playbook of how to fail at making a useable product, EX has no packet counters (cli or snmp) available for L3 vlan interfaces. It DOES have working counters if you do traditional Juniper subinterface style vlans (interface blah, vlan-tagging, unit 123 vlan-id 123), but it does NOT work if you have to do RVI style (vlan blah l3-interface vlan.123 and then put vlan blah in an ethernet switching interface). Subinterface style is my preference anyways, so as long as you only ever use vlans on point-to-point links this isn't a problem, but the instant you need to put a VLAN on more than one port you no longer get packet counters. * Related to the issue above, you can't mix subinterface style and RVI style vlans on the same trunk port. The instant you need to do anything more than classic subinterface style vlans, you have to convert everything on the trunk to vlan/rvi style. For example, where I might otherwise be able to get away with doing interface xe-1/0/0 unit 123 vlan-id 123 family inet blah, if I want to trunk a layer 2 vlan on that same interface I now have to convert unit 123 to RVI style. One possible workaround I have yet to test is doing a CCC instead of a vlan, to keep the subinterface style. This would only work with 2-port member vlans though, and I have yet to test the implications for mixing tagged and untagged ports on EX, so this may not actually work for anyone at all. * There is a hardcoded default maximum data memory per process of 512MB on the EX8200 series, which isn't enough memory for rpd to run any kind of complex BGP configuration. Juniper says they tested it to 3.2 million paths and it only used ~400MB, but I guess they found some artificially low test scenario (I still wonder how they did this, even with no communities and no as-path attributes it's only a ~10% memory difference), and they didn't bother to ask the rpd developers how much memory usage to expect, because real world usage is obviously FAR higher. In our testing rpd coredumped at about 900k BGP paths, which is probably a bad thing if you actually want to route on these boxes. Juniper is working on this issue, but as a temporary workaround, edit your /boot/loader.conf file as root, replace the kern.maxdsiz=512M with something more sensible like 1536M, and reboot. This will of course be blown away every time you upgrade JUNOS, so it helps to have dual REs so you can pre-stage things. :) * Firewall filters are still a bit of a mess. You can't count or log anything, you can't use policers on either control plane or egress filters (heck you can't even commit a firewall filter with a policer in it if applied as an output filter), you can't match frags, etc, etc. Basically if you're coming from another Juniper router, be prepared to completely redesign all of your firewall filters across the board. For example, you can't currently do a match like from port 179 on EX, you have to do from source-port 179 and from destination-port 179, which means splitting what was previously a single term into two terms. This is expected to get better with future sw, but my feeling is probably in small increments over the next year+ or so. * I don't know who thought 2GB of storage on an RE was sufficient, but it isn't. The best idea I've come up with so far is to grab some small USB flash devices like http://www.geckoandfly.com/tag/small-usb/ and deploy them on every RE so you have a little bit of working space. Other than that we haven't found any
Re: [j-nsp] EX 8200 deployment
Richard, I agree with you with the EX platform.. It's really buggy. I personally think that Juniper is moving too slow bringing new features and bug fixes to the platform.. We're deploying the new Cisco Nexus platforms in our data centers at this point. Cisco is moving at light speed with these platforms, while Juniper is crawling to bring their aging boxes into the lime light left by the Cisco Cat6500 days. On another note. I know you're upset about the limit on the routing that the EX series can do, but a better question is why are you using this box for that high end routing solution? In my opinion, the EX's 8200's are a switch built for the data center, they shouldn't require much more than a default route and/or a few hundred routes to your core. Discussion? On Sat, Mar 20, 2010 at 11:03 AM, Richard A Steenbergen r...@e-gerbil.netwrote: On Sat, Mar 20, 2010 at 06:50:58PM +0500, Fahad Khan wrote: Hi Folks, Please share your experiences regarding the deployment of EX 8200, I need to deploy two chassis in VRRP. Please let shed some light on the following point - Any trick in power/power requirements?? - stability - best design( like Virtual routers are needed or not) - possible caveats - Best junos version Add any trick or issue which you have found out? We just deployed our first EX8208 a few days ago, running 10.1R1. Gotchas so far: * Obviously this is a very different architecture from Juniper's normal boxes, so be prepared for vlan space being shared across the entire box, not a per-interface basis. * In a move straight out of Foundry's playbook of how to fail at making a useable product, EX has no packet counters (cli or snmp) available for L3 vlan interfaces. It DOES have working counters if you do traditional Juniper subinterface style vlans (interface blah, vlan-tagging, unit 123 vlan-id 123), but it does NOT work if you have to do RVI style (vlan blah l3-interface vlan.123 and then put vlan blah in an ethernet switching interface). Subinterface style is my preference anyways, so as long as you only ever use vlans on point-to-point links this isn't a problem, but the instant you need to put a VLAN on more than one port you no longer get packet counters. * Related to the issue above, you can't mix subinterface style and RVI style vlans on the same trunk port. The instant you need to do anything more than classic subinterface style vlans, you have to convert everything on the trunk to vlan/rvi style. For example, where I might otherwise be able to get away with doing interface xe-1/0/0 unit 123 vlan-id 123 family inet blah, if I want to trunk a layer 2 vlan on that same interface I now have to convert unit 123 to RVI style. One possible workaround I have yet to test is doing a CCC instead of a vlan, to keep the subinterface style. This would only work with 2-port member vlans though, and I have yet to test the implications for mixing tagged and untagged ports on EX, so this may not actually work for anyone at all. * There is a hardcoded default maximum data memory per process of 512MB on the EX8200 series, which isn't enough memory for rpd to run any kind of complex BGP configuration. Juniper says they tested it to 3.2 million paths and it only used ~400MB, but I guess they found some artificially low test scenario (I still wonder how they did this, even with no communities and no as-path attributes it's only a ~10% memory difference), and they didn't bother to ask the rpd developers how much memory usage to expect, because real world usage is obviously FAR higher. In our testing rpd coredumped at about 900k BGP paths, which is probably a bad thing if you actually want to route on these boxes. Juniper is working on this issue, but as a temporary workaround, edit your /boot/loader.conf file as root, replace the kern.maxdsiz=512M with something more sensible like 1536M, and reboot. This will of course be blown away every time you upgrade JUNOS, so it helps to have dual REs so you can pre-stage things. :) * Firewall filters are still a bit of a mess. You can't count or log anything, you can't use policers on either control plane or egress filters (heck you can't even commit a firewall filter with a policer in it if applied as an output filter), you can't match frags, etc, etc. Basically if you're coming from another Juniper router, be prepared to completely redesign all of your firewall filters across the board. For example, you can't currently do a match like from port 179 on EX, you have to do from source-port 179 and from destination-port 179, which means splitting what was previously a single term into two terms. This is expected to get better with future sw, but my feeling is probably in small increments over the next year+ or so. * I don't know who thought 2GB of storage on an RE was sufficient, but it isn't. The best idea I've come up with so far is to grab some small USB flash devices like
Re: [j-nsp] EX 8200 deployment
On Mar 20, 2010, at 5:31 PM, Chris Evans wrote: Richard, I agree with you with the EX platform.. It's really buggy. I personally think that Juniper is moving too slow bringing new features and bug fixes to the platform.. We're deploying the new Cisco Nexus platforms in our data centers at this point. Cisco is moving at light speed with these platforms, while Juniper is crawling to bring their aging boxes into the lime light left by the Cisco Cat6500 days. On another note. I know you're upset about the limit on the routing that the EX series can do, but a better question is why are you using this box for that high end routing solution? In my opinion, the EX's 8200's are a switch built for the data center, they shouldn't require much more than a default route and/or a few hundred routes to your core. Discussion? Agreed. High end routing with dense port configurations is called an MX. On Sat, Mar 20, 2010 at 11:03 AM, Richard A Steenbergen r...@e-gerbil.netwrote: On Sat, Mar 20, 2010 at 06:50:58PM +0500, Fahad Khan wrote: Hi Folks, Please share your experiences regarding the deployment of EX 8200, I need to deploy two chassis in VRRP. Please let shed some light on the following point - Any trick in power/power requirements?? - stability - best design( like Virtual routers are needed or not) - possible caveats - Best junos version Add any trick or issue which you have found out? We just deployed our first EX8208 a few days ago, running 10.1R1. Gotchas so far: * Obviously this is a very different architecture from Juniper's normal boxes, so be prepared for vlan space being shared across the entire box, not a per-interface basis. * In a move straight out of Foundry's playbook of how to fail at making a useable product, EX has no packet counters (cli or snmp) available for L3 vlan interfaces. It DOES have working counters if you do traditional Juniper subinterface style vlans (interface blah, vlan-tagging, unit 123 vlan-id 123), but it does NOT work if you have to do RVI style (vlan blah l3-interface vlan.123 and then put vlan blah in an ethernet switching interface). Subinterface style is my preference anyways, so as long as you only ever use vlans on point-to-point links this isn't a problem, but the instant you need to put a VLAN on more than one port you no longer get packet counters. * Related to the issue above, you can't mix subinterface style and RVI style vlans on the same trunk port. The instant you need to do anything more than classic subinterface style vlans, you have to convert everything on the trunk to vlan/rvi style. For example, where I might otherwise be able to get away with doing interface xe-1/0/0 unit 123 vlan-id 123 family inet blah, if I want to trunk a layer 2 vlan on that same interface I now have to convert unit 123 to RVI style. One possible workaround I have yet to test is doing a CCC instead of a vlan, to keep the subinterface style. This would only work with 2-port member vlans though, and I have yet to test the implications for mixing tagged and untagged ports on EX, so this may not actually work for anyone at all. * There is a hardcoded default maximum data memory per process of 512MB on the EX8200 series, which isn't enough memory for rpd to run any kind of complex BGP configuration. Juniper says they tested it to 3.2 million paths and it only used ~400MB, but I guess they found some artificially low test scenario (I still wonder how they did this, even with no communities and no as-path attributes it's only a ~10% memory difference), and they didn't bother to ask the rpd developers how much memory usage to expect, because real world usage is obviously FAR higher. In our testing rpd coredumped at about 900k BGP paths, which is probably a bad thing if you actually want to route on these boxes. Juniper is working on this issue, but as a temporary workaround, edit your /boot/loader.conf file as root, replace the kern.maxdsiz=512M with something more sensible like 1536M, and reboot. This will of course be blown away every time you upgrade JUNOS, so it helps to have dual REs so you can pre-stage things. :) * Firewall filters are still a bit of a mess. You can't count or log anything, you can't use policers on either control plane or egress filters (heck you can't even commit a firewall filter with a policer in it if applied as an output filter), you can't match frags, etc, etc. Basically if you're coming from another Juniper router, be prepared to completely redesign all of your firewall filters across the board. For example, you can't currently do a match like from port 179 on EX, you have to do from source-port 179 and from destination-port 179, which means splitting what was previously a single term into two terms. This is expected to get better with future sw, but my feeling is probably in small increments over the next year+ or so. * I don't know who thought 2GB
Re: [j-nsp] EX 8200 deployment
On Sat, Mar 20, 2010 at 05:31:36PM -0400, Chris Evans wrote: Richard, I agree with you with the EX platform.. It's really buggy. I personally think that Juniper is moving too slow bringing new features and bug fixes to the platform.. We're deploying the new Cisco Nexus platforms in our data centers at this point. Cisco is moving at light speed with these platforms, while Juniper is crawling to bring their aging boxes into the lime light left by the Cisco Cat6500 days. Nexus 7000 is not a bad platform, and definitely deserves consideration for the same kind of role. Obviously each has its own specific advantages and disadvantages, but at a high level Nexus 7k an EX8200 are *VERY* comperable in both price and features (and even their roadmaps look a lot alike). Yes n7k is probably a little more mature and stable than EX at the moment, but there is a lot to be said for the benefits of working with JUNOS too. An importand consideration is not only how well it works today, but how well it will work in the future, and like I said JUNOS earns the EX a LOT of leeway (to a point :P). On another note. I know you're upset about the limit on the routing that the EX series can do, but a better question is why are you using this box for that high end routing solution? In my opinion, the EX's 8200's are a switch built for the data center, they shouldn't require much more than a default route and/or a few hundred routes to your core. Clearly we have different definitions of a datacenter role. Let me be clear, EX is absolutely *NOT* a replacement for the very capable and mature MX platform, nor does it try to be. MX does things like MPLS, VPLS, large (2mil+) FIBs, large memory (4GB) for multiple RIBs, it supports complex vlan tag manipulation that EX will never do, it has unique vlan IDs per interface not shared across the box, it has large packet buffers, support for SONET cards, XFP support for working with long reach optics in a carrier ethernet role, much more robust firewall features, services card support, and a host of other things that EX will never do. That said, EX has a role for simpler work in the datacenter where the full functionality of an MX is simply not worth the extra money and the features aren't necessary. EX is also (or at least is intended to be) a competitor of datacenter boxes like 6500 and Nexus 7000, both of which support a full routing table and multiple paths via BGP. Your datacenter may not make use of a full BGP routing table, but mine does, and clearly a lot of other people's do too or Cisco wouldn't be making full-table 6500s or Nexus 7000s. Also, my use of routing on the EX was well within the intended support of the platform, it was simply a mistake in the code that caused it to fail at much lower levels than they intended. My real gripe was that the failure should have been perfectly obvious to them, and yet it wasn't. I know for a fact that Juniper employs many very smart and capable people who would have known better, it's just that the EX guys didn't consult them. :) -- Richard A Steenbergen r...@e-gerbil.net http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
On Saturday 20 March 2010 11:03:13 pm Richard A Steenbergen wrote: * Obviously this is a very different architecture from Juniper's normal boxes, so be prepared for vlan space being shared across the entire box, not a per-interface basis. Terrible. Mark. signature.asc Description: This is a digitally signed message part. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] EX 8200 deployment
Chassis-wide VLAN space? Great!... Juniper just managed to build a Cisco 6509 ;) Stefan Fouant Sent from my Verizon Wireless BlackBerry -Original Message- From: Mark Tinka mti...@globaltransit.net Date: Sun, 21 Mar 2010 03:23:41 To: juniper-nsp@puck.nether.net Cc: Richard A Steenbergenr...@e-gerbil.net Subject: Re: [j-nsp] EX 8200 deployment ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
