Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled

[Richard McGovern via juniper-nsp]

--- Begin Message ---
Thanks Chuck

Richard McGovern
Sr Sales Engineer, Juniper Networks 
978-618-3342
 
I’d rather be lucky than good, as I know I am not good
I don’t make the news, I just report it
 

On 4/21/20, 11:53 AM, "Chuck Anderson"  wrote:

[External Email. Be cautious of content]


As I said, I'm not excluding any protocols from MACsec.  With that 
configuration, LLDP apparently doesn't work "outside the tunnel"--I never see 
any directly attached neighbors.  LLDP does work between the MACsec 
endpoints--they show as if they are directly connected neighbors.  I'm fine 
with that result in my case.

The solution to eliminate the spurious framing errors appears to be: Ask 
your carrier to shut off LLDP/CDP/any other L2 protocols running on their 
interfaces directly attached to your MACsec endpoint devices.

On Tue, Apr 21, 2020 at 02:59:53PM +, Richard McGovern wrote:
> Based upon Chuck’s reply:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on 
the EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end 
between the EX3400s just fine.
>
> Don’t exclude LLDP from MACSEC and either stop or block LLDP from the 
Carrier/ISP.  In Chuck’s case the Carrier (to the EX3400s) was his MX.  Then 
EX3400s should see each other via LLDP, but not see the carrier.  I am not sure 
if today, you have an LLDP neighbor with your Carrier/ISP or not?
>
> This is the way I now read his response.
>
> Yes?
>
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
> [signature_1140633420]
>
> From: james list 
> Date: Tuesday, April 21, 2020 at 10:53 AM
> To: Richard McGovern 
> Cc: Chuck Anderson , Juniper List 

> Subject: Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
>
> [External Email. Be cautious of content]
>
> Hi Richard
> lldp and lacp are excluded:
>
>   > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode 
static-cak
> > > set security macsec connectivity-association MAC pre-shared-key 
ckn 
> > > set security macsec connectivity-association MAC pre-shared-key 
cak
> > > "t"
> > > set security macsec connectivity-association MAC exclude-protocol 
lldp
> > > set security macsec connectivity-association MAC exclude-protocol 
lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association 
MAC
>
> I did not catch the connection with Framing errors counter...
>
> Please detail if you can.
>
> Cheers
>
>
> Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern 
mailto:rmcgov...@juniper.net>> ha scritto:
> Chuck, I thought you were running both LLDP and LACP outside the MACSEC 
tunnel, no?
>
> (Optional) Exclude a protocol from MACsec:
> [edit security macsec connectivity-association 
connectivity-association-name]
> user@switch# set exclude-protocol protocol-name
> For instance, if you did not want Link Level Discovery Protocol (LLDP) to 
be secured using MACsec:
>
> [edit security macsec connectivity-association ca-dynamic1]
> user@switch# set exclude-protocol lldp
> When this option is enabled, MACsec is disabled for all packets of the 
specified protocol—in this case, LLDP—that are sent or received on the link.
>
> BEST PRACTICEWe recommend that any protocol other than MACsec being used 
on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing 
protocols, should be excluded and moved outside of the MACsec tunnel.
>
> Is this not working properly for LLDP?
>
> Rich
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
>
> On 4/19/20, 7:31 PM, "Chuck Anderson" mailto:c...@wpi.edu>> 
wrote:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on 
the EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end 
between the EX3400s just fine.
>
> Check if the carrier is running LLDP or CDP or similar.
>
>
> On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> > Yes, I see CRC errors on EX3400s with MACsec termination, but only 
on one side.
>

Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled

[Chuck Anderson]

As I said, I'm not excluding any protocols from MACsec.  With that 
configuration, LLDP apparently doesn't work "outside the tunnel"--I never see 
any directly attached neighbors.  LLDP does work between the MACsec 
endpoints--they show as if they are directly connected neighbors.  I'm fine 
with that result in my case.

The solution to eliminate the spurious framing errors appears to be: Ask your 
carrier to shut off LLDP/CDP/any other L2 protocols running on their interfaces 
directly attached to your MACsec endpoint devices.

On Tue, Apr 21, 2020 at 02:59:53PM +, Richard McGovern wrote:
> Based upon Chuck’s reply:
> 
> Well, that was an easy fix on my MX480s:
> 
> set protocols lldp interface xe-0/0/1 disable
> 
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the 
> EX3400s connected directly to the MX480s.
> 
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end between 
> the EX3400s just fine.
> 
> Don’t exclude LLDP from MACSEC and either stop or block LLDP from the 
> Carrier/ISP.  In Chuck’s case the Carrier (to the EX3400s) was his MX.  Then 
> EX3400s should see each other via LLDP, but not see the carrier.  I am not 
> sure if today, you have an LLDP neighbor with your Carrier/ISP or not?
> 
> This is the way I now read his response.
> 
> Yes?
> 
> 
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
> 
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
> 
> [signature_1140633420]
> 
> From: james list 
> Date: Tuesday, April 21, 2020 at 10:53 AM
> To: Richard McGovern 
> Cc: Chuck Anderson , Juniper List 
> Subject: Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
> 
> [External Email. Be cautious of content]
> 
> Hi Richard
> lldp and lacp are excluded:
> 
>   > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode 
> static-cak
> > > set security macsec connectivity-association MAC pre-shared-key ckn 
> 
> > > set security macsec connectivity-association MAC pre-shared-key cak
> > > "t"
> > > set security macsec connectivity-association MAC exclude-protocol lldp
> > > set security macsec connectivity-association MAC exclude-protocol lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association MAC
> 
> I did not catch the connection with Framing errors counter...
> 
> Please detail if you can.
> 
> Cheers
> 
> 
> Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern 
> mailto:rmcgov...@juniper.net>> ha scritto:
> Chuck, I thought you were running both LLDP and LACP outside the MACSEC 
> tunnel, no?
> 
> (Optional) Exclude a protocol from MACsec:
> [edit security macsec connectivity-association connectivity-association-name]
> user@switch# set exclude-protocol protocol-name
> For instance, if you did not want Link Level Discovery Protocol (LLDP) to be 
> secured using MACsec:
> 
> [edit security macsec connectivity-association ca-dynamic1]
> user@switch# set exclude-protocol lldp
> When this option is enabled, MACsec is disabled for all packets of the 
> specified protocol—in this case, LLDP—that are sent or received on the link.
> 
> BEST PRACTICEWe recommend that any protocol other than MACsec being used on 
> the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing protocols, 
> should be excluded and moved outside of the MACsec tunnel.
> 
> Is this not working properly for LLDP?
> 
> Rich
> 
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
> 
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
> 
> 
> On 4/19/20, 7:31 PM, "Chuck Anderson" mailto:c...@wpi.edu>> 
> wrote:
> 
> Well, that was an easy fix on my MX480s:
> 
> set protocols lldp interface xe-0/0/1 disable
> 
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the 
> EX3400s connected directly to the MX480s.
> 
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end between 
> the EX3400s just fine.
> 
> Check if the carrier is running LLDP or CDP or similar.
> 
> 
> On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> > Yes, I see CRC errors on EX3400s with MACsec termination, but only on 
> one side.
> >
> > Here is my topology:
> >
> > From A to B:
> >
> > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 
> vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
> >   MACsec L2 connection  L2 
> xconnectMACsec
> >
> > From B to A:
> >
> > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 
> vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
> >   MACsec L2 connection  L2 
> xconnectMACsec
> >
> > I also have a redundant path with EX3400-C (different local switch) 

Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled

[james list]

Hi Richard
lldp and lacp are excluded:

  > > @EX4300-A> show configuration security macsec | display set
> > set security macsec connectivity-association MAC security-mode
static-cak
> > set security macsec connectivity-association MAC pre-shared-key ckn

> > set security macsec connectivity-association MAC pre-shared-key cak
> > "t"
> > set security macsec connectivity-association MAC exclude-protocol
lldp
> > set security macsec connectivity-association MAC exclude-protocol
lacp
> > set security macsec interfaces ge-0/0/0 connectivity-association
MAC

I did not catch the connection with Framing errors counter...

Please detail if you can.

Cheers


Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern <
rmcgov...@juniper.net> ha scritto:

> Chuck, I thought you were running both LLDP and LACP outside the MACSEC
> tunnel, no?
>
> (Optional) Exclude a protocol from MACsec:
> [edit security macsec connectivity-association
> connectivity-association-name]
> user@switch# set exclude-protocol protocol-name
> For instance, if you did not want Link Level Discovery Protocol (LLDP) to
> be secured using MACsec:
>
> [edit security macsec connectivity-association ca-dynamic1]
> user@switch# set exclude-protocol lldp
> When this option is enabled, MACsec is disabled for all packets of the
> specified protocol—in this case, LLDP—that are sent or received on the link.
>
> BEST PRACTICEWe recommend that any protocol other than MACsec being used
> on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing
> protocols, should be excluded and moved outside of the MACsec tunnel.
>
> Is this not working properly for LLDP?
>
> Rich
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
>
> On 4/19/20, 7:31 PM, "Chuck Anderson"  wrote:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the
> EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end
> between the EX3400s just fine.
>
> Check if the carrier is running LLDP or CDP or similar.
>
>
> On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> > Yes, I see CRC errors on EX3400s with MACsec termination, but only
> on one side.
> >
> > Here is my topology:
> >
> > From A to B:
> >
> > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2
> vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
> >   MACsec L2 connection  L2
> xconnectMACsec
> >
> > From B to A:
> >
> > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2
> vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
> >   MACsec L2 connection  L2
> xconnectMACsec
> >
> > I also have a redundant path with EX3400-C (different local switch)
> and EX3400-B (same remote switch).
> >
> > I see the CRC errors increasing at a rate of about 2-3 per minute,
> but only on EX3400-A and EXX3400-C.
> >
> > All EX3400s were initially running 15.1X53-D57.  Now A and C are
> running 18.2R3-S2 and B is running 15.1X53-D592.  But the problem has been
> consistent throughout all releases, no improvement with upgrades.
> >
> > I wonder if something the carrier's ASR9k is sending down the VLAN
> towards EX3400-A and -C is causing this?  If not, maybe it is the MX480s
> sending something locally to EX3400-A and -C?
> >
> > The following PRs don't seem relevant--I'm not doing anywhere close
> to 60% utilization:
> >
> >
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567
> >
> > And I'm not seeing "runts":
> >
> >
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663
> >
> > I'm only seeing Framing errors (CRC/Align errors):
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791
> > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0,
> Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch
> timeouts: 0, FIFO errors: 0, Resource errors: 0
> > CRC/Align errors2279110
> >
> > A few seconds later, it increased to 227913:
> >
> >   MAC statistics:  Receive Transmit
> > Total octets179536471171563221741316352
> > Total packets  13200126465   7010832956
> > Unicast packets13194022205   7004785539
> > Broadcast packets 52720
> > Multicast packets  6098988  6047417
> > CRC/Align errors

Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled

[Richard McGovern via juniper-nsp]

--- Begin Message ---
Chuck, I thought you were running both LLDP and LACP outside the MACSEC tunnel, 
no?

(Optional) Exclude a protocol from MACsec:
[edit security macsec connectivity-association connectivity-association-name]
user@switch# set exclude-protocol protocol-name
For instance, if you did not want Link Level Discovery Protocol (LLDP) to be 
secured using MACsec:

[edit security macsec connectivity-association ca-dynamic1]
user@switch# set exclude-protocol lldp
When this option is enabled, MACsec is disabled for all packets of the 
specified protocol—in this case, LLDP—that are sent or received on the link.

BEST PRACTICEWe recommend that any protocol other than MACsec being used on the 
MACsec connection, such as LLDP, LACP, STP, or layer 3 routing protocols, 
should be excluded and moved outside of the MACsec tunnel.

Is this not working properly for LLDP?

Rich

Richard McGovern
Sr Sales Engineer, Juniper Networks 
978-618-3342
 
I’d rather be lucky than good, as I know I am not good
I don’t make the news, I just report it
 

On 4/19/20, 7:31 PM, "Chuck Anderson"  wrote:

Well, that was an easy fix on my MX480s:

set protocols lldp interface xe-0/0/1 disable

Now I'm not seeing CRC errors incrementing 2-3 times per minute on the 
EX3400s connected directly to the MX480s.

I'm not excluding any protocols from MACsec--LLDP runs end-to-end between 
the EX3400s just fine.

Check if the carrier is running LLDP or CDP or similar.


On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> Yes, I see CRC errors on EX3400s with MACsec termination, but only on one 
side.
> 
> Here is my topology:
> 
> From A to B:
> 
> [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 
vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
>   MACsec L2 connection  L2 
xconnectMACsec
> 
> From B to A:
> 
> [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 
vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
>   MACsec L2 connection  L2 
xconnectMACsec
> 
> I also have a redundant path with EX3400-C (different local switch) and 
EX3400-B (same remote switch).
> 
> I see the CRC errors increasing at a rate of about 2-3 per minute, but 
only on EX3400-A and EXX3400-C.
> 
> All EX3400s were initially running 15.1X53-D57.  Now A and C are running 
18.2R3-S2 and B is running 15.1X53-D592.  But the problem has been consistent 
throughout all releases, no improvement with upgrades.
> 
> I wonder if something the carrier's ASR9k is sending down the VLAN 
towards EX3400-A and -C is causing this?  If not, maybe it is the MX480s 
sending something locally to EX3400-A and -C?
> 
> The following PRs don't seem relevant--I'm not doing anywhere close to 
60% utilization:
> 
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567
> 
> And I'm not seeing "runts":
> 
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663
> 
> I'm only seeing Framing errors (CRC/Align errors):
> 
> admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791 
> Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed 
discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, 
FIFO errors: 0, Resource errors: 0
> CRC/Align errors2279110
> 
> A few seconds later, it increased to 227913:
> 
>   MAC statistics:  Receive Transmit
> Total octets179536471171563221741316352
> Total packets  13200126465   7010832956
> Unicast packets13194022205   7004785539
> Broadcast packets 52720
> Multicast packets  6098988  6047417
> CRC/Align errors2279130
> FIFO errors  00
> MAC control frames   00
> MAC pause frames 00
> Oversized frames 0
> Jabber frames0
> Fragment frames  0
> VLAN tagged frames 13196813130
> Code violations  0
> 
> Rate is only 24 Mbps, 2200 pps:
> 
> admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps" 
>   Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU 
Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, 
MAC-REWRITE Error: None,
>Input  bytes  :   17954037433802 24242136 bps
>Output bytes  :3221749625049

Re: [j-nsp] Buffer Size

[John Kristoff]

On Mon, 20 Apr 2020 20:58:02 +
Mohammad Khalil  wrote:

> Am trying to conduct a comparison for campus refresh , my end customer is
> deeply interested in deep details.
> He is interested to know the buffer size of Juniper switches (EX series)
> and I could not find such a piece of information in any place.

I'm not sure how well maintained this page is today and you'll
probably want to verify the info if you can, but it may at least give
you some leads:

  

John
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Buffer Size

[Mark Tinka]

On 21/Apr/20 11:49, Tore Anderson wrote:

>
> The vendor is usually irrelevant. Implying that Arista have better/bigger 
> buffers than Juniper is misleading - it depends on the ASIC used. For 
> example, the Juniper EX4600 and the Arista 7050X both contain a TD2 ASIC, so 
> they both have a 12 MB large buffer.
>
> https://people.ucsc.edu/~warner/buffer.html is an excellent resource.

See my response to Saku.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Buffer Size

[Mark Tinka]

On 21/Apr/20 11:49, Saku Ytti wrote:

> You probably didn't mean it, but people will read this as you imply
> ANET does something different to JNPR. If you are comparing EX4600
> (on-chip only) to Arista Jericho (off-chip), that is an entirely
> unfair comparison. ANET also has on-chip buffer models, JNPR also has
> off-chip buffer models.
>
> But yes, EX4600 has like 12MB of buffer, which is not enough to do
> speed step downs or handle multiple interfaces sending to one.

Oh no, we did look at other Juniper models that had large buffers, but
physical and price properties were way out.

The Arista options that lay in the same range as the EX4600 had the
larger buffers we needed without the size and price penalty.

It was mostly annoyance with Juniper for mucking up the EX4600 with ELS,
refusing to listen to us about fixing it until 2021, and the tiny
buffers on both those switches.

I'm not God, this group is intelligent enough to do their own research
before making a purchase, same way I did. That is why my example was
only about the EX4550 and EX4600, which implies Juniper has other boxes
that have deep buffers, if the OP is insistent on maintaining Juniper.

Mark.

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Buffer Size

[Tore Anderson]

* Mark Tinka

> On 20/Apr/20 22:58, Mohammad Khalil wrote:
> 
>> Hi all
>> Am trying to conduct a comparison for campus refresh , my end customer is
>> deeply interested in deep details.
>> He is interested to know the buffer size of Juniper switches (EX series)
>> and I could not find such a piece of information in any place.
>>
>> If anyone has an idea it would be appreciated.
> 
> Atrocious - on the 2 we ran; EX4550 and EX4600.
> 
> We dumped it and went with Arista.

The vendor is usually irrelevant. Implying that Arista have better/bigger 
buffers than Juniper is misleading - it depends on the ASIC used. For example, 
the Juniper EX4600 and the Arista 7050X both contain a TD2 ASIC, so they both 
have a 12 MB large buffer.

https://people.ucsc.edu/~warner/buffer.html is an excellent resource.

Tore
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Buffer Size

[Saku Ytti]

On Tue, 21 Apr 2020 at 12:31, Mark Tinka  wrote:

> > If anyone has an idea it would be appreciated.
>
> Atrocious - on the 2 we ran; EX4550 and EX4600.
>
> We dumped it and went with Arista.

You probably didn't mean it, but people will read this as you imply
ANET does something different to JNPR. If you are comparing EX4600
(on-chip only) to Arista Jericho (off-chip), that is an entirely
unfair comparison. ANET also has on-chip buffer models, JNPR also has
off-chip buffer models.

But yes, EX4600 has like 12MB of buffer, which is not enough to do
speed step downs or handle multiple interfaces sending to one.

-- 
  ++ytti
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Buffer Size

[Mark Tinka]

On 20/Apr/20 22:58, Mohammad Khalil wrote:

> Hi all
> Am trying to conduct a comparison for campus refresh , my end customer is
> deeply interested in deep details.
> He is interested to know the buffer size of Juniper switches (EX series)
> and I could not find such a piece of information in any place.
>
> If anyone has an idea it would be appreciated.

Atrocious - on the 2 we ran; EX4550 and EX4600.

We dumped it and went with Arista.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp