Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Nick W]

1.9.7 definitely applies to Infinity:

ER-8-XG:
https://dl.ubnt.com/firmwares/edgemax/v1.9.7/ER-e1000.v1.9.7+hotfix.1.5005858.tar
(SHA256:b1a16900e3fbe1eef3876548ac7eda12a95ef849d4328f22b478459e2a506b92)



On Tue, Aug 8, 2017 at 9:07 PM, Josh Reynolds <j...@kyneticwifi.com> wrote:

> Forgot reply all...
>
> That does not apply to the infinity. Those shipped with 1.9.8dev.
>
>
> On Aug 8, 2017 8:03 PM, "Mike Hammett" <na...@ics-il.net> wrote:
>
> > 1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released on
> > May 1st.
> >
> > https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-
> > EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
> >
> >
> >
> >
> > -
> > Mike Hammett
> > Intelligent Computing Solutions
> > http://www.ics-il.com
> >
> > Midwest-IX
> > http://www.midwest-ix.com
> >
> > - Original Message -
> >
> > From: "Nick W" <nickdwh...@gmail.com>
> > To: nanog@nanog.org
> > Sent: Thursday, July 20, 2017 10:55:28 PM
> > Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?
> >
> > Tried the Infinity, unsuccessfully. Several of them. Ended up pulling
> them
> > all, sitting in my homelab for now. Multiple full tables, nothing fancy
> for
> > firewall or QOS, but ran into issues with random ribd/bgpd crashes and
> > kernel panics. I've submitted a lot of logs and core dumps to UBNT. I
> would
> > personally stay away from them until they are out of beta, and possibly
> > even another 6-12 months after that.
> >
> > The current stable EdgeMax version (1.9.1.1) is relatively stable, but
> > using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF,
> BGP)
> > - nothing too major, but can be annoying. Probably okay for what you
> > described. Depending on how much throughput you need, an ERPro, or
> Mikrotik
> > would probably be fine. If you need 10G, load up VyOS on some cheap
> servers
> > with an Intel or Solarflare card... probably cheaper than a beta Infinity
> > or Mikrotik.
> >
> > On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders <j...@instituut.net> wrote:
> >
> > > Dear NANOG,
> > >
> > > Some friends of mine are operating a nonprofit (on shoe string) and
> > looking
> > > to connect some CDN caches to an IX fabric. A BGP speaking device is
> > needed
> > > between the caches and the BGP peers connected to the fabric. The BGP
> > > speaker is needed to present the peers on the IX with a unified view of
> > the
> > > assemblage of CDN nodes.
> > >
> > > I was wondering whether anyone was experience with the "EdgeRouter
> > Infinity
> > > XG" device, specifically in the role of a simple peering router for a
> > > couple of tens of thousands of routes. (I'd point default to the left
> and
> > > take just the on-net routes on the right to reduce the table size
> > > requirement).
> > >
> > > I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
> > > automatable (supports idempotency), can forward IMIX at line-rate,
> *flow,
> > > and exposes some telemetry via SNMP.
> > >
> > > Any note sharing would be appreciated!
> > >
> > > Kind regards,
> > >
> > > Job
> > >
> >
> >
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Brielle Bruns]

On 8/11/2017 10:30 PM, Josh Reynolds wrote:

I'm dumb, Brielle is right.

1.9.0, 1.9.5, 1.9.7h1

1.9.8dev and 1.9.8b1 are for two other newer products.



Ubiquiti has been pretty active in developing improvements lately.

I do recommend anyone who does use the Edge* series in production that 
they join the beta program on the Ubnt forums to keep current on whats 
being worked on.  It pays off in the long run, and gives you a chance to 
test out new features and fixes before deploying them in real world 
scenarios.


The Infinity isn't a perfect fit for everyone, but it is a 'disruptive' 
device and worth a look if you have > 1G needs.  Now that its 'out 
there' in production, hopefully those that can put it to use will be 
able to give out real world numbers.


(Note:  I don't work for Ubnt, but I've used many of their products in 
the Edge and Unifi lines.  I only speak for myself.)



--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org/ http://www.ahbl.org

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

I'm dumb, Brielle is right.

1.9.0, 1.9.5, 1.9.7h1

1.9.8dev and 1.9.8b1 are for two other newer products.

On Aug 11, 2017 11:16 PM, "Brielle Bruns"  wrote:

> On 8/11/2017 9:34 AM, Josh Reynolds wrote:
>
>> As an additional note, sometimes drivers get backported, this is how
>> 1.9.7hotfix1 works on Infinity. There are multiple trees in various stages
>> of dev at any given time.
>>
>
>
>
> The Infinity started out on 1.9.0 (which is what my Infinity alpha
> hardware had) and went up from there as the various in between releases
> came up, up until the current 1.9.7 release.
>
> The 1.9.8 dev releases are not currently for the Infinity or any previous
> ER hardware.  That's about all I can say.
>
>
> --
> Brielle Bruns
> The Summit Open Source Development Group
> http://www.sosdg.org/ http://www.ahbl.org
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Brielle Bruns]

On 8/11/2017 9:34 AM, Josh Reynolds wrote:

As an additional note, sometimes drivers get backported, this is how
1.9.7hotfix1 works on Infinity. There are multiple trees in various stages
of dev at any given time.




The Infinity started out on 1.9.0 (which is what my Infinity alpha 
hardware had) and went up from there as the various in between releases 
came up, up until the current 1.9.7 release.


The 1.9.8 dev releases are not currently for the Infinity or any 
previous ER hardware.  That's about all I can say.



--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org/ http://www.ahbl.org

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

As an additional note, sometimes drivers get backported, this is how
1.9.7hotfix1 works on Infinity. There are multiple trees in various stages
of dev at any given time.

On Aug 11, 2017 10:29 AM, "Josh Reynolds" <j...@kyneticwifi.com> wrote:

> Since it's been announced now...
>
> I have an alpha unit. It came with 1.9.8dev straight from $manufacturer.
> My box still has markings from customs all over it.
>
> Expect a new version with some minor fixes soon. A lot of firmware work
> going on at the moment on various edgeOS product lines.
>
> On Aug 11, 2017 10:03 AM, "Nick W" <nickdwh...@gmail.com> wrote:
>
>> 1.9.7 definitely applies to Infinity:
>>
>> ER-8-XG:
>> https://dl.ubnt.com/firmwares/edgemax/v1.9.7/ER-e1000.v1.9.7
>> +hotfix.1.5005858.tar
>> (SHA256:b1a16900e3fbe1eef3876548ac7eda12a95ef849d4328f22b478459e2a506b92)
>>
>>
>>
>> On Tue, Aug 8, 2017 at 9:07 PM, Josh Reynolds <j...@kyneticwifi.com>
>> wrote:
>>
>>> Forgot reply all...
>>>
>>> That does not apply to the infinity. Those shipped with 1.9.8dev.
>>>
>>>
>>> On Aug 8, 2017 8:03 PM, "Mike Hammett" <na...@ics-il.net> wrote:
>>>
>>> > 1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released
>>> on
>>> > May 1st.
>>> >
>>> > https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-
>>> > EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
>>> >
>>> >
>>> >
>>> >
>>> > -
>>> > Mike Hammett
>>> > Intelligent Computing Solutions
>>> > http://www.ics-il.com
>>> >
>>> > Midwest-IX
>>> > http://www.midwest-ix.com
>>> >
>>> > - Original Message -
>>> >
>>> > From: "Nick W" <nickdwh...@gmail.com>
>>> > To: nanog@nanog.org
>>> > Sent: Thursday, July 20, 2017 10:55:28 PM
>>> > Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?
>>> >
>>> > Tried the Infinity, unsuccessfully. Several of them. Ended up pulling
>>> them
>>> > all, sitting in my homelab for now. Multiple full tables, nothing
>>> fancy for
>>> > firewall or QOS, but ran into issues with random ribd/bgpd crashes and
>>> > kernel panics. I've submitted a lot of logs and core dumps to UBNT. I
>>> would
>>> > personally stay away from them until they are out of beta, and possibly
>>> > even another 6-12 months after that.
>>> >
>>> > The current stable EdgeMax version (1.9.1.1) is relatively stable, but
>>> > using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF,
>>> BGP)
>>> > - nothing too major, but can be annoying. Probably okay for what you
>>> > described. Depending on how much throughput you need, an ERPro, or
>>> Mikrotik
>>> > would probably be fine. If you need 10G, load up VyOS on some cheap
>>> servers
>>> > with an Intel or Solarflare card... probably cheaper than a beta
>>> Infinity
>>> > or Mikrotik.
>>> >
>>> > On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders <j...@instituut.net>
>>> wrote:
>>> >
>>> > > Dear NANOG,
>>> > >
>>> > > Some friends of mine are operating a nonprofit (on shoe string) and
>>> > looking
>>> > > to connect some CDN caches to an IX fabric. A BGP speaking device is
>>> > needed
>>> > > between the caches and the BGP peers connected to the fabric. The BGP
>>> > > speaker is needed to present the peers on the IX with a unified view
>>> of
>>> > the
>>> > > assemblage of CDN nodes.
>>> > >
>>> > > I was wondering whether anyone was experience with the "EdgeRouter
>>> > Infinity
>>> > > XG" device, specifically in the role of a simple peering router for a
>>> > > couple of tens of thousands of routes. (I'd point default to the
>>> left and
>>> > > take just the on-net routes on the right to reduce the table size
>>> > > requirement).
>>> > >
>>> > > I hope the device can do at least 2xLACP trunks, has a sizable FIB,
>>> is
>>> > > automatable (supports idempotency), can forward IMIX at line-rate,
>>> *flow,
>>> > > and exposes some telemetry via SNMP.
>>> > >
>>> > > Any note sharing would be appreciated!
>>> > >
>>> > > Kind regards,
>>> > >
>>> > > Job
>>> > >
>>> >
>>> >
>>>
>>
>>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

Since it's been announced now...

I have an alpha unit. It came with 1.9.8dev straight from $manufacturer. My
box still has markings from customs all over it.

Expect a new version with some minor fixes soon. A lot of firmware work
going on at the moment on various edgeOS product lines.

On Aug 11, 2017 10:03 AM, "Nick W" <nickdwh...@gmail.com> wrote:

> 1.9.7 definitely applies to Infinity:
>
> ER-8-XG:
> https://dl.ubnt.com/firmwares/edgemax/v1.9.7/ER-e1000.v1.9.
> 7+hotfix.1.5005858.tar
> (SHA256:b1a16900e3fbe1eef3876548ac7eda12a95ef849d4328f22b478459e2a506b92)
>
>
>
> On Tue, Aug 8, 2017 at 9:07 PM, Josh Reynolds <j...@kyneticwifi.com>
> wrote:
>
>> Forgot reply all...
>>
>> That does not apply to the infinity. Those shipped with 1.9.8dev.
>>
>>
>> On Aug 8, 2017 8:03 PM, "Mike Hammett" <na...@ics-il.net> wrote:
>>
>> > 1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released
>> on
>> > May 1st.
>> >
>> > https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-
>> > EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
>> >
>> >
>> >
>> >
>> > -
>> > Mike Hammett
>> > Intelligent Computing Solutions
>> > http://www.ics-il.com
>> >
>> > Midwest-IX
>> > http://www.midwest-ix.com
>> >
>> > - Original Message -
>> >
>> > From: "Nick W" <nickdwh...@gmail.com>
>> > To: nanog@nanog.org
>> > Sent: Thursday, July 20, 2017 10:55:28 PM
>> > Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?
>> >
>> > Tried the Infinity, unsuccessfully. Several of them. Ended up pulling
>> them
>> > all, sitting in my homelab for now. Multiple full tables, nothing fancy
>> for
>> > firewall or QOS, but ran into issues with random ribd/bgpd crashes and
>> > kernel panics. I've submitted a lot of logs and core dumps to UBNT. I
>> would
>> > personally stay away from them until they are out of beta, and possibly
>> > even another 6-12 months after that.
>> >
>> > The current stable EdgeMax version (1.9.1.1) is relatively stable, but
>> > using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF,
>> BGP)
>> > - nothing too major, but can be annoying. Probably okay for what you
>> > described. Depending on how much throughput you need, an ERPro, or
>> Mikrotik
>> > would probably be fine. If you need 10G, load up VyOS on some cheap
>> servers
>> > with an Intel or Solarflare card... probably cheaper than a beta
>> Infinity
>> > or Mikrotik.
>> >
>> > On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders <j...@instituut.net> wrote:
>> >
>> > > Dear NANOG,
>> > >
>> > > Some friends of mine are operating a nonprofit (on shoe string) and
>> > looking
>> > > to connect some CDN caches to an IX fabric. A BGP speaking device is
>> > needed
>> > > between the caches and the BGP peers connected to the fabric. The BGP
>> > > speaker is needed to present the peers on the IX with a unified view
>> of
>> > the
>> > > assemblage of CDN nodes.
>> > >
>> > > I was wondering whether anyone was experience with the "EdgeRouter
>> > Infinity
>> > > XG" device, specifically in the role of a simple peering router for a
>> > > couple of tens of thousands of routes. (I'd point default to the left
>> and
>> > > take just the on-net routes on the right to reduce the table size
>> > > requirement).
>> > >
>> > > I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
>> > > automatable (supports idempotency), can forward IMIX at line-rate,
>> *flow,
>> > > and exposes some telemetry via SNMP.
>> > >
>> > > Any note sharing would be appreciated!
>> > >
>> > > Kind regards,
>> > >
>> > > Job
>> > >
>> >
>> >
>>
>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Jeff Waddell]

When I lasted checked in with Ubiquiti on these issues for that and the
ER-Pros - they told me that everything was to be resolved in 2.0

We shall see...

On Tue, Aug 8, 2017 at 9:20 PM, Mike Hammett <na...@ics-il.net> wrote:

> Ah, okay. I haven't used one yet.
>
> Also, I don't talk about beta outside of beta. ;-)
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Josh Reynolds" <j...@kyneticwifi.com>
> To: "Mike Hammett" <na...@ics-il.net>
> Cc: "NANOG" <nanog@nanog.org>
> Sent: Tuesday, August 8, 2017 8:07:36 PM
> Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?
>
>
> Forgot reply all...
>
>
> That does not apply to the infinity. Those shipped with 1.9.8dev.
>
>
>
>
>
> On Aug 8, 2017 8:03 PM, "Mike Hammett" < na...@ics-il.net > wrote:
>
>
> 1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released on
> May 1st.
>
> https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-
> EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Nick W" < nickdwh...@gmail.com >
> To: nanog@nanog.org
> Sent: Thursday, July 20, 2017 10:55:28 PM
> Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?
>
> Tried the Infinity, unsuccessfully. Several of them. Ended up pulling them
> all, sitting in my homelab for now. Multiple full tables, nothing fancy for
> firewall or QOS, but ran into issues with random ribd/bgpd crashes and
> kernel panics. I've submitted a lot of logs and core dumps to UBNT. I would
> personally stay away from them until they are out of beta, and possibly
> even another 6-12 months after that.
>
> The current stable EdgeMax version (1.9.1.1) is relatively stable, but
> using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF, BGP)
> - nothing too major, but can be annoying. Probably okay for what you
> described. Depending on how much throughput you need, an ERPro, or Mikrotik
> would probably be fine. If you need 10G, load up VyOS on some cheap servers
> with an Intel or Solarflare card... probably cheaper than a beta Infinity
> or Mikrotik.
>
> On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders < j...@instituut.net > wrote:
>
> > Dear NANOG,
> >
> > Some friends of mine are operating a nonprofit (on shoe string) and
> looking
> > to connect some CDN caches to an IX fabric. A BGP speaking device is
> needed
> > between the caches and the BGP peers connected to the fabric. The BGP
> > speaker is needed to present the peers on the IX with a unified view of
> the
> > assemblage of CDN nodes.
> >
> > I was wondering whether anyone was experience with the "EdgeRouter
> Infinity
> > XG" device, specifically in the role of a simple peering router for a
> > couple of tens of thousands of routes. (I'd point default to the left and
> > take just the on-net routes on the right to reduce the table size
> > requirement).
> >
> > I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
> > automatable (supports idempotency), can forward IMIX at line-rate, *flow,
> > and exposes some telemetry via SNMP.
> >
> > Any note sharing would be appreciated!
> >
> > Kind regards,
> >
> > Job
> >
>
>
>
>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Mike Hammett]

Ah, okay. I haven't used one yet. 

Also, I don't talk about beta outside of beta. ;-) 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Josh Reynolds" <j...@kyneticwifi.com> 
To: "Mike Hammett" <na...@ics-il.net> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Tuesday, August 8, 2017 8:07:36 PM 
Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"? 


Forgot reply all... 


That does not apply to the infinity. Those shipped with 1.9.8dev. 





On Aug 8, 2017 8:03 PM, "Mike Hammett" < na...@ics-il.net > wrote: 


1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released on May 
1st. 

https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message - 

From: "Nick W" < nickdwh...@gmail.com > 
To: nanog@nanog.org 
Sent: Thursday, July 20, 2017 10:55:28 PM 
Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"? 

Tried the Infinity, unsuccessfully. Several of them. Ended up pulling them 
all, sitting in my homelab for now. Multiple full tables, nothing fancy for 
firewall or QOS, but ran into issues with random ribd/bgpd crashes and 
kernel panics. I've submitted a lot of logs and core dumps to UBNT. I would 
personally stay away from them until they are out of beta, and possibly 
even another 6-12 months after that. 

The current stable EdgeMax version (1.9.1.1) is relatively stable, but 
using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF, BGP) 
- nothing too major, but can be annoying. Probably okay for what you 
described. Depending on how much throughput you need, an ERPro, or Mikrotik 
would probably be fine. If you need 10G, load up VyOS on some cheap servers 
with an Intel or Solarflare card... probably cheaper than a beta Infinity 
or Mikrotik. 

On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders < j...@instituut.net > wrote: 

> Dear NANOG, 
> 
> Some friends of mine are operating a nonprofit (on shoe string) and looking 
> to connect some CDN caches to an IX fabric. A BGP speaking device is needed 
> between the caches and the BGP peers connected to the fabric. The BGP 
> speaker is needed to present the peers on the IX with a unified view of the 
> assemblage of CDN nodes. 
> 
> I was wondering whether anyone was experience with the "EdgeRouter Infinity 
> XG" device, specifically in the role of a simple peering router for a 
> couple of tens of thousands of routes. (I'd point default to the left and 
> take just the on-net routes on the right to reduce the table size 
> requirement). 
> 
> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is 
> automatable (supports idempotency), can forward IMIX at line-rate, *flow, 
> and exposes some telemetry via SNMP. 
> 
> Any note sharing would be appreciated! 
> 
> Kind regards, 
> 
> Job 
> 

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

Forgot reply all...

That does not apply to the infinity. Those shipped with 1.9.8dev.


On Aug 8, 2017 8:03 PM, "Mike Hammett" <na...@ics-il.net> wrote:

> 1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released on
> May 1st.
>
> https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-
> EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Nick W" <nickdwh...@gmail.com>
> To: nanog@nanog.org
> Sent: Thursday, July 20, 2017 10:55:28 PM
> Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?
>
> Tried the Infinity, unsuccessfully. Several of them. Ended up pulling them
> all, sitting in my homelab for now. Multiple full tables, nothing fancy for
> firewall or QOS, but ran into issues with random ribd/bgpd crashes and
> kernel panics. I've submitted a lot of logs and core dumps to UBNT. I would
> personally stay away from them until they are out of beta, and possibly
> even another 6-12 months after that.
>
> The current stable EdgeMax version (1.9.1.1) is relatively stable, but
> using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF, BGP)
> - nothing too major, but can be annoying. Probably okay for what you
> described. Depending on how much throughput you need, an ERPro, or Mikrotik
> would probably be fine. If you need 10G, load up VyOS on some cheap servers
> with an Intel or Solarflare card... probably cheaper than a beta Infinity
> or Mikrotik.
>
> On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders <j...@instituut.net> wrote:
>
> > Dear NANOG,
> >
> > Some friends of mine are operating a nonprofit (on shoe string) and
> looking
> > to connect some CDN caches to an IX fabric. A BGP speaking device is
> needed
> > between the caches and the BGP peers connected to the fabric. The BGP
> > speaker is needed to present the peers on the IX with a unified view of
> the
> > assemblage of CDN nodes.
> >
> > I was wondering whether anyone was experience with the "EdgeRouter
> Infinity
> > XG" device, specifically in the role of a simple peering router for a
> > couple of tens of thousands of routes. (I'd point default to the left and
> > take just the on-net routes on the right to reduce the table size
> > requirement).
> >
> > I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
> > automatable (supports idempotency), can forward IMIX at line-rate, *flow,
> > and exposes some telemetry via SNMP.
> >
> > Any note sharing would be appreciated!
> >
> > Kind regards,
> >
> > Job
> >
>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Mike Hammett]

1.9.7+hotfix.1 is the currently available stable. 1.9.1.1 was released on May 
1st. 

https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-EdgeRouter-software-security-release-v1-9-7-hotfix-1/ba-p/2019161
 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Nick W" <nickdwh...@gmail.com> 
To: nanog@nanog.org 
Sent: Thursday, July 20, 2017 10:55:28 PM 
Subject: Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"? 

Tried the Infinity, unsuccessfully. Several of them. Ended up pulling them 
all, sitting in my homelab for now. Multiple full tables, nothing fancy for 
firewall or QOS, but ran into issues with random ribd/bgpd crashes and 
kernel panics. I've submitted a lot of logs and core dumps to UBNT. I would 
personally stay away from them until they are out of beta, and possibly 
even another 6-12 months after that. 

The current stable EdgeMax version (1.9.1.1) is relatively stable, but 
using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF, BGP) 
- nothing too major, but can be annoying. Probably okay for what you 
described. Depending on how much throughput you need, an ERPro, or Mikrotik 
would probably be fine. If you need 10G, load up VyOS on some cheap servers 
with an Intel or Solarflare card... probably cheaper than a beta Infinity 
or Mikrotik. 

On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders <j...@instituut.net> wrote: 

> Dear NANOG, 
> 
> Some friends of mine are operating a nonprofit (on shoe string) and looking 
> to connect some CDN caches to an IX fabric. A BGP speaking device is needed 
> between the caches and the BGP peers connected to the fabric. The BGP 
> speaker is needed to present the peers on the IX with a unified view of the 
> assemblage of CDN nodes. 
> 
> I was wondering whether anyone was experience with the "EdgeRouter Infinity 
> XG" device, specifically in the role of a simple peering router for a 
> couple of tens of thousands of routes. (I'd point default to the left and 
> take just the on-net routes on the right to reduce the table size 
> requirement). 
> 
> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is 
> automatable (supports idempotency), can forward IMIX at line-rate, *flow, 
> and exposes some telemetry via SNMP. 
> 
> Any note sharing would be appreciated! 
> 
> Kind regards, 
> 
> Job 
> 

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Nick W]

Tried the Infinity, unsuccessfully. Several of them. Ended up pulling them
all, sitting in my homelab for now. Multiple full tables, nothing fancy for
firewall or QOS, but ran into issues with random ribd/bgpd crashes and
kernel panics. I've submitted a lot of logs and core dumps to UBNT. I would
personally stay away from them until they are out of beta, and possibly
even another 6-12 months after that.

The current stable EdgeMax version (1.9.1.1) is relatively stable, but
using an outdated ZebOS (1.2.0?) with a number of issues (MPLS, OSPF, BGP)
- nothing too major, but can be annoying. Probably okay for what you
described. Depending on how much throughput you need, an ERPro, or Mikrotik
would probably be fine. If you need 10G, load up VyOS on some cheap servers
with an Intel or Solarflare card... probably cheaper than a beta Infinity
or Mikrotik.

On Mon, Jul 3, 2017 at 3:07 PM, Job Snijders  wrote:

> Dear NANOG,
>
> Some friends of mine are operating a nonprofit (on shoe string) and looking
> to connect some CDN caches to an IX fabric. A BGP speaking device is needed
> between the caches and the BGP peers connected to the fabric. The BGP
> speaker is needed to present the peers on the IX with a unified view of the
> assemblage of CDN nodes.
>
> I was wondering whether anyone was experience with the "EdgeRouter Infinity
> XG" device, specifically in the role of a simple peering router for a
> couple of tens of thousands of routes. (I'd point default to the left and
> take just the on-net routes on the right to reduce the table size
> requirement).
>
> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
> automatable (supports idempotency), can forward IMIX at line-rate, *flow,
> and exposes some telemetry via SNMP.
>
> Any note sharing would be appreciated!
>
> Kind regards,
>
> Job
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Jérôme Nicolle]

Hello Jeremy,

Le 04/07/2017 à 01:10, Jeremy Austin a écrit :
> can certainly handle a few tens of thousands of
> routes fine (single core BGP though), 

It can take multiple full views. It's also faster than an MX104.

> but I can't vouch for its ability to
> do IMIX or *flow at line rate

I wouldn't load one to 80g, but at 10-20G, it creates no bottleneck.

The entire packet-pipeline is in software. IPFIX is not sampled, it's
1:1 only AFAIK. It's also lacking some features, meaning you'd need to
filter through pmacct to add BGP informations.

Best regards,

-- 
Jérôme Nicolle

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Tim Pozar]

BTW...  At Fandor (before I left) we got one of the last /24s that ARIN
had.   Our transit providers at the office were Monkey Brains (wireless)
and Zayo (fiber).  We purchased a ER Pro, upgraded the memory and were
peering v4 with both on this box.  MB didn't have V6 at that point.  We
did nail up our V6 announcement with Zayo and got it that way.

If folks need config examples.

Tim

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

6.3 ;)

- Josh

On Jul 5, 2017 2:10 PM, "Paul Gear"  wrote:

> On 04/07/17 12:28, Hugo Slabbert wrote:
> >
> > ...
> >>>
> >>> As far as automation, it's a JunOS-like CLI originally based on vyatta,
> >>> which AT now owns - and one of the main reasons is it's
> scriptability,
> >>> use of Ansible and other tools right on the device, python, etc.
> >
> > Technically I believe it's based on VyOS rather than Vyatta.  Same base,
> > but just delineating that VyOS is open source and I don't believe AT
> > wields any control over it.
>
> EdgeOS was forked from Vyatta well before (around Vyatta Core 6.2?) VyOS
> took up the last public Vyatta release.  It has therefore diverged
> somewhat from current VyOS releases, but the two are still
> mostly-compatible.
>
> Paul
>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Paul Gear]

On 04/07/17 12:28, Hugo Slabbert wrote:
> 
> ...
>>>
>>> As far as automation, it's a JunOS-like CLI originally based on vyatta,
>>> which AT now owns - and one of the main reasons is it's scriptability,
>>> use of Ansible and other tools right on the device, python, etc.
> 
> Technically I believe it's based on VyOS rather than Vyatta.  Same base,
> but just delineating that VyOS is open source and I don't believe AT
> wields any control over it.

EdgeOS was forked from Vyatta well before (around Vyatta Core 6.2?) VyOS
took up the last public Vyatta release.  It has therefore diverged
somewhat from current VyOS releases, but the two are still
mostly-compatible.

Paul

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Jared Geiger]

The RAM is upgradeable but it can support quite a few full tables out of
the box. The routing software under the hood got upgraded by Ubiquiti to
ZebOS https://www.ipinfusion.com/products/zebos/ from the VyOS code.

There is a Cavium bug regarding UDP packets though that can be nasty if you
hit it.
https://community.ubnt.com/t5/EdgeMAX/UDP-packet-loss-with-EdgeRouter-Lite/m-p/1343012

Even though the thread starts by talking about the Lite, all of the Cavium
EdgeRouters  currently have the problem. The beta work around is to
restrict packet forwarding to only use one of the CPU cores. This is with
or without hardware offloading enabled. Hopefully Cavium will have a real
fix soon. I have two of these I'm itching to put into production once the
bugs are worked out.

On Mon, Jul 3, 2017 at 8:28 PM, Josh Reynolds  wrote:

> I kinda feel the same way.  I wish FRR was a big more mature at this
> point though.
>
> On Mon, Jul 3, 2017 at 10:21 PM, Baldur Norddahl
>  wrote:
> > Why not use a Linux or BSD computer for this? It is cheap and you know
> > exactly what you are getting. It will forward 10 gig at line rate at
> least
> > for normal traffic.
> >
> > Regards
> >
> > Baldur
> >
> > Den 3. jul. 2017 21.08 skrev "Job Snijders" :
> >
> >> Dear NANOG,
> >>
> >> Some friends of mine are operating a nonprofit (on shoe string) and
> looking
> >> to connect some CDN caches to an IX fabric. A BGP speaking device is
> needed
> >> between the caches and the BGP peers connected to the fabric. The BGP
> >> speaker is needed to present the peers on the IX with a unified view of
> the
> >> assemblage of CDN nodes.
> >>
> >> I was wondering whether anyone was experience with the "EdgeRouter
> Infinity
> >> XG" device, specifically in the role of a simple peering router for a
> >> couple of tens of thousands of routes. (I'd point default to the left
> and
> >> take just the on-net routes on the right to reduce the table size
> >> requirement).
> >>
> >> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
> >> automatable (supports idempotency), can forward IMIX at line-rate,
> *flow,
> >> and exposes some telemetry via SNMP.
> >>
> >> Any note sharing would be appreciated!
> >>
> >> Kind regards,
> >>
> >> Job
> >>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

I kinda feel the same way.  I wish FRR was a big more mature at this
point though.

On Mon, Jul 3, 2017 at 10:21 PM, Baldur Norddahl
 wrote:
> Why not use a Linux or BSD computer for this? It is cheap and you know
> exactly what you are getting. It will forward 10 gig at line rate at least
> for normal traffic.
>
> Regards
>
> Baldur
>
> Den 3. jul. 2017 21.08 skrev "Job Snijders" :
>
>> Dear NANOG,
>>
>> Some friends of mine are operating a nonprofit (on shoe string) and looking
>> to connect some CDN caches to an IX fabric. A BGP speaking device is needed
>> between the caches and the BGP peers connected to the fabric. The BGP
>> speaker is needed to present the peers on the IX with a unified view of the
>> assemblage of CDN nodes.
>>
>> I was wondering whether anyone was experience with the "EdgeRouter Infinity
>> XG" device, specifically in the role of a simple peering router for a
>> couple of tens of thousands of routes. (I'd point default to the left and
>> take just the on-net routes on the right to reduce the table size
>> requirement).
>>
>> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
>> automatable (supports idempotency), can forward IMIX at line-rate, *flow,
>> and exposes some telemetry via SNMP.
>>
>> Any note sharing would be appreciated!
>>
>> Kind regards,
>>
>> Job
>>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Baldur Norddahl]

Why not use a Linux or BSD computer for this? It is cheap and you know
exactly what you are getting. It will forward 10 gig at line rate at least
for normal traffic.

Regards

Baldur

Den 3. jul. 2017 21.08 skrev "Job Snijders" :

> Dear NANOG,
>
> Some friends of mine are operating a nonprofit (on shoe string) and looking
> to connect some CDN caches to an IX fabric. A BGP speaking device is needed
> between the caches and the BGP peers connected to the fabric. The BGP
> speaker is needed to present the peers on the IX with a unified view of the
> assemblage of CDN nodes.
>
> I was wondering whether anyone was experience with the "EdgeRouter Infinity
> XG" device, specifically in the role of a simple peering router for a
> couple of tens of thousands of routes. (I'd point default to the left and
> take just the on-net routes on the right to reduce the table size
> requirement).
>
> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
> automatable (supports idempotency), can forward IMIX at line-rate, *flow,
> and exposes some telemetry via SNMP.
>
> Any note sharing would be appreciated!
>
> Kind regards,
>
> Job
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

EdgeOS was forked and employees were poached from Vyatta before it was
purchased by Broadcom, when it was open source.  I think a few things
came down from VyOS after that, but not many.

On Mon, Jul 3, 2017 at 9:28 PM, Hugo Slabbert  wrote:
>
> On Mon 2017-Jul-03 19:26:17 -0500, Josh Reynolds 
> wrote:
>
>> On Jul 3, 2017 7:23 PM, "Josh Reynolds"  wrote:
>>
>>> Specs...
>>>
>>>
>>>- MIPS64 16 Core 1.8 GHz
>>>- 16 GB DDR4 RAM
>>>- 8 MB NOR Flash 4 GB eMMC NAND Flash
>>>- Data Ports: (1) RJ45 Serial Port, (8) SFP+ Ports (1) RJ45 Gigabit
>>>Ethernet Port
>>>- 2 hotswap power supplies
>>>
>>>
>>> No LACP. ECMP is currently broken. MPLS/VPLS is currently broken and not
>>> done in hardware - this may eventually change. As far as the other stuff,
>>> "telemetry" etc - no.
>>>
>>> As far as BGP crunching, plenty of routes, etc - it would easily and
>>> happily be fine with that.
>>>
>>> As far as automation, it's a JunOS-like CLI originally based on vyatta,
>>> which AT now owns - and one of the main reasons is it's scriptability,
>>> use of Ansible and other tools right on the device, python, etc.
>
>
> Technically I believe it's based on VyOS rather than Vyatta.  Same base, but
> just delineating that VyOS is open source and I don't believe AT wields
> any control over it.
>
>>>
>>> - Josh
>>>
>
> --
> Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
> pgp key: B178313E   | also on Signal
>
>
>>> On Jul 3, 2017 2:09 PM, "Job Snijders"  wrote:
>>>
 Dear NANOG,

 Some friends of mine are operating a nonprofit (on shoe string) and
 looking
 to connect some CDN caches to an IX fabric. A BGP speaking device is
 needed
 between the caches and the BGP peers connected to the fabric. The BGP
 speaker is needed to present the peers on the IX with a unified view of
 the
 assemblage of CDN nodes.

 I was wondering whether anyone was experience with the "EdgeRouter
 Infinity
 XG" device, specifically in the role of a simple peering router for a
 couple of tens of thousands of routes. (I'd point default to the left
 and
 take just the on-net routes on the right to reduce the table size
 requirement).

 I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
 automatable (supports idempotency), can forward IMIX at line-rate,
 *flow,
 and exposes some telemetry via SNMP.

 Any note sharing would be appreciated!

 Kind regards,

 Job

>>>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Hugo Slabbert]

On Mon 2017-Jul-03 19:26:17 -0500, Josh Reynolds  wrote:


On Jul 3, 2017 7:23 PM, "Josh Reynolds"  wrote:


Specs...


   - MIPS64 16 Core 1.8 GHz
   - 16 GB DDR4 RAM
   - 8 MB NOR Flash 4 GB eMMC NAND Flash
   - Data Ports: (1) RJ45 Serial Port, (8) SFP+ Ports (1) RJ45 Gigabit
   Ethernet Port
   - 2 hotswap power supplies


No LACP. ECMP is currently broken. MPLS/VPLS is currently broken and not
done in hardware - this may eventually change. As far as the other stuff,
"telemetry" etc - no.

As far as BGP crunching, plenty of routes, etc - it would easily and
happily be fine with that.

As far as automation, it's a JunOS-like CLI originally based on vyatta,
which AT now owns - and one of the main reasons is it's scriptability,
use of Ansible and other tools right on the device, python, etc.


Technically I believe it's based on VyOS rather than Vyatta.  Same base, 
but just delineating that VyOS is open source and I don't believe AT 
wields any control over it.




- Josh



--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal


On Jul 3, 2017 2:09 PM, "Job Snijders"  wrote:


Dear NANOG,

Some friends of mine are operating a nonprofit (on shoe string) and
looking
to connect some CDN caches to an IX fabric. A BGP speaking device is
needed
between the caches and the BGP peers connected to the fabric. The BGP
speaker is needed to present the peers on the IX with a unified view of
the
assemblage of CDN nodes.

I was wondering whether anyone was experience with the "EdgeRouter
Infinity
XG" device, specifically in the role of a simple peering router for a
couple of tens of thousands of routes. (I'd point default to the left and
take just the on-net routes on the right to reduce the table size
requirement).

I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
automatable (supports idempotency), can forward IMIX at line-rate, *flow,
and exposes some telemetry via SNMP.

Any note sharing would be appreciated!

Kind regards,

Job





signature.asc
Description: Digital signature

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Josh Reynolds]

- Josh

On Jul 3, 2017 7:23 PM, "Josh Reynolds"  wrote:

> Specs...
>
>
>- MIPS64 16 Core 1.8 GHz
>- 16 GB DDR4 RAM
>- 8 MB NOR Flash 4 GB eMMC NAND Flash
>- Data Ports: (1) RJ45 Serial Port, (8) SFP+ Ports (1) RJ45 Gigabit
>Ethernet Port
>- 2 hotswap power supplies
>
>
> No LACP. ECMP is currently broken. MPLS/VPLS is currently broken and not
> done in hardware - this may eventually change. As far as the other stuff,
> "telemetry" etc - no.
>
> As far as BGP crunching, plenty of routes, etc - it would easily and
> happily be fine with that.
>
> As far as automation, it's a JunOS-like CLI originally based on vyatta,
> which AT now owns - and one of the main reasons is it's scriptability,
> use of Ansible and other tools right on the device, python, etc.
>
> - Josh
>
> On Jul 3, 2017 2:09 PM, "Job Snijders"  wrote:
>
>> Dear NANOG,
>>
>> Some friends of mine are operating a nonprofit (on shoe string) and
>> looking
>> to connect some CDN caches to an IX fabric. A BGP speaking device is
>> needed
>> between the caches and the BGP peers connected to the fabric. The BGP
>> speaker is needed to present the peers on the IX with a unified view of
>> the
>> assemblage of CDN nodes.
>>
>> I was wondering whether anyone was experience with the "EdgeRouter
>> Infinity
>> XG" device, specifically in the role of a simple peering router for a
>> couple of tens of thousands of routes. (I'd point default to the left and
>> take just the on-net routes on the right to reduce the table size
>> requirement).
>>
>> I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
>> automatable (supports idempotency), can forward IMIX at line-rate, *flow,
>> and exposes some telemetry via SNMP.
>>
>> Any note sharing would be appreciated!
>>
>> Kind regards,
>>
>> Job
>>
>

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Jeremy Austin]

On Mon, Jul 3, 2017 at 2:44 PM, Seth Mattinen  wrote:

>
> EdgeRouter is... meh. If I was looking at that class of gear I'd go with a
> Mikrotik.


Job,

There is a bit of a price differential here, depending on whether you need
SFP+; the Infinity is "dead cheap", and has fairly opaque BGP
daemon+debugging tools. Also still technically a beta product. Not sure if
it meets your automation requirements. I wouldn't want to be deploying them
in a redundant pair, myself, but just when you say something can't be done…

Mikrotik's CCR1072: 10-gig router (shipping, not anything that's just been
announced) has an API, can certainly handle a few tens of thousands of
routes fine (single core BGP though), but I can't vouch for its ability to
do IMIX or *flow at line rate. This has probably been stress tested by
somebody. I doubt the sampling is in hardware.

If you don't need 10G ports then your options expand considerably. Do you
have a target throughput?

-- 
Jeremy Austin

(907) 895-2311 office
(907) 803-5422 cell
jhaus...@gmail.com

Heritage NetWorks
Whitestone Power & Communications
Vertical Broadband, LLC

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Seth Mattinen]

On 7/3/17 12:07 PM, Job Snijders wrote:

I was wondering whether anyone was experience with the "EdgeRouter Infinity
XG" device, specifically in the role of a simple peering router for a
couple of tens of thousands of routes. (I'd point default to the left and
take just the on-net routes on the right to reduce the table size
requirement).



EdgeRouter is... meh. If I was looking at that class of gear I'd go with 
a Mikrotik.


~Seth

Re: EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Mike Hammett]

I'm Ubiquiti's biggest critic. I'll check with my colleagues. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Job Snijders" <j...@instituut.net> 
To: nanog@nanog.org 
Sent: Monday, July 3, 2017 2:07:28 PM 
Subject: EdgeRouter Infinity as medium-sized "IXP Peering Router"? 

Dear NANOG, 

Some friends of mine are operating a nonprofit (on shoe string) and looking 
to connect some CDN caches to an IX fabric. A BGP speaking device is needed 
between the caches and the BGP peers connected to the fabric. The BGP 
speaker is needed to present the peers on the IX with a unified view of the 
assemblage of CDN nodes. 

I was wondering whether anyone was experience with the "EdgeRouter Infinity 
XG" device, specifically in the role of a simple peering router for a 
couple of tens of thousands of routes. (I'd point default to the left and 
take just the on-net routes on the right to reduce the table size 
requirement). 

I hope the device can do at least 2xLACP trunks, has a sizable FIB, is 
automatable (supports idempotency), can forward IMIX at line-rate, *flow, 
and exposes some telemetry via SNMP. 

Any note sharing would be appreciated! 

Kind regards, 

Job 

EdgeRouter Infinity as medium-sized "IXP Peering Router"?

[Job Snijders]

Dear NANOG,

Some friends of mine are operating a nonprofit (on shoe string) and looking
to connect some CDN caches to an IX fabric. A BGP speaking device is needed
between the caches and the BGP peers connected to the fabric. The BGP
speaker is needed to present the peers on the IX with a unified view of the
assemblage of CDN nodes.

I was wondering whether anyone was experience with the "EdgeRouter Infinity
XG" device, specifically in the role of a simple peering router for a
couple of tens of thousands of routes. (I'd point default to the left and
take just the on-net routes on the right to reduce the table size
requirement).

I hope the device can do at least 2xLACP trunks, has a sizable FIB, is
automatable (supports idempotency), can forward IMIX at line-rate, *flow,
and exposes some telemetry via SNMP.

Any note sharing would be appreciated!

Kind regards,

Job