On 2018-07-15 06:09, Jérôme Nicolle wrote:
Hi Baldur,
Le 14/07/2018 à 14:13, Baldur Norddahl a écrit :
I am investigating Linux as a BNG
As we say in France, it's like your trying to buttfuck flies (a local
saying standing for "reinventing the wheel for no practical reason").
You can say that about whole opensource ecosystem, why to bother, if
*proprietary solution name* exists. It is endless flamewar topic.
Linux' kernel networking stack is not made for this kind of job. 6WIND
or fd.io may be right on the spot, but it's still a lot of dark magic
for something that has been done over and over for the past 20 years by
most vendors.
And it just works.
Linux developers are working continuously to improve this, for example
latest feature, XDP, able to process several Mpps on <$1000 server.
Ask yourself, why Cloudflare "buttfuck flies" and doesn't buy some
proprietary vendor who 20 years does filtering in hardware?
https://blog.cloudflare.com/how-to-drop-10-million-packets/
I am doing experiments with XDP as well, to terminate PPPoE, and it is
doing that quite well over XDP.
DHCP (implying straight L2 from the CPE to the BNG) may be an option
bust most codebases are still young. PPP, on the other hand, is
field-tested for extremely large scale deployments with most vendors.
DHCP here, at least from RFC 2131 existence in March 1997.
Quite old, isn't it?
When you stick to PPPoE, you tie yourself with necessary layers of
encapsulation/decapsulation, and this is seriously degrading performance
at
_user_ level at least. With some development experience of firmware for
routers,
i can tell that hardware offloading of ipv4 routing (DHCP) obviousl
is much easier and cheaper, than offloading PPPoE encap/decap+ipv4
routing.
Also Vendors keep screwing up their routers with PPP, and for example
one of them failed processing properly PADO in newest firmware revision.
Another problem, with PPPoE you subscribe to headache called reduced
mtu, that also
will give a lot of unpleasant hours for ISP support.
If I were in you shooes, and I don't say I'd want to (my BNGs are
scaled
to less than a few thousand of subscribers, with 1-4 concurrent session
each), I'd stick to plain old bitstream (PPP) model, with a decent
subscriber framework on my BNGs (I mostly use Juniper MXs, but I also
like Nokia's and Cisco's for some features).
I am consulting operators from few hundreds to hundreds of thousands.
It is very rare, when Linux bng doesn't suit them.
But let's say we would want to go forward and ditch legacy /
proprietary
code to surf on the NFV bullshit-wave. What would you actually need ?
Linux does soft-recirculation at every encapsulation level by memory
copy. You can't scale anything with that. You need to streamline
decapsulation with 6wind's turborouter or fd.io frameworks. It'll cost
you a few thousand of man-hours to implement your first prototype.
6wind/fd.io is great solutions, but not suitable for mentioned task.
They are mostly created for very tailor made tasks or even as core of
some
vendor solution. Implementing your BNG based on such frameworks, or
DPDK, is really
reinventing the wheel, unless you will sell it or can save by that
millions
of US$.
Let's say you got a woking framework to treat subsequent headers on the
fly (because decapsulation is not really needed, what you want is just
to forward the payload, right ?)… Well, you'd need to address
provisionning protocols on the same layers. Who would want to rebase a
DHCP server with alien packet forms incoming ? I gess no one.
accel-ppp does all that and exactly for IPoE termination, and no black
magic
there.
Well, I could dissert on the topic for hours, because I've already
spent
months to address such design issues in scalable ISP networks, and the
conclusion is :
- PPPoE is simple and proven. Its rigid structure alleviates most of
the
dual-stack issues. It is well supported and largelly deployed.
PPPoE has VERY serious flaws.
1)Security of PPPoE sucks big time. Anybody who run rogue PPPoE server
in your network
will create significant headache for you, while with DHCP you have at
least "DHCP snooping".
DHCP snooping supported in very many vendors switches, while for PPPoE
most of them
have nothing, except... you stick each user to his own vlan.
Why to pppox them then?
2)DHCP can send some circuit information in Option 82, this is very
useful for
billing and very cost efficient on last stage of access switches.
3)Modern FTTX(GPON) solutions are built with QinQ in mind, so IPoE fits
there flawlessly.
- DHCP requires hacks (in the form of undocummented options from
several
vendors) to seemingly work on IPv4, but the multicast boundaries for
NDP
are a PITA to handle, so no one implemented that properly yet. So it is
to avoid for now.
While you can do multicast(mostly for IPTV, yes it is not easy, and need
some
vendor magic on "native" layer (DHCP), with PPP you can forget about
multicast
entirely.
- Subscriber frameworks, be it Juniper's, Cisco's or Nokia's, are at
the
core of the largest residentioal ISPs out there. It Just Works. Trust
them.
Sticking to "It Just Works" means "zero innovation" as well.
For example, while everybody said so, Mikrotik guys appeared, and very
possible
in total numbers their solutions serving now more users than Cisco or
Nokia,
for much lower cost.
But sure they have their own market niche, Mikrotik doesn't fit well for
large deployments.
That being said, I love the idea of NFV-ing all the things, let it be
BNGs first because those bricks in the wall are the most fragile we
have
to maintain.
But I cleraly won't stand for an alternative to traditionnal offerings
just yet : it's too critical, and it's a PITA to build from scratch and
scale.
Best regards,
A lot of people who just sit on their warm, monthly salary chair and
care about their
personal stability only(but not employer profit) -
will ask employer to pay for expensive *vendor name* solution, as it's
safest bet for them.
So, if person who implement solution is just "corporate screw" - he will
say *vendor*.
Nicolas Taleb "Skin in the game" books perfectly explains, why they will
do worst choice.
If person is entrepreneur - he will start feasibility study.