Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-25 Thread Jamal Hadi Salim
On 16-09-23 10:14 AM, Tom Herbert wrote: On Fri, Sep 23, 2016 at 4:13 AM, Jamal Hadi Salim wrote: On 16-09-20 06:00 PM, Tom Herbert wrote: Tom, perused the thread and it seems you are serious ;-> Are we heading towards Frankenstein Avenue? I'm surprised. I thought you'd like democratizing

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-25 Thread Jamal Hadi Salim
On 16-09-23 09:00 AM, Jesper Dangaard Brouer wrote: On Fri, 23 Sep 2016 07:13:30 -0400 Jamal Hadi Salim wrote: [..] Tom, perused the thread and it seems you are serious ;-> Are we heading towards Frankenstein Avenue? The whole point behind letting in XDP is so that _small programs_ can be w

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-23 Thread Alexei Starovoitov
On 9/23/16 6:00 AM, Jesper Dangaard Brouer wrote: To support Tom's case, with eBPF, I think we would have to implement specific eBPF helper functions that can do the ILA table lookups. And then need a userspace component to load the eBPF program. Why add all this, when we could contain all this

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-23 Thread Tom Herbert
On Fri, Sep 23, 2016 at 4:13 AM, Jamal Hadi Salim wrote: > On 16-09-20 06:00 PM, Tom Herbert wrote: >> >> This patch creates an infrastructure for registering and running code at >> XDP hooks in drivers. This is based on the orignal XDP?BPF and borrows >> heavily from the techniques used by netfil

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-23 Thread Jesper Dangaard Brouer
On Fri, 23 Sep 2016 07:13:30 -0400 Jamal Hadi Salim wrote: > On 16-09-20 06:00 PM, Tom Herbert wrote: > > This patch creates an infrastructure for registering and running code at > > XDP hooks in drivers. This is based on the orignal XDP?BPF and borrows > > heavily from the techniques used by net

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-23 Thread Jamal Hadi Salim
On 16-09-20 06:00 PM, Tom Herbert wrote: This patch creates an infrastructure for registering and running code at XDP hooks in drivers. This is based on the orignal XDP?BPF and borrows heavily from the techniques used by netfilter to make generic nfhooks. An XDP hook is defined by the xdp_hook_

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-22 Thread Eric Dumazet
On Thu, 2016-09-22 at 15:14 +0200, Jesper Dangaard Brouer wrote: > On Wed, 21 Sep 2016 21:56:58 +0200 > Jesper Dangaard Brouer wrote: > > > > > I'm not opposed to running non-BPF code at XDP. I'm against adding > > > > a linked list of hook consumers. > > > > I also worry about the performance

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-22 Thread Jesper Dangaard Brouer
On Wed, 21 Sep 2016 21:56:58 +0200 Jesper Dangaard Brouer wrote: > > > I'm not opposed to running non-BPF code at XDP. I'm against adding > > > a linked list of hook consumers. > > I also worry about the performance impact of a linked list. We should > simple benchmark it instead of discussin

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Jesper Dangaard Brouer
On Wed, 21 Sep 2016 08:08:34 -0700 Tom Herbert wrote: > On Wed, Sep 21, 2016 at 7:48 AM, Thomas Graf wrote: > > On 09/21/16 at 07:19am, Tom Herbert wrote: > >> certain design that because of constraints on one kernel interface. As > >> a kernel developer I want flexibility on how we design an

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Thomas Graf
On 09/21/16 at 11:50am, Tom Herbert wrote: > 50 lines in one driver is not a big deal, 50 lines in a hundred > drivers is! I learned this lesson in BQL which was well abstracted out > to be minimally invasive but we still saw many issues because of the > pecularities of different drivers. You want

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Jakub Kicinski
On Wed, 21 Sep 2016 11:50:06 -0700, Tom Herbert wrote: > On Wed, Sep 21, 2016 at 11:45 AM, Jakub Kicinski wrote: > > On Wed, 21 Sep 2016 10:39:40 -0700, Tom Herbert wrote: > >> On Wed, Sep 21, 2016 at 10:26 AM, Jakub Kicinski wrote: > >> > On Tue, 20 Sep 2016 17:01:39 -0700, Alexei Starovoito

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Tom Herbert
On Wed, Sep 21, 2016 at 11:45 AM, Jakub Kicinski wrote: > On Wed, 21 Sep 2016 10:39:40 -0700, Tom Herbert wrote: >> On Wed, Sep 21, 2016 at 10:26 AM, Jakub Kicinski wrote: >> > On Tue, 20 Sep 2016 17:01:39 -0700, Alexei Starovoitov wrote: >> >> > - Reduces the amount of code and complexity need

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Jakub Kicinski
On Wed, 21 Sep 2016 10:39:40 -0700, Tom Herbert wrote: > On Wed, Sep 21, 2016 at 10:26 AM, Jakub Kicinski wrote: > > On Tue, 20 Sep 2016 17:01:39 -0700, Alexei Starovoitov wrote: > >> > - Reduces the amount of code and complexity needed in drivers to > >> >manage XDP > >> > >> hmm: > >>

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Tom Herbert
On Wed, Sep 21, 2016 at 10:26 AM, Jakub Kicinski wrote: > On Tue, 20 Sep 2016 17:01:39 -0700, Alexei Starovoitov wrote: >> > - Reduces the amount of code and complexity needed in drivers to >> >manage XDP >> >> hmm: >> 534 insertions(+), 144 deletions(-) >> looks like increase in complexity

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Jakub Kicinski
On Tue, 20 Sep 2016 17:01:39 -0700, Alexei Starovoitov wrote: > > - Reduces the amount of code and complexity needed in drivers to > >manage XDP > > hmm: > 534 insertions(+), 144 deletions(-) > looks like increase in complexity instead. and more to come to tie this with HW offloads.

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Jakub Kicinski
On Wed, 21 Sep 2016 13:55:45 +0200, Thomas Graf wrote: > > I am looking at using this for ILA router. The problem I am hitting is > > that not all packets that we need to translate go through the XDP > > path. Some would go through the kernel path, some through XDP path but > > When you say kern

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Alexei Starovoitov
On 9/20/16 11:39 PM, Jesper Dangaard Brouer wrote: We are the early stages of XDP development. Users cannot consider XDP a stable UAPI yet. I added a big fat warning to the docs here[1]. If you already consider this a stable API, then I will suggest that we disable XDP or rip the hole thing out

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Alexei Starovoitov
On 9/21/16 7:19 AM, Tom Herbert wrote: #1: Should we allow alternate code to run in XDP other than BPF? separate nft hook - yes generic hook - no since it's one step away from kernel modules abusing this hook. pass/drop/tx of raw buffer at the driver level is a perfect interface to bypass every

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Tom Herbert
On Wed, Sep 21, 2016 at 7:48 AM, Thomas Graf wrote: > On 09/21/16 at 07:19am, Tom Herbert wrote: >> certain design that because of constraints on one kernel interface. As >> a kernel developer I want flexibility on how we design and implement >> things! > > Perfectly valid argument. I reviewed you

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Thomas Graf
On 09/21/16 at 07:19am, Tom Herbert wrote: > certain design that because of constraints on one kernel interface. As > a kernel developer I want flexibility on how we design and implement > things! Perfectly valid argument. I reviewed your ILA changes and did not object to them. > I think there a

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Tom Herbert
On Wed, Sep 21, 2016 at 4:55 AM, Thomas Graf wrote: > On 09/20/16 at 04:59pm, Tom Herbert wrote: >> Well, need to measure to ascertain the cost. As for complexity, this >> actually reduces complexity needed for XDP in the drivers which is a >> good thing because that's where most of the support an

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Thomas Graf
On 09/20/16 at 04:59pm, Tom Herbert wrote: > Well, need to measure to ascertain the cost. As for complexity, this > actually reduces complexity needed for XDP in the drivers which is a > good thing because that's where most of the support and development > pain will be. I'm not objecting to anythi

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-21 Thread Daniel Borkmann
On 09/21/2016 08:39 AM, Jesper Dangaard Brouer wrote: On Tue, 20 Sep 2016 17:01:39 -0700 Alexei Starovoitov wrote: > - Provides a more structured environment that is extensible to new >features while being mostly transparent to the drivers don't see that in these patches either. Thin

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Jesper Dangaard Brouer
On Tue, 20 Sep 2016 17:01:39 -0700 Alexei Starovoitov wrote: > > - Provides a more structured environment that is extensible to new > >features while being mostly transparent to the drivers > > don't see that in these patches either. > Things like packet size change (that we're working

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Alexei Starovoitov
On 9/20/16 4:59 PM, Tom Herbert wrote: I am looking at using this for ILA router. The problem I am hitting is that not all packets that we need to translate go through the XDP path. Some would go through the kernel path, some through XDP path but that would mean I need parallel lookup tables to b

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Alexei Starovoitov
On 9/20/16 3:00 PM, Tom Herbert wrote: +static inline int __xdp_hook_run(struct list_head *list_head, +struct xdp_buff *xdp) +{ + struct xdp_hook_ops *elem; + int ret = XDP_PASS; + + list_for_each_entry(elem, list_head, list) { + ret

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Tom Herbert
On Tue, Sep 20, 2016 at 4:43 PM, Thomas Graf wrote: > On 09/20/16 at 04:18pm, Tom Herbert wrote: >> This allows other use cases than BPF inserting code into the data >> path. This gives XDP potential more utility and more users so that we >> can motivate more driver implementations. For instance,

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Thomas Graf
On 09/20/16 at 04:18pm, Tom Herbert wrote: > This allows other use cases than BPF inserting code into the data > path. This gives XDP potential more utility and more users so that we > can motivate more driver implementations. For instance, I thinks it's > totally reasonable if the nftables guys wa

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Daniel Borkmann
On 09/21/2016 01:09 AM, Thomas Graf wrote: On 09/20/16 at 03:49pm, Tom Herbert wrote: On Tue, Sep 20, 2016 at 3:44 PM, Thomas Graf wrote: On 09/20/16 at 03:00pm, Tom Herbert wrote: +static inline int __xdp_hook_run(struct list_head *list_head, + struct xdp_buff *x

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Tom Herbert
On Tue, Sep 20, 2016 at 4:09 PM, Thomas Graf wrote: > On 09/20/16 at 03:49pm, Tom Herbert wrote: >> On Tue, Sep 20, 2016 at 3:44 PM, Thomas Graf wrote: >> > On 09/20/16 at 03:00pm, Tom Herbert wrote: >> >> +static inline int __xdp_hook_run(struct list_head *list_head, >> >> +

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Thomas Graf
On 09/20/16 at 03:49pm, Tom Herbert wrote: > On Tue, Sep 20, 2016 at 3:44 PM, Thomas Graf wrote: > > On 09/20/16 at 03:00pm, Tom Herbert wrote: > >> +static inline int __xdp_hook_run(struct list_head *list_head, > >> + struct xdp_buff *xdp) > >> +{ > >> + struct xd

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Tom Herbert
On Tue, Sep 20, 2016 at 3:44 PM, Thomas Graf wrote: > On 09/20/16 at 03:00pm, Tom Herbert wrote: >> +static inline int __xdp_hook_run(struct list_head *list_head, >> + struct xdp_buff *xdp) >> +{ >> + struct xdp_hook_ops *elem; >> + int ret = XDP_PASS; >> + >>

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Thomas Graf
On 09/20/16 at 03:00pm, Tom Herbert wrote: > +static inline int __xdp_hook_run(struct list_head *list_head, > + struct xdp_buff *xdp) > +{ > + struct xdp_hook_ops *elem; > + int ret = XDP_PASS; > + > + list_for_each_entry(elem, list_head, list) { > +

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Tom Herbert
On Tue, Sep 20, 2016 at 3:37 PM, Eric Dumazet wrote: > On Tue, 2016-09-20 at 15:00 -0700, Tom Herbert wrote: > >> diff --git a/net/core/xdp.c b/net/core/xdp.c >> new file mode 100644 >> index 000..815ead8 >> --- /dev/null >> +++ b/net/core/xdp.c >> @@ -0,0 +1,211 @@ >> +/* >> + * Kernel Connec

Re: [PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Eric Dumazet
On Tue, 2016-09-20 at 15:00 -0700, Tom Herbert wrote: > diff --git a/net/core/xdp.c b/net/core/xdp.c > new file mode 100644 > index 000..815ead8 > --- /dev/null > +++ b/net/core/xdp.c > @@ -0,0 +1,211 @@ > +/* > + * Kernel Connection Multiplexor > + * > + * Copyright (c) 2016 Tom Herbert > +

[PATCH RFC 1/3] xdp: Infrastructure to generalize XDP

2016-09-20 Thread Tom Herbert
This patch creates an infrastructure for registering and running code at XDP hooks in drivers. This is based on the orignal XDP?BPF and borrows heavily from the techniques used by netfilter to make generic nfhooks. An XDP hook is defined by the xdp_hook_ops. This structure contains the ops of an