[bpf-next PATCH v3 11/18] bpf: sockmap sample, add option to attach SK_MSG program

2018-03-18 Thread John Fastabend
Add sockmap option to use SK_MSG program types. Signed-off-by: John Fastabend Acked-by: David S. Miller --- samples/bpf/bpf_load.c|8 +++ samples/sockmap/sockmap_kern.c| 52 +++ samples/sockmap/sockmap_user.c| 67

[bpf-next PATCH v3 10/18] bpf: add verifier tests for BPF_PROG_TYPE_SK_MSG

2018-03-18 Thread John Fastabend
Test read and writes for BPF_PROG_TYPE_SK_MSG. Signed-off-by: John Fastabend Acked-by: David S. Miller --- tools/testing/selftests/bpf/test_verifier.c | 54 +++ 1 file changed, 54 insertions(+) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing

[bpf-next PATCH v3 08/18] bpf: sk_msg program helper bpf_sk_msg_pull_data

2018-03-18 Thread John Fastabend
gth). This is required to ensure any writes by the BPF program are correctly transmitted. Lastly this operation will invalidate any previous data checks so BPF programs will have to revalidate pointers after making this BPF call. Signed-off-by: John Fastab

[bpf-next PATCH v3 17/18] bpf: sockmap sample test for bpf_msg_pull_data

2018-03-18 Thread John Fastabend
noisy variants) to run pull_data inline with those tests. By giving user direct control over the variables we can easily do negative testing as well as positive tests. Signed-off-by: John Fastabend Acked-by: David S. Miller --- samples/sockmap/sockmap_kern.c| 79

Re: [bpf-next PATCH v3 07/18] bpf: sockmap, add msg_cork_bytes() helper

2018-03-19 Thread John Fastabend
On 03/19/2018 09:30 AM, Alexei Starovoitov wrote: > On Sun, Mar 18, 2018 at 12:57:20PM -0700, John Fastabend wrote: >> In the case where we need a specific number of bytes before a >> verdict can be assigned, even if the data spans multiple sendmsg >> or sendfile calls. Th

Re: [bpf-next PATCH v3 08/18] bpf: sk_msg program helper bpf_sk_msg_pull_data

2018-03-19 Thread John Fastabend
On 03/19/2018 01:24 PM, Alexei Starovoitov wrote: > On Sun, Mar 18, 2018 at 12:57:25PM -0700, John Fastabend wrote: >> Currently, if a bpf sk msg program is run the program >> can only parse data that the (start,end) pointers already >> consumed. For sendmsg hooks thi

Re: [bug, bisected] pfifo_fast causes packet reordering

2018-03-21 Thread John Fastabend
On 03/21/2018 03:01 AM, Jakob Unterwurzacher wrote: > On 16.03.18 11:26, Jakob Unterwurzacher wrote: >> On 15.03.18 23:30, John Fastabend wrote: >>>> I have reproduced it using two USB network cards connected to each other. >>>> The test tool sends UDP packets co

Re: [bug, bisected] pfifo_fast causes packet reordering

2018-03-21 Thread John Fastabend
On 03/21/2018 12:44 PM, Jakob Unterwurzacher wrote: > On 21.03.18 19:43, John Fastabend wrote: >> Thats my theory at least. Are you able to test a patch if I generate >> one to fix this? > > Yes, no problem. Can you try this, diff --git a/include/net/sch_generic.h b/incl

Re: [bug, bisected] pfifo_fast causes packet reordering

2018-03-24 Thread John Fastabend
On 03/22/2018 03:16 AM, Jakob Unterwurzacher wrote: > On 21.03.18 21:52, John Fastabend wrote: >> Can you try this, >> >> diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h >> index d4907b5..1e596bd 100644 >> --- a/include/net/sch_generic.h &

[net PATCH] net: sched, fix OOO packets with pfifo_fast

2018-03-24 Thread John Fastabend
queue NICs we can still have a dequeue in-flight per qdisc, which is typically per CPU. Fixes: c5ad119fb6c0 ("net: sched: pfifo_fast use skb_array") Reported-by: Jakob Unterwurzacher Signed-off-by: John Fastabend --- include/net/sch_generic.h |1 + net/sched/sch_generic.c | 13 +

Re: [net PATCH] net: sched, fix OOO packets with pfifo_fast

2018-03-24 Thread John Fastabend
On 03/24/2018 02:15 PM, Eric Dumazet wrote: > > > On 03/24/2018 01:13 PM, John Fastabend wrote: >> After the qdisc lock was dropped in pfifo_fast we allow multiple >> enqueue threads and dequeue threads to run in parallel. On the >> enqueue side the skb bit ooo_okay is

[net PATCH v2] net: sched, fix OOO packets with pfifo_fast

2018-03-24 Thread John Fastabend
queue NICs we can still have a dequeue in-flight per qdisc, which is typically per CPU. Fixes: c5ad119fb6c0 ("net: sched: pfifo_fast use skb_array") Reported-by: Jakob Unterwurzacher Signed-off-by: John Fastabend --- include/net/sch_generic.h |1 + net/sched/sch_gen

Re: [net PATCH v2] net: sched, fix OOO packets with pfifo_fast

2018-03-26 Thread John Fastabend
On 03/26/2018 09:36 AM, David Miller wrote: > From: John Fastabend > Date: Sat, 24 Mar 2018 22:25:06 -0700 > >> After the qdisc lock was dropped in pfifo_fast we allow multiple >> enqueue threads and dequeue threads to run in parallel. On the >> enqueue side the

[bpf-next PATCH 1/4] bpf: sockmap redirect ingress support

2018-03-26 Thread John Fastabend
write side) so that we have a way to wake up polling socks when a scatterlist is redirected to that sock. After this all that is needed is for the redirect helper to push the scatterlist into the psock receive queue. Signed-off-by: John Fastabend --- include/linux/filter.h |1 include/net

[bpf-next PATCH 2/4] bpf: sockmap, add BPF_F_INGRESS tests

2018-03-26 Thread John Fastabend
Add a set of tests to verify ingress flag in redirect helpers works correctly with various msg sizes. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_kern.c | 41 +-- samples/sockmap/sockmap_test.sh | 22 - samples/sockmap

[bpf-next PATCH 0/4] bpf, sockmap BPF_F_INGRESS support

2018-03-26 Thread John Fastabend
and four add tests for the corresponding ingress redirect hooks. Follow on patches can address busy polling support, but next series from me will move the sockmap sample program into selftests. Thanks, John --- John Fastabend (4): bpf: sockmap redirect ingress support bpf: sockmap

[bpf-next PATCH 3/4] bpf: sockmap, BPF_F_INGRESS flag for BPF_SK_SKB_STREAM_VERDICT:

2018-03-26 Thread John Fastabend
Add support for the BPF_F_INGRESS flag in skb redirect helper. To do this convert skb into a scatterlist and push into ingress queue. This is the same logic that is used in the sk_msg redirect helper so it should feel familiar. Signed-off-by: John Fastabend --- include/linux/filter.h |1

[bpf-next PATCH 4/4] bpf: sockmap, more BPF_SK_SKB_STREAM_VERDICT tests

2018-03-26 Thread John Fastabend
Add BPF_SK_SKB_STREAM_VERDICT tests for ingress hook. While we do this also bring stream tests in-line with MSG based testing. A map for skb options is added for userland to push options at BPF programs. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_kern.c | 21

Re: [net PATCH v2] net: sched, fix OOO packets with pfifo_fast

2018-03-26 Thread John Fastabend
On 03/26/2018 10:30 AM, Cong Wang wrote: > On Sat, Mar 24, 2018 at 10:25 PM, John Fastabend > wrote: >> After the qdisc lock was dropped in pfifo_fast we allow multiple >> enqueue threads and dequeue threads to run in parallel. On the >> enqueue side the skb bit ooo_ok

Re: [PATCH bpf-next] bpf: sockmap: initialize sg table entries properly

2018-03-26 Thread John Fastabend
; @@ -774,7 +778,6 @@ static int bpf_tcp_sendpage(struct sock *sk, struct page > *page, > goto out_err; > > psock->sg_size += size; > - sg = &m->sg_data[m->sg_end]; > sg_set_page(sg, page, size, offset); > get_page(page); > m->sg_copy[m->sg_end] = true; > Nice, catch. I probably should audit though code paths as well and run the test suite with CONFIG_DEBUG_SG. There might be a couple other spots where I open coded the sg elements. Thanks, John

[net-next PATCH 00/14] lockless qdisc series

2017-12-07 Thread John Fastabend
m, I left out lockdep annotation for a follow on series to add lockdep more completely, rather than just in code I touched. Comments and feedback welcome. Thanks, John --- John Fastabend (14): net: sched: cleanup qdisc_run and __qdisc_run semantics net: sched: allow qdiscs to handle locking

[net-next PATCH 01/14] net: sched: cleanup qdisc_run and __qdisc_run semantics

2017-12-07 Thread John Fastabend
Currently __qdisc_run calls qdisc_run_end() but does not call qdisc_run_begin(). This makes it hard to track pairs of qdisc_run_{begin,end} across function calls. To simplify reading these code paths this patch moves begin/end calls into qdisc_run(). Signed-off-by: John Fastabend --- include

[net-next PATCH 02/14] net: sched: allow qdiscs to handle locking

2017-12-07 Thread John Fastabend
doing the enqueue/dequeue operations when tested with pktgen. Signed-off-by: John Fastabend --- include/net/sch_generic.h |1 + net/core/dev.c| 26 ++ net/sched/sch_generic.c | 30 -- 3 files changed, 43 insertions(+), 14

[net-next PATCH 04/14] net: sched: provide per cpu qstat helpers

2017-12-07 Thread John Fastabend
The per cpu qstats support was added with per cpu bstat support which is currently used by the ingress qdisc. This patch adds a set of helpers needed to make other qdiscs that use qstats per cpu as well. Signed-off-by: John Fastabend --- include/net/sch_generic.h | 35

[net-next PATCH 03/14] net: sched: remove remaining uses for qdisc_qlen in xmit path

2017-12-07 Thread John Fastabend
ow it returns true. However in this case all call sites of sch_direct_xmit will implement a dequeue() and get a null skb and abort. This trades tracking qlen in the hotpath for an extra dequeue operation. Overall this seems to be good for performance. Signed-off-by: John Fastabend --- includ

[net-next PATCH 05/14] net: sched: a dflt qdisc may be used with per cpu stats

2017-12-07 Thread John Fastabend
ate the qdisc object so we don't have dangling allocations after qdisc init. Signed-off-by: John Fastabend --- include/net/sch_generic.h |1 + net/sched/sch_generic.c | 16 2 files changed, 17 insertions(+) diff --git a/include/net/sch_generic.h b/include/net/sch_g

[net-next PATCH 06/14] net: sched: explicit locking in gso_cpu fallback

2017-12-07 Thread John Fastabend
once its possible to have multiple sk_buffs here so we turn gso_skb into a queue. This should be the edge case and if we see this frequently then the netdev/qdisc layer needs to back off. Signed-off-by: John Fastabend --- include/net/sch_generic.h | 20 ++- net/sched/sch_generic.c

[net-next PATCH 08/14] net: sched: use skb list for skb_bad_tx

2017-12-07 Thread John Fastabend
Similar to how gso is handled use skb list for skb_bad_tx this is required with lockless qdiscs because we may have multiple cores attempting to push skbs into skb_bad_tx concurrently Signed-off-by: John Fastabend --- include/net/sch_generic.h |2 - net/sched/sch_generic.c | 106

[net-next PATCH 09/14] net: sched: check for frozen queue before skb_bad_txq check

2017-12-07 Thread John Fastabend
I can not think of any reason to pull the bad txq skb off the qdisc if the txq we plan to send this on is still frozen. So check for frozen queue first and abort before dequeuing either skb_bad_txq skb or normal qdisc dequeue() skb. Signed-off-by: John Fastabend --- net/sched/sch_generic.c

[net-next PATCH 10/14] net: sched: helpers to sum qlen and qlen for per cpu logic

2017-12-07 Thread John Fastabend
-off-by: John Fastabend --- include/net/sch_generic.h | 20 net/sched/sch_api.c |3 ++- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 4717c4b..2fbae2c9 100644 --- a/include/net

[net-next PATCH 11/14] net: sched: add support for TCQ_F_NOLOCK subqueues to sch_mq

2017-12-07 Thread John Fastabend
this case add a check when calculating stats and aggregate the per cpu stats if needed. Also exports __gnet_stats_copy_queue() to use as a helper function. Signed-off-by: John Fastabend --- include/net/gen_stats.h |3 +++ net/core/gen_stats.c|9 + net/sched/sch_mq.c

[net-next PATCH 07/14] net: sched: drop qdisc_reset from dev_graft_qdisc

2017-12-07 Thread John Fastabend
ce period and letting the qdisc_destroy operation clean up the qdisc correctly. Note, a refcnt greater than 1 would cause the destroy operation to be aborted however if this ever happened the reference to the qdisc would be lost and we would have a memory leak. Signed-off-by: John Fastabend --- net/s

[net-next PATCH 13/14] net: skb_array: expose peek API

2017-12-07 Thread John Fastabend
This adds a peek routine to skb_array.h for use with qdisc. Signed-off-by: John Fastabend --- include/linux/skb_array.h |5 + 1 file changed, 5 insertions(+) diff --git a/include/linux/skb_array.h b/include/linux/skb_array.h index 8621ffd..c7addf3 100644 --- a/include/linux/skb_array.h

[net-next PATCH 12/14] net: sched: add support for TCQ_F_NOLOCK subqueues to sch_mqprio

2017-12-07 Thread John Fastabend
andle this case add a check when calculating stats and aggregate the per cpu stats if needed. Signed-off-by: John Fastabend --- net/sched/sch_mq.c | 35 +++- net/sched/sch_mqprio.c | 69 +--- 2 files changed, 69 insertions(+), 35

[net-next PATCH 14/14] net: sched: pfifo_fast use skb_array

2017-12-07 Thread John Fastabend
graft operation occurs. This also removes the logic used to pick the next band to dequeue from and instead just checks a per priority array for packets from top priority to lowest. This might need to be a bit more clever but seems to work for now. Signed-off-by: John Fastabend --- net/sched

Re: Huge memory leak with 4.15.0-rc2+

2017-12-11 Thread John Fastabend
sage graph: > https://ibb.co/hU97kG > > And there is rising slab_unrecl - Amount of unreclaimable memory used > for slab kernel allocations > > > Forgot to add that im using hfsc and qdiscs like pfifo on classes. > > Maybe some error case I missed in the qdisc patches I'm looking into it. Thanks, John

Re: Huge memory leak with 4.15.0-rc2+

2017-12-13 Thread John Fastabend
On 12/12/2017 09:57 AM, Paweł Staszewski wrote: > > > W dniu 2017-12-11 o 23:27, Paweł Staszewski pisze: >> >> >> W dniu 2017-12-11 o 23:15, John Fastabend pisze: >>> On 12/11/2017 01:48 PM, Paweł Staszewski wrote: >>>> >>>>

Re: ixgbe tuning reset when XDP is setup

2017-12-15 Thread John Fastabend
ly should _not_ be a side effect of enabling XDP on a device. > Agreed, CC Emil and Alex we should restore these settings after the reconfiguration done to support a queue per core. .John

Re: [Patch net-next] net_sched: properly check for empty skb array on error path

2017-12-18 Thread John Fastabend
: pfifo_fast use skb_array") > Reported-by: syzbot > Cc: John Fastabend > Signed-off-by: Cong Wang > --- > net/sched/sch_generic.c | 8 +++- > 1 file changed, 7 insertions(+), 1 deletion(-) >

Re: [Patch net-next] net_sched: properly check for empty skb array on error path

2017-12-18 Thread John Fastabend
On 12/18/2017 06:20 PM, Cong Wang wrote: > On Mon, Dec 18, 2017 at 5:25 PM, John Fastabend > wrote: >> On 12/18/2017 02:34 PM, Cong Wang wrote: >>> First, the check of &q->ring.queue against NULL is wrong, it >>> is always false. We should check the value rath

Re: [Patch net-next] net_sched: properly check for empty skb array on error path

2017-12-18 Thread John Fastabend
On 12/18/2017 08:31 PM, Cong Wang wrote: > On Mon, Dec 18, 2017 at 7:58 PM, John Fastabend > wrote: >> On 12/18/2017 06:20 PM, Cong Wang wrote: >>> On Mon, Dec 18, 2017 at 5:25 PM, John Fastabend >>> wrote: >>>> On 12/18/2017 02:34 PM, Cong Wang wrote

Re: [PATCH bpf] bpf: samples/sockmap fix Makefile for build error

2018-02-13 Thread John Fastabend
>> Signed-off-by: Prashant Bhole > > Applied to bpf-next, thanks Prashant! > Thanks! Acked-by: John Fastabend

Re: [PATCH bpf] bpf: samples/sockmap detach sock ops program

2018-02-13 Thread John Fastabend
Bhole > > Also applied to bpf-next considering that John is planning to move > the whole file into selftests soon. > Thanks for the fix here as well. Acked-by: John Fastabend

Re: [bpf-next PATCH] net: avoid including xdp.h in filter.h

2018-02-13 Thread John Fastabend
On 02/13/2018 05:57 PM, Alexei Starovoitov wrote: > On Tue, Feb 13, 2018 at 02:15:36PM +0100, Jesper Dangaard Brouer wrote: >> If is sufficient with a forward declaration of struct xdp_rxq_info in >> linux/filter.h, which avoids including net/xdp.h. This was originally >&

Re: [PATCH bpf-next] bpf: fix sock_map_alloc() error path

2018-02-13 Thread John Fastabend
t;>> a map_alloc() handler to return an error, not a NULL pointer, >>> otherwise we crash later in find_and_alloc_map() >>> >>> Fixes: 174a79ff9515 ("bpf: sockmap with sk redirect support") >>> Signed-off-by: Eric Dumazet >>> Reported-b

Re: [net-next] samples/bpf: adjust rlimit RLIMIT_MEMLOCK for xdp_redirect

2018-02-13 Thread John Fastabend
gt; is already runnig. > > Signed-off-by: Tushar Dave > --- Looks good to me. Acked-by: John Fastabend

Re: [RFC net PATCH] virtio_net: disable XDP_REDIRECT in receive_mergeable() case

2018-02-16 Thread John Fastabend
a physical device (ixgbe) into a > guest device. Ran a benchmark: > - 0.5 Mpps with normal code path into device with driver tun > - 3.7 Mpps with XDP_REDIRECT from ixgbe into same device Yep also very interesting but a different use case. This is accelerating the hypervisor vswitch.

Re: [RFC net PATCH] virtio_net: disable XDP_REDIRECT in receive_mergeable() case

2018-02-20 Thread John Fastabend
On 02/20/2018 03:17 AM, Jesper Dangaard Brouer wrote: > On Fri, 16 Feb 2018 09:19:02 -0800 > John Fastabend wrote: > >> On 02/16/2018 07:41 AM, Jesper Dangaard Brouer wrote: >>> On Fri, 16 Feb 2018 13:31:37 +0800 >>> Jason Wang wrote: >>> >>

Re: [net PATCH 1/4] virtio_net: disable XDP_REDIRECT in receive_mergeable() case

2018-02-20 Thread John Fastabend
; function when attaching an XDP program, and also change the memory > model to be compatible with XDP when attaching an XDP prog. > > Fixes: 186b3c998c50 ("virtio-net: support XDP_REDIRECT") > Signed-off-by: Jesper Dangaard Brouer > --- Acked-by: John Fastabend Seems re

Re: [net PATCH 2/4] virtio_net: fix XDP code path in receive_small()

2018-02-20 Thread John Fastabend
> > Fixes: bb91accf2733 ("virtio-net: XDP support for small buffers") > Signed-off-by: Jesper Dangaard Brouer > --- Acked-by: John Fastabend Thanks! > drivers/net/virtio_net.c |2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/driver

Re: [net PATCH 3/4] virtio_net: fix memory leak in XDP_REDIRECT

2018-02-20 Thread John Fastabend
t follow API of xdp_do_redirect(), which on errors leave it up to > the caller to free the page, of the failed send operation. > > Fixes: 186b3c998c50 ("virtio-net: support XDP_REDIRECT") > Signed-off-by: Jesper Dangaard Brouer > --- Acked-by: John Fastabend

Re: [net PATCH 4/4] virtio_net: fix ndo_xdp_xmit crash towards dev not ready for XDP

2018-02-20 Thread John Fastabend
Signed-off-by: Jesper Dangaard Brouer > --- Yep, nice catch. I caught this in ixgbe but didn't manage to pull it into virtio. Acked-by: John Fastabend > drivers/net/virtio_net.c | 12 +++- > 1 file changed, 11 insertions(+), 1 deletion(-) > > diff --git a/driv

Re: [RFC PATCH bpf-next 05/12] bpf/verifier: detect loops dynamically rather than statically

2018-02-23 Thread John Fastabend
On 02/23/2018 09:40 AM, Edward Cree wrote: > Add in a new chain of parent states, which does not cross function-call > boundaries, and check whether our current insn_idx appears anywhere in > the chain. Since all jump targets have state-list marks (now placed > by prepare_cfg_marks(), which rep

Re: [RFC PATCH bpf-next 07/12] bpf/verifier: allow bounded loops with JLT/true back-edge

2018-02-23 Thread John Fastabend
On 02/23/2018 09:41 AM, Edward Cree wrote: > Where the register umin_value is increasing sufficiently fast, the loop > will terminate after a reasonable number of iterations, so we can allow > to keep walking it. Continuing to walk the loop is problematic because we hit the complexity limit. Wha

Re: [bpf-next PATCH 1/7] bpf: refactor sockmap sample program update for arg parsing

2018-01-09 Thread John Fastabend
On 01/09/2018 05:30 AM, Jesper Dangaard Brouer wrote: > On Mon, 08 Jan 2018 10:05:07 -0800 > John Fastabend wrote: > >> sockmap sample program takes arguments from cmd line but it reads them >> in using offsets into the array. Because we want to add more arguments >> i

Re: [bpf-next PATCH 4/7] bpf: sockmap sample, report bytes/sec

2018-01-09 Thread John Fastabend
On 01/09/2018 05:27 AM, Jesper Dangaard Brouer wrote: > On Mon, 08 Jan 2018 10:05:58 -0800 > John Fastabend wrote: > >> Report bytes/sec sent as well as total bytes. Useful to get rough >> idea how different configurations and usage patterns perform with >> sockmap

Re: [PATCH] ptr_ring: document usage around __ptr_ring_peek

2018-01-10 Thread John Fastabend
On 01/10/2018 06:03 AM, Michael S. Tsirkin wrote: > This explains why is the net usage of __ptr_ring_peek > actually ok without locks. > > Signed-off-by: Michael S. Tsirkin > --- > > John - I think this is more or less what you meant. Is that right? Yep, thanks for fo

[bpf-next PATCH v2 0/7] sockmap sample update

2018-01-10 Thread John Fastabend
seful, the reporting is bare bones, etc. But, IMO lets push this now rather than sit on it for weeks until I get time to do the above improvements. Additional patches can address the other limitations/issues. v2: removed bogus file added by patch 3/7 --- John Fastabend (7): bpf: refactor sock

[bpf-next PATCH v2 1/7] bpf: refactor sockmap sample program update for arg parsing

2018-01-10 Thread John Fastabend
future. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 142 +--- 1 file changed, 103 insertions(+), 39 deletions(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c index 7cc9d22..5cbe7a5 100644 --- a/samples/sockmap

[bpf-next PATCH v2 2/7] bpf: add sendmsg option for testing BPF programs

2018-01-10 Thread John Fastabend
supported, but more can be added as needed. The new help argument gives the following, Usage: ./sockmap --cgroup options: --help -h --cgroup -c --rate -r --verbose -v --iov_count-i --length -l --test -t Signed-off-by: John Fastabend

[bpf-next PATCH v2 4/7] bpf: sockmap sample, report bytes/sec

2018-01-10 Thread John Fastabend
Report bytes/sec sent as well as total bytes. Useful to get rough idea how different configurations and usage patterns perform with sockmap. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 37 - 1 file changed, 32 insertions(+), 5

[bpf-next PATCH v2 3/7] bpf: sockmap sample, use fork() for send and recv

2018-01-10 Thread John Fastabend
get many GBps of data which helps exercise the sockmap code. Signed-off-by: John Fastabend --- samples/sockmap/Makefile |2 + samples/sockmap/sockmap_user.c | 58 +--- 2 files changed, 43 insertions(+), 17 deletions(-) diff --git a/samples/sockmap

[bpf-next PATCH v2 5/7] bpf: sockmap sample add base test without any BPF for comparison

2018-01-10 Thread John Fastabend
Add a base test that does not use BPF hooks to test baseline case. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 26 +- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c

[bpf-next PATCH v2 6/7] bpf: sockmap put client sockets in blocking mode

2018-01-10 Thread John Fastabend
-by: John Fastabend --- samples/sockmap/sockmap_user.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c index eb19d14..9496b2c 100644 --- a/samples/sockmap/sockmap_user.c +++ b/samples/sockmap/sockmap_user.c

[bpf-next PATCH v2 7/7] bpf: sockmap set rlimit

2018-01-10 Thread John Fastabend
Avoid extra step of setting limit from cmdline and do it directly in the program. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c |7 +++ 1 file changed, 7 insertions(+) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c index 9496b2c..16c19c5

Re: [bpf-next PATCH v2 1/7] bpf: refactor sockmap sample program update for arg parsing

2018-01-11 Thread John Fastabend
On 01/11/2018 01:05 PM, Martin KaFai Lau wrote: > On Wed, Jan 10, 2018 at 10:39:04AM -0800, John Fastabend wrote: >> sockmap sample program takes arguments from cmd line but it reads them >> in using offsets into the array. Because we want to add more arguments >> in the

Re: [bpf-next PATCH v2 3/7] bpf: sockmap sample, use fork() for send and recv

2018-01-11 Thread John Fastabend
On 01/11/2018 01:08 PM, Martin KaFai Lau wrote: > On Wed, Jan 10, 2018 at 10:39:37AM -0800, John Fastabend wrote: >> Currently for SENDMSG tests first send completes then recv runs. This >> does not work well for large data sizes and/or many iterations. So >> fork the recv and

Re: [bpf-next PATCH v2 5/7] bpf: sockmap sample add base test without any BPF for comparison

2018-01-11 Thread John Fastabend
On 01/11/2018 01:10 PM, Martin KaFai Lau wrote: > On Wed, Jan 10, 2018 at 10:40:11AM -0800, John Fastabend wrote: >> Add a base test that does not use BPF hooks to test baseline case. >> >> Signed-off-by: John Fastabend >> --- >> samp

Re: [bpf-next PATCH v2 1/7] bpf: refactor sockmap sample program update for arg parsing

2018-01-11 Thread John Fastabend
On 01/10/2018 05:25 PM, Daniel Borkmann wrote: > On 01/10/2018 07:39 PM, John Fastabend wrote: >> sockmap sample program takes arguments from cmd line but it reads them >> in using offsets into the array. Because we want to add more arguments >> in the future lets do pro

Re: [bpf-next PATCH v2 3/7] bpf: sockmap sample, use fork() for send and recv

2018-01-11 Thread John Fastabend
On 01/10/2018 05:31 PM, Daniel Borkmann wrote: > On 01/10/2018 07:39 PM, John Fastabend wrote: >> Currently for SENDMSG tests first send completes then recv runs. This >> does not work well for large data sizes and/or many iterations. So >> fork the recv and send handler so

Re: [bpf-next PATCH v2 1/7] bpf: refactor sockmap sample program update for arg parsing

2018-01-11 Thread John Fastabend
On 01/11/2018 08:31 PM, John Fastabend wrote: > On 01/10/2018 05:25 PM, Daniel Borkmann wrote: >> On 01/10/2018 07:39 PM, John Fastabend wrote: >>> sockmap sample program takes arguments from cmd line but it reads them >>> in using offsets into the array. Because we

[bpf-next PATCH v3 0/7] sockmap sample update

2018-01-11 Thread John Fastabend
sighandler update, 2/7 free iov in error cases 3/7 fix bogus makefile change, bail out early on errors Thanks Daniel and Martin for the reviews! --- John Fastabend (7): bpf: refactor sockmap sample program update for arg parsing bpf: add sendmsg option for testing BPF programs

[bpf-next PATCH v3 1/7] bpf: refactor sockmap sample program update for arg parsing

2018-01-11 Thread John Fastabend
future. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 164 1 file changed, 113 insertions(+), 51 deletions(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c index 7cc9d22..17400d4 100644 --- a/samples/sockmap

[bpf-next PATCH v3 3/7] bpf: sockmap sample, use fork() for send and recv

2018-01-11 Thread John Fastabend
get many GBps of data which helps exercise the sockmap code. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 58 +--- 1 file changed, 42 insertions(+), 16 deletions(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap

[bpf-next PATCH v3 5/7] bpf: sockmap sample add base test without any BPF for comparison

2018-01-11 Thread John Fastabend
Add a base test that does not use BPF hooks to test baseline case. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 26 +- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c

[bpf-next PATCH v3 2/7] bpf: add sendmsg option for testing BPF programs

2018-01-11 Thread John Fastabend
supported, but more can be added as needed. The new help argument gives the following, Usage: ./sockmap --cgroup options: --help -h --cgroup -c --rate -r --verbose -v --iov_count-i --length -l --test -t Signed-off-by: John Fastabend

[bpf-next PATCH v3 6/7] bpf: sockmap put client sockets in blocking mode

2018-01-11 Thread John Fastabend
-by: John Fastabend --- samples/sockmap/sockmap_user.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c index c3295a7..818766b 100644 --- a/samples/sockmap/sockmap_user.c +++ b/samples/sockmap/sockmap_user.c

[bpf-next PATCH v3 7/7] bpf: sockmap set rlimit

2018-01-11 Thread John Fastabend
Avoid extra step of setting limit from cmdline and do it directly in the program. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c |7 +++ 1 file changed, 7 insertions(+) diff --git a/samples/sockmap/sockmap_user.c b/samples/sockmap/sockmap_user.c index 818766b..a6dab97

[bpf-next PATCH v3 4/7] bpf: sockmap sample, report bytes/sec

2018-01-11 Thread John Fastabend
Report bytes/sec sent as well as total bytes. Useful to get rough idea how different configurations and usage patterns perform with sockmap. Signed-off-by: John Fastabend --- samples/sockmap/sockmap_user.c | 37 - 1 file changed, 32 insertions(+), 5

[bpf-next PATCH 0/7] Add BPF_PROG_TYPE_SK_MSG and attach pt

2018-01-12 Thread John Fastabend
e automated side. We can push this as an independent patch set. --- John Fastabend (7): net: add a UID to use for ULP socket assignment sock: make static tls function alloc_sg generic sock helper sockmap: convert refcnt to an atomic refcnt net: do_tcp_sendpages flag to avo

[bpf-next PATCH 1/7] net: add a UID to use for ULP socket assignment

2018-01-12 Thread John Fastabend
ULP is known and done on the kernel side. In this case the named lookup is not needed. Remove pr_notice, user gets an error code back and should check that rather than rely on logs. Signed-off-by: John Fastabend --- include/net/tcp.h |5 + net/ipv4/tcp_ulp.c | 51

[bpf-next PATCH 3/7] sockmap: convert refcnt to an atomic refcnt

2018-01-12 Thread John Fastabend
sk_callback_lock() here (on every send) seems less than ideal, also it may sleep in cases where we hit memory pressure. Instead of dealing with these issues in some clever way simply make the reference counting a refcnt_t type and do proper atomic ops. Signed-off-by: John Fastabend --- kernel/bpf

[bpf-next PATCH 2/7] sock: make static tls function alloc_sg generic sock helper

2018-01-12 Thread John Fastabend
The TLS ULP module builds scatterlists from a sock using page_frag_refill(). This is going to be useful for other ULPs so move it into sock file for more general use. In the process remove useless goto at end of while loop. Signed-off-by: John Fastabend --- include/net/sock.h |4 +++ net

[bpf-next PATCH 4/7] net: do_tcp_sendpages flag to avoid SKBTX_SHARED_FRAG

2018-01-12 Thread John Fastabend
When calling do_tcp_sendpages() from in kernel and we know the data has no references from user side we can omit SKBTX_SHARED_FRAG flag. This patch adds an internal flag, NO_SKBTX_SHARED_FRAG that can be used to omit setting SKBTX_SHARED_FRAG. Signed-off-by: John Fastabend --- include/linux

[bpf-next PATCH 5/7] bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data

2018-01-12 Thread John Fastabend
list. Currently, we use MAX_SKB_FRAGS simply because this was being used already in the TLS case. Future work to improve the kernel sk APIs to tune this depending on workload may be useful. This is a trade-off between memory usage and B/s performance. Signed-off-by: John Fastabend --- include/li

[bpf-next PATCH 6/7] bpf: add map tests for BPF_PROG_TYPE_SK_MSG

2018-01-12 Thread John Fastabend
Add map tests to attach BPF_PROG_TYPE_SK_MSG types to a sockmap. Signed-off-by: John Fastabend --- tools/include/uapi/linux/bpf.h | 16 ++ tools/testing/selftests/bpf/Makefile |3 + tools/testing/selftests/bpf/bpf_helpers.h |2 + tools

[bpf-next PATCH 7/7] bpf: add verifier tests for BPF_PROG_TYPE_SK_MSG

2018-01-12 Thread John Fastabend
Test read and writes for BPF_PROG_TYPE_SK_MSG. Signed-off-by: John Fastabend --- tools/testing/selftests/bpf/test_verifier.c | 54 +++ 1 file changed, 54 insertions(+) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf

Re: [bpf-next PATCH 4/7] net: do_tcp_sendpages flag to avoid SKBTX_SHARED_FRAG

2018-01-12 Thread John Fastabend
On 01/12/2018 12:10 PM, Eric Dumazet wrote: > On Fri, 2018-01-12 at 10:10 -0800, John Fastabend wrote: >> When calling do_tcp_sendpages() from in kernel and we know the data >> has no references from user side we can omit SKBTX_SHARED_FRAG flag. >> This patch

Re: [bpf-next PATCH 4/7] net: do_tcp_sendpages flag to avoid SKBTX_SHARED_FRAG

2018-01-12 Thread John Fastabend
On 01/12/2018 12:46 PM, Eric Dumazet wrote: > On Fri, 2018-01-12 at 12:26 -0800, John Fastabend wrote: >> On 01/12/2018 12:10 PM, Eric Dumazet wrote: >>> On Fri, 2018-01-12 at 10:10 -0800, John Fastabend wrote: >>>> When calling do_tcp_sendpages() from in kernel an

Re: [PATCH net] ibmvnic: Fix pending MAC address changes

2018-01-15 Thread John Allen
ED" state before setting the > MAC address. > > Fixes: c26eba03e407 ("ibmvnic: Update reset infrastructure to support tunable > parameters") > Signed-off-by: Thomas Falcon > --- Reviewed-by: John Allen > drivers/net/ethernet/ibm/ibmvnic.c | 13 +++-

[PATCH 0/3 net] ibmvnic: Reset behavior fixes

2018-01-15 Thread John Allen
ix pending MAC address changes" which is currently submitted awaiting acceptance to the net tree. John Allen (3): ibmvnic: Modify buffer size on failover ibmvnic: Revert to previous mtu when unsupported value requested ibmvnic: Allocate and request vpd in init_resources drivers/n

[PATCH 1/3 net] ibmvnic: Modify buffer size on failover

2018-01-15 Thread John Allen
requirement, the vnic server will fail to post rx buffers due to inadequate space in our rx pool. This patch fixes the issue by checking whether or not the buffer size has changed on a reset and if it has, reallocate the buffer. Signed-off-by: John Allen --- diff --git a/drivers/net/ethernet/ibm/ibmvnic.c

[PATCH 2/3 net] ibmvnic: Revert to previous mtu when unsupported value requested

2018-01-15 Thread John Allen
tu to some completely different value most of the time. This patch fixes the issue by logging in with the previously used mtu value and printing an error message saying that the given mtu is unsupported. Signed-off-by: John Allen --- diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/driver

[PATCH 3/3 net] ibmvnic: Allocate and request vpd in init_resources

2018-01-15 Thread John Allen
it will be symmetrically freed during release resources. Signed-off-by: John Allen --- diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index bb56460..f0dbb76 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -867,7

Re: [PATCH 1/3 net] ibmvnic: Modify buffer size on failover

2018-01-15 Thread John Allen
On 01/15/2018 03:11 PM, John Allen wrote: > Using newer backing devices can cause the required padding at the end of > rx buffers to change. Currently we assume that the size of buffers will > never change, but in the case that we failover from a backing device with > smaller padding

Re: [bpf-next PATCH 5/7] bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data

2018-01-16 Thread John Fastabend
On 01/16/2018 06:25 PM, Alexei Starovoitov wrote: > On Fri, Jan 12, 2018 at 10:11:11AM -0800, John Fastabend wrote: >> This implements a BPF ULP layer to allow policy enforcement and >> monitoring at the socket layer. In order to support this a new >> program type BPF_PROG_T

Re: [bpf-next PATCH 5/7] bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data

2018-01-17 Thread John Fastabend
On 01/16/2018 10:20 PM, Alexei Starovoitov wrote: > On Tue, Jan 16, 2018 at 09:49:16PM -0800, John Fastabend wrote: >> >>> but this program will see only first SG ? >> >> Correct, to read further into the msg we would need to have a helper >> or some other way

Re: [bpf-next PATCH 5/7] bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data

2018-01-18 Thread John Fastabend
On 01/17/2018 02:04 PM, Martin KaFai Lau wrote: > On Fri, Jan 12, 2018 at 10:11:11AM -0800, John Fastabend wrote: >> This implements a BPF ULP layer to allow policy enforcement and >> monitoring at the socket layer. In order to support this a new >> program type BPF_PROG_TYPE_

[PATCH net v2 0/3] ibmvnic: Reset behavior fixes

2018-01-18 Thread John Allen
variables to the adapter struct to track how many rx/tx pools have actually been allocated and modify the release pools routines to use these values rather than the possibly incorrect req_rx/tx_queues values. John Allen (3): ibmvnic: Modify buffer size and number of queues on failover ibmvnic

[PATCH net v2 1/3] ibmvnic: Modify buffer size and number of queues on failover

2018-01-18 Thread John Allen
server, attempt to free long term buffers that are no longer there, or not free long term buffers that should be freed. This patch resolves the issue by checking whether any of these values change, and if so perform the necessary re-allocations. Signed-off-by: John Allen --- v2: Added the line to

<    4   5   6   7   8   9   10   11   12   13   >