Re: ipsec impact on performance

2015-12-08 Thread Sowmini Varadhan
On (12/08/15 12:32), Steffen Klassert wrote: > > Would be nice if you could share the results. Comments are Sure, not a problem. Give me some time though, I'm also looking into the skb_cow_data and other memory-management issues that were flagged on this thread. I'll have all this info by netde

Re: ipsec impact on performance

2015-12-08 Thread Steffen Klassert
On Mon, Dec 07, 2015 at 06:27:48AM -0500, Sowmini Varadhan wrote: > On (12/07/15 09:40), Steffen Klassert wrote: > > > > I've pushed it to > > > > https://git.kernel.org/cgit/linux/kernel/git/klassert/linux-stk.git/log/?h=net-next-ipsec-offload > > > > It is just example code, nothing that I wou

Re: ipsec impact on performance

2015-12-07 Thread Sowmini Varadhan
On (12/07/15 09:40), Steffen Klassert wrote: > > I've pushed it to > > https://git.kernel.org/cgit/linux/kernel/git/klassert/linux-stk.git/log/?h=net-next-ipsec-offload > > It is just example code, nothing that I would show usually. > But you asked for it, so here is it :) that's fine, I dont e

Re: ipsec impact on performance

2015-12-07 Thread Steffen Klassert
On Thu, Dec 03, 2015 at 06:38:20AM -0500, Sowmini Varadhan wrote: > On (12/03/15 09:45), Steffen Klassert wrote: > > pcrypt(echainiv(authenc(hmac(sha1-ssse3),cbc-aes-aesni))) > > > > Result: > > > > iperf -c 10.0.0.12 -t 60 > > > > Clie

Re: ipsec impact on performance

2015-12-03 Thread Sandy Harris
This article is old (turn of the century) but it may have numbers worth comparing to http://www.freeswan.org/freeswan_trees/CURRENT-TREE/doc/performance.html -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo inf

Re: ipsec impact on performance

2015-12-03 Thread Sowmini Varadhan
On (12/03/15 14:33), David Miller wrote: > > Doesn't skb_cow_data() contribute significantly to the ESP base cost, > especially for TCP packets? Indeed. For esp-null, it's about half of the total time spent in esp_output (for one run that I just instrumented with perf tracepoints 2.5 ms compared

Re: ipsec impact on performance

2015-12-03 Thread Eric Dumazet
On Thu, 2015-12-03 at 14:33 -0500, David Miller wrote: > From: Sowmini Varadhan > Date: Tue, 1 Dec 2015 12:59:53 -0500 > > > I instrumented iperf with and without ipsec, just using esp-null, > > and 1 thread, to keep things simple. I'm seeing some pretty dismal > > performance numbers with ipse

Re: ipsec impact on performance

2015-12-03 Thread David Miller
From: Sowmini Varadhan Date: Tue, 1 Dec 2015 12:59:53 -0500 > I instrumented iperf with and without ipsec, just using esp-null, > and 1 thread, to keep things simple. I'm seeing some pretty dismal > performance numbers with ipsec, and trying to think of ways to > improve this. Here are my findi

Re: ipsec impact on performance

2015-12-03 Thread Steffen Klassert
On Thu, Dec 03, 2015 at 06:38:20AM -0500, Sowmini Varadhan wrote: > On (12/03/15 09:45), Steffen Klassert wrote: > > pcrypt(echainiv(authenc(hmac(sha1-ssse3),cbc-aes-aesni))) > > > > Result: > > > > iperf -c 10.0.0.12 -t 60 > > > > Clie

Re: ipsec impact on performance

2015-12-03 Thread Sowmini Varadhan
On (12/03/15 09:45), Steffen Klassert wrote: > pcrypt(echainiv(authenc(hmac(sha1-ssse3),cbc-aes-aesni))) > > Result: > > iperf -c 10.0.0.12 -t 60 > > Client connecting to 10.0.0.12, TCP port 5001 > TCP window size: 45.0 KByte (default)

Re: ipsec impact on performance

2015-12-03 Thread Steffen Klassert
On Wed, Dec 02, 2015 at 07:05:38AM -0500, Sowmini Varadhan wrote: > On (12/02/15 07:53), Steffen Klassert wrote: > > > > I'm currently working on a GRO/GSO codepath for IPsec too. The GRO part > > works already. I decapsulate/decrypt the packets on layer2 with a esp GRO > > callback function and r

Re: ipsec impact on performance

2015-12-02 Thread Rick Jones
On 12/02/2015 03:56 AM, David Laight wrote: From: Sowmini Varadhan Sent: 01 December 2015 18:37 ... I was using esp-null merely to not have the crypto itself perturb the numbers (i.e., just focus on the s/w overhead for now), but here are the numbers for the stock linux kernel stack

Re: ipsec impact on performance

2015-12-02 Thread Eric Dumazet
On Wed, 2015-12-02 at 16:12 -0500, Sowmini Varadhan wrote: > IPv6 would be an interesting academic exercise Really, you made my day ! -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.ke

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 14:01), Tom Herbert wrote: > No, please don't persist is this myopic "we'll get to IPv6 later" > model! IPv6 is a real protocol, it has significant deployment of the > Internet, and there are now whole data centers that are IPv6 only > (e.g. FB), and there are plenty of use cases of IP

Re: ipsec impact on performance

2015-12-02 Thread Tom Herbert
On Wed, Dec 2, 2015 at 1:47 PM, Sowmini Varadhan wrote: > On (12/02/15 13:44), Tom Herbert wrote: >> > IPv6 would be an interesting academic exercise, but it's going >> > to be a while before we get RDS-TCP to go over IPv6. >> > >> Huh? Who said anything about RDS-TCP? I thought you were trying to

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 13:44), Tom Herbert wrote: > > IPv6 would be an interesting academic exercise, but it's going > > to be a while before we get RDS-TCP to go over IPv6. > > > Huh? Who said anything about RDS-TCP? I thought you were trying to > improve IPsec performance... yes, and it would be nice to f

Re: ipsec impact on performance

2015-12-02 Thread Tom Herbert
On Wed, Dec 2, 2015 at 1:12 PM, Sowmini Varadhan wrote: > On (12/02/15 13:07), Tom Herbert wrote: >> That's easy enough to add to flow dissector, but is SPI really >> intended to be used an L4 entropy value? We would need to consider the > > yes. To quote https://en.wikipedia.org/wiki/Security_Par

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 13:07), Tom Herbert wrote: > That's easy enough to add to flow dissector, but is SPI really > intended to be used an L4 entropy value? We would need to consider the yes. To quote https://en.wikipedia.org/wiki/Security_Parameter_Index "This works like port numbers in TCP and UDP connec

Re: ipsec impact on performance

2015-12-02 Thread Tom Herbert
On Wed, Dec 2, 2015 at 12:50 PM, Sowmini Varadhan wrote: > On (12/02/15 12:41), David Laight wrote: >> You are getting 0.7 Gbps with ass-ccm-a-128, scale the esp-null back to >> that and it would use 7/18*71 = 27% of the cpu. >> So 69% of the cpu in the a-128 case is probably caused by the >> encr

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 12:41), David Laight wrote: > You are getting 0.7 Gbps with ass-ccm-a-128, scale the esp-null back to > that and it would use 7/18*71 = 27% of the cpu. > So 69% of the cpu in the a-128 case is probably caused by the > encryption itself. > Even if the rest of the code cost nothing you'd

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 12:41), David Laight wrote: > > Also what/how are you measuring cpu use. > I'm not sure anything on Linux gives you a truly accurate value > when processes are running for very short periods. I was using mpstat, while running iperf. Should I be using something else? or running it for

RE: ipsec impact on performance

2015-12-02 Thread David Laight
From: Sowmini Varadhan > Sent: 02 December 2015 12:12 > On (12/02/15 11:56), David Laight wrote: > > > Gbps peak cpu util > > > esp-null 1.8 71% > > > aes-gcm-c-2561.6 79% > > > aes-ccm-a-1280.7 96% > > > > > > That trend made me think that if we can get esp-n

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 11:56), David Laight wrote: > > Gbps peak cpu util > > esp-null 1.8 71% > > aes-gcm-c-2561.6 79% > > aes-ccm-a-1280.7 96% > > > > That trend made me think that if we can get esp-null to be as close > > as possible to GSO/GRO, the rest will follow

Re: ipsec impact on performance

2015-12-02 Thread Sowmini Varadhan
On (12/02/15 07:53), Steffen Klassert wrote: > > I'm currently working on a GRO/GSO codepath for IPsec too. The GRO part > works already. I decapsulate/decrypt the packets on layer2 with a esp GRO > callback function and reinject them into napi_gro_receive(). So in case > the decapsulated packet i

RE: ipsec impact on performance

2015-12-02 Thread David Laight
From: Sowmini Varadhan > Sent: 01 December 2015 18:37 ... > I was using esp-null merely to not have the crypto itself perturb > the numbers (i.e., just focus on the s/w overhead for now), but here > are the numbers for the stock linux kernel stack > Gbps peak cpu util > esp-null

Re: ipsec impact on performance

2015-12-01 Thread Steffen Klassert
On Tue, Dec 01, 2015 at 12:59:53PM -0500, Sowmini Varadhan wrote: > > I instrumented iperf with and without ipsec, just using esp-null, > and 1 thread, to keep things simple. I'm seeing some pretty dismal > performance numbers with ipsec, and trying to think of ways to > improve this. Here are m

Re: ipsec impact on performance

2015-12-01 Thread David Ahern
On 12/1/15 5:09 PM, Sowmini Varadhan wrote: The not-so-great news is that I see that just adding perf tracepoints (not even enabling them!) seems to make a small diff (3 Gbps vs 3.2 Gbps) to my numbers. Is that mere standard-deviation, or something one should be aware of, about perf? existence

Re: ipsec impact on performance

2015-12-01 Thread Sowmini Varadhan
On (12/01/15 16:56), David Ahern wrote: > > Using iperf3 and AH with NULL algorithm between 2 peers connected by > a 10G link. > I'm using esp-null, not AH, and iperf2, which I understand is quite different from, and more aggressive than, iperf3 (though I'm not sure that it matters for this sin

Re: ipsec impact on performance

2015-12-01 Thread David Ahern
On 12/1/15 10:17 AM, Rick Jones wrote: On 12/01/2015 09:59 AM, Sowmini Varadhan wrote: But these are all still relatively small things - tweaking them doesnt get me significantly past the 3 Gbps limit. Any suggestions on how to make this budge (or design criticism of the patch) would be welcome.

Re: ipsec impact on performance

2015-12-01 Thread Sowmini Varadhan
On (12/01/15 10:50), Rick Jones wrote: > > Something of a longshot, but are you sure you are still getting > effective CKO/GRO on the receiver? Good question. With ipsec, GRO (like GSO) gets implicitly disabled. But when I explictly disable GRO on receiver, leaving only GSO on sender, I can stil

Re: ipsec impact on performance

2015-12-01 Thread Rick Jones
On 12/01/2015 10:45 AM, Sowmini Varadhan wrote: On (12/01/15 10:17), Rick Jones wrote: What do the perf profiles show? Presumably, loss of TSO/GSO means an increase in the per-packet costs, but if the ipsec path significantly increases the per-byte costs... For ESP-null, there's actually ver

Re: ipsec impact on performance

2015-12-01 Thread Sowmini Varadhan
On (12/01/15 10:17), Rick Jones wrote: > > What do the perf profiles show? Presumably, loss of TSO/GSO means > an increase in the per-packet costs, but if the ipsec path > significantly increases the per-byte costs... For ESP-null, there's actually very little work to do - we just need to add th

Re: ipsec impact on performance

2015-12-01 Thread Sowmini Varadhan
On (12/01/15 10:18), Tom Herbert wrote: > > 8.5-9.5 Gbps clear traffic, TSO disabled, so GSO, GRO is in effect > > 3-4 Gbps clear traffic, with both TSO/GSO disabled > > 1.8-2 Gbps for esp-null. > > Are you losing checksum offload also? I tried with both checksum offload on and off

Re: ipsec impact on performance

2015-12-01 Thread Tom Herbert
On Tue, Dec 1, 2015 at 9:59 AM, Sowmini Varadhan wrote: > > I instrumented iperf with and without ipsec, just using esp-null, > and 1 thread, to keep things simple. I'm seeing some pretty dismal > performance numbers with ipsec, and trying to think of ways to > improve this. Here are my findings,

Re: ipsec impact on performance

2015-12-01 Thread Rick Jones
On 12/01/2015 09:59 AM, Sowmini Varadhan wrote: But these are all still relatively small things - tweaking them doesnt get me significantly past the 3 Gbps limit. Any suggestions on how to make this budge (or design criticism of the patch) would be welcome. What do the perf profiles show? Pres

ipsec impact on performance

2015-12-01 Thread Sowmini Varadhan
I instrumented iperf with and without ipsec, just using esp-null, and 1 thread, to keep things simple. I'm seeing some pretty dismal performance numbers with ipsec, and trying to think of ways to improve this. Here are my findings, please share feedback. I suspect that a big part of the problem