Re: [vpp-dev] getting segfault in execution of vpp 19.01 having nat configuration with trex

[Rubina Bianchi]

Hi John.

Thanks for your response. I built my own VPP on Debian 8 and my gcc version 
"gcc (Debian 4.9.2-10) 4.9.2". Since I had problem in building vpp with my gcc, 
I made changes on code which are available in the following link.
https://paste.ubuntu.com/p/kn34fxwSjb/

I don't know exactly what debug image is. But I changed -O2 parameters in 
vpp.mk file into -O0.  After doing this, I ran the test again and there was no 
SIGSEGV.

I took my test again with a vpp which was built by -O2 parameter and again I 
saw SIGSEGV signal. The following link shows the back-trace of new test.
https://paste.ubuntu.com/p/x98qjjxtBS/

Here is my startup.conf.
https://paste.ubuntu.com/p/G5pHbW4PGY/

Do you want me to send my vpp deb files?


Best regards

From: John DeNisco (jdenisco) 
Sent: Tuesday, March 12, 2019 3:14 PM
To: Rubina Bianchi; Ole Troan
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] getting segfault in execution of vpp 19.01 having nat 
configuration with trex


Hi Rubina,



I tried to repro this using the VPP image you specified and could not. I was 
able to configure trex the way you had, but was able to run the test without a 
crash overnight.



Could you send us your vpp startup.conf?



Also, is this your own private build or downloaded from package cloud?



Would you be able to build and run a debug image?



Thanks,



John



From:  on behalf of Rubina Bianchi 
Date: Tuesday, March 12, 2019 at 4:10 AM
To: Ole Troan 
Cc: "vpp-dev@lists.fd.io" 
Subject: Re: [vpp-dev] getting segfault in execution of vpp 19.01 having nat 
configuration with trex



Hi Ole,





Thanks for your response. I did apply the patch you'd sent on vpp 19.01. Again 
I got SIGSEGV by running the same test, but it occurred in a different part of 
the code.



Here is the back-trace of gdb:

https://paste.ubuntu.com/p/fpmynt3BCm/



best regards



From: Ole Troan 
Sent: Monday, March 11, 2019 2:36 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] getting segfault in execution of vpp 19.01 having nat 
configuration with trex



Hi Rubina,

This looks like a fault in a local UDP application (BFD) registering to receive 
UDP packets, even when not configured.
That should be fixed in:
https://gerrit.fd.io/r/#/c/18188/

Great if you can verify when that is merged.

Cheers,
Ole


> I configured nat on vpp 19.01 and wanted to test it using trex. After a 
> while, vpp got SIGSEGV and all packets were dropped. My vpp version was 
> v19.01.1-2~gd0dcf7685.
> Here is my vpp configuration:
> https://paste.ubuntu.com/p/czRf8g6Nt6/
>
> This is trex config file:
> https://paste.ubuntu.com/p/DXHdQf77mC/
> This is my trex yaml file containing pcaps:
> https://paste.ubuntu.com/p/Xdngz3NWRP/
>
> I used this command to start trex test.
> ./t-rex-64 -f cap2/sfr.yaml --cfg cfg/trex_config_nat.yaml -c 4 -m 8 
> --learn-mode 3 -d 3600
>
> Here is back-trace of gdb.
> https://paste.ubuntu.com/p/qxbHP8PVYS/
>
> It would be appreciated if you can help me to fix this SIGSEGV problem.
>
> Best regards
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#12479): https://lists.fd.io/g/vpp-dev/message/12479
> Mute This Topic: https://lists.fd.io/mt/30386509/675193
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [otr...@employees.org]
> -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12563): https://lists.fd.io/g/vpp-dev/message/12563
Mute This Topic: https://lists.fd.io/mt/30386509/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] getting segfault in execution of vpp 19.01 having nat configuration with trex

[Rubina Bianchi]

Hi, dear VPP

I configured nat on vpp 19.01 and wanted to test it using trex. After a while, 
vpp got SIGSEGV and all packets were dropped. My vpp version was 
v19.01.1-2~gd0dcf7685.
Here is my vpp configuration:
https://paste.ubuntu.com/p/czRf8g6Nt6/

This is trex config file:
https://paste.ubuntu.com/p/DXHdQf77mC/
This is my trex yaml file containing pcaps:
https://paste.ubuntu.com/p/Xdngz3NWRP/

I used this command to start trex test.
./t-rex-64 -f cap2/sfr.yaml --cfg cfg/trex_config_nat.yaml -c 4 -m 8 
--learn-mode 3 -d 3600

Here is back-trace of gdb.
https://paste.ubuntu.com/p/qxbHP8PVYS/

It would be appreciated if you can help me to fix this SIGSEGV problem.

Best regards

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12479): https://lists.fd.io/g/vpp-dev/message/12479
Mute This Topic: https://lists.fd.io/mt/30386509/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Problem with adding second IP to interface

[Rubina Bianchi]

Hi Dear VPP

I configured a topology using two clients and one DUT (VPP).
Topology: https://paste.ubuntu.com/p/gNy3ZZyTfD/

In my scenario Client1 ping Clinet2 and Client2 ping Client1. While it is 
operational, I add second IP (0.2.0.3/8) to GigabitEthernet3/0/0. I see both 
pings are dropped for a while (about 30 secs).
My question is why this happen? I expected that pings were working continuously 
during second IP adding.

Thanks,
Sincerely
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11532): https://lists.fd.io/g/vpp-dev/message/11532
Mute This Topic: https://lists.fd.io/mt/28682722/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP and TCP half-open connection

[Rubina Bianchi]

Hi Dear VPP

Is there any mechanism in VPP to restrict TCP half-open connection count? If 
answer is yes how it can be configured?

Thanks,
Sincerely
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11312): https://lists.fd.io/g/vpp-dev/message/11312
Mute This Topic: https://lists.fd.io/mt/28241090/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP coredump question

[Rubina Bianchi]

Hi Dear VPP

I have two question about vpp coredump.
How can I use full-coredump parameter in my startup.conf to make a dump if vpp 
crashes?
After crash, where this dump will be saved?

Here is my startup.conf:
unix {
nodaemon
log /tmp/vpp.log
full-coredump
cli-listen /run/vpp/cli.sock
startup-config /etc/vpp/init.conf
}

api-trace {
on
}

api-segment {
gid vpp
}

cpu {
main-core 1
corelist-workers 2
workers 1
}

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11311): https://lists.fd.io/g/vpp-dev/message/11311
Mute This Topic: https://lists.fd.io/mt/28241075/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Odd problem in adding new session on vpp when its session table is full

[Rubina Bianchi]

Thanks for your following up. 

From: Andrew  Yourtchenko 
Sent: Wednesday, September 26, 2018 12:31 AM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its 
session table is full

Dear Rubina,

On 9/25/18, Rubina Bianchi  wrote:
> Dear Andrew,
>
> Actually it's not a RFC standard , or known attack, we extracted the
> restriction using experiment on linux netfiltter and tried to implement its
> behavior.

hmm okay. so my position about implementing it still stands then :-)

Also, another point - if you send two echo requests at once with the
same code (which is fine, because they are differentiated by ICMP ID
and sequence number), then the second request will overwrite bihash
values on activation (which is fine) and then the first reply that
comes will erase those upon deactivation, so the second request will
be dropped. This can make the troubleshooting harder in some cases
since this is confusing.  One way to deal with that is to keep some
sort of reference counter, or extend the bihash key, but again in the
absence of real-world threat model this looks like shooting pigeons
from ICBMs... (the only attack I know for this scenario is smurf and
the existing implementation covers it fine)

>
> I checked your patch, throughput is fine. After about 2000 seconds my
> throughput was still maximum and there was no drop-rate on trex. Also,
> rx-miss was very little in this patch.

Excellent, thanks a lot!

> However, one thing is attracting for
> me is that, in the same scenario in previous discussion, vpp served this
> traffic with 3.6 million sessions, but in current patch it serves with 3.9
> million sessions. Is this related to your last changes?

Yeah, it could be...  the new change  make the expiry somewhat longer:
the session in any given timeout list is checked twice per timeout,
rather than every X seconds (X being the shortest list timeout). So if
e.g. a UDP session timeout is 100 seconds, and there is a packet on it
10 seconds before the session is checked, the next check will be 50
seconds later, at which point the idle time on the session (60
seconds) is still smaller than 100, so the session will be expunged
only 50 seconds later, with the idle time of 110 seconds.

Contrast this with the previous behavior where all the lists might be
checked as often as 2 seconds - this would have resulted in a more
precise timing out, but at an expense of a lot of CPU usage.

But then all of the TCP sessions will transition into the tcp
transient state upon closure, so the only impact in real world will be
with the UDP connections, which should be relatively small.

All that said: I am considering  to move the timeout infra to the
tw_timers (which weren't available back when I was writing the code) -
then in theory we can get both the precise expiry and the efficiency.
But that is a larger change, and I am not sure of it yet, so I wanted
to get this simpler solution in first.

--a

>
> Thanks,
> Sincerely
> 
> From: Andrew  Yourtchenko 
> Sent: Monday, September 24, 2018 6:49 PM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its
> session table is full
>
> Dear Rubina,
>
> On 9/24/18, Rubina Bianchi  wrote:
>> Dear Andrew,
>>
>> It's hardcoded as it was simple and fast solution for our default
>> scenario
>> implementation.
>> As you correctly mentioned in previous email one of the bug fixes was the
>> restriction. Also another one is preventing reply packets pass through
>> even
>
> ok. that's more a "featurette" - I deliberately did not attempt to
> implement the "strict checking" because I had difficult time finding
> the attack vector. (rather than maybe some kind of "compliance" checks
> for the reasons of compliance ?)
>
>> if those packets are matched with an acl rule. In another word these
>> reply
>> packets are not belong to any echo request in reverse direction.
>
> hmm so you are sort-of making a "protocol inspection engine" there ? :-)
>
> Anyway, so far I haven't managed to recreate this condition - though
> if you were running the 18.07 rather than 18.07.1 code, then the bug
> related to hash acl manipulation on ACL changes might have caused that
> effect... I will experiment a bit more, though.
>
> Also, remember the other thread we discussed a while ago about the
> throughput getting lower over time.. I have made
> https://gerrit.fd.io/r/#/c/14821/ which should significantly reduce
> the amount of session list shuffling work in normal case scenarios.
> Before I commit it, could you give it a shot to see if it indeed
> behaves as I would expect it to b

Re: [vpp-dev] Odd problem in adding new session on vpp when its session table is full

[Rubina Bianchi]

Dear Andrew,

Actually it's not a RFC standard or known attack, we extracted the restriction 
using experiment on linux netfiltter and tried to implement its behavior.

I checked your patch, throughput is fine. After about 2000 seconds my 
throughput was still maximum and there was no drop-rate on trex. Also, rx-miss 
was very little in this patch. However, one thing is attracting for me is that, 
in the same scenario in previous discussion, vpp served this traffic with 3.6 
million sessions, but in current patch it serves with 3.9 million sessions. Is 
this related to your last changes?

Thanks,
Sincerely

From: Andrew  Yourtchenko 
Sent: Monday, September 24, 2018 6:49 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its 
session table is full

Dear Rubina,

On 9/24/18, Rubina Bianchi  wrote:
> Dear Andrew,
>
> It's hardcoded as it was simple and fast solution for our default scenario
> implementation.
> As you correctly mentioned in previous email one of the bug fixes was the
> restriction. Also another one is preventing reply packets pass through even

ok. that's more a "featurette" - I deliberately did not attempt to
implement the "strict checking" because I had difficult time finding
the attack vector. (rather than maybe some kind of "compliance" checks
for the reasons of compliance ?)

> if those packets are matched with an acl rule. In another word these reply
> packets are not belong to any echo request in reverse direction.

hmm so you are sort-of making a "protocol inspection engine" there ? :-)

Anyway, so far I haven't managed to recreate this condition - though
if you were running the 18.07 rather than 18.07.1 code, then the bug
related to hash acl manipulation on ACL changes might have caused that
effect... I will experiment a bit more, though.

Also, remember the other thread we discussed a while ago about the
throughput getting lower over time.. I have made
https://gerrit.fd.io/r/#/c/14821/ which should significantly reduce
the amount of session list shuffling work in normal case scenarios.
Before I commit it, could you give it a shot to see if it indeed
behaves as I would expect it to behave ?  Thanks a lot!

--a

> Thanks,
> Sincerely
> 
> From: Andrew  Yourtchenko 
> Sent: Tuesday, September 18, 2018 4:06 PM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its
> session table is full
>
> Dear Rubina,
>
> On 9/18/18, Rubina Bianchi  wrote:
>> Dear Andrew,
>>
>> Our changes is provided to you by creating a patch which is attached to
>> this
>> email.
>> I didn't commit it to gerrit due to our specific scenario (permit+reflect
>> on
>> all inputs, permit+reflect or deny on all outputs).
>
> Why do you hardcode it as opposed to making it part of configuration ?
> permit+reflect in one direction and deny except established sessions
> is a fairly standard config.
>
>> In addition to ICMP timeout handling, our code fixes some ICMP bugs.
>
> Do you mean the "strict"  enforcement of the one-request-one-response
> policy for ICMP that this code does ?
>
> --a
>
>> Although, I think code is clear for you, I can explain it in details if
>> you
>> ask.
>>
>> Thanks,
>> Sincerely
>> 
>> From: Andrew  Yourtchenko 
>> Sent: Tuesday, September 18, 2018 11:27 AM
>> To: Rubina Bianchi
>> Cc: vpp-dev@lists.fd.io
>> Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its
>> session table is full
>>
>>
>>
>>
>> Hi Rubina,
>>
>> On 18 Sep 2018, at 11:14, Rubina Bianchi
>> mailto:r_bian...@outlook.com>> wrote:
>>
>> Hi Dear Andrew
>>
>> 1) I just attached my init.conf to this email. As you guessed session
>> table
>> size is 100. This problem is occurred on vpp stable/1807.
>>
>> Ah, cool, that helps, thanks!
>>
>>
>> 2) Yes, there is 6 timeout list. We added a list for handling icmp
>> timeouts.
>>
>> That is not the stable/1807, then ☺️ would you mind submitting the change
>> to
>> gerrit so we could take a look at it and ideally incorporate into the
>> master
>> ?
>>
>> —a
>>
>>
>> 
>> From: Andrew  Yourtchenko
>> mailto:ayour...@gmail.com>>
>> Sent: Monday, September 17, 2018 8:03 PM
>> To: Rubina Bianchi
>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
>> Subject: Re: [vpp-dev] Odd problem in adding 

Re: [vpp-dev] Odd problem in adding new session on vpp when its session table is full

[Rubina Bianchi]

Dear Andrew,

It's hardcoded as it was simple and fast solution for our default scenario 
implementation.
As you correctly mentioned in previous email one of the bug fixes was the 
restriction. Also another one is preventing reply packets pass through even if 
those packets are matched with an acl rule. In another word these reply packets 
are not belong to any echo request in reverse direction.

Thanks,
Sincerely

From: Andrew  Yourtchenko 
Sent: Tuesday, September 18, 2018 4:06 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its 
session table is full

Dear Rubina,

On 9/18/18, Rubina Bianchi  wrote:
> Dear Andrew,
>
> Our changes is provided to you by creating a patch which is attached to this
> email.
> I didn't commit it to gerrit due to our specific scenario (permit+reflect on
> all inputs, permit+reflect or deny on all outputs).

Why do you hardcode it as opposed to making it part of configuration ?
permit+reflect in one direction and deny except established sessions
is a fairly standard config.

> In addition to ICMP timeout handling, our code fixes some ICMP bugs.

Do you mean the "strict"  enforcement of the one-request-one-response
policy for ICMP that this code does ?

--a

> Although, I think code is clear for you, I can explain it in details if you
> ask.
>
> Thanks,
> Sincerely
> 
> From: Andrew  Yourtchenko 
> Sent: Tuesday, September 18, 2018 11:27 AM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its
> session table is full
>
>
>
>
> Hi Rubina,
>
> On 18 Sep 2018, at 11:14, Rubina Bianchi
> mailto:r_bian...@outlook.com>> wrote:
>
> Hi Dear Andrew
>
> 1) I just attached my init.conf to this email. As you guessed session table
> size is 100. This problem is occurred on vpp stable/1807.
>
> Ah, cool, that helps, thanks!
>
>
> 2) Yes, there is 6 timeout list. We added a list for handling icmp
> timeouts.
>
> That is not the stable/1807, then ☺️ would you mind submitting the change to
> gerrit so we could take a look at it and ideally incorporate into the master
> ?
>
> —a
>
>
> 
> From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
> Sent: Monday, September 17, 2018 8:03 PM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
> Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its
> session table is full
>
> Dear Rubina,
>
> looking at the outputs, there are a few anomalies that hopefully you
> can clarify:
>
> 1) the max session count is 100. The latest master has the default
> limit of 50, and I do not see any startup config parameters
> changing that. Which version are you testing with/building off ?
>
> 2) there are 6 fa_conn_list_head elements in each worker for your
> outputs. That number was initially 3, and in the early spring when I
> introduced the purgatory list and the reserved unused list this number
> has increased to 5. The vectors are initialized at a start with a
> constant, so I am wondering why your outputs have a different number.
>
> Would be able to comment on these observations ?
>
> Thank you!
>
> --a
>
> On 9/17/18, Rubina Bianchi
> mailto:r_bian...@outlook.com>> wrote:
>>   *   Dear VPP
>>
>> I ran a test on VPP configured with permit+reflect ACl rules with t-rex.
>> In
>> this test, I put two interfaces on one bridge-domain and had an ACL on
>> all
>> of its input and output interfaces. The ACL had just one rule which was
>> allowing any traffic. I ran my test until VPP's session table was full. I
>> run t-rex whith following command:
>>
>> "./t-rex-64 -f cap2/sfr.yaml-m 10  -d  1"
>>
>>
>> After a couple of days, I took another test  on VPP. I tried to establish
>> a
>> ssh session between two clients via my VPP. But session could not be
>> established. When I checked VPP trace, All of my ssh packets where
>> dropped
>> due to following error:
>>
>> "acl-plugin-in-ip4-l2: too many sessions to add new"
>>
>> when I checked VPP's session table, I realized that it was full. No
>> session
>> where deleted since my previous test and no session where going to be
>> added
>> to session table.I also checked my /var/log/hawk.log file and saw
>> following
>> error:
>>
>> "acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple
>> collision!"
>>
>> I could not fix this problem so

Re: [vpp-dev] Odd problem in adding new session on vpp when its session table is full

[Rubina Bianchi]

Dear Andrew,

Our changes is provided to you by creating a patch which is attached to this 
email.
I didn't commit it to gerrit due to our specific scenario (permit+reflect on 
all inputs, permit+reflect or deny on all outputs).
In addition to ICMP timeout handling, our code fixes some ICMP bugs. Although, 
I think code is clear for you, I can explain it in details if you ask.

Thanks,
Sincerely

From: Andrew  Yourtchenko 
Sent: Tuesday, September 18, 2018 11:27 AM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its 
session table is full




Hi Rubina,

On 18 Sep 2018, at 11:14, Rubina Bianchi 
mailto:r_bian...@outlook.com>> wrote:

Hi Dear Andrew

1) I just attached my init.conf to this email. As you guessed session table 
size is 100. This problem is occurred on vpp stable/1807.

Ah, cool, that helps, thanks!


2) Yes, there is 6 timeout list. We added a list for handling icmp timeouts.

That is not the stable/1807, then ☺️ would you mind submitting the change to 
gerrit so we could take a look at it and ideally incorporate into the master ?

—a



From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Monday, September 17, 2018 8:03 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its 
session table is full

Dear Rubina,

looking at the outputs, there are a few anomalies that hopefully you
can clarify:

1) the max session count is 100. The latest master has the default
limit of 50, and I do not see any startup config parameters
changing that. Which version are you testing with/building off ?

2) there are 6 fa_conn_list_head elements in each worker for your
outputs. That number was initially 3, and in the early spring when I
introduced the purgatory list and the reserved unused list this number
has increased to 5. The vectors are initialized at a start with a
constant, so I am wondering why your outputs have a different number.

Would be able to comment on these observations ?

Thank you!

--a

On 9/17/18, Rubina Bianchi 
mailto:r_bian...@outlook.com>> wrote:
>   *   Dear VPP
>
> I ran a test on VPP configured with permit+reflect ACl rules with t-rex. In
> this test, I put two interfaces on one bridge-domain and had an ACL on all
> of its input and output interfaces. The ACL had just one rule which was
> allowing any traffic. I ran my test until VPP's session table was full. I
> run t-rex whith following command:
>
> "./t-rex-64 -f cap2/sfr.yaml-m 10  -d  1"
>
>
> After a couple of days, I took another test  on VPP. I tried to establish a
> ssh session between two clients via my VPP. But session could not be
> established. When I checked VPP trace, All of my ssh packets where dropped
> due to following error:
>
> "acl-plugin-in-ip4-l2: too many sessions to add new"
>
> when I checked VPP's session table, I realized that it was full. No session
> where deleted since my previous test and no session where going to be added
> to session table.I also checked my /var/log/hawk.log file and saw following
> error:
>
> "acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple
> collision!"
>
> I could not fix this problem so I restarted my VPP service. After that,
> I could not reproduce this state again. Does anyone have any idea on
> what my problem on VPP was?
>
> I attached my hawk.log, vpp trace, "vppctl sh acl-plugin sessions" output
> and startup.conf file to this email.
>
>
>
>
>

diff --git a/src/plugins/acl/acl.c b/src/plugins/acl/acl.c
index 2076e5cf..9320b1cd 100644
--- a/src/plugins/acl/acl.c
+++ b/src/plugins/acl/acl.c
@@ -4185,7 +4185,8 @@ acl_init (vlib_main_t * vm)
 TCP_SESSION_IDLE_TIMEOUT_SEC;
   am->session_timeout_sec[ACL_TIMEOUT_UDP_IDLE] =
 UDP_SESSION_IDLE_TIMEOUT_SEC;
-
+  am->session_timeout_sec[ACL_TIMEOUT_ICMP_IDLE] =
+ICMP_SESSION_IDLE_TIMEOUT_SEC;
   am->fa_conn_table_hash_num_buckets =
 ACL_FA_CONN_TABLE_DEFAULT_HASH_NUM_BUCKETS;
   am->fa_conn_table_hash_memory_size =
diff --git a/src/plugins/acl/acl.h b/src/plugins/acl/acl.h
index d030681a..1c2cf129 100644
--- a/src/plugins/acl/acl.h
+++ b/src/plugins/acl/acl.h
@@ -39,6 +39,7 @@
 #define UDP_SESSION_IDLE_TIMEOUT_SEC 600
 #define TCP_SESSION_IDLE_TIMEOUT_SEC (3600*24)
 #define TCP_SESSION_TRANSIENT_TIMEOUT_SEC 120
+#define ICMP_SESSION_IDLE_TIMEOUT_SEC 30
 
 #define SESSION_PURGATORY_TIMEOUT_USEC 10
 
@@ -57,6 +58,7 @@ enum acl_timeout_e {
   ACL_TIMEOUT_UDP_IDLE,
   ACL_TIMEOUT_TCP_IDLE,
   ACL_TIMEOUT_TCP_TRANSIENT,
+  ACL_TIMEOUT_ICMP_IDLE,
   ACL_N_USER_TIMEOUTS,
   ACL_TIMEOUT_PURGATORY = ACL_N_USER_TIMEOUTS, /* a special-case queue for deletion-in-progress sessions */
   A

Re: [vpp-dev] Odd problem in adding new session on vpp when its session table is full

[Rubina Bianchi]

Hi Dear Andrew

1) I just attached my init.conf to this email. As you guessed session table 
size is 100. This problem is occurred on vpp stable/1807.

2) Yes, there is 6 timeout list. We added a list for handling icmp timeouts.

From: Andrew  Yourtchenko 
Sent: Monday, September 17, 2018 8:03 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Odd problem in adding new session on vpp when its 
session table is full

Dear Rubina,

looking at the outputs, there are a few anomalies that hopefully you
can clarify:

1) the max session count is 100. The latest master has the default
limit of 50, and I do not see any startup config parameters
changing that. Which version are you testing with/building off ?

2) there are 6 fa_conn_list_head elements in each worker for your
outputs. That number was initially 3, and in the early spring when I
introduced the purgatory list and the reserved unused list this number
has increased to 5. The vectors are initialized at a start with a
constant, so I am wondering why your outputs have a different number.

Would be able to comment on these observations ?

Thank you!

--a

On 9/17/18, Rubina Bianchi  wrote:
>   *   Dear VPP
>
> I ran a test on VPP configured with permit+reflect ACl rules with t-rex. In
> this test, I put two interfaces on one bridge-domain and had an ACL on all
> of its input and output interfaces. The ACL had just one rule which was
> allowing any traffic. I ran my test until VPP's session table was full. I
> run t-rex whith following command:
>
> "./t-rex-64 -f cap2/sfr.yaml-m 10  -d  1"
>
>
> After a couple of days, I took another test  on VPP. I tried to establish a
> ssh session between two clients via my VPP. But session could not be
> established. When I checked VPP trace, All of my ssh packets where dropped
> due to following error:
>
> "acl-plugin-in-ip4-l2: too many sessions to add new"
>
> when I checked VPP's session table, I realized that it was full. No session
> where deleted since my previous test and no session where going to be added
> to session table.I also checked my /var/log/hawk.log file and saw following
> error:
>
> "acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple
> collision!"
>
> I could not fix this problem so I restarted my VPP service. After that,
> I could not reproduce this state again. Does anyone have any idea on
> what my problem on VPP was?
>
> I attached my hawk.log, vpp trace, "vppctl sh acl-plugin sessions" output
> and startup.conf file to this email.
>
>
>
>
>


init.conf
Description: init.conf
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10533): https://lists.fd.io/g/vpp-dev/message/10533
Mute This Topic: https://lists.fd.io/mt/25722080/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Odd problem in adding new session on vpp when its session table is full

[Rubina Bianchi]

  *   Dear VPP

I ran a test on VPP configured with permit+reflect ACl rules with t-rex. In 
this test, I put two interfaces on one bridge-domain and had an ACL on all of 
its input and output interfaces. The ACL had just one rule which was allowing 
any traffic. I ran my test until VPP's session table was full. I run t-rex 
whith following command:

"./t-rex-64 -f cap2/sfr.yaml-m 10  -d  1"


After a couple of days, I took another test  on VPP. I tried to establish a ssh 
session between two clients via my VPP. But session could not be established. 
When I checked VPP trace, All of my ssh packets where dropped due to following 
error:

"acl-plugin-in-ip4-l2: too many sessions to add new"

when I checked VPP's session table, I realized that it was full. No session 
where deleted since my previous test and no session where going to be added to 
session table.I also checked my /var/log/hawk.log file and saw following error:

"acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!"

I could not fix this problem so I restarted my VPP service. After that,
I could not reproduce this state again. Does anyone have any idea on
what my problem on VPP was?

I attached my hawk.log, vpp trace, "vppctl sh acl-plugin sessions" output and 
startup.conf file to this email.




2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 5-tuple collision!
2018 Sep 11 15:14:39 rubina acl_fa_node_fn:516: BUG: session LSB16(sw_if_index) and 

Re: [vpp-dev] VPP Memory usage

[Rubina Bianchi]

Hi dear Andrew

I once again ran previous scenario and logged previous outputs plus "vppctl 
show acl memory".
I also send you T-rex configs, 60s and 1500s output.

As you can see during this period RSS is increasing and Total-rx is decreasing.
I assume I didn't get my answer. Why does my Total-rx decrease during my test 
and RSS still increases?

Thanks,
Sincerely

From: Andrew  Yourtchenko 
Sent: Monday, August 20, 2018 10:25 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] VPP Memory usage

Dear Rubina,

On 8/20/18, Rubina Bianchi  wrote:
> Hi dear Andrew
>
> What we were talked before was about "Worker Thread Deadlock".

We had that discussion in march or may. :-)

The one I had in mind was another thread, starting with your mail on
January 30. I forwarded to you unicast :-)

>
> I tried to test scenario as you explained and started with 1M entry and
> after that I doubled it at each run.
> When I test with 4M entry size, I logged two things:
> 1. ps aux | grep vpp
> 2. First 5 lines of "vppctl show acl-plugin session"
>
> At first, I've run VPP and configured it with script that I attached to
> previous email.
> After that I run my logger script.
> Finally I run Trex with this command: ./t-rex-64 --cfg cfg/trex_config.yaml
> -f cap2/sfr.yaml -m 50 -c 3 -d 1 -p
> After tracing VPP logs I found some signs of leakage. I mean in the logs of
> VPP, RSS (6th parameter in ps aux command) is increasing continuously
> (sometimes more and sometimes less) but on the other side, Trex Total-Rx is
> decreasing at the same time.
> After about 3000 seconds, I stopped Trex and wait until session table being
> cleared. But no change in RSS happens.
> Then, I run Trex again without any change and again I saw the increase of
> RSS while the Trex Total-Rx is decreasing.

Based on the counters, in this test we are continuously churning
through the half-open sessions, because we are hitting the maximum
session limit. Session creation is quite expensive (at least at this
point, I did not optimize that code much yet).

>
> This is my ram status when vpp is stop:
> root@debian-hp:~# free -m
>  total   used   free sharedbuffers cached
> Mem:129135   3414 125721 12 99591
> -/+ buffers/cache:   2723 126412
> Swap: 2518  0   2518
>
> I also attached my logs to this email. This logs are gathered every 20
> seconds.
>
> In 40M entry size I saw this behavior too, but It happens much faster than
> 4M entry size.

Yes, because you create more sessions and use more buckets, I think
(though this is a speculation at this point, since we dont have the
memory outputs).

What i sthe maximum amount of simultaneous sessions on the T-rex and
what is the connection per second rate ?

> I also have a question about your phrase  of "Using this method you can
> arrive to the number of maximum connections that your memory configuration
> can support".
> Is there any formula to config init.conf in an efficient way? Because VPP
> didn't return any error about misconfiguration.

No, there is no formula, unfortunately - hence I can not print an
error about a misconfiguration.

You can use the "show acl memory" as I described in the other mail, to
see what the memory usage in the session bihash is and what is the
number of active elements - could you have a look at doing that ?

--a

>
> Thanks,
> Sincerely
>
>
>
>
> 
> From: Andrew  Yourtchenko 
> Sent: Sunday, August 19, 2018 8:28 AM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] VPP Memory usage
>
> Dear Rubina,
>
> The ACL plugin does all the necessary allocations at startup for all data
> structures except the connection bihash.
>
> You would need to check the current number of the connections as your test
> progresses. I believe we had a communication a while ago regarding the
> gradual growth of background memory usage within the bihash data structure
> as you churn through random addresses. Since then there were some changes
> aimed to address this. Please verify what does the current total session
> count look like in “show acl-plugin sessions” as your test progresses -
> based on what you described I think it continuously increases.
>
> If the bihash memory requirement for active connections goes above of what
> is available from the OS, then there is no feedback to the user code (acl
> plugin) other than a full crash.
>
> The  only safeguard I could come up against this situation is the maximum
> connection count, which is checked before attempting to insert an entry into
> the bihash.
>
&

Re: [vpp-dev] VPP Memory usage

[Rubina Bianchi]

Hi dear Andrew

What we were talked before was about "Worker Thread Deadlock".

I tried to test scenario as you explained and started with 1M entry and after 
that I doubled it at each run.
When I test with 4M entry size, I logged two things:
1. ps aux | grep vpp
2. First 5 lines of "vppctl show acl-plugin session"

At first, I've run VPP and configured it with script that I attached to 
previous email.
After that I run my logger script.
Finally I run Trex with this command: ./t-rex-64 --cfg cfg/trex_config.yaml  -f 
cap2/sfr.yaml -m 50 -c 3 -d 1 -p
After tracing VPP logs I found some signs of leakage. I mean in the logs of 
VPP, RSS (6th parameter in ps aux command) is increasing continuously 
(sometimes more and sometimes less) but on the other side, Trex Total-Rx is 
decreasing at the same time.
After about 3000 seconds, I stopped Trex and wait until session table being 
cleared. But no change in RSS happens.
Then, I run Trex again without any change and again I saw the increase of RSS 
while the Trex Total-Rx is decreasing.

This is my ram status when vpp is stop:
root@debian-hp:~# free -m
 total   used   free sharedbuffers cached
Mem:129135   3414 125721 12 99591
-/+ buffers/cache:   2723 126412
Swap: 2518  0   2518

I also attached my logs to this email. This logs are gathered every 20 seconds.

In 40M entry size I saw this behavior too, but It happens much faster than 4M 
entry size.
I also have a question about your phrase  of "Using this method you can arrive 
to the number of maximum connections that your memory configuration can 
support".
Is there any formula to config init.conf in an efficient way? Because VPP 
didn't return any error about misconfiguration.

Thanks,
Sincerely





From: Andrew  Yourtchenko 
Sent: Sunday, August 19, 2018 8:28 AM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] VPP Memory usage

Dear Rubina,

The ACL plugin does all the necessary allocations at startup for all data 
structures except the connection bihash.

You would need to check the current number of the connections as your test 
progresses. I believe we had a communication a while ago regarding the gradual 
growth of background memory usage within the bihash data structure as you churn 
through random addresses. Since then there were some changes aimed to address 
this. Please verify what does the current total session count look like in 
“show acl-plugin sessions” as your test progresses - based on what you 
described I think it continuously increases.

If the bihash memory requirement for active connections goes above of what is 
available from the OS, then there is no feedback to the user code (acl plugin) 
other than a full crash.

The  only safeguard I could come up against this situation is the maximum 
connection count, which is checked before attempting to insert an entry into 
the bihash.

Your current value is 40 million which is quite a lot, while the hash table 
heap size is 17 gigabytes. This might not be enough to hold all the 40 million 
entries as the churn progresses and you need to create more buckets.

I suggest you keep all the other parameters as they are and start with the 
value of maximum connections of 1 million and rerun the test, and monitor the 
memory usage within the ACL plugin heap (“show acl-plugin memory”) - it should 
stabilize over time at some value and there should be no crash. The exact usage 
will depend on the distribution of session entries over bucket (note that in 
the worst case you may have one entry per bucket which may give a lot of 
overhead). Note that value.

If you stop the traffic, as the session count goes down to zero, the memory 
should get released.

Then double the max conn count and recheck the behavior same as above - the 
usage probably would be about double of the previous one.

Using this method you can arrive to the number of maximum connections that your 
memory configuration can support, and get a gauge of how much memory you would 
need for the target amount of connections.

If in the initial iteration test you observe the memory usage never stabilizing 
or if you see that the memory is not being released as the connection count 
goes down to zero, then it would be a bug, which we will need to further 
troubleshoot - though from your description so far it seems more a case of 
tuning the parameters. So please apply the method above and let me know how it 
goes! Thanks!

--a

On 19 Aug 2018, at 07:26, Rubina Bianchi 
mailto:r_bian...@outlook.com>> wrote:


Hi dear VPP


I configured vpp stable/1807 and added permit+reflect acl on input and output 
of my network interfaces. I configured vpp with 9 cpu (1 main and 8 worker 
cpu). My init.conf is:


vppctl>

set acl-plugin session table max-entries 4000
set acl-plugin session table hash-table-buckets 10

[vpp-dev] VPP Memory usage

[Rubina Bianchi]

Hi dear VPP


I configured vpp stable/1807 and added permit+reflect acl on input and output 
of my network interfaces. I configured vpp with 9 cpu (1 main and 8 worker 
cpu). My init.conf is:


vppctl>

set acl-plugin session table max-entries 4000
set acl-plugin session table hash-table-buckets 100
set acl-plugin session table hash-table-memory 17179869184
set acl-plugin session timeout udp idle 20
set acl-plugin session timeout tcp idle 120
set acl-plugin session timeout tcp transient 30


vpp_api_test>

acl_add_replace permit
acl_add_replace permit+reflect

acl_interface_add_del TenGigabitEthernet3/0/0 add output acl 1
acl_interface_add_del TenGigabitEthernet3/0/1 add output acl 1
acl_interface_add_del TenGigabitEthernet3/0/0 add input acl 1
acl_interface_add_del TenGigabitEthernet3/0/1 add input acl 1

exec set interface l2 bridge TenGigabitEthernet3/0/0 1
exec set interface l2 bridge TenGigabitEthernet3/0/1 1
exec set int state TenGigabitEthernet3/0/0 up
exec set int state TenGigabitEthernet3/0/1 up

My startup.conf is pasted in this link: https://paste.ubuntu.com/p/MhQDyqF6Xd/


I used Trex as traffic generator as following:

./t-rex-64 --cfg cfg/trex_config.yaml  -f cap2/sfr.yaml -m 50 -c 3 -d 3600 -p


During execution of my test, Total-rx continuously decreased and after a while, 
it reached to 0. I checked vpp status and it got SIGKILL signal from OS.

I monitored vpp memory and it was increasing until it crashed.

Does acl_plugin session management have any memory leak problem?


Regards,

Rubina
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10213): https://lists.fd.io/g/vpp-dev/message/10213
Mute This Topic: https://lists.fd.io/mt/24729023/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Error in VPP DPDK plugin api

[Rubina Bianchi]

Thanks for your prompt answer

As you mentioned I checked your clue and I find out that there is no #define 
REPLY_MSG_ID_BASE in dpdk plugin. So I added #define REPLY_MSG_ID_BASE 
dpdk_main.msg_id_base and my first problem solved.

So what's your opinion on my second question in my first email?

From: Dave Barach (dbarach) 
Sent: Wednesday, August 1, 2018 2:17 PM
To: Rubina Bianchi; vpp-dev@lists.fd.io
Subject: RE: Error in VPP DPDK plugin api


Seems like you may have forgotten to add my_main->msg_id_base to mp->_vl_msg_id 
when sending the reply message from your data plane plugin.



From: vpp-dev@lists.fd.io  On Behalf Of Rubina Bianchi
Sent: Wednesday, August 1, 2018 6:42 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Error in VPP DPDK plugin api



Hi Dear VPP



I added an api to DPDK plugin in VPP src/plugin/dpdk.

My api functionality is OK but my reply handler is never called 
(_reply_t_handler is implemented for api and its function signature added to 
foreach_api_msg).

When I call my api in vpp_api_test, it returns "msg_handler_internal:432: no 
handler for msg id 9" and it doesn't return retval (its job is done correctly, 
just reply has problem).



How could I fix this warning?



I also developed a client based on dpdk_test.c like as VAT, but I faced with 
missing "dpdk/api/dpdk.api.h". I checked /usr/include/vpp and /usr/include/dpdk 
and didn't find missed header. So I added dpdk/api/dpdk.api.h to 
nobase_include_HEADERS in VPP src/plugins/dpdk.am.



What is the best solution? Do you think my approach to handle it, is right?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10006): https://lists.fd.io/g/vpp-dev/message/10006
Mute This Topic: https://lists.fd.io/mt/24003854/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Error in VPP DPDK plugin api

[Rubina Bianchi]

Hi Dear VPP

I added an api to DPDK plugin in VPP src/plugin/dpdk.
My api functionality is OK but my reply handler is never called 
(_reply_t_handler is implemented for api and its function signature added to 
foreach_api_msg).
When I call my api in vpp_api_test, it returns "msg_handler_internal:432: no 
handler for msg id 9" and it doesn't return retval (its job is done correctly, 
just reply has problem).

How could I fix this warning?

I also developed a client based on dpdk_test.c like as VAT, but I faced with 
missing "dpdk/api/dpdk.api.h". I checked /usr/include/vpp and /usr/include/dpdk 
and didn't find missed header. So I added dpdk/api/dpdk.api.h to 
nobase_include_HEADERS in VPP src/plugins/dpdk.am.

What is the best solution? Do you think my approach to handle it, is right?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10002): https://lists.fd.io/g/vpp-dev/message/10002
Mute This Topic: https://lists.fd.io/mt/24003854/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Interface Mib II counters

[Rubina Bianchi]

Dear Ole

Thanks for your great guide, I want a tool that works on vpp stable/18.04.

I find a dump_stats_table command in vpp_api_test, but when I used it in 
vpp_api_test it returns:
"dump_stats_table:6242: dump_stats_table supported only in JSON format".

How can I use this api?

Thanks,
Sincerely

From: Ole Troan 
Sent: Monday, July 16, 2018 12:46 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Interface Mib II counters

Hi Rubina,

> I wanna extract stats of vpp interfaces (specially information about traffic, 
> ie. bytes out, bytes in, ...) and set them to relevant snmp oid.
> Is there any VPP infrastructure to do that?
> If not, what is the efficient way to extract data?

The most efficient (and new) way to extract stats is through the shared memory 
segment for counters.
See commit 048a4e5a.

Then you could build an external SNMP agent scraping counters. There is an 
example usage in the stats_client.c file.
I'm also working on building more client support, including for Python. I just 
need to get back from holidays first.

Cheers,
Ole
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9847): https://lists.fd.io/g/vpp-dev/message/9847
Mute This Topic: https://lists.fd.io/mt/23483393/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Interface Mib II counters

[Rubina Bianchi]

Hi Dear VPP

I wanna extract stats of vpp interfaces (specially information about traffic, 
ie. bytes out, bytes in, ...) and set them to relevant snmp oid.
Is there any VPP infrastructure to do that?
If not, what is the efficient way to extract data?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9845): https://lists.fd.io/g/vpp-dev/message/9845
Mute This Topic: https://lists.fd.io/mt/23483393/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] CPU Usage in VPP

[Rubina Bianchi]

Hi Damjan

I run T-rex on a VPP acl stateful machine with 4 core and run a watch on  
vppctl show run.
Average vectors/node value per each thread increases at first until they reach 
around 60 (20% of 255 that you mentioned).
After that, I saw drop in my T-rex report but average vectors/node value is 
still around 60.
I stopped T-rex and I expected average vectors/node value should be decreased. 
But It didn't change an it was still around 60.
I wait until all acl session to be deleted but average vectors/node didn't go 
down?

Am I missed anything from your last email?

From: John DeNisco (jdenisco) 
Sent: Thursday, June 28, 2018 4:47 PM
To: Damjan Marion; Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] CPU Usage in VPP


Rubina,

Damjan,



I am working on a project to enhance the VPP documentation. I think these are 
great questions and answers.



Do you mind if I use them for my documents?



Thanks,



John



From:  on behalf of Damjan Marion 
Date: Tuesday, June 26, 2018 at 6:28 AM
To: Rubina Bianchi 
Cc: "vpp-dev@lists.fd.io" 
Subject: Re: [vpp-dev] CPU Usage in VPP





Yes, good indicator is "average vectors/node" in show run. Bigger number means 
VPP is more busy but also more efficient.

Maximum value us 255 (unless you change VLIB_FRAME_SIZE in code). It basically 
means how many packets are processed

in batch. If VPP is not loaded it will likely poll so fast that it will just 
get one or few packets from the rx queue, as load goes up

vpp will have more work to do, so it will poll less frequently, and that will 
result in more packets waiting in rx queue. More packets

will result in more efficient execution of the code so number of clock cycles / 
packet will go down.

When "average vectors/node" goes up close to 255, you will likely start 
observing  rx queue tail drops.



Hope this explains...



On 26 Jun 2018, at 07:57, Rubina Bianchi 
mailto:r_bian...@outlook.com>> wrote:



Thanks Dear Damjan



So, Is there any way to compute VPP load or power?

Actually, what I'm looking for is a metric to determine how much is VPP load in 
different cases.

In other word I want to know how much VPP is busy.



Thanks,

Sincerely





From: Damjan Marion mailto:dmar...@me.com>>
Sent: Monday, June 25, 2018 8:43 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] CPU Usage in VPP





With at least one interface in polling mode, VPP CPU is always 100% utilised.

For memory usage, you can use "show memory [verbose]" debug cli.





On 25 Jun 2018, at 13:51, Rubina Bianchi 
mailto:r_bian...@outlook.com>> wrote:



Hi Dear VPP



Is there any command or tools in vpp to show real cpu and memory usage on 
runtime? I used top and htop in linux kernel but it shows pre-allocated 
amounts. I also tried vppctl sho runtime, but its frequency is less than what I 
expected (on high traffic and low traffic its reported clocks are same or have 
low frequency).



Thanks,

Sincerely



_._,_._,_



Links:

You receive all messages sent to this group.

View/Reply Online (#9705)<https://lists.fd.io/g/vpp-dev/message/9705> | Reply 
To 
Sender<mailto:dmar...@me.com?subject=Private:%20Re:%20Re%3A%20%5Bvpp-dev%5D%20CPU%20Usage%20in%20VPP>
 | Reply To 
Group<mailto:vpp-dev@lists.fd.io?subject=Re:%20Re%3A%20%5Bvpp-dev%5D%20CPU%20Usage%20in%20VPP>
 | Mute This Topic<https://lists.fd.io/mt/22674849/675681> | New 
Topic<https://lists.fd.io/g/vpp-dev/post>

Your Subscription<https://lists.fd.io/g/vpp-dev/editsub/675681> | Contact Group 
Owner<mailto:vpp-dev+ow...@lists.fd.io> | 
Unsubscribe<https://lists.fd.io/g/vpp-dev/unsub> [jdeni...@cisco.com]

_._,_._,_


acls
Description: acls


init.conf
Description: init.conf


startup.conf
Description: startup.conf


show run output
Description: show run output
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9749): https://lists.fd.io/g/vpp-dev/message/9749
Mute This Topic: https://lists.fd.io/mt/22674849/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] CPU Usage in VPP

[Rubina Bianchi]

Thanks Dear Damjan

So, Is there any way to compute VPP load or power?
Actually, what I'm looking for is a metric to determine how much is VPP load in 
different cases.
In other word I want to know how much VPP is busy.

Thanks,
Sincerely


From: Damjan Marion 
Sent: Monday, June 25, 2018 8:43 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] CPU Usage in VPP


With at least one interface in polling mode, VPP CPU is always 100% utilised.
For memory usage, you can use "show memory [verbose]" debug cli.


On 25 Jun 2018, at 13:51, Rubina Bianchi 
mailto:r_bian...@outlook.com>> wrote:

Hi Dear VPP

Is there any command or tools in vpp to show real cpu and memory usage on 
runtime? I used top and htop in linux kernel but it shows pre-allocated 
amounts. I also tried vppctl sho runtime, but its frequency is less than what I 
expected (on high traffic and low traffic its reported clocks are same or have 
low frequency).

Thanks,
Sincerely



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9703): https://lists.fd.io/g/vpp-dev/message/9703
Mute This Topic: https://lists.fd.io/mt/22674849/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CPU Usage in VPP

[Rubina Bianchi]

Hi Dear VPP

Is there any command or tools in vpp to show real cpu and memory usage on 
runtime? I used top and htop in linux kernel but it shows pre-allocated 
amounts. I also tried vppctl sho runtime, but its frequency is less than what I 
expected (on high traffic and low traffic its reported clocks are same or have 
low frequency).

Thanks,
Sincerely

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9686): https://lists.fd.io/g/vpp-dev/message/9686
Mute This Topic: https://lists.fd.io/mt/22674849/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Support for TCP flag

[Rubina Bianchi]

As Regards to your question, the case that I'm testing is connection tracking 
in stateful firewall which its functionality is the same as Linux conntrack. Do 
you have any plan to provide VPP as an appropriate infrastructure for firewall 
applications?

From: Andrew  Yourtchenko 
Sent: Tuesday, May 29, 2018 1:10 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Support for TCP flag

Hi Rubina,

I designed the stateful mode to be just a bit more than the ACL, with
a "diode" state, rather than going for the fully fledged firewall
model - as a balance between the simplicity and the functionality.

The full tracking of the TCP state machine was not in scope - getting
into that territory properly requires also TCP sequence number
tracking, etc. - and there the complexity would far outweigh the
usefulness for most practical cases.

So I needed to primarily differentiate the session state from the
timeout perspective - when to remove it.

For that purpose, there are  two types of TCP sessions, decided by
taking by the combination of SYN,FIN,RST,ACK TCP flag bits seen from
each side:

1) Those that has seen SYN+ACK on both sides are fully open (this is
where the "tcp idle" timeout applies, which is usually rather long.

2)  Those that had seen any other combination of the flags (this is
where the "tcp transient" timeout applies, which is default to 2
minutes)

As we receive the packets, we update the seen flags, and we may change
the current idle timeout based on the accumulated seen flags.

Additionally, if we run out of sessions when creating the new ones,
then the sessions in the transient state will be cleaned up and reused
in the FIFO manner - so as to simulate a simple mechanism against the
resource starvation for the higher session rate.

This is a deliberate design choice, and unless there is some
operational issues with it (i.e. where the resource clean-up does not
happen where it should, etc...), I did not have any plans to change
it.

So, could you expand a bit more on what kind of use case you are
looking for, to discuss further ?

--a

On 5/29/18, Rubina Bianchi  wrote:
> Hi
> I have a question about vpp stateful mode. It seems that vpp stateful mode
> hasn't implemented completely. I mean there aren't any feature same as
> contrack in linux kernel. So, vpp doesn't have any mechanism to handle TCP
> sessions based on different flags. For example I sent TCP three way
> handshaking packets in different order (ack -> syn -> syn-ack), in this case
> an idle session is added to session table. Do you have any plan to develop
> it?
>

Re: [vpp-dev] Rx stuck to 0 after a while

[Rubina Bianchi]

Dear Andrew

Sorry for delayed response. I checked your second patch and here is my test 
result:

Best case is still the best and vpp throughput is Maximum (18.5 Gbps) in my 
scenario.
Worst case is getting better than past. I never see deadlock again and 
throughput increases from 50 Mbps to 5.5 Gbps. I also added my T-Rex result.

-Per port stats table
  ports |   0 |   1
 
-
   opackets |  1119818503 |  1065627562
 obytes |490687253990 |471065675962
   ipackets |   274437415 |   391504529
 ibytes |120020261974 |170214837563
ierrors |   0 |   0
oerrors |   0 |   0
  Tx Bw |   9.48 Gbps |   9.08 Gbps

-Global stats enabled
 Cpu Utilization : 88.4  %  7.0 Gb/core
 Platform_factor : 1.0
 Total-Tx:  18.56 Gbps
 Total-Rx:   5.78 Gbps
 Total-PPS   :   5.27 Mpps
 Total-CPS   :  79.51 Kcps

 Expected-PPS:   9.02 Mpps
 Expected-CPS: 135.31 Kcps
 Expected-BPS:  31.77 Gbps

 Active-flows:88840  Clients :  252   Socket-util : 0.5598 %
 Open-flows  : 33973880  Servers :65532   Socket :88840 
Socket/Clients :  352.5
 drop-rate   :  12.79 Gbps
 current time: 423.4 sec
 test duration   : 99576.6 sec

One point that I missed and would be helpful is that I run T-Rex with '-p' 
parameter:
./t-rex-64 -c 6 -d 10 -f cap2/sfr.yaml --cfg cfg/trex_cfg.yaml -m 30 -p

Thanks,
Sincerely


From: Andrew  Yourtchenko 
Sent: Wednesday, May 30, 2018 12:08 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Rx stuck to 0 after a while

Dear Rubina,

Thanks for checking it!

yeah actually that patch was leaking the sessions in the session reuse
path. I have got the setup in the lab locally yesterday and am working
on a better way to do it...

Will get back to you when I am happy with the way the code works..

--a



On 5/29/18, Rubina Bianchi  wrote:
> Dear Andrew
>
> I cleaned everything and created a new deb packages by your patch once
> again. With your patch I never see deadlock again, but still I have
> throughput problem in my scenario.
>
> -Per port stats table
>   ports |   0 |   1
> -
>opackets |   474826597 |   452028770
>  obytes |207843848531 |199591809555
>ipackets |71010677 |72028456
>  ibytes | 31441646551 | 31687562468
> ierrors |   0 |   0
> oerrors |   0 |   0
>   Tx Bw |   9.56 Gbps |   9.16 Gbps
>
> -Global stats enabled
>  Cpu Utilization : 88.4  %  7.1 Gb/core
>  Platform_factor : 1.0
>  Total-Tx:  18.72 Gbps
>  Total-Rx:  59.30 Mbps
>  Total-PPS   :   5.31 Mpps
>  Total-CPS   :  79.79 Kcps
>
>  Expected-PPS:   9.02 Mpps
>  Expected-CPS: 135.31 Kcps
>  Expected-BPS:  31.77 Gbps
>
>  Active-flows:88837  Clients :  252   Socket-util : 0.5598 %
>  Open-flows  : 14708455  Servers :65532   Socket :88837
> Socket/Clients :  352.5
>  Total_queue_full : 328355248
>  drop-rate   :  18.66 Gbps
>  current time: 180.9 sec
>  test duration   : 99819.1 sec
>
> In best case (4 interface in one numa that only 2 of them has acl) my device
> (HP DL380 G9) throughput is maximum (18.72Gbps) but in worst case (4
> interface in one numa that all of them has acl) my device throughput will
> decrease from maximum to around 60Mbps. Actually patch just prevent deadlock
> in my case but throughput is same as before.
>
> 
> From: Andrew  Yourtchenko 
> Sent: Tuesday, May 29, 2018 10:11 AM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Rx stuck to 0 after a while
>
> Dear Rubina,
>
> thank you for quickly checking it!
>
> Judging by the logs the VPP quits, so I would say there should be a
> core file, could you check ?
>
> If you find it (doublecheck by the timestamps that it is indeed the
> fresh one), you can load it in gdb (using gdb 'path-to-vpp-binary'
> 'path-to-core') and then get the backtrace using 'bt', this will give
> more idea on what is going on.
>
> --a
>
> On 5/29/18, Rubina Bianchi  wrote:
>> Dear Andrew
>>
>> I tested your patch and my problem still exist, but my service status
>> changed and now there isn't any information about deadlock problem. Do
>> you
>> have any idea about how I can provide you more information?
>>
>> 

Re: [vpp-dev] Support for TCP flag

[Rubina Bianchi]

Thanks a lot for your prompt answer. :)

From: Andrew  Yourtchenko 
Sent: Tuesday, May 29, 2018 1:10 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Support for TCP flag

Hi Rubina,

I designed the stateful mode to be just a bit more than the ACL, with
a "diode" state, rather than going for the fully fledged firewall
model - as a balance between the simplicity and the functionality.

The full tracking of the TCP state machine was not in scope - getting
into that territory properly requires also TCP sequence number
tracking, etc. - and there the complexity would far outweigh the
usefulness for most practical cases.

So I needed to primarily differentiate the session state from the
timeout perspective - when to remove it.

For that purpose, there are  two types of TCP sessions, decided by
taking by the combination of SYN,FIN,RST,ACK TCP flag bits seen from
each side:

1) Those that has seen SYN+ACK on both sides are fully open (this is
where the "tcp idle" timeout applies, which is usually rather long.

2)  Those that had seen any other combination of the flags (this is
where the "tcp transient" timeout applies, which is default to 2
minutes)

As we receive the packets, we update the seen flags, and we may change
the current idle timeout based on the accumulated seen flags.

Additionally, if we run out of sessions when creating the new ones,
then the sessions in the transient state will be cleaned up and reused
in the FIFO manner - so as to simulate a simple mechanism against the
resource starvation for the higher session rate.

This is a deliberate design choice, and unless there is some
operational issues with it (i.e. where the resource clean-up does not
happen where it should, etc...), I did not have any plans to change
it.

So, could you expand a bit more on what kind of use case you are
looking for, to discuss further ?

--a

On 5/29/18, Rubina Bianchi  wrote:
> Hi
> I have a question about vpp stateful mode. It seems that vpp stateful mode
> hasn't implemented completely. I mean there aren't any feature same as
> contrack in linux kernel. So, vpp doesn't have any mechanism to handle TCP
> sessions based on different flags. For example I sent TCP three way
> handshaking packets in different order (ack -> syn -> syn-ack), in this case
> an idle session is added to session table. Do you have any plan to develop
> it?
>

Re: [vpp-dev] Rx stuck to 0 after a while

[Rubina Bianchi]

Dear Andrew

I cleaned everything and created a new deb packages by your patch once again. 
With your patch I never see deadlock again, but still I have throughput problem 
in my scenario.

-Per port stats table
  ports |   0 |   1
 
-
   opackets |   474826597 |   452028770
 obytes |207843848531 |199591809555
   ipackets |71010677 |72028456
 ibytes | 31441646551 | 31687562468
ierrors |   0 |   0
oerrors |   0 |   0
  Tx Bw |   9.56 Gbps |   9.16 Gbps

-Global stats enabled
 Cpu Utilization : 88.4  %  7.1 Gb/core
 Platform_factor : 1.0
 Total-Tx:  18.72 Gbps
 Total-Rx:  59.30 Mbps
 Total-PPS   :   5.31 Mpps
 Total-CPS   :  79.79 Kcps

 Expected-PPS:   9.02 Mpps
 Expected-CPS: 135.31 Kcps
 Expected-BPS:  31.77 Gbps

 Active-flows:88837  Clients :  252   Socket-util : 0.5598 %
 Open-flows  : 14708455  Servers :65532   Socket :88837 
Socket/Clients :  352.5
 Total_queue_full : 328355248
 drop-rate   :  18.66 Gbps
 current time: 180.9 sec
 test duration   : 99819.1 sec

In best case (4 interface in one numa that only 2 of them has acl) my device 
(HP DL380 G9) throughput is maximum (18.72Gbps) but in worst case (4 interface 
in one numa that all of them has acl) my device throughput will decrease from 
maximum to around 60Mbps. Actually patch just prevent deadlock in my case but 
throughput is same as before.


From: Andrew  Yourtchenko 
Sent: Tuesday, May 29, 2018 10:11 AM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Rx stuck to 0 after a while

Dear Rubina,

thank you for quickly checking it!

Judging by the logs the VPP quits, so I would say there should be a
core file, could you check ?

If you find it (doublecheck by the timestamps that it is indeed the
fresh one), you can load it in gdb (using gdb 'path-to-vpp-binary'
'path-to-core') and then get the backtrace using 'bt', this will give
more idea on what is going on.

--a

On 5/29/18, Rubina Bianchi  wrote:
> Dear Andrew
>
> I tested your patch and my problem still exist, but my service status
> changed and now there isn't any information about deadlock problem. Do you
> have any idea about how I can provide you more information?
>
> root@MYRB:~# service vpp status
> * vpp.service - vector packet processing engine
>Loaded: loaded (/lib/systemd/system/vpp.service; disabled; vendor preset:
> enabled)
>Active: inactive (dead)
>
> May 29 09:27:06 MYRB /usr/bin/vpp[30805]: load_one_vat_plugin:67: Loaded
> plugin: udp_ping_test_plugin.so
> May 29 09:27:06 MYRB /usr/bin/vpp[30805]: load_one_vat_plugin:67: Loaded
> plugin: stn_test_plugin.so
> May 29 09:27:06 MYRB vpp[30805]: /usr/bin/vpp[30805]: dpdk: EAL init args:
> -c 1ff -n 4 --huge-dir /run/vpp/hugepages --file-prefix vpp -w :08:00.0
> -w :08:00.1 -w :08
> May 29 09:27:06 MYRB /usr/bin/vpp[30805]: dpdk: EAL init args: -c 1ff -n 4
> --huge-dir /run/vpp/hugepages --file-prefix vpp -w :08:00.0 -w
> :08:00.1 -w :08:00.2 -w 000
> May 29 09:27:07 MYRB vnet[30805]: dpdk_ipsec_process:1012: not enough DPDK
> crypto resources, default to OpenSSL
> May 29 09:27:13 MYRB vnet[30805]: unix_signal_handler:124: received signal
> SIGCONT, PC 0x7fa535dfbac0
> May 29 09:27:13 MYRB vnet[30805]: received SIGTERM, exiting...
> May 29 09:27:13 MYRB systemd[1]: Stopping vector packet processing
> engine...
> May 29 09:27:13 MYRB vnet[30805]: unix_signal_handler:124: received signal
> SIGTERM, PC 0x7fa534121867
> May 29 09:27:13 MYRB systemd[1]: Stopped vector packet processing engine.
>
>
> ________
> From: Andrew  Yourtchenko 
> Sent: Monday, May 28, 2018 5:58 PM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Rx stuck to 0 after a while
>
> Dear Rubina,
>
> Thanks for catching and reporting this!
>
> I suspect what might be happening is my recent change of using two
> unidirectional sessions in bihash vs. the single one triggered a race,
> whereby as the owning worker is deleting the session,
> the non-owning worker is trying to update it. That would logically
> explain the "BUG: .." line (since you don't change the interfaces nor
> moving the traffic around, the 5 tuples should not collide), and as
> well the later stop.
>
> To take care of this issue, I think I will split the deletion of the
> session in two stages:
> 1) deactivation of the bihash entries that steer the traffic
> 2) freeing up the per-worker session structure
>
> and have a little pause time inbetween these two s

Re: [vpp-dev] Rx stuck to 0 after a while

[Rubina Bianchi]

Dear Andrew

I tested your patch and my problem still exist, but my service status changed 
and now there isn't any information about deadlock problem. Do you have any 
idea about how I can provide you more information?

root@MYRB:~# service vpp status
* vpp.service - vector packet processing engine
   Loaded: loaded (/lib/systemd/system/vpp.service; disabled; vendor preset: 
enabled)
   Active: inactive (dead)

May 29 09:27:06 MYRB /usr/bin/vpp[30805]: load_one_vat_plugin:67: Loaded 
plugin: udp_ping_test_plugin.so
May 29 09:27:06 MYRB /usr/bin/vpp[30805]: load_one_vat_plugin:67: Loaded 
plugin: stn_test_plugin.so
May 29 09:27:06 MYRB vpp[30805]: /usr/bin/vpp[30805]: dpdk: EAL init args: -c 
1ff -n 4 --huge-dir /run/vpp/hugepages --file-prefix vpp -w :08:00.0 -w 
:08:00.1 -w :08
May 29 09:27:06 MYRB /usr/bin/vpp[30805]: dpdk: EAL init args: -c 1ff -n 4 
--huge-dir /run/vpp/hugepages --file-prefix vpp -w :08:00.0 -w :08:00.1 
-w :08:00.2 -w 000
May 29 09:27:07 MYRB vnet[30805]: dpdk_ipsec_process:1012: not enough DPDK 
crypto resources, default to OpenSSL
May 29 09:27:13 MYRB vnet[30805]: unix_signal_handler:124: received signal 
SIGCONT, PC 0x7fa535dfbac0
May 29 09:27:13 MYRB vnet[30805]: received SIGTERM, exiting...
May 29 09:27:13 MYRB systemd[1]: Stopping vector packet processing engine...
May 29 09:27:13 MYRB vnet[30805]: unix_signal_handler:124: received signal 
SIGTERM, PC 0x7fa534121867
May 29 09:27:13 MYRB systemd[1]: Stopped vector packet processing engine.



From: Andrew  Yourtchenko 
Sent: Monday, May 28, 2018 5:58 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Rx stuck to 0 after a while

Dear Rubina,

Thanks for catching and reporting this!

I suspect what might be happening is my recent change of using two
unidirectional sessions in bihash vs. the single one triggered a race,
whereby as the owning worker is deleting the session,
the non-owning worker is trying to update it. That would logically
explain the "BUG: .." line (since you don't change the interfaces nor
moving the traffic around, the 5 tuples should not collide), and as
well the later stop.

To take care of this issue, I think I will split the deletion of the
session in two stages:
1) deactivation of the bihash entries that steer the traffic
2) freeing up the per-worker session structure

and have a little pause time inbetween these two so that the
workers-in-progress could
finish updating the structures.

The below gerrit is the first cut:

https://gerrit.fd.io/r/#/c/12770/

It passes the make test right now but I did not kick its tires too
much yet, will do tomorrow.

You can try this change out in your test setup as well and tell me how it feels.

--a

On 5/28/18, Rubina Bianchi  wrote:
> Hi
>
> I run vpp v18.07-rc0~237-g525c9d0f with only 2 interface in stateful acl
> (permit+reflect) and generated sfr traffic using trex v2.27. My rx will
> become 0 after a short while, about 300 sec in my machine. Here is vpp
> status:
>
> root@MYRB:~# service vpp status
> * vpp.service - vector packet processing engine
>Loaded: loaded (/lib/systemd/system/vpp.service; disabled; vendor preset:
> enabled)
>Active: failed (Result: signal) since Mon 2018-05-28 11:35:03 +0130; 37s
> ago
>   Process: 32838 ExecStopPost=/bin/rm -f /dev/shm/db /dev/shm/global_vm
> /dev/shm/vpe-api (code=exited, status=0/SUCCESS)
>   Process: 31754 ExecStart=/usr/bin/vpp -c /etc/vpp/startup.conf
> (code=killed, signal=ABRT)
>   Process: 31750 ExecStartPre=/sbin/modprobe uio_pci_generic (code=exited,
> status=0/SUCCESS)
>   Process: 31747 ExecStartPre=/bin/rm -f /dev/shm/db /dev/shm/global_vm
> /dev/shm/vpe-api (code=exited, status=0/SUCCESS)
>  Main PID: 31754 (code=killed, signal=ABRT)
>
> May 28 16:32:47 MYRB vnet[31754]: acl_fa_node_fn:210: BUG: session
> LSB16(sw_if_index) and 5-tuple collision!
> May 28 16:35:02 MYRB vnet[31754]: unix_signal_handler:124: received signal
> SIGCONT, PC 0x7f1fb591cac0
> May 28 16:35:02 MYRB vnet[31754]: received SIGTERM, exiting...
> May 28 16:35:02 MYRB systemd[1]: Stopping vector packet processing
> engine...
> May 28 16:35:02 MYRB vnet[31754]: unix_signal_handler:124: received signal
> SIGTERM, PC 0x7f1fb3c40867
> May 28 16:35:03 MYRB vpp[31754]: vlib_worker_thread_barrier_sync_int: worker
> thread deadlock
> May 28 16:35:03 MYRB systemd[1]: vpp.service: Main process exited,
> code=killed, status=6/ABRT
> May 28 16:35:03 MYRB systemd[1]: Stopped vector packet processing engine.
> May 28 16:35:03 MYRB systemd[1]: vpp.service: Unit entered failed state.
> May 28 16:35:03 MYRB systemd[1]: vpp.service: Failed with result 'signal'.
>
> I attach my vpp configs to this email. I also run this test with the same
> config and added 4 interface instead of two. But in this case nothing
> happened to vpp and it was functional for a long time.
>
> Thanks,
> RB
>

[vpp-dev] Rx stuck to 0 after a while

[Rubina Bianchi]

Hi

I run vpp v18.07-rc0~237-g525c9d0f with only 2 interface in stateful acl 
(permit+reflect) and generated sfr traffic using trex v2.27. My rx will become 
0 after a short while, about 300 sec in my machine. Here is vpp status:

root@MYRB:~# service vpp status
* vpp.service - vector packet processing engine
   Loaded: loaded (/lib/systemd/system/vpp.service; disabled; vendor preset: 
enabled)
   Active: failed (Result: signal) since Mon 2018-05-28 11:35:03 +0130; 37s ago
  Process: 32838 ExecStopPost=/bin/rm -f /dev/shm/db /dev/shm/global_vm 
/dev/shm/vpe-api (code=exited, status=0/SUCCESS)
  Process: 31754 ExecStart=/usr/bin/vpp -c /etc/vpp/startup.conf (code=killed, 
signal=ABRT)
  Process: 31750 ExecStartPre=/sbin/modprobe uio_pci_generic (code=exited, 
status=0/SUCCESS)
  Process: 31747 ExecStartPre=/bin/rm -f /dev/shm/db /dev/shm/global_vm 
/dev/shm/vpe-api (code=exited, status=0/SUCCESS)
 Main PID: 31754 (code=killed, signal=ABRT)

May 28 16:32:47 MYRB vnet[31754]: acl_fa_node_fn:210: BUG: session 
LSB16(sw_if_index) and 5-tuple collision!
May 28 16:35:02 MYRB vnet[31754]: unix_signal_handler:124: received signal 
SIGCONT, PC 0x7f1fb591cac0
May 28 16:35:02 MYRB vnet[31754]: received SIGTERM, exiting...
May 28 16:35:02 MYRB systemd[1]: Stopping vector packet processing engine...
May 28 16:35:02 MYRB vnet[31754]: unix_signal_handler:124: received signal 
SIGTERM, PC 0x7f1fb3c40867
May 28 16:35:03 MYRB vpp[31754]: vlib_worker_thread_barrier_sync_int: worker 
thread deadlock
May 28 16:35:03 MYRB systemd[1]: vpp.service: Main process exited, code=killed, 
status=6/ABRT
May 28 16:35:03 MYRB systemd[1]: Stopped vector packet processing engine.
May 28 16:35:03 MYRB systemd[1]: vpp.service: Unit entered failed state.
May 28 16:35:03 MYRB systemd[1]: vpp.service: Failed with result 'signal'.

I attach my vpp configs to this email. I also run this test with the same 
config and added 4 interface instead of two. But in this case nothing happened 
to vpp and it was functional for a long time.

Thanks,
RB


init.conf
Description: init.conf


startup.conf
Description: startup.conf


vat
Description: vat


vppctl
Description: vppctl

Re: [vpp-dev] VPP Scalability Problem

[Rubina Bianchi]

Thanks for your prompt response.
If it would be possible I want to know about the maximum throughput which you 
take from vpp in stateful mode (permit+reflect acl rule, session table) ?

Does vpp work scalable? I think the session table is bottleneck in stateful 
scenario and multi-threading? what 's your opinion?

Sent from Outlook<http://aka.ms/weboutlook>

From: Andrew Yourtchenko <ayour...@gmail.com>
Sent: Wednesday, May 9, 2018 2:47 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] VPP Scalability Problem

Dear Rubina,

You could take a look at “perf top” to see what could be going on. if you need 
help, let me know, I would be happy to look at it together.

Also, now as part of the work for 18.07 I am testing a couple of different 
approaches to change the processing for more performance, would you be 
interested to give it a try ?

--a

On 9 May 2018, at 13:19, Rubina Bianchi 
<r_bian...@outlook.com<mailto:r_bian...@outlook.com>> wrote:

Dear VPP Folks,
I have a problem in vpp scalability in statefull mode (by permit+reflect acl 
rules). I installed vpp on HPE ProLiant DL380 Gen9 Server in order to check the 
vpp performance and throughput scalability based on number of vpp worker 
threads.
Our appliance have 2 cpus that each cpu have 22 core (44 core in hyperthread 
mode). In our Scenario, 6 interfaces is considered. Each 2 interfaces is 
bridged and so we have 3 pair interfaces. Startup config file and other configs 
are added to attachment. With increasing the number of worker threads, I 
expected to see more throughput but it is not happened. I mean for example, 
with 18 workers (current config) I expect to see 45 Gbps (Because the best case 
throughput in sfr scenario is almost 15 Gbps per each pair interface) 
throughput (rx) on our DUT with 6 interfaces but the maximum throughput which 
is observed is approximately 24. And after that, increasing the number of 
worker threads not only did not have any effect on throughput but also in some 
cases decrease overall throughput catastrophically. Why?



Sent from Outlook<http://aka.ms/weboutlook>

Re: [vpp-dev] MTU in vpp

[Rubina Bianchi]

Here is my "vppctl show hardware-interfaces detail"

vpp# show hardware-interfaces detail
  NameIdx   Link  Hardware
GigabitEthernet0/9/0   1 up   GigabitEthernet0/9/0
  Ethernet address 52:54:00:e2:39:c1
  Intel 82540EM (e1000)
carrier up full duplex speed 1000 mtu 1500  promisc
flags: admin-up promisc pmd maybe-multiseg tx-offload intel-phdr-cksum
pci id:device 8086:100e subsystem 1af4:1100
pci address:   :00:09.00
max rx packet len: 16128
max num of queues: rx 1 tx 1
promiscuous:   unicast on all-multicast on
vlan offload:  strip off filter off qinq off
rx offload caps:   vlan-strip ipv4-cksum udp-cksum tcp-cksum
tx offload caps:   vlan-insert ipv4-cksum udp-cksum tcp-cksum
rss active:ipv4-frag ipv4-udp ipv6-udp-ex ipv6-udp ipv6-other
   ipv6-ex ipv6
rss supported: none
rx queues 1, rx desc 1024, tx queues 1, tx desc 1024
cpu socket 0

tx frames ok 822
rx frames ok  36
extended stats:
  rx good packets 36
  tx good packets822
  rx good bytes0
  tx good bytes0
  rx missed errors 0
  rx errors0
  tx errors0
  rx mbuf allocation errors0
  rx q0packets 0
  rx q0bytes   0
  rx q0errors  0
  tx q0packets 0
  tx q0bytes   0
GigabitEthernet0/a/0   2 up   GigabitEthernet0/a/0
  Ethernet address 52:54:00:4a:1c:c9
  Intel 82540EM (e1000)
carrier up full duplex speed 1000 mtu 1500  promisc
flags: admin-up promisc pmd maybe-multiseg tx-offload intel-phdr-cksum
pci id:device 8086:100e subsystem 1af4:1100
pci address:   :00:0a.00
max rx packet len: 16128
max num of queues: rx 1 tx 1
promiscuous:   unicast on all-multicast on
vlan offload:  strip off filter off qinq off
rx offload caps:   vlan-strip ipv4-cksum udp-cksum tcp-cksum
tx offload caps:   vlan-insert ipv4-cksum udp-cksum tcp-cksum
rss active:ipv4-frag ipv4-udp ipv6-udp-ex ipv6-udp ipv6-other
   ipv6-ex ipv6
rss supported: none
rx queues 1, rx desc 1024, tx queues 1, tx desc 1024
cpu socket 0

tx frames ok   5
rx frames ok 853
extended stats:
  rx good packets853
  tx good packets  5
  rx good bytes0
  tx good bytes0
  rx missed errors 0
  rx errors0
  tx errors0
  rx mbuf allocation errors0
  rx q0packets 0
  rx q0bytes   0
  rx q0errors  0
  tx q0packets 0
  tx q0bytes   0



Sent from Outlook<http://aka.ms/weboutlook>
________
From: Rubina Bianchi
Sent: Tuesday, May 8, 2018 2:24 PM
To: vpp-dev@lists.fd.io
Subject: MTU in vpp

Hi

I set mtu to all of my interfaces using "vppctl set interface mtu 1500 
".
But when I generate packets larger than mtu size, all packet pass through vpp. 
Is there any parameter that I must set?

I generate large packet using "ping -M do -s 9472 200.200.200.40".

My vpp version: master v18.07-rc0~147-g7220f42c

Thanks,
Sincerely

Sent from Outlook<http://aka.ms/weboutlook>

[vpp-dev] MTU in vpp

[Rubina Bianchi]

Hi

I set mtu to all of my interfaces using "vppctl set interface mtu 1500 
".
But when I generate packets larger than mtu size, all packet pass through vpp. 
Is there any parameter that I must set?

I generate large packet using "ping -M do -s 9472 200.200.200.40".

My vpp version: master v18.07-rc0~147-g7220f42c

Thanks,
Sincerely

Sent from Outlook

Re: [vpp-dev] Freezing Session Deletion Operation

[Rubina Bianchi]

Dear Andrew

My Trex config is uploaded; I also tested the scenario with your Trex config.
The stability of vpp in your run is strange. When I run this scenario, vpp 
crashes in my DUT machine after about 200 second of running Trex.
In this period I see #del sessions is 0 until session pool becomes full, after 
that session deletion starts. But its rate is lower than the one I see when I 
run vpp on single core.

Could you please check my configs once again for any misconfiguration?
Is vpp or dpdk compatible or incompatible with any specified device?

Thanks,
Sincerely

Sent from Outlook<http://aka.ms/weboutlook>

From: Andrew  Yourtchenko <ayour...@gmail.com>
Sent: Monday, March 12, 2018 1:50 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Freezing Session Deletion Operation

Dear Rubina,

I've tried the test locally using the data that you sent, here is the
output from my trex after 10 minutes running:

-Per port stats table
  ports |   0 |   1
 
-
   opackets |   312605970 |   312191927
 obytes |100919855857 |174147108346
   ipackets |   311329098 |   277120788
 ibytes |173666531289 | 76492053900
ierrors |   0 |   0
oerrors |   0 |   0
  Tx Bw |   1.17 Gbps |   2.01 Gbps

-Global stats enabled
 Cpu Utilization : 21.2  %  30.0 Gb/core
 Platform_factor : 1.0
 Total-Tx:   3.18 Gbps
 Total-Rx:   2.89 Gbps
 Total-PPS   : 901.93 Kpps
 Total-CPS   :  13.52 Kcps

 Expected-PPS: 901.92 Kpps
 Expected-CPS:  13.53 Kcps
 Expected-BPS:   3.18 Gbps

 Active-flows: 8883  Clients :  255   Socket-util : 0.0553 %
 Open-flows  :  9425526  Servers :65535   Socket : 8883
Socket/Clients :  34.8
 drop-rate   :   0.00  bps
 current time: 702.8 sec
 test duration   : 2897.2 sec

So, in my setup worked I could not see the behavior you describe...

But we have at least one more thing that may be different between our
setups - is the trex config.

Here is what mine looks like:

- version: 2
  interfaces: ['03:00.0', '03:00.1']
  port_limit: 2
  memory:
   dp_flows: 200
  port_info:
   - ip: 1.1.1.1
 default_gw: 1.1.1.2
   - ip: 1.1.1.2
 default_gw: 1.1.1.1


Could you send me your trex config to see if that might be the
difference between our setups, so I could try it locally ?

Thanks!

--a

On 3/12/18, Rubina Bianchi <r_bian...@outlook.com> wrote:
> Hi Dear Andrew
>
> I repeated once again my scenarios with short timeouts and upload all
> configs and outputs for your consideration.
> I am clear about that session cleaner process doesn't work properly and my
> Trex throughput  stuck at 0.
> Please repeat this scenario to verify this (Unfortunately vpp is just stable
> for 200 second and after that vpp will be down).
>
> Thanks,
> Sincerely
>
> Sent from Outlook<http://aka.ms/weboutlook>
> 
> From: Andrew  Yourtchenko <ayour...@gmail.com>
> Sent: Sunday, March 11, 2018 3:48 PM
> To: Rubina Bianchi
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Freezing Session Deletion Operation
>
> Hi Rubina,
>
> I am assuming you are observing this both in single core and multicore
> scenario ?
>
> Based on the outputs, this is what I think might be going on:
>
> I am seeing the total# of sessions is 100, and no TCP transient
> sessions - thus the packets that require a a session are dropped.
>
> What is a bit peculiar, is that the session delete# per-worker are
> non-zero, yet the the delete counters are zero. To me this indicates
> there was a fair bit of transient sessions, which also then got
> recycled by the TCP sessions properly established, before the idle
> timeout has expired.
>
> And at the moment of taking the show command output the connection
> cleaner activity has not yet kicked in - I do not see either any
> session deleted by idle timeout nor its timer restarted. Which makes
> me think that the time interval in which you are testing must be
> relatively short...
>
> So, assuming the time between the start of the traffic and the time
> you have 1m sessions is quite short, this is simply using up all of
> the connection pool, a classic inherent resource management issue with
> any stateful scenario.
>
> You can verify that the sessions delete and start building again if
> you issue "clear acl-plugin sessions".
>
> Also, changing the session timeouts to more aggressive values (say, 10
> seconds), should kick off the aggressive connection cleaning, thus
> should unlock this condition. Of course

Re: [vpp-dev] Freezing Session Deletion Operation

[Rubina Bianchi]

Hi Dear Andrew

I repeated once again my scenarios with short timeouts and upload all configs 
and outputs for your consideration.
I am clear about that session cleaner process doesn't work properly and my Trex 
throughput  stuck at 0.
Please repeat this scenario to verify this (Unfortunately vpp is just stable 
for 200 second and after that vpp will be down).

Thanks,
Sincerely

Sent from Outlook<http://aka.ms/weboutlook>

From: Andrew  Yourtchenko <ayour...@gmail.com>
Sent: Sunday, March 11, 2018 3:48 PM
To: Rubina Bianchi
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Freezing Session Deletion Operation

Hi Rubina,

I am assuming you are observing this both in single core and multicore
scenario ?

Based on the outputs, this is what I think might be going on:

I am seeing the total# of sessions is 100, and no TCP transient
sessions - thus the packets that require a a session are dropped.

What is a bit peculiar, is that the session delete# per-worker are
non-zero, yet the the delete counters are zero. To me this indicates
there was a fair bit of transient sessions, which also then got
recycled by the TCP sessions properly established, before the idle
timeout has expired.

And at the moment of taking the show command output the connection
cleaner activity has not yet kicked in - I do not see either any
session deleted by idle timeout nor its timer restarted. Which makes
me think that the time interval in which you are testing must be
relatively short...

So, assuming the time between the start of the traffic and the time
you have 1m sessions is quite short, this is simply using up all of
the connection pool, a classic inherent resource management issue with
any stateful scenario.

You can verify that the sessions delete and start building again if
you issue "clear acl-plugin sessions".

Also, changing the session timeouts to more aggressive values (say, 10
seconds), should kick off the aggressive connection cleaning, thus
should unlock this condition. Of course, shorter idle time means
potentially useful connections removed.  (the commands are "set
acl-plugin session timeout <udp|tcp> idle ").

*if* neither of  the above does not adequately describe what you are
seeing, the cleaner node
may for whatever reason ceases to kick in every half a second.

To see the dynamics of conn cleaner node, you can use the debug command
"set acl-plugin session event-trace 1" before the start of the test.
This will produce the event trace, which you can view by "show
event-logger all" - this should give a reasonable idea about what the
cleaner node is up to.

Please let me know.

--a





On 3/11/18, Rubina Bianchi <r_bian...@outlook.com> wrote:
> Hi,
>
> I am testing vpp_18.01.1-124~g302ef0b5 (commit:
> 696e0da1cde7e2bcda5a781f9ad286672de8dcbf) and vpp_18.04-rc0~322-g30787378
> (commit: 30787378f49714319e75437b347b7027a949700d) using Trex with sfr
> scenario in one core and multicore state.
> After a while I saw session deletion rate decreases and vpp throughput
> becomes 0 bps.
> All configuration files and outputs are attached.
>
> Thanks,
> Sincerely
>
> Sent from Outlook<http://aka.ms/weboutlook>
>


acls
Description: acls


init.conf
Description: init.conf


show sessions
Description: show sessions


startup.conf
Description: startup.conf


trex
Description: trex

[vpp-dev] Freezing Session Deletion Operation

[Rubina Bianchi]

Hi,

I am testing vpp_18.01.1-124~g302ef0b5 (commit: 
696e0da1cde7e2bcda5a781f9ad286672de8dcbf) and vpp_18.04-rc0~322-g30787378 
(commit: 30787378f49714319e75437b347b7027a949700d) using Trex with sfr scenario 
in one core and multicore state.
After a while I saw session deletion rate decreases and vpp throughput becomes 
0 bps.
All configuration files and outputs are attached.

Thanks,
Sincerely

Sent from Outlook


acls
Description: acls


init.conf
Description: init.conf


show sessions
Description: show sessions


startup.conf
Description: startup.conf


trex
Description: trex

[vpp-dev] VPP Crash in stress test when configuring stateful ACL

[Rubina Bianchi]

Hi

I run a stress test on stateful VPP. After a while it crash and raise an os 
panic signal.
I run this test on both single and multicore vpp and every time it happens.

My Scenario:

1.
VPP:
Default startup.conf configuration (main and worker on same core)
Session timeouts are configured as follows:
vppctl set acl-plugin session timeout udp idle 5
vppctl set acl-plugin session timeout tcp idle 10
vppctl set acl-plugin session timeout tcp transient 5
2 interface in brdige mode
add permit + reflect acl rule on both Input & Output of all interfaces
Trex:
./t-rex-64 -f cap2/my_custom_test.yaml --cfg 
cfg/concurrent_connection_trex_config.yaml -c 8 -m 3000

2.
VPP:
1 main core + 1 worker core
Session timeouts are configured as follows:
vppctl set acl-plugin session timeout udp idle 5
vppctl set acl-plugin session timeout tcp idle 10
vppctl set acl-plugin session timeout tcp transient 5
2 interface in brdige mode
add permit + reflect acl rule on both Input & Output of all interfaces
Trex:
./t-rex-64 -f cap2/my_custom_test.yaml --cfg 
cfg/concurrent_connection_trex_config.yaml -c 8 -m 3000


3.
VPP:
1 main core + 4 worker core
Session timeouts are configured as follows:
vppctl set acl-plugin session timeout udp idle 5
vppctl set acl-plugin session timeout tcp idle 10
vppctl set acl-plugin session timeout tcp transient 5
2 interface in brdige mode
add permit + reflect acl rule on both Input & Output of all interfaces
Trex:
./t-rex-64 -f cap2/my_custom_test.yaml --cfg 
cfg/concurrent_connection_trex_config.yaml -c 8 -m 3000

I also attached my Trex yaml and config files.

Sent from Outlook


my_custom_test.yaml
Description: my_custom_test.yaml


new_connection_test_pcap.pcap
Description: new_connection_test_pcap.pcap


concurrent_connection_trex_config.yaml
Description: concurrent_connection_trex_config.yaml
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev