Hi Stanislav,
I repeated test with disabled api trace
vpp# show api trace-status
RX Trace disabled
TX Trace disabled
vpp# show threads
ID Name Type LWP Sched Policy (Priority) lcore
Core Socket State
0 vpp_main 178961 other (0) n/a
n/a n/a
Before Test
vpp# show memory main-heap verbose
Thread 0 vpp_main
base 0x7f5f6b4aa000, size 8g, locked, unmap-on-destroy, name 'main heap'
page stats: page-size 4K, total 2097152, mapped 40974, not-mapped 1525072,
unknown 531106
numa 0: 18302 pages, 71.49m bytes
numa 1: 22672 pages, 88.56m bytes
total: 7.99G, used: 36.12M, free: 7.96G, trimmable: 7.96G
free chunks 150 free fastbin blks 0
max total allocated 7.99G
After several hours of execution
vpp# show memory main-heap verbose
Thread 0 vpp_main
base 0x7f5f6b4aa000, size 8g, locked, unmap-on-destroy, name 'main heap'
page stats: page-size 4K, total 2097152, mapped 563066, not-mapped 1002980,
unknown 531106
numa 0: 545742 pages, 2.08g bytes
numa 1: 17324 pages, 67.67m bytes
total: 7.99G, used: 1.04G, free: 6.96G, trimmable: 6.96G
free chunks 1 free fastbin blks 0
max total allocated 7.99G
Best Regards,
Eugene
From: [email protected] <[email protected]> On Behalf Of Stanislav Zaikin
Sent: Wednesday, February 22, 2023 11:46 AM
To: [email protected]
Subject: Re: [vpp-dev] VPP Policer API Memory Leak
Hi Eugene,
Could you run again with disabled api trace, wait until let's say 1g is
consumed and then show us the output of show memory main-heap verbose?
On Tue, 21 Feb 2023 at 20:13, <[email protected]
<mailto:[email protected]> > wrote:
Hi Steven,
Thanks for response.
I added “nitems 65535” and repeated test. The main-heap usage still growing up.
Also I completely disabled the “api-trace” and nothing changed =(
Best Regards,
Eugene
From: [email protected] <mailto:[email protected]> <[email protected]
<mailto:[email protected]> > On Behalf Of steven luong via lists.fd.io
<http://lists.fd.io>
Sent: Tuesday, February 21, 2023 8:58 PM
To: [email protected] <mailto:[email protected]>
Subject: Re: [vpp-dev] VPP Policer API Memory Leak
I bet you didn’t limit the number of API trace entries. Try limit the number of
API trace entries that VPP keeps with nitems and give it a reasonable number.
api-trace {
on
nitems 65535
}
Steven
From: <[email protected] <mailto:[email protected]> > on behalf of
"[email protected] <mailto:[email protected]> "
<[email protected] <mailto:[email protected]> >
Reply-To: "[email protected] <mailto:[email protected]> "
<[email protected] <mailto:[email protected]> >
Date: Tuesday, February 21, 2023 at 7:14 AM
To: "[email protected] <mailto:[email protected]> " <[email protected]
<mailto:[email protected]> >
Subject: [vpp-dev] VPP Policer API Memory Leak
Hi Dear Developers,
I am testing creating and deleting of policers and it looks that there is a
memory leak
VPP Version: v22.10-release
My simple script:
#!/bin/env python
from vpp_papi import VPPApiClient
from vpp_papi import VppEnum
import os
import fnmatch
import sys
from time import sleep
vpp_json_dir = '/usr/share/vpp/api/'
# construct a list of all the json api files
jsonfiles = []
for root, dirnames, filenames in os.walk(vpp_json_dir):
for filename in fnmatch.filter(filenames, '*.api.json'):
jsonfiles.append(os.path.join(root, filename))
vpp = VPPApiClient(apifiles=jsonfiles, server_address='/run/vpp/api.sock')
vpp.connect("test-client")
r = vpp.api.show_version()
print('VPP version is %s' % r.version)
while True:
### Create 10 policers
for i in range (10):
name = "policer_" + str(i)
policer_add_del = vpp.api.policer_add_del(is_add=True, name=name,
cb=2500,cir=1000, eb=3000,eir=0,rate_type=0,round_type=1,type=1)
print(policer_add_del)
### Delete 10 policers
for i in range (10):
name = "policer_" + str(i)
policer_add_del = vpp.api.policer_add_del(is_add=False, name=name,
cb=2500,cir=1000, eb=3000,eir=0,rate_type=0,round_type=1,type=1)
print(policer_add_del)
The memory usage is growing permanently and very fast. It takes less than 10
minutes to spend ~ 100Mb of main-heap.
vpp# show memory main-heap
Thread 0 vpp_main
base 0x7efb0a117000, size 8g, locked, unmap-on-destroy, traced, name 'main
heap'
page stats: page-size 4K, total 2097152, mapped 116134, not-mapped 1450398,
unknown 530620
numa 0: 115788 pages, 452.29m bytes
numa 1: 346 pages, 1.35m bytes
total: 7.99G, used: 188.26M, free: 7.82G, trimmable: 7.82G
Bytes Count Sample Traceback
1774488 14781 0x7efb15d59570 _vec_alloc_internal + 0x6b
vl_msg_api_trace + 0x4a4
vl_msg_api_socket_handler + 0x10f
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
291272 1 0x7efb15cf4190 _vec_realloc_internal + 0x89
vl_msg_api_trace + 0x529
vl_msg_api_socket_handler + 0x10f
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
178928 7390 0x7efb15d595f0 _vec_alloc_internal + 0x6b
va_format + 0x2318
format + 0x83
0x7efd0a896b91
vl_msg_api_socket_handler + 0x226
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
85800 1 0x7efb135ca840 _vec_realloc_internal + 0x89
vl_socket_api_send + 0x720
vl_api_sockclnt_create_t_handler + 0x2e2
vl_msg_api_socket_handler + 0x226
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
4104 1 0x7efb13dcf220 _vec_alloc_internal + 0x6b
0x7efd0a5e0965
0x7efd0a5f05c4
0x7efd0a584978
0x7efd0a5845f5
0x7efd0a5f213b
0x7efd0a48d6a8
1920 16 0x7efb13e62a40 _vec_realloc_internal + 0x89
0x7efd0a482d1d
va_format + 0xf62
format + 0x83
va_format + 0x1041
format + 0x83
va_format + 0x1041
vlib_log + 0x2c6
0x7efb08b033aa
0x7efb08b031c9
0x7efb08b0cc6d
0x7efb08b988ee
vpp# show memory main-heap verbose
Thread 0 vpp_main
base 0x7efb0a117000, size 8g, locked, unmap-on-destroy, traced, name 'main
heap'
page stats: page-size 4K, total 2097152, mapped 170152, not-mapped 1396380,
unknown 530620
numa 0: 169806 pages, 663.30m bytes
numa 1: 346 pages, 1.35m bytes
total: 7.99G, used: 289.51M, free: 7.72G, trimmable: 7.71G
free chunks 93 free fastbin blks 0
max total allocated 7.99G
Bytes Count Sample Traceback
29822832 248408 0x7efb18bbfaf0 _vec_alloc_internal + 0x6b
vl_msg_api_trace + 0x4a4
vl_msg_api_socket_handler + 0x10f
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
22669680 822640 0x7efb19210510 _vec_alloc_internal + 0x6b
va_format + 0x2318
format + 0x83
0x7efd0a896b91
vl_msg_api_socket_handler + 0x226
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
2212040 1 0x7efb17ee2220 _vec_realloc_internal + 0x89
vl_msg_api_trace + 0x529
vl_msg_api_socket_handler + 0x10f
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
85800 1 0x7efb135ca840 _vec_realloc_internal + 0x89
vl_socket_api_send + 0x720
vl_api_sockclnt_create_t_handler + 0x2e2
vl_msg_api_socket_handler + 0x226
vl_socket_process_api_msg + 0x1d
0x7efd0c177171
0x7efd0a588837
0x7efd0a48d6a8
Thanks in Advance
Best Regards,
Eugene
--
Best regards
Stanislav Zaikin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22623): https://lists.fd.io/g/vpp-dev/message/22623
Mute This Topic: https://lists.fd.io/mt/97110527/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
