Hi Matt, I’ll let Neale provide more details but I think that what you’re looking for is to start vpp with ip {heap-size <value>}
Cheers, Florin > On Jan 29, 2018, at 1:24 PM, Matthew Smith <mgsm...@netgate.com> wrote: > > > Hi, > > I’ve been trying to insert a large set of routes into VPP’s FIB. VPP keeps > exiting after adding a very specific amount of them (64278). If I attach to > the vpp process with gdb and start adding the routes, after a few seconds an > ABRT is received. The backtrace (see below) shows that os_out_of_memory() was > called as a result of trying to resize a vector containing FIB data > structures. > > I was loading the routes originally using a BGP daemon that has been modified > to communicate with the VPP API. When I had problems with that, I stopped > using the BGP daemon and tried adding the same routes using both vppctl and > vpp_api_test. They all ended up at the same result. > > The procedure I have been using to load routes is: > > systemctl start vpp > vppctl set int ip addr TenGigabitEthernet2/0/0 198.51.100.1/24 > vppctl set int state TenGigabitEthernet2/0/0 up > vppctl exec <file_with_routes> > > An example of the commands I used to add routes with vppctl is: > > ip route add 10.20.30.0/24 table 0 via 198.51.100.2 TenGigabitEthernet2/0/0 > > All routes are added to the same table and all have the same next-hop address > and interface. Only the destination prefix varies. I noticed that if I omit > the command to set an IP address on the interface, that a larger number of > routes - about 103k - can be processed before VPP dies. The system I am > testing on is an Atom C2758 with 16 GB of RAM. Just to make sure it wasn’t an > issue specific to that system, I repeated the test on a Xeon D-1541 with 32 > GB of RAM also and saw the exact same behavior (died after adding 64278 > routes). Neither system has anything else running on it. > > Is there any memory tuning required in order to add a large number of routes > to a FIB? Or is this a bug? I tried increasing the hugepages memory (to 4 GB > plus increased vm.max_map_count and kernel.shmmax) and the DPDK socket-mem > (to 1 GB) because those were the memory parameters that I knew of that could > be tuned. Does anyone know what I’m doing wrong or have any ideas for where I > should look? > > Thanks! > -Matt > > > Backtrace: > > (gdb) bt > #0 0x00007feece2761f7 in __GI_raise (sig=sig@entry=6) at raise.c:56 > #1 0x00007feece2778e8 in __GI_abort () at abort.c:90 > #2 0x0000000000405fa3 in os_panic () > #3 0x00007feecef8c7b2 in clib_mem_alloc_aligned_at_offset > (os_out_of_memory_on_failure=1, align_offset=<optimized out>, align=64, > size=13676394) at mem.h:105 > #4 vec_resize_allocate_memory (v=v@entry=0x7fee50be3480, > length_increment=length_increment@entry=1, data_bytes=<optimized out>, > header_bytes=<optimized out>, header_bytes@entry=48, > data_align=data_align@entry=64) at vec.c:84 > #5 0x00007feecfbfa0cf in _vec_resize (data_align=64, header_bytes=48, > data_bytes=<optimized out>, length_increment=1, v=<optimized out>) at > vec.h:142 > #6 ply_create (init_leaf=init_leaf@entry=3, leaf_prefix_len=16, > ply_base_len=ply_base_len@entry=16, m=<optimized out>) at ip4_mtrie.c:183 > #7 0x00007feecfbfa6f7 in set_root_leaf (a=0x7fee8f7fa880, m=0x7fee4fa70600) > at ip4_mtrie.c:495 > #8 ip4_fib_mtrie_route_add (m=0x7fee4fa70600, > dst_address=dst_address@entry=0x7fee8e640668, dst_address_length=<optimized > out>, adj_index=<optimized out>) at ip4_mtrie.c:646 > #9 0x00007feecff356a8 in ip4_fib_table_fwding_dpo_update (fib=<optimized > out>, addr=addr@entry=0x7fee8e640668, len=<optimized out>, dpo=<optimized > out>) at ip4_fib.c:386 > #10 0x00007feecff3a554 in fib_table_fwding_dpo_update (fib_index=<optimized > out>, prefix=prefix@entry=0x7fee8e640658, dpo=dpo@entry=0x7fee8e640674) at > fib_table.c:252 > #11 0x00007feecff45f98 in fib_entry_src_action_install > (fib_entry=0x7fee8e64064c, source=<optimized out>) at fib_entry_src.c:606 > #12 0x00007feecff43a60 in fib_entry_create (fib_index=fib_index@entry=0, > prefix=prefix@entry=0x7fee8f7faa80, source=source@entry=FIB_SOURCE_CLI, > flags=flags@entry=FIB_ENTRY_FLAG_NONE, paths=paths@entry=0x7fee8f6e0e48) at > fib_entry.c:704 > #13 0x00007feecff3ac40 in fib_table_entry_path_add2 > (fib_index=fib_index@entry=0, prefix=prefix@entry=0x7fee8f7faa80, > source=source@entry=FIB_SOURCE_CLI, flags=flags@entry=FIB_ENTRY_FLAG_NONE, > rpath=rpath@entry=0x7fee8f6e0e48) at fib_table.c:568 > #14 0x00007feecfc56f5a in vnet_ip_route_cmd (vm=0x7feed05522c0 > <vlib_global_main>, main_input=<optimized out>, cmd=<optimized out>) at > lookup.c:523 > #15 0x00007feed02f0ba1 in vlib_cli_dispatch_sub_commands > (vm=vm@entry=0x7feed05522c0 <vlib_global_main>, cm=cm@entry=0x7feed05524a0 > <vlib_global_main+480>, input=input@entry=0x7fee8f7facb0, > parent_command_index=<optimized out>) at cli.c:588 > #16 0x00007feed02f0f57 in vlib_cli_dispatch_sub_commands > (vm=vm@entry=0x7feed05522c0 <vlib_global_main>, cm=cm@entry=0x7feed05524a0 > <vlib_global_main+480>, input=input@entry=0x7fee8f7facb0, > parent_command_index=parent_command_index@entry=0) at cli.c:566 > #17 0x00007feed02f1190 in vlib_cli_input (vm=vm@entry=0x7feed05522c0 > <vlib_global_main>, input=input@entry=0x7fee8f7facb0, > function=function@entry=0x0, function_arg=function_arg@entry=0) at cli.c:662 > #18 0x00007feed032bc79 in unix_cli_exec (vm=0x7feed05522c0 > <vlib_global_main>, input=<optimized out>, cmd=<optimized out>) at cli.c:3002 > #19 0x00007feed02f0ba1 in vlib_cli_dispatch_sub_commands > (vm=vm@entry=0x7feed05522c0 <vlib_global_main>, cm=cm@entry=0x7feed05524a0 > <vlib_global_main+480>, input=input@entry=0x7fee8f7faed0, > parent_command_index=parent_command_index@entry=0) at cli.c:588 > #20 0x00007feed02f1190 in vlib_cli_input (vm=0x7feed05522c0 > <vlib_global_main>, input=input@entry=0x7fee8f7faed0, > function=function@entry=0x7feed0332030 <unix_vlib_cli_output>, > function_arg=function_arg@entry=0) at cli.c:662 > #21 0x00007feed033319b in unix_cli_process_input (cli_file_index=0, > cm=0x7feed0552160 <unix_cli_main>) at cli.c:2308 > #22 0x00007feed0336c46 in unix_cli_process (vm=0x7feed05522c0 > <vlib_global_main>, rt=0x7fee8f7ea000, f=<optimized out>) at cli.c:2420 > #23 0x00007feed02fe856 in vlib_process_bootstrap (_a=<optimized out>) at > main.c:1231 > #24 0x00007feecef57918 in clib_calljmp () at longjmp.S:110 > #25 0x00007fee9022bc20 in ?? () > #26 0x00007feed02ffb99 in vlib_process_startup (f=0x0, p=0x7fee8f7ea000, > vm=0x7feed05522c0 <vlib_global_main>) at main.c:1253 > #27 dispatch_process (vm=0x7feed05522c0 <vlib_global_main>, p=0x7fee8f7ea000, > last_time_stamp=0, f=0x0) at main.c:1296 > #28 0x0000000000000001 in ?? () > #29 0x0000000000000000 in ?? () > (gdb) > > _______________________________________________ > vpp-dev mailing list > vpp-dev@lists.fd.io > https://lists.fd.io/mailman/listinfo/vpp-dev _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev