Re: [vpp-dev] Create big tables on huge-page
Hi Nitin,
Both 2M and 1G are supported by pmalloc, but they are not supported at same
time, as system is configured with either 2M or 1G hugepages, not both at the
same time.
Thanks.
From: vpp-dev@lists.fd.io On Behalf Of Nitin Saxena via
lists.fd.io
Sent: 2020年7月29日 15:23
To: Honnappa Nagarahalli ; Damjan Marion
Cc: Lijian Zhang ; vpp-dev ; nd
; Govindarajan Mohandoss ;
Jieqiang Wang
Subject: Re: [vpp-dev] Create big tables on huge-page
Hi Lijian,
+1 on the finding. It would be interested to know how much is the performance
gain.
Having said that, correct me if I am wrong, I think pmalloc module works only
on single hugepage size (pm->def_log2_page_sz) which means either 1G or 2M and
not both
Thanks,
Nitin
From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
mailto:vpp-dev@lists.fd.io>> On Behalf Of Honnappa
Nagarahalli
Sent: Thursday, July 23, 2020 10:53 PM
To: Damjan Marion mailto:dmar...@me.com>>
Cc: Lijian Zhang mailto:lijian.zh...@arm.com>>; vpp-dev
mailto:vpp-dev@lists.fd.io>>; nd
mailto:n...@arm.com>>; Govindarajan Mohandoss
mailto:govindarajan.mohand...@arm.com>>;
Jieqiang Wang mailto:jieqiang.w...@arm.com>>; Honnappa
Nagarahalli mailto:honnappa.nagaraha...@arm.com>>
Subject: [EXT] Re: [vpp-dev] Create big tables on huge-page
External Email
Sure. We will create couple of patches (in the areas we are analyzing
currently) and we can decide from there.
Thanks,
Honnappa
From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 12:17 PM
To: Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>>
Cc: Lijian Zhang mailto:lijian.zh...@arm.com>>; vpp-dev
mailto:vpp-dev@lists.fd.io>>; nd
mailto:n...@arm.com>>; Govindarajan Mohandoss
mailto:govindarajan.mohand...@arm.com>>;
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page
Hard to say without seeing the patch. Can you summarize what the changes will
be in each particular .c file?
On 23 Jul 2020, at 18:00, Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>> wrote:
Hi Damjan,
Thank you. Till your patch is ready, would you accept patches
that would enable creating these tables in 1G huge pages as temporary solution?
Thanks,
Honnappa
From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 7:15 AM
To: Lijian Zhang mailto:lijian.zh...@arm.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd
mailto:n...@arm.com>>; Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>>;
Govindarajan Mohandoss
mailto:govindarajan.mohand...@arm.com>>;
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page
I started working on patch which addresses most of this points, few weeks ago,
but likely I will not have it completed for 20.09.
Even if it is completed, it is probably bad idea to merge it so late in the
release process….
—
Damjan
On 23 Jul 2020, at 10:45, Lijian Zhang
mailto:lijian.zh...@arm.com>> wrote:
Hi Maintainers,
From VPP source code, ip4-mtrie table is created on huge-page only when below
parameters are set in configuration file.
While adjacency table is created on normal-page always.
36 ip {
37 heap-size 256M
38 mtrie-hugetlb
39 }
In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and 10K
entries in adjacency table.
By creating ip4-mtrie table on 1G huge-page with above parameters set and
similarly create adjacency table on 1G huge-page, although I don’t observe
obvious throughput performance improvement, but TLB misses are dramatically
reduced.
Do you think configuration of 10K routing entries + 10K adjacency entries is a
reasonable and possible config, or normally it would be 10K routing entries +
only several adjacency entries?
Does it make sense to create adjacency table on huge-pages?
Another problem is although above assigned heap-size is 256M, but on 1G
huge-page system, it seems to occupy a huge-page completely, other memory space
within that huge-page seems will not be used by other tables.
Same as the bihash based tables, only 2M huge-page system is supported. To
support creating bihash based tables on 1G huge-page system, each table will
occupy a 1G huge-page completely, but that will waste a lot of memories.
Is it possible just like pmalloc module, reserving a big memory space on 1G/2M
huge-pages in initialization stage, and then allocate memory pieces per
requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
could be created on huge-pages and will fully utilize the huge-pages.
I tried to create MAC table on 1G huge-page, and it does improve throughput
performance.
vpp# show bihash
Name Actual Configured
GBP Endpoints - MAC/BD 1m 1m
b4s
Re: [vpp-dev] Create big tables on huge-page
Hi Lijian,
+1 on the finding. It would be interested to know how much is the performance
gain.
Having said that, correct me if I am wrong, I think pmalloc module works only
on single hugepage size (pm->def_log2_page_sz) which means either 1G or 2M and
not both
Thanks,
Nitin
From: vpp-dev@lists.fd.io On Behalf Of Honnappa
Nagarahalli
Sent: Thursday, July 23, 2020 10:53 PM
To: Damjan Marion
Cc: Lijian Zhang ; vpp-dev ; nd
; Govindarajan Mohandoss ;
Jieqiang Wang ; Honnappa Nagarahalli
Subject: [EXT] Re: [vpp-dev] Create big tables on huge-page
External Email
Sure. We will create couple of patches (in the areas we are analyzing
currently) and we can decide from there.
Thanks,
Honnappa
From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 12:17 PM
To: Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>>
Cc: Lijian Zhang mailto:lijian.zh...@arm.com>>; vpp-dev
mailto:vpp-dev@lists.fd.io>>; nd
mailto:n...@arm.com>>; Govindarajan Mohandoss
mailto:govindarajan.mohand...@arm.com>>;
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page
Hard to say without seeing the patch. Can you summarize what the changes will
be in each particular .c file?
On 23 Jul 2020, at 18:00, Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>> wrote:
Hi Damjan,
Thank you. Till your patch is ready, would you accept patches
that would enable creating these tables in 1G huge pages as temporary solution?
Thanks,
Honnappa
From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 7:15 AM
To: Lijian Zhang mailto:lijian.zh...@arm.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd
mailto:n...@arm.com>>; Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>>;
Govindarajan Mohandoss
mailto:govindarajan.mohand...@arm.com>>;
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page
I started working on patch which addresses most of this points, few weeks ago,
but likely I will not have it completed for 20.09.
Even if it is completed, it is probably bad idea to merge it so late in the
release process….
—
Damjan
On 23 Jul 2020, at 10:45, Lijian Zhang
mailto:lijian.zh...@arm.com>> wrote:
Hi Maintainers,
From VPP source code, ip4-mtrie table is created on huge-page only when below
parameters are set in configuration file.
While adjacency table is created on normal-page always.
36 ip {
37 heap-size 256M
38 mtrie-hugetlb
39 }
In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and 10K
entries in adjacency table.
By creating ip4-mtrie table on 1G huge-page with above parameters set and
similarly create adjacency table on 1G huge-page, although I don’t observe
obvious throughput performance improvement, but TLB misses are dramatically
reduced.
Do you think configuration of 10K routing entries + 10K adjacency entries is a
reasonable and possible config, or normally it would be 10K routing entries +
only several adjacency entries?
Does it make sense to create adjacency table on huge-pages?
Another problem is although above assigned heap-size is 256M, but on 1G
huge-page system, it seems to occupy a huge-page completely, other memory space
within that huge-page seems will not be used by other tables.
Same as the bihash based tables, only 2M huge-page system is supported. To
support creating bihash based tables on 1G huge-page system, each table will
occupy a 1G huge-page completely, but that will waste a lot of memories.
Is it possible just like pmalloc module, reserving a big memory space on 1G/2M
huge-pages in initialization stage, and then allocate memory pieces per
requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
could be created on huge-pages and will fully utilize the huge-pages.
I tried to create MAC table on 1G huge-page, and it does improve throughput
performance.
vpp# show bihash
Name Actual Configured
GBP Endpoints - MAC/BD 1m 1m
b4s 64m 64m
b4s 64m 64m
in2out 10.12m 10.12m
in2out 10.12m 10.12m
ip4-dr 2m 2m
ip4-dr 2m 2m
ip6 FIB fwding table32m 32m
ip6 FIB non-fwding table32m 32m
ip6 mFIB table 32m 32m
l2fib mac table512m 512m
mapping_by_as4 64m 64m
out2in 128m 128m
out2in 128m 128m
out2in 10.12m 10.12m
out2in 10.12m 10.12m
pppoe link table 8m 8m
pppoe session table 8m 8m
static_mapping_by_external 6
Re: [vpp-dev] Create big tables on huge-page
Sure. We will create couple of patches (in the areas we are analyzing
currently) and we can decide from there.
Thanks,
Honnappa
From: Damjan Marion
Sent: Thursday, July 23, 2020 12:17 PM
To: Honnappa Nagarahalli
Cc: Lijian Zhang ; vpp-dev ; nd
; Govindarajan Mohandoss ;
Jieqiang Wang
Subject: Re: [vpp-dev] Create big tables on huge-page
Hard to say without seeing the patch. Can you summarize what the changes will
be in each particular .c file?
On 23 Jul 2020, at 18:00, Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>> wrote:
Hi Damjan,
Thank you. Till your patch is ready, would you accept patches
that would enable creating these tables in 1G huge pages as temporary solution?
Thanks,
Honnappa
From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 7:15 AM
To: Lijian Zhang mailto:lijian.zh...@arm.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd
mailto:n...@arm.com>>; Honnappa Nagarahalli
mailto:honnappa.nagaraha...@arm.com>>;
Govindarajan Mohandoss
mailto:govindarajan.mohand...@arm.com>>;
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page
I started working on patch which addresses most of this points, few weeks ago,
but likely I will not have it completed for 20.09.
Even if it is completed, it is probably bad idea to merge it so late in the
release process….
—
Damjan
On 23 Jul 2020, at 10:45, Lijian Zhang
mailto:lijian.zh...@arm.com>> wrote:
Hi Maintainers,
From VPP source code, ip4-mtrie table is created on huge-page only when below
parameters are set in configuration file.
While adjacency table is created on normal-page always.
36 ip {
37 heap-size 256M
38 mtrie-hugetlb
39 }
In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and 10K
entries in adjacency table.
By creating ip4-mtrie table on 1G huge-page with above parameters set and
similarly create adjacency table on 1G huge-page, although I don’t observe
obvious throughput performance improvement, but TLB misses are dramatically
reduced.
Do you think configuration of 10K routing entries + 10K adjacency entries is a
reasonable and possible config, or normally it would be 10K routing entries +
only several adjacency entries?
Does it make sense to create adjacency table on huge-pages?
Another problem is although above assigned heap-size is 256M, but on 1G
huge-page system, it seems to occupy a huge-page completely, other memory space
within that huge-page seems will not be used by other tables.
Same as the bihash based tables, only 2M huge-page system is supported. To
support creating bihash based tables on 1G huge-page system, each table will
occupy a 1G huge-page completely, but that will waste a lot of memories.
Is it possible just like pmalloc module, reserving a big memory space on 1G/2M
huge-pages in initialization stage, and then allocate memory pieces per
requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
could be created on huge-pages and will fully utilize the huge-pages.
I tried to create MAC table on 1G huge-page, and it does improve throughput
performance.
vpp# show bihash
Name Actual Configured
GBP Endpoints - MAC/BD 1m 1m
b4s 64m 64m
b4s 64m 64m
in2out 10.12m 10.12m
in2out 10.12m 10.12m
ip4-dr 2m 2m
ip4-dr 2m 2m
ip6 FIB fwding table32m 32m
ip6 FIB non-fwding table32m 32m
ip6 mFIB table 32m 32m
l2fib mac table512m 512m
mapping_by_as4 64m 64m
out2in 128m 128m
out2in 128m 128m
out2in 10.12m 10.12m
out2in 10.12m 10.12m
pppoe link table 8m 8m
pppoe session table 8m 8m
static_mapping_by_external 64m 64m
static_mapping_by_local 64m 64m
stn addresses1m 1m
users 648k 648k
users 648k 648k
vip_index_per_port 64m 64m
vxlan4 1m 1m
vxlan4-gbp 1m 1m
Total 1.28g 1.28g
Thanks.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17072): https://lists.fd.io/g/vpp-dev/message/17072
Mute This Topic: https://lists.fd.io/mt/75742152/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Create big tables on huge-page
Hard to say without seeing the patch. Can you summarize what the changes will
be in each particular .c file?
> On 23 Jul 2020, at 18:00, Honnappa Nagarahalli
> wrote:
>
> Hi Damjan,
> Thank you. Till your patch is ready, would you accept patches
> that would enable creating these tables in 1G huge pages as temporary
> solution?
>
> Thanks,
> Honnappa
>
> From: Damjan Marion mailto:dmar...@me.com>>
> Sent: Thursday, July 23, 2020 7:15 AM
> To: Lijian Zhang mailto:lijian.zh...@arm.com>>
> Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd
> mailto:n...@arm.com>>; Honnappa Nagarahalli
> mailto:honnappa.nagaraha...@arm.com>>;
> Govindarajan Mohandoss <mailto:govindarajan.mohand...@arm.com>>; Jieqiang Wang
> mailto:jieqiang.w...@arm.com>>
> Subject: Re: [vpp-dev] Create big tables on huge-page
>
>
> I started working on patch which addresses most of this points, few weeks
> ago, but likely I will not have it completed for 20.09.
> Even if it is completed, it is probably bad idea to merge it so late in the
> release process….
>
> —
> Damjan
>
>
>
> On 23 Jul 2020, at 10:45, Lijian Zhang <mailto:lijian.zh...@arm.com>> wrote:
>
> Hi Maintainers,
> From VPP source code, ip4-mtrie table is created on huge-page only when below
> parameters are set in configuration file.
> While adjacency table is created on normal-page always.
> 36 ip {
> 37 heap-size 256M
> 38 mtrie-hugetlb
> 39 }
> In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and
> 10K entries in adjacency table.
> By creating ip4-mtrie table on 1G huge-page with above parameters set and
> similarly create adjacency table on 1G huge-page, although I don’t observe
> obvious throughput performance improvement, but TLB misses are dramatically
> reduced.
> Do you think configuration of 10K routing entries + 10K adjacency entries is
> a reasonable and possible config, or normally it would be 10K routing entries
> + only several adjacency entries?
> Does it make sense to create adjacency table on huge-pages?
> Another problem is although above assigned heap-size is 256M, but on 1G
> huge-page system, it seems to occupy a huge-page completely, other memory
> space within that huge-page seems will not be used by other tables.
>
> Same as the bihash based tables, only 2M huge-page system is supported. To
> support creating bihash based tables on 1G huge-page system, each table will
> occupy a 1G huge-page completely, but that will waste a lot of memories.
> Is it possible just like pmalloc module, reserving a big memory space on
> 1G/2M huge-pages in initialization stage, and then allocate memory pieces per
> requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
> could be created on huge-pages and will fully utilize the huge-pages.
> I tried to create MAC table on 1G huge-page, and it does improve throughput
> performance.
> vpp# show bihash
> Name Actual Configured
> GBP Endpoints - MAC/BD 1m 1m
> b4s 64m 64m
> b4s 64m 64m
> in2out 10.12m 10.12m
> in2out 10.12m 10.12m
> ip4-dr 2m 2m
> ip4-dr 2m 2m
> ip6 FIB fwding table32m 32m
> ip6 FIB non-fwding table32m 32m
> ip6 mFIB table 32m 32m
> l2fib mac table512m 512m
> mapping_by_as4 64m 64m
> out2in 128m 128m
> out2in 128m 128m
> out2in 10.12m 10.12m
> out2in 10.12m 10.12m
> pppoe link table 8m 8m
> pppoe session table 8m 8m
> static_mapping_by_external 64m 64m
> static_mapping_by_local 64m 64m
> stn addresses1m 1m
> users 648k 648k
> users 648k 648k
> vip_index_per_port 64m 64m
> vxlan4 1m 1m
> vxlan4-gbp 1m 1m
> Total 1.28g 1.28g
>
> Thanks.
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17071): https://lists.fd.io/g/vpp-dev/message/17071
Mute This Topic: https://lists.fd.io/mt/75742152/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Create big tables on huge-page
Hi Damjan,
Thank you. Till your patch is ready, would you accept patches
that would enable creating these tables in 1G huge pages as temporary solution?
Thanks,
Honnappa
From: Damjan Marion
Sent: Thursday, July 23, 2020 7:15 AM
To: Lijian Zhang
Cc: vpp-dev ; nd ; Honnappa Nagarahalli
; Govindarajan Mohandoss
; Jieqiang Wang
Subject: Re: [vpp-dev] Create big tables on huge-page
I started working on patch which addresses most of this points, few weeks ago,
but likely I will not have it completed for 20.09.
Even if it is completed, it is probably bad idea to merge it so late in the
release process….
—
Damjan
On 23 Jul 2020, at 10:45, Lijian Zhang
mailto:lijian.zh...@arm.com>> wrote:
Hi Maintainers,
From VPP source code, ip4-mtrie table is created on huge-page only when below
parameters are set in configuration file.
While adjacency table is created on normal-page always.
36 ip {
37 heap-size 256M
38 mtrie-hugetlb
39 }
In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and 10K
entries in adjacency table.
By creating ip4-mtrie table on 1G huge-page with above parameters set and
similarly create adjacency table on 1G huge-page, although I don’t observe
obvious throughput performance improvement, but TLB misses are dramatically
reduced.
Do you think configuration of 10K routing entries + 10K adjacency entries is a
reasonable and possible config, or normally it would be 10K routing entries +
only several adjacency entries?
Does it make sense to create adjacency table on huge-pages?
Another problem is although above assigned heap-size is 256M, but on 1G
huge-page system, it seems to occupy a huge-page completely, other memory space
within that huge-page seems will not be used by other tables.
Same as the bihash based tables, only 2M huge-page system is supported. To
support creating bihash based tables on 1G huge-page system, each table will
occupy a 1G huge-page completely, but that will waste a lot of memories.
Is it possible just like pmalloc module, reserving a big memory space on 1G/2M
huge-pages in initialization stage, and then allocate memory pieces per
requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
could be created on huge-pages and will fully utilize the huge-pages.
I tried to create MAC table on 1G huge-page, and it does improve throughput
performance.
vpp# show bihash
Name Actual Configured
GBP Endpoints - MAC/BD 1m 1m
b4s 64m 64m
b4s 64m 64m
in2out 10.12m 10.12m
in2out 10.12m 10.12m
ip4-dr 2m 2m
ip4-dr 2m 2m
ip6 FIB fwding table32m 32m
ip6 FIB non-fwding table32m 32m
ip6 mFIB table 32m 32m
l2fib mac table512m 512m
mapping_by_as4 64m 64m
out2in 128m 128m
out2in 128m 128m
out2in 10.12m 10.12m
out2in 10.12m 10.12m
pppoe link table 8m 8m
pppoe session table 8m 8m
static_mapping_by_external 64m 64m
static_mapping_by_local 64m 64m
stn addresses1m 1m
users 648k 648k
users 648k 648k
vip_index_per_port 64m 64m
vxlan4 1m 1m
vxlan4-gbp 1m 1m
Total 1.28g 1.28g
Thanks.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17069): https://lists.fd.io/g/vpp-dev/message/17069
Mute This Topic: https://lists.fd.io/mt/75742152/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Create big tables on huge-page
I started working on patch which addresses most of this points, few weeks ago,
but likely I will not have it completed for 20.09.
Even if it is completed, it is probably bad idea to merge it so late in the
release process….
—
Damjan
> On 23 Jul 2020, at 10:45, Lijian Zhang wrote:
>
> Hi Maintainers,
> From VPP source code, ip4-mtrie table is created on huge-page only when below
> parameters are set in configuration file.
> While adjacency table is created on normal-page always.
> 36 ip {
> 37 heap-size 256M
> 38 mtrie-hugetlb
> 39 }
> In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and
> 10K entries in adjacency table.
> By creating ip4-mtrie table on 1G huge-page with above parameters set and
> similarly create adjacency table on 1G huge-page, although I don’t observe
> obvious throughput performance improvement, but TLB misses are dramatically
> reduced.
> Do you think configuration of 10K routing entries + 10K adjacency entries is
> a reasonable and possible config, or normally it would be 10K routing entries
> + only several adjacency entries?
> Does it make sense to create adjacency table on huge-pages?
> Another problem is although above assigned heap-size is 256M, but on 1G
> huge-page system, it seems to occupy a huge-page completely, other memory
> space within that huge-page seems will not be used by other tables.
>
> Same as the bihash based tables, only 2M huge-page system is supported. To
> support creating bihash based tables on 1G huge-page system, each table will
> occupy a 1G huge-page completely, but that will waste a lot of memories.
> Is it possible just like pmalloc module, reserving a big memory space on
> 1G/2M huge-pages in initialization stage, and then allocate memory pieces per
> requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
> could be created on huge-pages and will fully utilize the huge-pages.
> I tried to create MAC table on 1G huge-page, and it does improve throughput
> performance.
> vpp# show bihash
> Name Actual Configured
> GBP Endpoints - MAC/BD 1m 1m
> b4s 64m 64m
> b4s 64m 64m
> in2out 10.12m 10.12m
> in2out 10.12m 10.12m
> ip4-dr 2m 2m
> ip4-dr 2m 2m
> ip6 FIB fwding table32m 32m
> ip6 FIB non-fwding table32m 32m
> ip6 mFIB table 32m 32m
> l2fib mac table512m 512m
> mapping_by_as4 64m 64m
> out2in 128m 128m
> out2in 128m 128m
> out2in 10.12m 10.12m
> out2in 10.12m 10.12m
> pppoe link table 8m 8m
> pppoe session table 8m 8m
> static_mapping_by_external 64m 64m
> static_mapping_by_local 64m 64m
> stn addresses1m 1m
> users 648k 648k
> users 648k 648k
> vip_index_per_port 64m 64m
> vxlan4 1m 1m
> vxlan4-gbp 1m 1m
> Total 1.28g 1.28g
>
> Thanks.
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17058): https://lists.fd.io/g/vpp-dev/message/17058
Mute This Topic: https://lists.fd.io/mt/75742152/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Create big tables on huge-page
Hi Maintainers,
>From VPP source code, ip4-mtrie table is created on huge-page only when below
>parameters are set in configuration file.
While adjacency table is created on normal-page always.
36 ip {
37 heap-size 256M
38 mtrie-hugetlb
39 }
In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and 10K
entries in adjacency table.
By creating ip4-mtrie table on 1G huge-page with above parameters set and
similarly create adjacency table on 1G huge-page, although I don't observe
obvious throughput performance improvement, but TLB misses are dramatically
reduced.
Do you think configuration of 10K routing entries + 10K adjacency entries is a
reasonable and possible config, or normally it would be 10K routing entries +
only several adjacency entries?
Does it make sense to create adjacency table on huge-pages?
Another problem is although above assigned heap-size is 256M, but on 1G
huge-page system, it seems to occupy a huge-page completely, other memory space
within that huge-page seems will not be used by other tables.
Same as the bihash based tables, only 2M huge-page system is supported. To
support creating bihash based tables on 1G huge-page system, each table will
occupy a 1G huge-page completely, but that will waste a lot of memories.
Is it possible just like pmalloc module, reserving a big memory space on 1G/2M
huge-pages in initialization stage, and then allocate memory pieces per
requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables
could be created on huge-pages and will fully utilize the huge-pages.
I tried to create MAC table on 1G huge-page, and it does improve throughput
performance.
vpp# show bihash
Name Actual Configured
GBP Endpoints - MAC/BD 1m 1m
b4s 64m 64m
b4s 64m 64m
in2out 10.12m 10.12m
in2out 10.12m 10.12m
ip4-dr 2m 2m
ip4-dr 2m 2m
ip6 FIB fwding table32m 32m
ip6 FIB non-fwding table32m 32m
ip6 mFIB table 32m 32m
l2fib mac table512m 512m
mapping_by_as4 64m 64m
out2in 128m 128m
out2in 128m 128m
out2in 10.12m 10.12m
out2in 10.12m 10.12m
pppoe link table 8m 8m
pppoe session table 8m 8m
static_mapping_by_external 64m 64m
static_mapping_by_local 64m 64m
stn addresses1m 1m
users 648k 648k
users 648k 648k
vip_index_per_port 64m 64m
vxlan4 1m 1m
vxlan4-gbp 1m 1m
Total 1.28g 1.28g
Thanks.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17051): https://lists.fd.io/g/vpp-dev/message/17051
Mute This Topic: https://lists.fd.io/mt/75742152/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
