Re: [PATCH 05/15] RDS: increase size of hash-table to 8K

[santosh shilimkar]

Hi Dave,

On 9/21/2015 4:55 PM, santosh shilimkar wrote:

On 9/21/2015 4:05 PM, David Miller wrote:

From: Santosh Shilimkar 
Date: Sat, 19 Sep 2015 19:04:42 -0400


Even with per bucket locking scheme, in a massive parallel
system with active rds sockets which could be in excess of multiple
of 10K, rds_bin_lookup() workload is siginificant because of smaller
hashtable size.

With some tests, it was found that we get modest but still nice
reduction in rds_bind_lookup with bigger bucket.

HashtableBaseline(1k)Delta
2048:8.28% -2.45%
4096:8.28%-4.60%
8192:8.28%-6.46%
16384:8.28%-6.75%

Based on the data, we set 8K as the bind hash-table size.

Signed-off-by: Santosh Shilimkar 
Signed-off-by: Santosh Shilimkar 


Like others I would strongly prefer that you use a dynamically sized
hash table.

Eating 8k just because a module just happened to get loaded is really
not appropriate.

And there are many other places that use such a scheme, one example is
the AF_NETLINK socket hash table.


OK. Thanks for AF_NETLINK pointer. I will look it up.


I will follow your advice on resizable hash table usage. It seems
to be neat and fits well. But I want to make sure that the
implementation works for all the workloads so it will take
some time. Hopefully I can get that ready with testing for 4.5.

So for now,lets just drop the $subject patch from this
series. Do you want me to resend the series with the $subject
patch dropped, or you can apply rest of the series except
this one.

Let me know. Thanks !!

Regards,
Santosh



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 05/15] RDS: increase size of hash-table to 8K

[David Miller]

From: Santosh Shilimkar 
Date: Sat, 19 Sep 2015 19:04:42 -0400

> Even with per bucket locking scheme, in a massive parallel
> system with active rds sockets which could be in excess of multiple
> of 10K, rds_bin_lookup() workload is siginificant because of smaller
> hashtable size.
> 
> With some tests, it was found that we get modest but still nice
> reduction in rds_bind_lookup with bigger bucket.
> 
>   Hashtable   Baseline(1k)Delta
>   2048:   8.28%   -2.45%
>   4096:   8.28%   -4.60%
>   8192:   8.28%   -6.46%
>   16384:  8.28%   -6.75%
> 
> Based on the data, we set 8K as the bind hash-table size.
> 
> Signed-off-by: Santosh Shilimkar 
> Signed-off-by: Santosh Shilimkar 

Like others I would strongly prefer that you use a dynamically sized
hash table.

Eating 8k just because a module just happened to get loaded is really
not appropriate.

And there are many other places that use such a scheme, one example is
the AF_NETLINK socket hash table.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 05/15] RDS: increase size of hash-table to 8K

[santosh shilimkar]

On 9/21/2015 4:05 PM, David Miller wrote:

From: Santosh Shilimkar 
Date: Sat, 19 Sep 2015 19:04:42 -0400


Even with per bucket locking scheme, in a massive parallel
system with active rds sockets which could be in excess of multiple
of 10K, rds_bin_lookup() workload is siginificant because of smaller
hashtable size.

With some tests, it was found that we get modest but still nice
reduction in rds_bind_lookup with bigger bucket.

Hashtable   Baseline(1k)Delta
2048:   8.28%   -2.45%
4096:   8.28%   -4.60%
8192:   8.28%   -6.46%
16384:  8.28%   -6.75%

Based on the data, we set 8K as the bind hash-table size.

Signed-off-by: Santosh Shilimkar 
Signed-off-by: Santosh Shilimkar 


Like others I would strongly prefer that you use a dynamically sized
hash table.

Eating 8k just because a module just happened to get loaded is really
not appropriate.

And there are many other places that use such a scheme, one example is
the AF_NETLINK socket hash table.


OK. Thanks for AF_NETLINK pointer. I will look it up.

Regards,
Santosh
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 05/15] RDS: increase size of hash-table to 8K

[santosh shilimkar]

On 9/21/2015 1:31 AM, David Laight wrote:

From: Santosh Shilimkar

Sent: 20 September 2015 00:05
Even with per bucket locking scheme, in a massive parallel
system with active rds sockets which could be in excess of multiple
of 10K, rds_bin_lookup() workload is significant because of smaller
hashtable size.

With some tests, it was found that we get modest but still nice
reduction in rds_bind_lookup with bigger bucket.

Hashtable   Baseline(1k)Delta
2048:   8.28%   -2.45%
4096:   8.28%   -4.60%
8192:   8.28%   -6.46%
16384:  8.28%   -6.75%

Based on the data, we set 8K as the bind hash-table size.


Can't you use of on the dynamically sizing hash tables?
8k hash table entries is OTT for a lot of systems.


Do you know an example in Linux kernel uses that ? What I
certainly don't want is over-head of re-sizing whenever
that happens in running systems running multiple databases.

Memory is certainly not an issue on the systems where RDS
has been deployed. I certainly don't want to over-use the
memory but in the system where RDS being used and also
amount of connection it needs to handle, it needs
bigger bucket.

Regards,
Santosh
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 05/15] RDS: increase size of hash-table to 8K

[David Laight]

From: Santosh Shilimkar
> Sent: 20 September 2015 00:05
> Even with per bucket locking scheme, in a massive parallel
> system with active rds sockets which could be in excess of multiple
> of 10K, rds_bin_lookup() workload is siginificant because of smaller
> hashtable size.
> 
> With some tests, it was found that we get modest but still nice
> reduction in rds_bind_lookup with bigger bucket.
> 
>   Hashtable   Baseline(1k)Delta
>   2048:   8.28%   -2.45%
>   4096:   8.28%   -4.60%
>   8192:   8.28%   -6.46%
>   16384:  8.28%   -6.75%
> 
> Based on the data, we set 8K as the bind hash-table size.

Can't you use of on the dynamically sizing hash tables?
8k hash table entries is OTT for a lot of systems.

David

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 05/15] RDS: increase size of hash-table to 8K

[Santosh Shilimkar]

Even with per bucket locking scheme, in a massive parallel
system with active rds sockets which could be in excess of multiple
of 10K, rds_bin_lookup() workload is siginificant because of smaller
hashtable size.

With some tests, it was found that we get modest but still nice
reduction in rds_bind_lookup with bigger bucket.

Hashtable   Baseline(1k)Delta
2048:   8.28%   -2.45%
4096:   8.28%   -4.60%
8192:   8.28%   -6.46%
16384:  8.28%   -6.75%

Based on the data, we set 8K as the bind hash-table size.

Signed-off-by: Santosh Shilimkar 
Signed-off-by: Santosh Shilimkar 
---
 net/rds/bind.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/rds/bind.c b/net/rds/bind.c
index bc6b93e..fb2d545 100644
--- a/net/rds/bind.c
+++ b/net/rds/bind.c
@@ -43,7 +43,7 @@ struct bind_bucket {
struct hlist_head   head;
 };
 
-#define BIND_HASH_SIZE 1024
+#define BIND_HASH_SIZE 8192
 static struct bind_bucket bind_hash_table[BIND_HASH_SIZE];
 
 static struct bind_bucket *hash_to_bucket(__be32 addr, __be16 port)
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html