Re: [PATCH v5 0/9] "Task_isolation" mode

2020-12-11 Thread Yury Norov
On Sun, Dec 06, 2020 at 12:25:45AM +0100, Thomas Gleixner wrote:
> Pavel,
> 
> On Sat, Dec 05 2020 at 21:40, Pavel Machek wrote:
> > So... what kind of guarantees does this aim to provide / what tasks it
> > is useful for?
> >
> > For real time response, we have other approaches.
> 
> Depends on your requirements. Some problems are actually better solved
> with busy polling. See below.
> 
> > If you want to guarantee performnace of the "isolated" task... I don't
> > see how that works. Other tasks on the system still compete for DRAM
> > bandwidth, caches, etc...
> 
> Applications which want to run as undisturbed as possible. There is
> quite a range of those:
> 
>   - Hardware in the loop simulation is today often done with that crude
> approach of "offlining" a CPU and then instead of playing dead
> jumping to a preloaded bare metal executable. That's a horrible hack
> and impossible to debug, but gives them the results they need to
> achieve. These applications are well optimized vs. cache and memory
> foot print, so they don't worry about these things too much and they
> surely don't run on SMI and BIOS value add inflicted machines.
> 
> Don't even think about waiting for an interrupt to achieve what
> these folks are doing. So no, there are problems which a general
> purpose realtime OS cannot solve ever.
> 
>   - HPC computations on large data sets. While the memory foot print is
> large the access patterns are cache optimized. 
> 
> The problem there is that any unnecessary IPI, tick interrupt or
> whatever nuisance is disturbing the carefully optimized cache usage
> and alone getting rid of the timer interrupt gained them measurable
> performance. Even very low single digit percentage of runtime saving
> is valuable for these folks because the compute time on such beasts
> is expensive.
> 
>   - Realtime guests in KVM. With posted interrupts and a fully populated
> host side page table there is no point in running host side
> interrupts or IPIs for random accounting or whatever purposes as
> they affect the latency in the guest. With all the side effects
> mitigated and a properly set up guest and host it is possible to get
> to a zero exit situation after the bootup phase which means pretty
> much matching bare metal behaviour.
> 
> Yes, you can do that with e.g. Jailhouse as well, but you lose lots
> of the fancy things KVM provides. And people care about these not
> just because they are fancy. They care because their application
> scenario needs them.
> 
> There are more reasons why people want to be able to get as much
> isolation from the OS as possible but at the same time have a sane
> execution environment, debugging, performance monitoring and the OS
> provided protection mechanisms instead of horrible hacks.
> 
> Isolation makes sense for a range of applications and there is no reason
> why Linux should not support them. 

One good client for the task isolation is Open Data Plane. There are
even some code stubs supposed to enable isolation where needed.

> > If you want to guarantee performnace of the "isolated" task... I don't
> > see how that works. Other tasks on the system still compete for DRAM
> > bandwidth, caches, etc...

My experiments say that typical delay caused by dry IPI or syscall is
2000-2 'ticks'. Typical delay caused by cache miss is 3-30 ticks.

To guarantee cache / memory bandwidth, one can use resctrl. Linux has
implementation of it for x86 only, but arm64 has support for for
resctrl on CPU side.

Thanks,
Yury

> Thanks,
> 
> tglx


Re: Regression in 39461af74125 "bitmap: replace bitmap_{from,to}_u32array"

2018-02-06 Thread Yury Norov
On Tue, Feb 06, 2018 at 11:42:18AM -0500, David Miller wrote:
> From: Yury Norov 
> Date: Tue, 6 Feb 2018 19:26:23 +0300
> 
> > On Tue, Feb 06, 2018 at 11:17:36AM -0500, David Miller wrote:
> >> From: Heiner Kallweit 
> >> Date: Mon, 5 Feb 2018 07:21:32 +0100
> >> 
> >> > Recently ethtool started to give incomplete values for supported and
> >> > advertised modes. There seems to be a regression in this commit:
> >> > The bit number parameter in the calls to bitmap_to_arr32() in
> >> > store_link_ksettings_for_user() should be __ETHTOOL_LINK_MODE_MASK_NBITS
> >> > instead of __ETHTOOL_LINK_MODE_MASK_NU32.
> >> > 
> >> > After having changed this ethtool behaves normally again.
> >> 
> >> I do not see the commit with the SHA1 ID from the Subject line in
> >> either Linus's nor my networking tree.
> >> 
> >> Where is that change?
> > 
> > It's in next-20180202.
> 
> Hmmm, then the fix should probably go into whatever tree that commit
> ultimately came from.

It's already in Andrew's tree. The original patch is not about networking,
and Andrew's tree is probably better place for fix. My apologies for noise
in your tree.

http://ozlabs.org/~akpm/mmots/broken-out/bitmap-replace-bitmap_fromto_u32array-fix-2.patch


Re: Regression in 39461af74125 "bitmap: replace bitmap_{from,to}_u32array"

2018-02-06 Thread Yury Norov
On Tue, Feb 06, 2018 at 11:17:36AM -0500, David Miller wrote:
> From: Heiner Kallweit 
> Date: Mon, 5 Feb 2018 07:21:32 +0100
> 
> > Recently ethtool started to give incomplete values for supported and
> > advertised modes. There seems to be a regression in this commit:
> > The bit number parameter in the calls to bitmap_to_arr32() in
> > store_link_ksettings_for_user() should be __ETHTOOL_LINK_MODE_MASK_NBITS
> > instead of __ETHTOOL_LINK_MODE_MASK_NU32.
> > 
> > After having changed this ethtool behaves normally again.
> 
> I do not see the commit with the SHA1 ID from the Subject line in
> either Linus's nor my networking tree.
> 
> Where is that change?

It's in next-20180202.

Yury


Re: Regression in 39461af74125 "bitmap: replace bitmap_{from,to}_u32array"

2018-02-04 Thread Yury Norov
On Mon, Feb 05, 2018 at 07:21:32AM +0100, Heiner Kallweit wrote:
> Recently ethtool started to give incomplete values for supported and
> advertised modes. There seems to be a regression in this commit:
> The bit number parameter in the calls to bitmap_to_arr32() in
> store_link_ksettings_for_user() should be __ETHTOOL_LINK_MODE_MASK_NBITS
> instead of __ETHTOOL_LINK_MODE_MASK_NU32.
> 
> After having changed this ethtool behaves normally again.
> 
> Heiner

Hi Heiner,

Thank you for discovering this, and sorry for my error. 
The patch is below.

Yury

>From af960f7ae1be8e86abfaa39300ba690c24a80e5b Mon Sep 17 00:00:00 2001
From: Yury Norov 
Date: Mon, 5 Feb 2018 10:01:59 +0300
Subject: [PATCH] Fix 39461af74125 "bitmap: replace bitmap_{from,to}_u32array"

During rework, bitmap_to_arr32() was erroneously passed with
__ETHTOOL_LINK_MODE_MASK_NU32 instead of __ETHTOOL_LINK_MODE_MASK_NBITS.

Fixes: 39461af74125fd268 (bitmap: replace bitmap_{from,to}_u32array)
Reported-by: Heiner Kallweit 
Signed-off-by: Yury Norov 
---
 net/core/ethtool.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/core/ethtool.c b/net/core/ethtool.c
index 1822335518ea..494e6a5d7306 100644
--- a/net/core/ethtool.c
+++ b/net/core/ethtool.c
@@ -642,13 +642,13 @@ store_link_ksettings_for_user(void __user *to,
memcpy(&link_usettings.base, &from->base, sizeof(link_usettings));
bitmap_to_arr32(link_usettings.link_modes.supported,
from->link_modes.supported,
-   __ETHTOOL_LINK_MODE_MASK_NU32);
+   __ETHTOOL_LINK_MODE_MASK_NBITS);
bitmap_to_arr32(link_usettings.link_modes.advertising,
from->link_modes.advertising,
-   __ETHTOOL_LINK_MODE_MASK_NU32);
+   __ETHTOOL_LINK_MODE_MASK_NBITS);
bitmap_to_arr32(link_usettings.link_modes.lp_advertising,
from->link_modes.lp_advertising,
-   __ETHTOOL_LINK_MODE_MASK_NU32);
+   __ETHTOOL_LINK_MODE_MASK_NBITS);
 
if (copy_to_user(to, &link_usettings, sizeof(link_usettings)))
return -EFAULT;
-- 
2.14.1



Re: next: arm64: LTP sendto01 test causes system crash in ilp32 mode

2017-10-11 Thread Yury Norov
Hi Eric,

On Wed, Oct 11, 2017 at 11:48:37AM -0700, Eric Dumazet wrote:
> On Wed, 2017-10-11 at 11:41 -0700, Eric Dumazet wrote:
> > On Wed, 2017-10-11 at 21:35 +0300, Yury Norov wrote:
> > > Hi all, 
> > > 
> > > It seems like next-20171009 with ilp32 patches crashes on LTP sendto01 
> > > test
> > > in sys_sendto() path, like this:
> > 
> > Thanks for the report.
> > Probably caused by one of my recent patches, so I am taking a look.
> 
> Yes, this was silly.
> 
> Please test this fix :
> 
> diff --git a/include/net/tcp.h b/include/net/tcp.h
> index 
> 5a95e5886b55e03e4a8bfeac3506c657a4f97dde..15163454174babdcb465904f725b919268dd1bc7
>  100644
> --- a/include/net/tcp.h
> +++ b/include/net/tcp.h
> @@ -1712,6 +1712,7 @@ static inline void tcp_insert_write_queue_before(struct 
> sk_buff *new,
>  
>  static inline void tcp_unlink_write_queue(struct sk_buff *skb, struct sock 
> *sk)
>  {
> + tcp_skb_tsorted_anchor_cleanup(skb);
>   __skb_unlink(skb, &sk->sk_write_queue);
>  }

The fix works for me, thanks.

Tested-by: Yury Norov 


next: arm64: LTP sendto01 test causes system crash in ilp32 mode

2017-10-11 Thread Yury Norov
Hi all, 

It seems like next-20171009 with ilp32 patches crashes on LTP sendto01 test
in sys_sendto() path, like this:

[  554.034021] [] 0x80003ccd5a58
[  554.034156] [] skb_release_all+0x14/0x30
[  554.034288] [] __kfree_skb+0x14/0x28
[  554.034409] [] tcp_sendmsg_locked+0x4dc/0xcc8
[  554.034541] [] tcp_sendmsg+0x34/0x58
[  554.034659] [] inet_sendmsg+0x2c/0xf8
[  554.034783] [] sock_sendmsg+0x18/0x30
[  554.034928] [] SyS_sendto+0x84/0xf8

I cannot reproduce it in lp64 mode, and test is passed in ilp32 mode
if I run it alone, even in infinite loop. But in ltplite scenario the
fail is always reproducible.

The brief analisys of dump shows that kernel crashes due to bad value
in ->destructor field of struct sk_buff, when tries to call
skb->destructor() in skb_release_all(). It looks very unusual,
comparing to typical ilp32 ABI bugs, and I suspect that here is generic
issue - maybe some race condition?

Kernel v4.14-rc4 works well. If no ideas, I'll bisect it a bit later.
Ooops log is below. Config is attached, and kernel sources are:
https://github.com/norov/linux/tree/ilp32-20171009

Yury

[  554.026522] Unable to handle kernel read from unreadable memory at virtual 
address 80003ccd5a58
[  554.027005] Mem abort info:
[  554.027124]   Exception class = IABT (current EL), IL = 32 bits
[  554.027292]   SET = 0, FnV = 0
[  554.027378]   EA = 0, S1PTW = 0
[  554.027537] swapper pgtable: 4k pages, 48-bit VAs, pgd = 09069000
[  554.027732] [80003ccd5a58] *pgd=7eff7003, *pud=7eff6003, 
*pmd=00f87cc00711
[  554.028128] Internal error: Oops: 860e [#1] PREEMPT SMP
[  554.028308] Modules linked in:
[  554.028480] CPU: 1 PID: 6388 Comm: send01 Not tainted 
4.14.0-rc4-next-20171009-00025-g6229c950955a #256
[  554.028684] Hardware name: linux,dummy-virt (DT)
[  554.028797] task: 80003b6d0e80 task.stack: 09d7
[  554.028959] PC is at 0x80003ccd5a58
[  554.029272] LR is at skb_release_head_state+0x5c/0xf8
[  554.029406] pc : [] lr : [] pstate: 
4145
[  554.029676] sp : 09d73c00
[  554.029806] x29: 09d73c00 x28: 800039a86c80 
[  554.030021] x27: 800039a86dd8 x26: fff2 
[  554.030139] x25: 80003ccd5a00 x24:  
[  554.030258] x23: 09d73de8 x22:  
[  554.030375] x21: 09d73df8 x20:  
[  554.030490] x19: 80003ccd5a00 x18: f7e73df8 
[  554.030606] x17: f7f40320 x16: 08886178 
[  554.030721] x15: 0126 x14: f7fea700 
[  554.030840] x13: f7e75b8c x12: f7e7e43c 
[  554.030959] x11: 6f732064696c6176 x10: 0101010101010101 
[  554.031060] x9 : 206d305b1b535341 x8 :  
[  554.031159] x7 : 80003b6d0e80 x6 : 80003c0aa910 
[  554.031256] x5 : 80003c0aad10 x4 :  
[  554.031354] x3 : 0001f809 x2 : 0700 
[  554.031452] x1 : 80003ccd5a58 x0 : 80003ccd5a00 
[  554.031566] Process send01 (pid: 6388, stack limit = 0x09d7)
[  554.031753] Call trace:
[  554.031870] Exception stack(0x09d73ac0 to 0x09d73c00)
[  554.032064] 3ac0: 80003ccd5a00 80003ccd5a58 0700 
0001f809
[  554.032224] 3ae0:  80003c0aad10 80003c0aa910 
80003b6d0e80
[  554.032380] 3b00:  206d305b1b535341 0101010101010101 
6f732064696c6176
[  554.032584] 3b20: f7e7e43c f7e75b8c f7fea700 
0126
[  554.032732] 3b40: 08886178 f7f40320 f7e73df8 
80003ccd5a00
[  554.032883] 3b60:  09d73df8  
09d73de8
[  554.033066] 3b80:  80003ccd5a00 fff2 
800039a86dd8
[  554.033233] 3ba0: 800039a86c80 09d73c00 0888fc84 
09d73c00
[  554.033386] 3bc0: 80003ccd5a58 4145 089a2a64 
0145
[  554.033656] 3be0: 0001 0888fd08 09d73c00 
80003ccd5a58
[  554.034021] [] 0x80003ccd5a58
[  554.034156] [] skb_release_all+0x14/0x30
[  554.034288] [] __kfree_skb+0x14/0x28
[  554.034409] [] tcp_sendmsg_locked+0x4dc/0xcc8
[  554.034541] [] tcp_sendmsg+0x34/0x58
[  554.034659] [] inet_sendmsg+0x2c/0xf8
[  554.034783] [] sock_sendmsg+0x18/0x30
[  554.034928] [] SyS_sendto+0x84/0xf8
[  554.035046] Exception stack(0x09d73ec0 to 0x09d74000)
[  554.035186] 3ec0: 0004  0400 

[  554.035334] 3ee0:   20203130646e6573 
1b20203220202020
[  554.035503] 3f00: 00ce 206d305b1b535341 0101010101010101 
6f732064696c6176
[  554.035657] 3f20: f7e7e43c f7e75b8c f7fea700 
0126
[  554.035825] 3f40: 004240e0 f7f40320 f7e73df8 
0040e000
[  554.035981] 3f60: f7feaea0 00424000 00424000 
00447000
[  5

Re: [PATCH v3] net: don't call strlen on non-terminated string in dev_set_alias()

2017-06-01 Thread Yury Norov
On Thu, Jun 01, 2017 at 03:50:33PM +0200, Alexander Potapenko wrote:
> On Thu, Jun 1, 2017 at 3:47 PM, Yury Norov  wrote:
> > On Thu, Jun 01, 2017 at 02:38:29PM +0200, Alexander Potapenko wrote:
> >> KMSAN reported a use of uninitialized memory in dev_set_alias(),
> >> which was caused by calling strlcpy() (which in turn called strlen())
> >> on the user-supplied non-terminated string.
> >>
> >> Signed-off-by: Alexander Potapenko 
> >> ---
> >> v3: removed the multi-line comment
> >> v2: fixed an off-by-one error spotted by Dmitry Vyukov
> >
> > [...]
> >
> >> ---
> >>  net/core/dev.c | 4 +++-
> >>  1 file changed, 3 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/net/core/dev.c b/net/core/dev.c
> >> index fca407b4a6ea..3e3b29133cc9 100644
> >> --- a/net/core/dev.c
> >> +++ b/net/core/dev.c
> >> @@ -1254,7 +1254,9 @@ int dev_set_alias(struct net_device *dev, const char 
> >> *alias, size_t len)
> >>   return -ENOMEM;
> >>   dev->ifalias = new_ifalias;
> >>
> >> - strlcpy(dev->ifalias, alias, len+1);
> >> + /* alias comes from the userspace and may not be zero-terminated. */
> >
> > So if the comment is correct, you'd use copy_from_user() instead.
> Well, the contents of |alias| have been previously copied from the
> userspace, but this is a pointer to a kernel buffer, as the function
> prototype tells.
> Do you think a confusion is possible here?

Yes, I think so... If pointer comes from userspace, it normally points to
userspace data. If you have the data already copied, just say:
+ /* alias may not be zero-terminated. */

Or say nothing, because at the first glance, using the strlcpy()
instead of simple memcpy() looks weird in this case - you know the
length of the string from the beginning, and should not recalculate it
again.

Yury

> >> + memcpy(dev->ifalias, alias, len);
> >> + dev->ifalias[len] = 0;
> >>   return len;
> >>  }
> >>
> >> --
> >> 2.13.0.219.gdb65acc882-goog
> 
> 
> 
> -- 
> Alexander Potapenko
> Software Engineer
> 
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
> 
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg


Re: [PATCH v3] net: don't call strlen on non-terminated string in dev_set_alias()

2017-06-01 Thread Yury Norov
On Thu, Jun 01, 2017 at 02:38:29PM +0200, Alexander Potapenko wrote:
> KMSAN reported a use of uninitialized memory in dev_set_alias(),
> which was caused by calling strlcpy() (which in turn called strlen())
> on the user-supplied non-terminated string.
> 
> Signed-off-by: Alexander Potapenko 
> ---
> v3: removed the multi-line comment
> v2: fixed an off-by-one error spotted by Dmitry Vyukov
 
[...]

> ---
>  net/core/dev.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/net/core/dev.c b/net/core/dev.c
> index fca407b4a6ea..3e3b29133cc9 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -1254,7 +1254,9 @@ int dev_set_alias(struct net_device *dev, const char 
> *alias, size_t len)
>   return -ENOMEM;
>   dev->ifalias = new_ifalias;
>  
> - strlcpy(dev->ifalias, alias, len+1);
> + /* alias comes from the userspace and may not be zero-terminated. */

So if the comment is correct, you'd use copy_from_user() instead.

> + memcpy(dev->ifalias, alias, len);
> + dev->ifalias[len] = 0;
>   return len;
>  }
>  
> -- 
> 2.13.0.219.gdb65acc882-goog


Re: arm64: next-20170428 hangs on boot

2017-04-28 Thread Yury Norov
On Fri, Apr 28, 2017 at 08:40:54AM -0700, Florian Fainelli wrote:
> On 04/28/2017 08:09 AM, Yury Norov wrote:
> > On Fri, Apr 28, 2017 at 03:52:34PM +0100, Mark Rutland wrote:
> >> On Fri, Apr 28, 2017 at 04:24:29PM +0300, Yury Norov wrote:
> >>> Hi all,
> >>
> >> Hi,
> >>
> >> [adding Dave Miller, netdev, lkml]
> > 
> > thanks
> > 
> >>> On QEMU the next-20170428 hangs on boot for me due to kernel panic in 
> >>> rtnetlink_init():
> >>>
> >>> void __init rtnetlink_init(void)
> >>> {
> >>> if (register_pernet_subsys(&rtnetlink_net_ops))
> >>> panic("rtnetlink_init: cannot initialize rtnetlink\n");
> >>>
> >>> ...
> >>> }
> >>
> >> I see the same thing with a next-20170428 arm64 defconfig, on a Juno R1
> >> system:
> >>
> >> [0.531949] Kernel panic - not syncing: rtnetlink_init: cannot 
> >> initialize rtnetlink
> >> [0.531949] 
> >> [0.541271] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 
> >> 4.11.0-rc8-next-20170428-2-g6ee3799 #10
> >> [0.550307] Hardware name: ARM Juno development board (r1) (DT)
> >> [0.556332] Call trace:
> >> [0.558833] [] dump_backtrace+0x0/0x238
> >> [0.564332] [] show_stack+0x14/0x20
> >> [0.569477] [] dump_stack+0x9c/0xc0
> >> [0.574622] [] panic+0x11c/0x28c
> >> [0.579505] [] rtnetlink_init+0x2c/0x1d0
> >> [0.585092] [] netlink_proto_init+0x14c/0x17c
> >> [0.591119] [] do_one_initcall+0x38/0x120
> >> [0.596796] [] kernel_init_freeable+0x1a0/0x240
> >> [0.603003] [] kernel_init+0x10/0x100
> >> [0.608324] [] ret_from_fork+0x10/0x50
> >> [0.613736] SMP: stopping secondary CPUs
> >> [0.617738] ---[ end Kernel panic - not syncing: rtnetlink_init: cannot 
> >> initialize rtnetlink
> >>
> >> If this isn't a known issue, it would be worth trying to bisect this.
> 
> It's fixed already by this commit in net-next:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/commit/?id=2d2ab658d2debcb4c0e29c9e6f18e5683f3077bf

Works for me, thank you.


Re: arm64: next-20170428 hangs on boot

2017-04-28 Thread Yury Norov
On Fri, Apr 28, 2017 at 03:52:34PM +0100, Mark Rutland wrote:
> On Fri, Apr 28, 2017 at 04:24:29PM +0300, Yury Norov wrote:
> > Hi all,
> 
> Hi,
> 
> [adding Dave Miller, netdev, lkml]

thanks

> > On QEMU the next-20170428 hangs on boot for me due to kernel panic in 
> > rtnetlink_init():
> > 
> > void __init rtnetlink_init(void)
> > {
> > if (register_pernet_subsys(&rtnetlink_net_ops))
> > panic("rtnetlink_init: cannot initialize rtnetlink\n");
> > 
> > ...
> > }
> 
> I see the same thing with a next-20170428 arm64 defconfig, on a Juno R1
> system:
> 
> [0.531949] Kernel panic - not syncing: rtnetlink_init: cannot initialize 
> rtnetlink
> [0.531949] 
> [0.541271] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 
> 4.11.0-rc8-next-20170428-2-g6ee3799 #10
> [0.550307] Hardware name: ARM Juno development board (r1) (DT)
> [0.556332] Call trace:
> [0.558833] [] dump_backtrace+0x0/0x238
> [0.564332] [] show_stack+0x14/0x20
> [0.569477] [] dump_stack+0x9c/0xc0
> [0.574622] [] panic+0x11c/0x28c
> [0.579505] [] rtnetlink_init+0x2c/0x1d0
> [0.585092] [] netlink_proto_init+0x14c/0x17c
> [0.591119] [] do_one_initcall+0x38/0x120
> [0.596796] [] kernel_init_freeable+0x1a0/0x240
> [0.603003] [] kernel_init+0x10/0x100
> [0.608324] [] ret_from_fork+0x10/0x50
> [0.613736] SMP: stopping secondary CPUs
> [0.617738] ---[ end Kernel panic - not syncing: rtnetlink_init: cannot 
> initialize rtnetlink
> 
> If this isn't a known issue, it would be worth trying to bisect this.

The exact function that fails is: 
include/linux/rhashtable.h
static inline void *__rhashtable_insert_fast(
struct rhashtable *ht, const void *key, struct rhash_head *obj,
const struct rhashtable_params params, bool rhlist)
{
...

data = ERR_PTR(-E2BIG);
if (unlikely(rht_grow_above_max(ht, tbl)))
   goto out;
...

out:
spin_unlock_bh(lock);
rcu_read_unlock();

return data;
}

And the backtrace:
#0  __rhashtable_insert_fast (rhlist=, params=..., 
obj=, 
key=, ht=) at ./include/linux/rhashtable.h:803
#1  rhashtable_lookup_insert_key (params=..., obj=, 
key=, 
ht=) at ./include/linux/rhashtable.h:980
#2  __netlink_insert (sk=, table=) at 
net/netlink/af_netlink.c:484
#3  netlink_insert (sk=0x80003da85000, portid=0) at 
net/netlink/af_netlink.c:548
#4  0x0876c5a0 in __netlink_kernel_create (net=, unit=0, 
module=0x0, 
cfg=0x80003d84fc60) at net/netlink/af_netlink.c:1996
#5  0x08756704 in netlink_kernel_create (cfg=, 
unit=, 
net=) at ./include/linux/netlink.h:62
#6  rtnetlink_net_init (net=0x08c7c100 ) at 
net/core/rtnetlink.c:4175
#7  0x08737a2c in ops_init (ops=0x08c7e268 , 
net=0x08c7c100 ) at net/core/net_namespace.c:117
#8  0x08738704 in __register_pernet_operations (ops=,
list=) at net/core/net_namespace.c:818
#9  register_pernet_operations (list=, ops=0x08c7e268
) at net/core/net_namespace.c:892
#10 0x087387fc in register_pernet_subsys (ops=0x08c7e268
) at net/core/net_namespace.c:934
#11 0x08b5b9b8 in rtnetlink_init () at net/core/rtnetlink.c:4195
#12 0x08b5be08 in netlink_proto_init () at net/netlink/af_netlink.c:2730
#13 0x08083158 in do_one_initcall (fn=0x08b5bcc4 
) at init/main.c:795
#14 0x08b20d04 in do_initcall_level (level=) at 
init/main.c:861
#15 do_initcalls () at init/main.c:869
#16 do_basic_setup () at init/main.c:887

Yury