Public bug reported:
The kernel seems to lock up when accessing a kerberos mounted nfs share
with a user holding a kerberos ticket. This is in a virtualbox vm(run by
vagrant), but it also occurs in a VMware vm from a standard ubuntu 14.04
server install disk.
To reproduce:
Join machine to windows active directory domain using sssd
Install nfs-kernel server, enable NEED_SVCGSSD
Enable NEED_GSSD for the client(can be same host as the server)
$ sudo mount -t nfs4 <fqdn>:/ /mnt/nfs -o sec=krb5
$ sudo ls -l /mnt/nfs # this works ok
$ kinit <a_domain_user>
$ ls /mnt/nfs
Permission denied error # I don't recall the exact wording
# wait a few moments, and the kernel starts reporting a soft lockup.
I think there are a few other things that trigger this error as well. It
basically makes nfs+kerberos unusable.
I have the kernel crash dump, and the debugsyms installed so if there is
any other information I can provide please let me know.
lsb_release -rd:
Description: Ubuntu 14.04.2 LTS
Release: 14.04
$ apt-cache policy nfs-common
nfs-common:
Installed: 1:1.2.8-6ubuntu1.1
Info from crash:
KERNEL: /usr/lib/debug/boot/vmlinux-3.13.0-53-generic
DUMPFILE: dump.201506181954 [PARTIAL DUMP]
CPUS: 2
DATE: Thu Jun 18 19:54:08 2015
UPTIME: 00:05:48
LOAD AVERAGE: 1.50, 0.45, 0.18
TASKS: 120
NODENAME: t-fileserver
RELEASE: 3.13.0-53-generic
VERSION: #89-Ubuntu SMP Wed May 20 10:34:39 UTC 2015
MACHINE: x86_64 (3581 Mhz)
MEMORY: 511.6 MB
PANIC: "Kernel panic - not syncing: softlockup: hung tasks"
PID: 1353
COMMAND: "rpc.svcgssd"
TASK: ffff880014dce000 [THREAD_INFO: ffff88001514e000]
CPU: 0
STATE: TASK_RUNNING (PANIC)
Backtrace:
PID: 1353 TASK: ffff880014dce000 CPU: 0 COMMAND: "rpc.svcgssd"
#0 [ffff88001fc03d18] machine_kexec at ffffffff8104ace2
#1 [ffff88001fc03d68] crash_kexec at ffffffff810e7423
#2 [ffff88001fc03e30] panic at ffffffff8171bcc4
#3 [ffff88001fc03ea8] watchdog_timer_fn at ffffffff8110dc85
#4 [ffff88001fc03ed8] __run_hrtimer at ffffffff8108e8c7
#5 [ffff88001fc03f18] hrtimer_interrupt at ffffffff8108f08f
#6 [ffff88001fc03f80] local_apic_timer_interrupt at ffffffff81043617
#7 [ffff88001fc03f98] smp_apic_timer_interrupt at ffffffff81735ccf
#8 [ffff88001fc03fb0] apic_timer_interrupt at ffffffff8173465d
--- <IRQ stack> ---
#9 [ffff88001514fd58] apic_timer_interrupt at ffffffff8173465d
[exception RIP: qword_addhex+176]
RIP: ffffffffa01c2df0 RSP: ffff88001514fe08 RFLAGS: 00000206
RAX: 0000000000000001 RBX: 0000000000000006 RCX: 00000000000001f6
RDX: ffff880015e54678 RSI: ffff88001514fe84 RDI: ffff88001514fe88
RBP: ffff88001514fe18 R8: ffff880015e57cf5 R9: 000000000000030b
R10: 0000000000000039 R11: 0000000000000027 R12: 0000000000000006
R13: ffffea0000422420 R14: ffffea00003e46e0 R15: ffff88001514fe98
ORIG_RAX: ffffffffffffff10 CS: 0010 SS: 0018
#10 [ffff88001514fe20] rsi_request at ffffffffa01f61bb [auth_rpcgss]
#11 [ffff88001514fe48] cache_read at ffffffffa01c5045 [sunrpc]
#12 [ffff88001514fec0] cache_read_procfs at ffffffffa01c51a1 [sunrpc]
#13 [ffff88001514fee8] proc_reg_read at ffffffff81224a6d
#14 [ffff88001514ff08] vfs_read at ffffffff811bdf55
#15 [ffff88001514ff40] sys_read at ffffffff811bea69
#16 [ffff88001514ff80] system_call_fastpath at ffffffff8173391d
RIP: 00007f85cf9f3810 RSP: 00007ffdba7dfe78 RFLAGS: 00000206
RAX: 0000000000000000 RBX: ffffffff8173391d RCX: ffffffffffffffff
RDX: 0000000000001000 RSI: 00000000008746f0 RDI: 0000000000000004
RBP: 00000000006083f8 R8: 0000000000000000 R9: 0000000000878820
R10: 00007f85cfcc67b8 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000004 R14: 0000000000608400 R15: 00000000008744b0
ORIG_RAX: 0000000000000000 CS: 0033 SS: 002b
Log:
[ 348.084011] BUG: soft lockup - CPU#0 stuck for 23s! [rpc.svcgssd:1353]
[ 348.084011] Modules linked in: cts vboxsf(OX) nfsv4 rpcsec_gss_krb5 nfsd
auth_rpcgss nfs_acl nfs lockd sunrpc fscache dm_crypt ip6t_REJECT ppdev xt_hl
ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT xt_comment xt_limit
xt_tcpudp xt_addrtype nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack
ip6table_filter parport_pc ip6_tables parport nf_conntrack_netbios_ns
nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack
iptable_filter serio_raw ip_tables x_tables vboxguest(OX) psmouse ahci libahci
e1000
[ 348.084011] CPU: 0 PID: 1353 Comm: rpc.svcgssd Tainted: G OX
3.13.0-53-generic #89-Ubuntu
[ 348.084011] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
VirtualBox 12/01/2006
[ 348.084011] task: ffff880014dce000 ti: ffff88001514e000 task.ti:
ffff88001514e000
[ 348.084011] RIP: 0010:[<ffffffffa01c2df0>] [<ffffffffa01c2df0>]
qword_addhex+0xb0/0x100 [sunrpc]
[ 348.084011] RSP: 0018:ffff88001514fe08 EFLAGS: 00000206
[ 348.084011] RAX: 0000000000000001 RBX: 0000000000000006 RCX: 00000000000001f6
[ 348.084011] RDX: ffff880015e54678 RSI: ffff88001514fe84 RDI: ffff88001514fe88
[ 348.084011] RBP: ffff88001514fe18 R08: ffff880015e57cf5 R09: 000000000000030b
[ 348.084011] R10: 0000000000000039 R11: 0000000000000027 R12: 0000000000000006
[ 348.084011] R13: ffffea0000422420 R14: ffffea00003e46e0 R15: ffff88001514fe98
[ 348.084011] FS: 00007f85d02f3740(0000) GS:ffff88001fc00000(0000)
knlGS:0000000000000000
[ 348.084011] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 348.084011] CR2: 0000000000408000 CR3: 00000000152f7000 CR4: 00000000000006f0
[ 348.084011] Stack:
[ 348.084011] ffff88001478d580 ffff88001514fe88 ffff88001514fe40
ffffffffa01f61bb
[ 348.084011] ffff8800150b2840 0000000000001000 ffff88001f48f500
ffff88001514feb8
[ 348.084011] ffffffffa01c5045 0002000000000001 ffff88000c084a60
ffff880015c2bb60
[ 348.084011] Call Trace:
[ 348.084011] [<ffffffffa01f61bb>] rsi_request+0x3b/0x50 [auth_rpcgss]
[ 348.084011] [<ffffffffa01c5045>] cache_read.isra.19+0x275/0x3a0 [sunrpc]
[ 348.084011] [<ffffffffa01c51a1>] cache_read_procfs+0x31/0x40 [sunrpc]
[ 348.084011] [<ffffffff81224a6d>] proc_reg_read+0x3d/0x80
[ 348.084011] [<ffffffff811bdf55>] vfs_read+0x95/0x160
[ 348.084011] [<ffffffff811bea69>] SyS_read+0x49/0xa0
[ 348.084011] [<ffffffff811d35e5>] ? SyS_poll+0x65/0x100
[ 348.084011] [<ffffffff8173391d>] system_call_fastpath+0x1a/0x1f
[ 348.084011] Code: e0 27 42 8d 44 20 30 41 88 40 fe 41 0f b6 c2 83 f8 0a 44
89 d8 0f 4c c3 41 83 e9 02 83 e9 01 46 8d 54 10 30 0f 95 c0 41 83 f9 01 <45> 88
50 ff 7f aa 45 85 c9 7f 1f 5b 41 5c 5d 41 b9 ff ff ff ff
[ 348.084011] Kernel panic - not syncing: softlockup: hung tasks
[ 348.084011] CPU: 0 PID: 1353 Comm: rpc.svcgssd Tainted: G OX
3.13.0-53-generic #89-Ubuntu
[ 348.084011] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
VirtualBox 12/01/2006
[ 348.084011] 000000000000012d ffff88001fc03e28 ffffffff81722e1e
ffffffff81a62b16
[ 348.084011] ffff88001fc03ea0 ffffffff8171bcbd 0000000000000008
ffff88001fc03eb0
[ 348.084011] ffff88001fc03e50 0000000000000086 0000000000000046
0000000000000007
[ 348.084011] Call Trace:
[ 348.084011] <IRQ> [<ffffffff81722e1e>] dump_stack+0x45/0x56
[ 348.084011] [<ffffffff8171bcbd>] panic+0xc8/0x1d7
[ 348.084011] [<ffffffff8110dc85>] watchdog_timer_fn+0x165/0x170
[ 348.084011] [<ffffffff8108e8c7>] __run_hrtimer+0x77/0x1d0
[ 348.084011] [<ffffffff8110db20>] ? watchdog_cleanup+0x10/0x10
[ 348.084011] [<ffffffff8108f08f>] hrtimer_interrupt+0xef/0x230
[ 348.084011] [<ffffffff81043617>] local_apic_timer_interrupt+0x37/0x60
[ 348.084011] [<ffffffff81735ccf>] smp_apic_timer_interrupt+0x3f/0x60
[ 348.084011] [<ffffffff8173465d>] apic_timer_interrupt+0x6d/0x80
[ 348.084011] <EOI> [<ffffffffa01c2df0>] ? qword_addhex+0xb0/0x100 [sunrpc]
[ 348.084011] [<ffffffffa01f61bb>] rsi_request+0x3b/0x50 [auth_rpcgss]
[ 348.084011] [<ffffffffa01c5045>] cache_read.isra.19+0x275/0x3a0 [sunrpc]
[ 348.084011] [<ffffffffa01c51a1>] cache_read_procfs+0x31/0x40 [sunrpc]
[ 348.084011] [<ffffffff81224a6d>] proc_reg_read+0x3d/0x80
[ 348.084011] [<ffffffff811bdf55>] vfs_read+0x95/0x160
[ 348.084011] [<ffffffff811bea69>] SyS_read+0x49/0xa0
[ 348.084011] [<ffffffff811d35e5>] ? SyS_poll+0x65/0x100
[ 348.084011] [<ffffffff8173391d>] system_call_fastpath+0x1a/0x1f
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: nfs-common 1:1.2.8-6ubuntu1.1
ProcVersionSignature: Ubuntu 3.13.0-53.89-generic 3.13.11-ckt19
Uname: Linux 3.13.0-53-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.11
Architecture: amd64
Date: Thu Jun 18 20:46:05 2015
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=en_US.UTF-8
SHELL=/bin/bash
SourcePackage: nfs-utils
UpgradeStatus: No upgrade log present (probably fresh install)
** Affects: nfs-utils (Ubuntu)
Importance: Undecided
Status: New
** Tags: amd64 apport-bug trusty uec-images
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1466654
Title:
kernel soft lockup on nfs server when using a kerberos mount
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1466654/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
