[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
[Expired for samba (Ubuntu) because there has been no activity for 60 days.] ** Changed in: samba (Ubuntu) Status: Incomplete = Expired -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
For apport-collect, try running: https_proxy=http://1.2.3.4:5678/ apport-collect 1100787 If your proxy needs authentication, then: https_proxy=http://username:password@1.2.3.4:5678/ apport-collect 1100787 (change 1.2.3.4:5678 to your proxy IP address and port number of course) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
I tried the commands above, first using the proxy's domain name, then with it's IP address but I still get the same error. ERROR: connecting to Launchpad failed: [Errno 110] Connection timed out Running env | grep proxy shows the proxy is set correctly. Looking at the other proxy's logs, it doesn't even look like it's trying to use the proxy. Back to the actual crash bug... I decided to install the debug symbols for libtdb and it looks like the signal handler gets called in __memmove_sse2. Stack trace is posted below. I also managed to get Winbind running under Valgrind but it didn't crash for the 2 days that I had it running. I'll attach the valgrind log to this post anyway in ccase there's any useful info in it. I've also setup another proxy server on the same ESXi host using Ubuntu 12.10. Settings are more or less identical to the 12.04 proxy, only thing that's different are the package versions. So far it hasn't had any problems at all. libtdb-dbg stack trace: [Thread debugging using libthread_db enabled] Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1. 0x7f4a2c6b443e in __libc_waitpid (pid=optimised out, stat_loc=0x7fff25d00030, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:32 #0 0x7f4a2c6b443e in __libc_waitpid (pid=optimised out, stat_loc=0x7fff25d00030, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:32 #1 0x7f4a2c63a29e in do_system (line=0x7f4a311f5110 /usr/share/samba/panic-action 12134) at ../sysdeps/posix/system.c:149 #2 0x7f4a2f1b546c in smb_panic (why=optimised out) at lib/util.c:1123 #3 0x7f4a2f1a6708 in fault_report (sig=7) at lib/fault.c:53 #4 sig_fault (sig=7) at lib/fault.c:76 #5 signal handler called #6 __memmove_sse2 (dest=0x7fff25d0062c, src=optimised out, len=optimised out) at ../string/memmove.c:82 #7 0x7f4a2cde0042 in tdb_read (tdb=0x7f4a311d0480, off=1339388, buf=0x7fff25d0062c, len=4, cv=0) at /usr/include/x86_64-linux-gnu/bits/string3.h:52 #8 0x7f4a2cde1bc4 in transaction_read (tdb=0x7f4a311d0480, off=1339388, buf=0x7fff25d0062c, len=4, cv=0) at ../common/transaction.c:176 #9 0x7f4a2cddf664 in tdb_free (tdb=0x7f4a311d0480, offset=1339392, rec=0x7fff25d00650) at ../common/freelist.c:148 #10 0x7f4a2cde03db in tdb_expand (tdb=0x7f4a311d0480, size=335872) at ../common/io.c:360 #11 0x7f4a2cddf9d9 in tdb_allocate (tdb=0x7f4a311d0480, length=104, rec=0x7fff25d00740) at ../common/freelist.c:358 #12 0x7f4a2cddca5c in _tdb_store (tdb=0x7f4a311d0480, key=..., dbuf=..., flag=0, hash=610244803) at ../common/tdb.c:561 #13 0x7f4a2cddcd07 in tdb_store (tdb=0x7f4a311d0480, key=..., dbuf=..., flag=optimised out) at ../common/tdb.c:618 #14 0x7f4a2f1c35ba in stabilize_fn (tdb=optimised out, key=..., val=..., priv=0x7fff25d009b0) at lib/gencache.c:561 #15 0x7f4a2cddd7bc in tdb_traverse_internal (tdb=0x7f4a311d0780, fn=0x7f4a2f1c3480 stabilize_fn, private_data=0x7fff25d009b0, tl=0x7fff25d00930) at ../common/traverse.c:190 #16 0x7f4a2cddd9d6 in tdb_traverse (tdb=0x7f4a311d0780, fn=0x7f4a2f1c3480 stabilize_fn, private_data=0x7fff25d009b0) at ../common/traverse.c:260 #17 0x7f4a2f1c382c in gencache_stabilize () at lib/gencache.c:494 #18 gencache_stabilize () at lib/gencache.c:457 #19 0x7f4a2f1c3d12 in gencache_set_data_blob (keystr=optimised out, blob=optimised out, timeout=1361184829) at lib/gencache.c:202 #20 0x7f4a2f1c3ee3 in gencache_set (keystr=0x7fff25d00bc0 IDMAP/SID2GID /S-1-5-21-1152652908-1908898795-1555891258-10921, value=0x7fff25d00cc0 20921, timeout=1361184829) at lib/gencache.c:641 #21 0x7f4a2f1e2437 in idmap_cache_set_sid2gid (sid=0x7f4a31203ed0, gid=20921) at lib/idmap_cache.c:247 #22 0x7f4a2f4a0af9 in idmap_sid_to_gid (domname=optimised out, sid=0x7f4a31203ed0, gid=0x7fff25d00e5c) at winbindd/idmap_util.c:290 #23 0x7f4a2f1092b7 in _wbint_Sid2Gid (p=optimised out, r=0x7f4a3122fd90) at winbindd/winbindd_dual_srv.c:130 #24 0x7f4a2f115454 in api_wbint_Sid2Gid (p=0x7fff25d00eb0) at librpc/gen_ndr/srv_wbint.c:476 #25 0x7f4a2f108f6c in winbindd_dual_ndrcmd (domain=optimised out, state=0x7fff25d01020) at winbindd/winbindd_dual_ndr.c:322 #26 0x7f4a2f107cb4 in child_process_request (state=0x7fff25d01020, child=optimised out) at winbindd/winbindd_dual.c:440 #27 fork_domain_child (child=0x7f4a2f981680) at winbindd/winbindd_dual.c:1554 #28 0x7f4a2f108765 in wb_child_request_trigger (req=0x7f4a31231bb0, private_data=optimised out) at winbindd/winbindd_dual.c:145 #29 0x7f4a2f1c6d32 in tevent_common_loop_immediate (ev=0x7f4a311c2c70) at ../lib/tevent/tevent_immediate.c:135 #30 0x7f4a2f1c4e28 in run_events_poll (ev=0x7f4a311c2c70, pollrtn=0, pfds=0x0, num_pfds=0) at lib/events.c:198 #31 0x7f4a2f1c5246 in s3_event_loop_once (ev=0x7f4a311c2c70, location=optimised out) at lib/events.c:331 #32 0x7f4a2f1c5e50 in _tevent_loop_once (ev=0x7f4a311c2c70, location=0x7f4a2f525f01
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
I ran tdbtool to check the tdb files while winbind was working properly and they all reported OK. I ran it again after killing winbind, after it crashed, and again they all showed up as being OK. I was also beginning to think it was a hardware memory fault as we recently added extra RAM to the server. But the Ubuntu server is running as a VM under VMWare ESXi and none of the other VMs (or the hypervisor itself for that matter) on the server are having errors. The physical server itself also uses ECC memory so I would have that would have caught any errors. We'll run a full memory diagnostic on the physical server when we get a chance during the next couple of days. I'm having a little trouble running apport-collect at the moment. I just get connection timed out errors. Googling the issue, it looks like it's because we are behind a proxy (our local squid proxy forwards all traffic to our ISPs proxy). I'll look into running winbind under valgrind. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
Had two crashes today so far and I've managed to pull some more info from the logs and reports. We use the squid Kerberos authentication plugin to authenticate the user and their password, then we use the wbinfo_group.pl script to check the users membership of an AD group. It appears that the checking for a group stage is where winbind crashes. Interestingly the Kerberos authentication stage carries on authenticating users properly even after the crash. Users that have had their authentication cached by squid can carry on accessing the internet as normal for a short time. Any new authentication attempts fail. The SID in each initial segfault report was the SID for the Domain Users group. The SIDs in the following segfaults are a mix of random groups and users, different after each crash. The last successful user to be looked up before each crash was also different. After the first segfault I had to force a server restart as it became totally unresponsive. The second time I managed to kill the Winbind service and restart it. No other faults so far but it does seem to be a quiet period at the moment, so the server isn't under a lot of load. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
Thank you for taking the time to report this bug and helping to make Ubuntu better. From your stack trace, it looks likely to me that the crash is in libtdb rather than winbind itself, unless winbind is corrupting memory somehow. Can you check all tdb files for corruption using tdbtool (I presume this needs to be done when winbind and samba are shut down), and if you are able then run winbind under valgrind? It isn't clear to me whether this really is a bug or local corruption or a hardware fault or something like that. These steps might give us something really obvious to help decide. Please also report exact package numbers. The command apport-collect 1100787 will do this for you automatically. Please change the bug status back to New when you are done. Thanks! ** Changed in: samba (Ubuntu) Status: New = Incomplete -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
Thought I'd add a little bit more info that may be useful. It seems to only crash when there is a high demand on the server, when a lot of users need authenticating. As we're a school we often get high loads at the start of a lesson when there could be upwards of 100+ students starting to access the internet at the same time. I thought it might be due to a specific user or group that was corrupt in AD but, looking at the SID in each of the stack traces in the reports, it's different each time. Some times it's a user, some times a group. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
** Changed in: samba (Ubuntu) Importance: Undecided = High -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1100787] Re: Samba 3.6.3 segfault or panic.
** Package changed: ubuntu = samba (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to samba in Ubuntu. https://bugs.launchpad.net/bugs/1100787 Title: Samba 3.6.3 segfault or panic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1100787/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
