[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Following up on the previous comment (#12), the following log output illustrates the random nature of the problem. # while : ; do su -c groups foo ; sleep 1 ; done domusers domusers domusers domusers domusers domadmins devel publish domusers domusers domusers domusers domusers domadmins devel publish domusers domusers domusers domusers domusers domusers domusers domusers domusers domusers domusers domusers domadmins devel publish domusers domadmins devel publish domusers domadmins devel publish domusers domadmins devel publish domusers domusers domusers domadmins devel publish domusers domadmins devel publish domusers domadmins devel publish domusers domusers domusers domusers domadmins devel publish domusers domadmins devel publish domusers domusers domusers domusers domadmins devel publish domusers domadmins devel publish domusers domusers domusers domusers domusers domadmins devel publish domusers domusers domusers Commenting out either one of the two lines quoted in comment #12 and restarting sssd and repeating the experiment gives something like: # while : ; do su -c groups foo ; sleep 1 ; done domusers domusers domusers domusers domusers domusers domusers domadmins devel publish domusers domadmins devel publish domusers domadmins devel publish domusers domusers domusers [... ad infinitum, or at least for several minutes] ** Summary changed: - sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd + sssd sometimes forgets all but one group memberships of a user -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd sometimes forgets all but one group memberships of a user To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
With the partial workaround ldap_purge_cache_timeout = 3 ldap_enumeration_refresh_timeout = 3 when foo logs in to the affected system, half the time groups shows foo to be a member of four groups; half the time groups shows only one group. Whether it's one or the other seems random. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Here's another possible clue. I have a machine running sssd 1.8.2-0ubuntu1. On startup with debug_level = 0x470 it logs the following. (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_get_groups_next_base] (0x0400): Searching for groups with base [dc=cmpny,dc=com] (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [((objectclass=group)(msSFU30Name=*)((gidNumber=*)(!(gidNumber=0][dc=cmpny,dc=com]. (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_get_generic_ext_done] (0x0400): Search result: Success(0), no errmsg set (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_get_groups_process] (0x0400): Search for groups, returned 4 results. (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group devel (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group publish (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domadmins (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domusers (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group devel (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group publish (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group domadmins (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group domusers (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_process_group_send] (0x0040): No Members. Done! (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group devel (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group publish (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domadmins (Sat Sep 15 21:16:05 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domusers Sssd 1.9.0~rc1-0ubuntu1 logs the following. (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_get_groups_next_base] (0x0400): Searching for groups with base [dc=cmpny,dc=com] (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [((objectclass=group)(msSFU30Name=*)((gidNumber=*)(!(gidNumber=0][dc=cmpny,dc=com]. (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_get_generic_ext_done] (0x0400): Search result: Success(0), no errmsg set (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_get_groups_process] (0x0400): Search for groups, returned 4 results. (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group devel (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group publish (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domadmins (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domusers (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group devel (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group publish (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0040): Failed to save user domadmins (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_groups] (0x0040): Failed to store group 2 members. (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_grpmem] (0x0400): Storing members for group domusers (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_process_group_send] (0x0040): No Members. Done! (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group devel (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group publish (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domadmins (Sat Sep 15 21:10:21 2012) [sssd[be[SAMBA]]] [sdap_save_group] (0x0400): Storing info for group domusers Notice that sdap_save_groups reported a failure for one of the four groups. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Here's a partial workaround. In [domain/SAMBA] I set ldap_purge_cache_timeout = 3 ldap_enumeration_refresh_timeout = 3 Then, although user foo still disappears from groups on login, it's back within a few seconds without my having to restart sssd. This may be an additional clue as to where the bug is. root@ellen:/# su -c pwd foo ; while : ; do groups foo ; sleep 1 ; done / foo : domusers foo : domusers devel publish domadmins ^C root@ellen:/# su -c pwd foo ; while : ; do groups foo ; sleep 1 ; done / foo : domusers devel publish domadmins ^C root@ellen:/# su -c pwd foo ; while : ; do groups foo ; sleep 1 ; done / foo : domusers foo : domusers foo : domusers foo : domusers foo : domusers foo : domusers foo : domusers devel publish domadmins ^C root@ellen:/# su -c pwd foo ; while : ; do groups foo ; sleep 1 ; done / foo : domusers foo : domusers foo : domusers foo : domusers foo : domusers devel publish domadmins ^C -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Just for the record, 'getent group username' and 'groups' should NOT be identical. Just plain groups will list the set of groups that were assigned to you *at login time*, whereas 'getent group username' or 'groups username' gives you the list of groups that would be assigned to you if you logged in right now. So it's expected that they would be different if you changed memberships since logging in. This is standard UNIX behavior, though certainly not what most people would expect. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
You can't compare 'getent group xxx' with 'groups'. The first lists user names, the second group names. Perhaps you meant to say that 'groups' and 'groups $USER' are not necessarily the same. The first returns a list that was created at login time, the second an up-to-date list. Good to point that out, especially in connection with my comment #6. However, the main issue here (bug #1049186) concerns incorrect output from getent group, getent group grpnm and groups foo, all of which (I believe) are supposed to report current information, not information collected at login time. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Jakub wrote in comment #1 Is the behaviour reproducable within a single SSSD session? In other words, if you log in after the ten seconds have passed and the getent command reports correct group memberships, does groups still show wrong membership? Sorry, Jakub, I didn't answer this question in my first reply to your comment. As Stephen has just pointed out, the output of groups doesn't ever change. It reports information collected at login time. The output of groups foo does change ten seconds after restarting sssd. root@ellen:/# su foo foo@ellen:/$ groups domusers foo@ellen:/$ groups foo foo : domusers foo@ellen:/$ # Restart sssd here foo@ellen:/$ groups domusers foo@ellen:/$ groups foo foo : domusers foo@ellen:/$ # Wait ten seconds foo@ellen:/$ groups domusers foo@ellen:/$ groups foo foo : domusers devel publish domadmins foo@ellen:/$ Logging in again causes foo to disappear again from all but one group. foo@ellen:/$ exit exit root@ellen:/# su foo foo@ellen:/$ groups domusers foo@ellen:/$ groups foo foo : domusers -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Sure, getent group and groups should yield the same results, the are just a different ways of reaching the same information -- getent group groupname retrieves members of a group, groups username performs an initgroups operation that retrieves the groups the user is a member of. I think we should debug the information some more... Can you raise the debug_level in the domain section of the sssd.conf to 8 perhaps, clear the SSSD caches and run both tests? Then please attach /var/log/sssd/sssd_SAMBA.log. Did exactly the same config file work with a previous SSSD version? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
I said that groups and getent are consistent with each other but here is proof that that is not always the case. After foo logs in and out, groups foo and getent group domadmins omit foo from domadmins whereas getent group still includes foo in domadmins. root@ellen:/# getent group |grep domadmins domadmins:*:512:bar,foo root@ellen:/# getent group domadmins domadmins:*:512:bar,foo root@ellen:/# groups foo foo : domusers devel publish domadmins root@ellen:/# su -c pwd foo / root@ellen:/# getent group |grep domadmins domadmins:*:512:bar,foo root@ellen:/# getent group domadmins domadmins:*:512:bar root@ellen:/# groups foo foo : domusers After restarting sssd, all methods initially agree (incorrectly) that foo is not a member of domadmins. root@ellen:/# restart sssd sssd start/running, process 6690 root@ellen:/# getent group | grep domadmins domadmins:*:512:bar root@ellen:/# getent group domadmins domadmins:*:512:bar root@ellen:/# groups foo foo : domusers But ten seconds later, groups foo and getent group domadmins include foo in domadmins whereas getent group still omits foo from domadmins. root@ellen:/# # Wait ten seconds root@ellen:/# getent group | grep domadmins domadmins:*:512:bar root@ellen:/# getent group domadmins domadmins:*:512:bar,foo root@ellen:/# groups foo foo : domusers devel publish domadmins This is, again, with enumerate = true in [domain/SAMBA]. Sometimes, though, nothing changes even after ten seconds. Sometimes restarting sssd causes getent group to include foo in domadmins again. The behavior varies. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
I wrote comment #4 before I saw your new comment #3. You wrote: Did exactly the same config file work with a previous SSSD version? Yes, I had no trouble with sssd 1.8.2-0ubuntu1. I'll gather the debugging info you asked for and mail it to you directly. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Another interesting phenomenon. I log in as foo. foo@ellen:~$ groups domusers foo@ellen:~$ groups foo foo : domusers devel publish domadmins foo@ellen:~$ id uid=10005(foo) gid=513(domusers) groups=513(domusers) foo@ellen:~$ id foo uid=10005(foo) gid=513(domusers) groups=513(domusers),601(devel),602(publish),512(domadmins) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Without the SSSD logs it's hard to tell for certain, but I suspect this is caused by enumerate=True in the sssd.conf config file. The reason why the groups seemingly appear after about ten seconds is that after the SSSD provider starts up, the enumerate task is scheduled. In general, it *should* block the NSS operations until the initial enumeration has completed, though. Is the behaviour reproducable within a single SSSD session? In other words, if you log in after the ten seconds have passed and the getent command reports correct group memberships, does groups still show wrong membership? Also, is there a particular reason to use enumerate=True? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1049186] Re: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd
Hi Jakub, You wrote: Without the SSSD logs it's hard to tell for certain, but I suspect this is caused by enumerate=True in the sssd.conf config file. If I comment out enumerate = True then the behavior is the same except that even after restarting sssd, getent group domadmins continues to fail to list the user even after ten seconds. root@ellen:/# getent group domadmins domadmins:*:512:bar,foo root@ellen:/# su -c pwd foo / root@ellen:/# getent group domadmins domadmins:*:512:bar root@ellen:/# restart sssd sssd start/running, process 5154 root@ellen:/# date ; getent group domadmins Wed Sep 12 00:42:14 CEST 2012 domadmins:*:512:bar root@ellen:/# date ; getent group domadmins Wed Sep 12 00:43:16 CEST 2012 domadmins:*:512:bar (Please note that su -c pwd foo doesn't open a new interactive shell; it just executes the pwd command in a short-lived shell process owned by foo.) Also, is there a particular reason to use enumerate=True? Well, the bug is worse without it than with it. :) Without it, sssd fails to remember that foo is a member of domadmins even after it's restarted. The reason why the groups seemingly appear after about ten seconds is that after the SSSD provider starts up, the enumerate task is scheduled. In general, it *should* block the NSS operations until the initial enumeration has completed, though. It doesn't block. If you think that this is a bug then please file a report. :) Is the behaviour reproducable within a single SSSD session? In other words, if you log in after the ten seconds have passed and the getent command reports correct group memberships, does groups still show wrong membership? With enumerate = true, after sssd has been restarted and ten seconds have passed, getent group domadmins reports foo as a member and groups foo shows domadmins as one of foo's groups. Before ten seconds have passed getent group domadmins does not show foo as a member and groups foo does not show domadmins as one of foo's groups. The getent and groups commands have always been consistent with each other so far as I have seen during my testing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1049186 Title: sssd forgets group memberships of foo when foo logs in; remembers them after ten seconds after restarting sssd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1049186/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
