On (28/07/16 12:58), Lukas Slebodnik wrote: >On (28/07/16 12:44), thierry bordaz wrote: >> >> >>On 07/28/2016 11:17 AM, Lukas Slebodnik wrote: >>> On (28/07/16 10:24), thierry bordaz wrote: >>> > >>> > On 07/28/2016 09:39 AM, Jakub Hrozek wrote: >>> > > On Wed, Jul 27, 2016 at 04:09:07PM +0200, thierry bordaz wrote: >>> > > > On 07/27/2016 03:36 PM, Jakub Hrozek wrote: >>> > > > > On Wed, Jul 27, 2016 at 02:55:37PM +0200, thierry bordaz wrote: >>> > > > > > On 07/27/2016 01:56 PM, Jakub Hrozek wrote: >>> > > > > > > On Wed, Jul 27, 2016 at 01:03:59PM +0200, Jakub Hrozek wrote: >>> > > > > > > > On Wed, Jul 27, 2016 at 12:22:46PM +0200, Lukas Slebodnik >>> > > > > > > > wrote: >>> > > > > > > > > On (27/07/16 12:08), Jakub Hrozek wrote: >>> > > > > > > > > > On Wed, Jul 27, 2016 at 12:02:24PM +0200, Jakub Hrozek >>> > > > > > > > > > wrote: >>> > > > > > > > > > > On Wed, Jul 27, 2016 at 11:54:16AM +0200, Lukas >>> > > > > > > > > > > Slebodnik wrote: >>> > > > > > > > > > > > ehlo, >>> > > > > > > > > > > > >>> > > > > > > > > > > > attached patch fixes acces denied after activating >>> > > > > > > > > > > > user in 389ds. >>> > > > > > > > > > > > Jakub had some comments/ideas in ticket but I think >>> > > > > > > > > > > > it's better to discuss >>> > > > > > > > > > > > about virtual attributes and timestamp cache on >>> > > > > > > > > > > > mailing list. >>> > > > > > > > > > > Yes, so the comment I have is that while this works, it >>> > > > > > > > > > > might break some >>> > > > > > > > > > > strange LDAP servers. >>> > > > > > > > > > > >>> > > > > > > > > > > We use modifyTimestamp as a 'positive' indicator that >>> > > > > > > > > > > the entry has not >>> > > > > > > > > > > changed -- if the modifyTimestamp didn't change, we >>> > > > > > > > > > > consider the cached >>> > > > > > > > > > > entry the same as what is on the server and only bump >>> > > > > > > > > > > the timestamp >>> > > > > > > > > > > cache. If the timestamp is different, we do a >>> > > > > > > > > > > deep-comparison of cached >>> > > > > > > > > > > attribute values with what is on the LDAP server and >>> > > > > > > > > > > write the sysdb >>> > > > > > > > > > > cache entry only if the attributes differ. >>> > > > > > > > > > > >>> > > > > > > > > > > I was wondering if we can use the modifyTimestamp at >>> > > > > > > > > > > all, then, because >>> > > > > > > > > > > even if it's the same, we might want to check the >>> > > > > > > > > > > attributes to see if >>> > > > > > > > > > > some of the values are different because some of the >>> > > > > > > > > > > attributes might be >>> > > > > > > > > > > this operational/virtual attribute.. >>> > > > > > > > > > Sorry, sent too soon. >>> > > > > > > > > > >>> > > > > > > > > > I think the questions are -- 1) can we enumerate the >>> > > > > > > > > > virtual attributes? >>> > > > > > > > > That might be a question for 389-ds developers. >>> > > > > > > > > But it's very likely it will be different on other LDAP >>> > > > > > > > > servers. >>> > > > > > > > > >>> > > > > > > > > > 2) Would different LDAP servers have different virtual >>> > > > > > > > > > attributes. >>> > > > > > > > > > >>> > > > > > > > > > For 2) maybe a possible solution might be to set a >>> > > > > > > > > > non-existing >>> > > > > > > > > > modifyTimestamp attribute value, but I would consider >>> > > > > > > > > > that only a >>> > > > > > > > > > kludge, we shouldn't break existing setups.. >>> > > > > > > > > I am not satisfied with this POC solution either. >>> > > > > > > > > >>> > > > > > > > > So should we remove usage of modifyTimestamp for detecting >>> > > > > > > > > changes? >>> > > > > > > > I would prefer to ask the DS developers before removing it >>> > > > > > > > completely. >>> > > > > > > > >>> > > > > > > > At least for large groups it might take a long time to >>> > > > > > > > compare all attribute >>> > > > > > > > values and IIRC we don't depend on any virtual attributes for >>> > > > > > > > groups. Maybe >>> > > > > > > > we could parametrize that part of the code and enable the >>> > > > > > > > fast way with >>> > > > > > > > modifyTimestamps for 'known' server types, that is for setups >>> > > > > > > > with AD and >>> > > > > > > > IPA providers. >>> > > > > > > > >>> > > > > > > > For users, there is typically not as many attributes so we >>> > > > > > > > should be >>> > > > > > > > fine deep-comparing all attributes. >>> > > > > > > I'm adding Thierry (so please reply-to-all to keep him in the >>> > > > > > > thread). >>> > > > > > > >>> > > > > > > Thierry, in the latest sssd version we tried to add a >>> > > > > > > performance >>> > > > > > > improvement related to how we store SSSD entries in the cache. >>> > > > > > > The short >>> > > > > > > version is that we store the modifyTimestamp attribute in the >>> > > > > > > cache and >>> > > > > > > when we fetch an entry, we compare the entry modifyTimestamp >>> > > > > > > with what >>> > > > > > > is on the server. When the two are the same, we say that the >>> > > > > > > entry did >>> > > > > > > not change and don't update the cache. >>> > > > > > > >>> > > > > > > This works fine for most attributes, but not for attributes like >>> > > > > > > nsAccountLock which do not change modifyTimestamp when they are >>> > > > > > > modified. So when an entry was already cached but then >>> > > > > > > nsAccountLock >>> > > > > > > changed, we treated the entry as the same and never read the new >>> > > > > > > nsAccountLock value. >>> > > > > > > >>> > > > > > > To fix this, I think we have several options: >>> > > > > > > 1) special-case the nsAccountLock. This seems a bit >>> > > > > > > dangerous, >>> > > > > > > because I'm not sure we can say that some other >>> > > > > > > attribute we are >>> > > > > > > interested in behaves the same as nsAccountLock. >>> > > > > > > 2) drop the modifyTimestamp optimization completely. >>> > > > > > > Then we fall >>> > > > > > > back to comparing the attribute values, which might >>> > > > > > > work, but for >>> > > > > > > huge objects like groups with thousands of members, >>> > > > > > > this might be >>> > > > > > > too expensive. >>> > > > > > > 3) only use the modifyTimestamp optimization for cases >>> > > > > > > where we know >>> > > > > > > we don't read any virtual attributes. >>> > > > > > > >>> > > > > > > And my question is -- can we, in general, know if the >>> > > > > > > modifyTimestamp >>> > > > > > > way of detecting changes is realiable for all LDAP servers? Or >>> > > > > > > do you >>> > > > > > > think it should only be used for cases where we know we are not >>> > > > > > > interested in any virtual attributes (that would mostly be >>> > > > > > > storing >>> > > > > > > groups from servers where we know exactly what is on the server >>> > > > > > > side, >>> > > > > > > like IPA or AD). >>> > > > > > Hello, >>> > > > > > >>> > > > > > Relying on modifytimestamp looks a good idea. Any MOD/MODRDN will >>> > > > > > update it, >>> > > > > > except I think it is unchanged when updating some password policy >>> > > > > > attributes. >>> > > > > > >>> > > > > > Regarding virtual attribute, the only one I know in IPA is >>> > > > > > nsaccountlock. >>> > > > > > nsaccountlock is an operational attribute (you need to request it >>> > > > > > to see it) >>> > > > > > and is also a virtual attribute BUT only for 'staged' and >>> > > > > > 'deleted' users. >>> > > > > > It is a stored attribute for regular users and we should update >>> > > > > > modifytimestamp when it is set. >>> > > > > > >>> > > > > > thanks >>> > > > > > thierry >>> > > > > OK, in that case it seems like we can special-case it. But do you >>> > > > > know >>> > > > > about any other attributes in any other LDAP servers? >>> > > > Any LDAP server following standard should provide modifytimestamp that >>> > > > reflect the last update of the entry. Now virtual attribute values >>> > > > may be >>> > > > "attached" to the entry and its value change without modification of >>> > > > modifytimestamp. >>> > > > For 389-ds and IPA it is fine as virtual value of nsaccountlock is >>> > > > changed >>> > > > only when the DN change. >>> > > > For others LDAP servers I suppose it exists the same ability to define >>> > > > service providers that return virtual attribute values. The >>> > > > difficulty is >>> > > > that the schema may not give any hint if the retrieved attributes >>> > > > values >>> > > > were stored or computed and consequently trust modifytimestamp to >>> > > > know if >>> > > > the values changed or not. >>> > > > For example in ODSEE, memberof is a virtual attribute. >>> > > Thank you, for the explanation Thierry. >>> > > >>> > > Then to be on the safe side I propose: >>> > > 1) We add an (probably undocumented?) flag that says whether to >>> > > use >>> > > modifyTimestamp to detect entry changes or not >>> > > 2) for the generic LDAP provider we always really compare the >>> > > attribute values, in other words the option would be set to >>> > > false. If there is anyone with performance issues with a >>> > > generic >>> > > setup, we tell them to flip the option. >>> > > 3) For the IPA and AD providers, we set this option to true and >>> > > use >>> > > the modifyTimestamp value to detect changes >>> > > 4) We special case nsAccountLock >>> > I am not sure I understand why nsAccountLock is a special case ? >>> > In IPA/389: >>> > >>> > * Stage user, it is always 'nsaccountlock: True' >>> > When stage entry is created or updated, 'modifytimestamp' is also >>> > updated. So you can rely on modifytimestamp to detect a change in a >>> > stage user. >>> > There is no way to update nsaccountlock for a stage entry >>> > * Deleted user. Idem Stage user >>> We haven't tested stage users yet. I ran just sssd related regression tests. >>> >>> > * Regular user. 'nsaccountlock' is *not* a virtual attribute, so if it >>> > is enable/disable you can rely on modifytimestamp to detect a change >>> > of 'nsaccountlock' for a regular user. >>> > Also any change on regular user will update 'modifytimestamp' so you >>> > can rely on it to detect a change. >>> > >>> My experience with ns-inactivate.pl and ns-activate.pl is different. >>> modifytimestamp is not changed even though nsaccountlock was changed. >>> >>> It was a plain 389-ds with id_provider ldap in sssd >>> >>> LS >>Hi Lukas, >> >> >>It may have change recently because I can not reproduce. What versions are >>you running ? >> >> # >> #initial entry >> # >> ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx -b >> "cn=accounts, <suffix>" 'uid=user0' nsaccountlock modifytimestamp >> createtimestamp >> dn: uid=user0,cn=users,cn=accounts,<suffix> >> modifytimestamp: *20160728103441Z* >> createtimestamp: 20160726160436Z >> >> # >> # Inactivate: modifytimestamp changed >> # >> /usr/sbin/ns-inactivate.pl -Z <instance> -D "cn=directory manager" >> -w xxx -I uid=user0,cn=users,cn=accounts,<suffix> >> uid=user0,cn=users,cn=accounts,<suffix> inactivated. >> >> ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx -b >> "cn=accounts, <suffix>" 'uid=user0' nsaccountlock modifytimestamp >> createtimestamp >> dn: uid=user0,cn=users,cn=accounts,<suffix> >> nsaccountlock: true >> modifytimestamp: *20160728103642Z* >> createtimestamp: 20160726160436Z >> >> # >> # activate: modifytimestamp changed >> # >> /usr/sbin/ns-activate.pl -Z <instance> -D "cn=directory manager" -w >> xxx -I uid=user0,cn=users,cn=accounts,<suffix> >> uid=user0,cn=users,cn=accounts,<suffix> activated. >> >> ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx -b >> "cn=accounts, <suffix>" 'uid=user0' nsaccountlock modifytimestamp >> createtimestamp >> dn: uid=user0,cn=users,cn=accounts,<suffix> >> modifytimestamp: *20160728103711Z* >> createtimestamp: 20160726160436Z >> >rhel7.3 >389-ds-base-1.3.5.10-5.el7.x86_64 > >But I didn't test directly with user but indirectly > >*Add Managed role > dn: cn=managed,ou=people,dc=example,dc=com > objectclass: top > objectclass: LdapSubEntry > objectclass: nsRoleDefinition > objectclass: nsSimpleRoleDefinition > objectclass: nsManagedRoleDefinition > cn: Managed > >*Authenticate with lockuser > >*Add user to managed role > dn: uid=lockuser,ou=people,dc=example,dc=com > changetype: modify > add: nsRoleDN > nsRoleDN: cn=managed,ou=people,dc=example,dc=com > >ns-inactivate.pl -D cn=Manager,dc=example,dc=com" -E -p 389 \ > -h $SERVER -I cn=managed,ou=people,dc=example,dc=com > >*Authenticate with lockuser > >ns-activate.pl -D cn=Manager,dc=example,dc=com" -E -p 389 \ > -h $SERVER -I cn=managed,ou=people,dc=example,dc=com > >*Authenticate with lockuser > >Should I test with rhel7.2 or is it an expected behaviour ? > I tested with rhel7.2 389-ds and there is the same behaviour.
The attribute modifytimestamp was not changed for user. LS _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://lists.fedorahosted.org/admin/lists/sssd-devel@lists.fedorahosted.org
