On (28/07/16 10:24), thierry bordaz wrote: > > >On 07/28/2016 09:39 AM, Jakub Hrozek wrote: >> On Wed, Jul 27, 2016 at 04:09:07PM +0200, thierry bordaz wrote: >> > >> > On 07/27/2016 03:36 PM, Jakub Hrozek wrote: >> > > On Wed, Jul 27, 2016 at 02:55:37PM +0200, thierry bordaz wrote: >> > > > On 07/27/2016 01:56 PM, Jakub Hrozek wrote: >> > > > > On Wed, Jul 27, 2016 at 01:03:59PM +0200, Jakub Hrozek wrote: >> > > > > > On Wed, Jul 27, 2016 at 12:22:46PM +0200, Lukas Slebodnik wrote: >> > > > > > > On (27/07/16 12:08), Jakub Hrozek wrote: >> > > > > > > > On Wed, Jul 27, 2016 at 12:02:24PM +0200, Jakub Hrozek wrote: >> > > > > > > > > On Wed, Jul 27, 2016 at 11:54:16AM +0200, Lukas Slebodnik >> > > > > > > > > wrote: >> > > > > > > > > > ehlo, >> > > > > > > > > > >> > > > > > > > > > attached patch fixes acces denied after activating user in >> > > > > > > > > > 389ds. >> > > > > > > > > > Jakub had some comments/ideas in ticket but I think it's >> > > > > > > > > > better to discuss >> > > > > > > > > > about virtual attributes and timestamp cache on mailing >> > > > > > > > > > list. >> > > > > > > > > Yes, so the comment I have is that while this works, it >> > > > > > > > > might break some >> > > > > > > > > strange LDAP servers. >> > > > > > > > > >> > > > > > > > > We use modifyTimestamp as a 'positive' indicator that the >> > > > > > > > > entry has not >> > > > > > > > > changed -- if the modifyTimestamp didn't change, we consider >> > > > > > > > > the cached >> > > > > > > > > entry the same as what is on the server and only bump the >> > > > > > > > > timestamp >> > > > > > > > > cache. If the timestamp is different, we do a >> > > > > > > > > deep-comparison of cached >> > > > > > > > > attribute values with what is on the LDAP server and write >> > > > > > > > > the sysdb >> > > > > > > > > cache entry only if the attributes differ. >> > > > > > > > > >> > > > > > > > > I was wondering if we can use the modifyTimestamp at all, >> > > > > > > > > then, because >> > > > > > > > > even if it's the same, we might want to check the attributes >> > > > > > > > > to see if >> > > > > > > > > some of the values are different because some of the >> > > > > > > > > attributes might be >> > > > > > > > > this operational/virtual attribute.. >> > > > > > > > Sorry, sent too soon. >> > > > > > > > >> > > > > > > > I think the questions are -- 1) can we enumerate the virtual >> > > > > > > > attributes? >> > > > > > > That might be a question for 389-ds developers. >> > > > > > > But it's very likely it will be different on other LDAP servers. >> > > > > > > >> > > > > > > > 2) Would different LDAP servers have different virtual >> > > > > > > > attributes. >> > > > > > > > >> > > > > > > > For 2) maybe a possible solution might be to set a non-existing >> > > > > > > > modifyTimestamp attribute value, but I would consider that >> > > > > > > > only a >> > > > > > > > kludge, we shouldn't break existing setups.. >> > > > > > > I am not satisfied with this POC solution either. >> > > > > > > >> > > > > > > So should we remove usage of modifyTimestamp for detecting >> > > > > > > changes? >> > > > > > I would prefer to ask the DS developers before removing it >> > > > > > completely. >> > > > > > >> > > > > > At least for large groups it might take a long time to compare all >> > > > > > attribute >> > > > > > values and IIRC we don't depend on any virtual attributes for >> > > > > > groups. Maybe >> > > > > > we could parametrize that part of the code and enable the fast way >> > > > > > with >> > > > > > modifyTimestamps for 'known' server types, that is for setups with >> > > > > > AD and >> > > > > > IPA providers. >> > > > > > >> > > > > > For users, there is typically not as many attributes so we should >> > > > > > be >> > > > > > fine deep-comparing all attributes. >> > > > > I'm adding Thierry (so please reply-to-all to keep him in the >> > > > > thread). >> > > > > >> > > > > Thierry, in the latest sssd version we tried to add a performance >> > > > > improvement related to how we store SSSD entries in the cache. The >> > > > > short >> > > > > version is that we store the modifyTimestamp attribute in the cache >> > > > > and >> > > > > when we fetch an entry, we compare the entry modifyTimestamp with >> > > > > what >> > > > > is on the server. When the two are the same, we say that the entry >> > > > > did >> > > > > not change and don't update the cache. >> > > > > >> > > > > This works fine for most attributes, but not for attributes like >> > > > > nsAccountLock which do not change modifyTimestamp when they are >> > > > > modified. So when an entry was already cached but then nsAccountLock >> > > > > changed, we treated the entry as the same and never read the new >> > > > > nsAccountLock value. >> > > > > >> > > > > To fix this, I think we have several options: >> > > > > 1) special-case the nsAccountLock. This seems a bit dangerous, >> > > > > because I'm not sure we can say that some other attribute we >> > > > > are >> > > > > interested in behaves the same as nsAccountLock. >> > > > > 2) drop the modifyTimestamp optimization completely. Then we >> > > > > fall >> > > > > back to comparing the attribute values, which might work, but >> > > > > for >> > > > > huge objects like groups with thousands of members, this >> > > > > might be >> > > > > too expensive. >> > > > > 3) only use the modifyTimestamp optimization for cases where >> > > > > we know >> > > > > we don't read any virtual attributes. >> > > > > >> > > > > And my question is -- can we, in general, know if the modifyTimestamp >> > > > > way of detecting changes is realiable for all LDAP servers? Or do you >> > > > > think it should only be used for cases where we know we are not >> > > > > interested in any virtual attributes (that would mostly be storing >> > > > > groups from servers where we know exactly what is on the server side, >> > > > > like IPA or AD). >> > > > Hello, >> > > > >> > > > Relying on modifytimestamp looks a good idea. Any MOD/MODRDN will >> > > > update it, >> > > > except I think it is unchanged when updating some password policy >> > > > attributes. >> > > > >> > > > Regarding virtual attribute, the only one I know in IPA is >> > > > nsaccountlock. >> > > > nsaccountlock is an operational attribute (you need to request it to >> > > > see it) >> > > > and is also a virtual attribute BUT only for 'staged' and 'deleted' >> > > > users. >> > > > It is a stored attribute for regular users and we should update >> > > > modifytimestamp when it is set. >> > > > >> > > > thanks >> > > > thierry >> > > OK, in that case it seems like we can special-case it. But do you know >> > > about any other attributes in any other LDAP servers? >> > Any LDAP server following standard should provide modifytimestamp that >> > reflect the last update of the entry. Now virtual attribute values may be >> > "attached" to the entry and its value change without modification of >> > modifytimestamp. >> > For 389-ds and IPA it is fine as virtual value of nsaccountlock is changed >> > only when the DN change. >> > For others LDAP servers I suppose it exists the same ability to define >> > service providers that return virtual attribute values. The difficulty is >> > that the schema may not give any hint if the retrieved attributes values >> > were stored or computed and consequently trust modifytimestamp to know if >> > the values changed or not. >> > For example in ODSEE, memberof is a virtual attribute. >> Thank you, for the explanation Thierry. >> >> Then to be on the safe side I propose: >> 1) We add an (probably undocumented?) flag that says whether to use >> modifyTimestamp to detect entry changes or not >> 2) for the generic LDAP provider we always really compare the >> attribute values, in other words the option would be set to >> false. If there is anyone with performance issues with a generic >> setup, we tell them to flip the option. >> 3) For the IPA and AD providers, we set this option to true and use >> the modifyTimestamp value to detect changes >> 4) We special case nsAccountLock >I am not sure I understand why nsAccountLock is a special case ? >In IPA/389: > > * Stage user, it is always 'nsaccountlock: True' > When stage entry is created or updated, 'modifytimestamp' is also > updated. So you can rely on modifytimestamp to detect a change in a > stage user. > There is no way to update nsaccountlock for a stage entry > * Deleted user. Idem Stage user We haven't tested stage users yet. I ran just sssd related regression tests.
> * Regular user. 'nsaccountlock' is *not* a virtual attribute, so if it > is enable/disable you can rely on modifytimestamp to detect a change > of 'nsaccountlock' for a regular user. > Also any change on regular user will update 'modifytimestamp' so you > can rely on it to detect a change. > My experience with ns-inactivate.pl and ns-activate.pl is different. modifytimestamp is not changed even though nsaccountlock was changed. It was a plain 389-ds with id_provider ldap in sssd LS _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://lists.fedorahosted.org/admin/lists/sssd-devel@lists.fedorahosted.org
