On Thu, 2009-08-13 at 08:54 -0400, Stephen Gallagher wrote: > One potential idea would be to have the SSSD automatically start an > enumeration at startup time if the cache is stale. Then, instead of > blocking updates waiting for subsequent enumerations, we could just go > immediately to the cache until the enumeration was complete.
See below, this is what I propose :) > > After some hard thinking I wrote down a few points I'd like the list > > opinion on. If people agree I will start acting on them. > > > > * stop performing enumerations on demand, and perform them in background > > if enumerations are activated (change the enumeration parameter from a > > bitfield to a true/flase boolean) > > > * perform a full user+group enumeration at startup (possibly using a > > paged or vlv search) > > * when possible request the modifyTimestamp attribute and save the > > highest modifyTimestamp into the domain entry as originalMaxTimestamp > > * on a tunable interval run a task that refreshes all users and groups > > in the background using a search filter that includes > > &(modifyTimestamp>$originalMaxtimestamp) > > * still do a full refresh every X minutes/hours > > * disable using a single huge transaction for enumerations (we might be > > ok doing a transaction for each page search if pages are small, > > otherwise just revert to the previous behavior of having a transaction > > per stored object) > > * Every time we update an entry we store the originalModifyTimestamp on > > it as a copy of the remote modifyTimestamp, this allows us to know if we > > actually need to touch the cached entry at all upon refresh (like when a > > getpwnam() is called, speeding up operations for entries that need no > > refresh (we will avoid data transformation and a writing to ldb). > > * Every time we run the general refresh task or we save a changed entry > > we store a LastUpdatedTimestamp > > * When the refresh task is completed successfully we run another cleanup > > task that searches our LDB for any entry that has a too old > > LastUpdatedTimestamp. If any is found, we double check against the > > remote server if the entry still exists (and update it if it does), and > > otherwise we delete it. > > I do like the idea of updating only the entries on the remote server > that have been updated since the previous enumeration. Yes hopefully all servers allow this kind of query, I guess we need to experiment and have options to fallback to full searches if this is not supported. > > NOTE: this means that until the first background enumeration is > > complete, a getent passwd or a getent group call may return incomplete > > results. I think this is acceptable as it will really happen only at > > startup, when the daemon caches are empty. > > > > I disagree. If we're going to have a startup enumeration, then we should > simply not enable handling NSS requests until that first enumeration is > complete. Incomplete results can be worse than no results. I assume NSS > has a return code for temporary failure? Internally, yes, but all it does it to return no results to the user space. Not returning results is == returning partial results. So I see no difference here. Note that this will happen only on the first startup when caches are empty as we otherwise always return whatever is in the cache. _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/sssd-devel
