Re: using nscd (ldap) makes passwd/group disappearing while installing ports
Am 01.02.2012 01:03, schrieb Benjamin Lee: > What's going on is: > > 1) The port checks if the group exists > 2) nscd caches that the group does not exist in its negative cache > 3) pw(8) creates the group then checks if it exists > 4) nscd returns the negative cache entry (group does not exist) > > This causes pw(8) to error since it expects the group that it just > created to exist. I had suggested before, that nscd be changed to only rely on cached negative entries, if they have repeatedly been seen in the underlying files/data-bases. E.g. only consider negative entries valid after they have been repeatedly stored into the cache (I'd think 3 times are a reasonable number). That way, the first lookup of an account or group will lead to an entry with count=1, which is found during cache lookup but which is ignored due to a too low "verification count". If the negative lookup is repeated 3 times within some reasonable time window (say 60 seconds), then it is to be considered verified. This will make the above sequence of queries and modifications of the passwd or group databases work with caching. This concept does not protect against scenarios where the negative cache entry is made active by several queries (e.g. manual checking for the presence of an account or group before the software installation repeats these tests). That is the reason to ask for more than 2 negative replies (3, perhaps better 5) before a negative cache entry is trusted. The main purpose of the cache (reduced latency for positive queries, limited load due to negative queries) will still be maintained. >> I also have this error very often when rebuilding/updating or even >> installing cups when "nscd" is enabled. A simple restart of nscd helps >> in most cases, most times I need to disable "cache" tag in >> /etc/nsswitch.conf, then everything runs smooth. >> >> Well, this behaviour is since a couple of years now, occurs sporadic. I >> have had in FreeBSD 7, 8, 9 and I see it in 10. What is it? >> >> I like the cache facility, since in domains with a lot of users >> searching LDAP takes some time and caching help keeping traffic and >> latency short. But the namservice caching mechanism seems to be >> unreliable. What is up there? > > You should put "files" before "cache" in /etc/nsswitch.conf, e.g.: > > group: files cache ldap > passwd: files cache ldap > > The problem is that tools that modify the passwd and group files, like > pw(8), don't invalidate nscd's negative cache entries when making > changes. You point out an alternative to making negative entries trusted only after they have been repeatedly entered into the cache: Tools that are used to modify the passwd/group databases might signal their changes to nscd. They could either purge the modified caches for the current or for all users, or they could just clear the single affected entry. In each case, nscd needs to re-fetch the modified (and possibly all other cached) entries. A simple implementation of such invalidation could be to invoke "nscd -I" after each modification performed by "pw". Still I think that the reduced trust in negative entries that have not been repeatedly tested is the best solution. I had looked into implementing such a logic in the cache, a few months back. It took more effort than I had hoped due to the way the cache is implemented, but I still think it should be possible without major changes to nscd. Regards, STefan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: using nscd (ldap) makes passwd/group disappearing while installing ports
On 01/02/2012, at 19:25, O. Hartmann wrote: >> The problem is that tools that modify the passwd and group files, like >> pw(8), don't invalidate nscd's negative cache entries when making >> changes. >> >> > > Thank you for the explanation. How feasible would it be for pw to try and notify nscd? Or for nscd to monitor the passwd & group files? Either would be somewhat racy though.. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: using nscd (ldap) makes passwd/group disappearing while installing ports
On 02/01/12 01:03, Benjamin Lee wrote: > On 01/31/2012 03:03 PM, O. Hartmann wrote: >> I'm using on a couple of servers the nameservice cache dameon nscd and >> cache "group", "passwd" and "sudoers". Backend is LDAP, but local files >> should searched first. then ldap. cache is searched the very first even >> before files. >> >> Well, I'd expect that if a group is present, like "cups" or "dhcp" and >> reside in the local file (/etc/group or /etc/passwd), they are cached. >> >> Installing net/isc-dhcp42-server fails with this error: >> >> >> gmake[1]: Leaving directory >> `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2/server' >> gmake[1]: Entering directory >> `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2' >> gmake[1]: Nothing to be done for `all-am'. >> gmake[1]: Leaving directory >> `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2' >> ===> Installing for isc-dhcp42-server-4.2.3_2 >> ===> Generating temporary packing list >> ===> Creating users and/or groups. >> Creating group `dhcpd' with gid `136'. >> pw: group disappeared during update >> *** Error code 70 >> >> Stop in /usr/ports/net/isc-dhcp42-server. >> *** Error code 1 >> >> Stop in /usr/ports/net/isc-dhcp42-server. > > What's going on is: > > 1) The port checks if the group exists > 2) nscd caches that the group does not exist in its negative cache > 3) pw(8) creates the group then checks if it exists > 4) nscd returns the negative cache entry (group does not exist) > > This causes pw(8) to error since it expects the group that it just > created to exist. > >> I also have this error very often when rebuilding/updating or even >> installing cups when "nscd" is enabled. A simple restart of nscd helps >> in most cases, most times I need to disable "cache" tag in >> /etc/nsswitch.conf, then everything runs smooth. >> >> Well, this behaviour is since a couple of years now, occurs sporadic. I >> have had in FreeBSD 7, 8, 9 and I see it in 10. What is it? >> >> I like the cache facility, since in domains with a lot of users >> searching LDAP takes some time and caching help keeping traffic and >> latency short. But the namservice caching mechanism seems to be >> unreliable. What is up there? > > You should put "files" before "cache" in /etc/nsswitch.conf, e.g.: > > group: files cache ldap > passwd: files cache ldap > > The problem is that tools that modify the passwd and group files, like > pw(8), don't invalidate nscd's negative cache entries when making > changes. > > Thank you for the explanation. Cheers, Oliver signature.asc Description: OpenPGP digital signature
Re: using nscd (ldap) makes passwd/group disappearing while installing ports
On 01/31/2012 03:03 PM, O. Hartmann wrote: > I'm using on a couple of servers the nameservice cache dameon nscd and > cache "group", "passwd" and "sudoers". Backend is LDAP, but local files > should searched first. then ldap. cache is searched the very first even > before files. > > Well, I'd expect that if a group is present, like "cups" or "dhcp" and > reside in the local file (/etc/group or /etc/passwd), they are cached. > > Installing net/isc-dhcp42-server fails with this error: > > > gmake[1]: Leaving directory > `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2/server' > gmake[1]: Entering directory > `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2' > gmake[1]: Nothing to be done for `all-am'. > gmake[1]: Leaving directory > `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2' > ===> Installing for isc-dhcp42-server-4.2.3_2 > ===> Generating temporary packing list > ===> Creating users and/or groups. > Creating group `dhcpd' with gid `136'. > pw: group disappeared during update > *** Error code 70 > > Stop in /usr/ports/net/isc-dhcp42-server. > *** Error code 1 > > Stop in /usr/ports/net/isc-dhcp42-server. What's going on is: 1) The port checks if the group exists 2) nscd caches that the group does not exist in its negative cache 3) pw(8) creates the group then checks if it exists 4) nscd returns the negative cache entry (group does not exist) This causes pw(8) to error since it expects the group that it just created to exist. > I also have this error very often when rebuilding/updating or even > installing cups when "nscd" is enabled. A simple restart of nscd helps > in most cases, most times I need to disable "cache" tag in > /etc/nsswitch.conf, then everything runs smooth. > > Well, this behaviour is since a couple of years now, occurs sporadic. I > have had in FreeBSD 7, 8, 9 and I see it in 10. What is it? > > I like the cache facility, since in domains with a lot of users > searching LDAP takes some time and caching help keeping traffic and > latency short. But the namservice caching mechanism seems to be > unreliable. What is up there? You should put "files" before "cache" in /etc/nsswitch.conf, e.g.: group: files cache ldap passwd: files cache ldap The problem is that tools that modify the passwd and group files, like pw(8), don't invalidate nscd's negative cache entries when making changes. -- Benjamin Lee http://www.b1c1l1.com/ signature.asc Description: OpenPGP digital signature
using nscd (ldap) makes passwd/group disappearing while installing ports
I'm using on a couple of servers the nameservice cache dameon nscd and cache "group", "passwd" and "sudoers". Backend is LDAP, but local files should searched first. then ldap. cache is searched the very first even before files. Well, I'd expect that if a group is present, like "cups" or "dhcp" and reside in the local file (/etc/group or /etc/passwd), they are cached. Installing net/isc-dhcp42-server fails with this error: gmake[1]: Leaving directory `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2/server' gmake[1]: Entering directory `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2' gmake[1]: Nothing to be done for `all-am'. gmake[1]: Leaving directory `/usr/ports/net/isc-dhcp42-server/work/dhcp-4.2.3-P2' ===> Installing for isc-dhcp42-server-4.2.3_2 ===> Generating temporary packing list ===> Creating users and/or groups. Creating group `dhcpd' with gid `136'. pw: group disappeared during update *** Error code 70 Stop in /usr/ports/net/isc-dhcp42-server. *** Error code 1 Stop in /usr/ports/net/isc-dhcp42-server. I also have this error very often when rebuilding/updating or even installing cups when "nscd" is enabled. A simple restart of nscd helps in most cases, most times I need to disable "cache" tag in /etc/nsswitch.conf, then everything runs smooth. Well, this behaviour is since a couple of years now, occurs sporadic. I have had in FreeBSD 7, 8, 9 and I see it in 10. What is it? I like the cache facility, since in domains with a lot of users searching LDAP takes some time and caching help keeping traffic and latency short. But the namservice caching mechanism seems to be unreliable. What is up there? signature.asc Description: OpenPGP digital signature
