Hi Ian I must apologize as I forgot to mention that I do in fact use browse mode. For the windows->Linux migration, it helps because we are moving to a case sensitive world, and people need to know the exact case of the mount points.
I will still try out submounts as you suggested to see if the mount storm is any better than the hierarchial mounts. Thanks once again. On Sat, Dec 7, 2019, 11:44 PM Ian Kent <ik...@redhat.com> wrote: > On Fri, 2019-12-06 at 07:34 -0500, Oguzhan Eris wrote: > > Thank you for thinking about this. I will investigate the submounts > > you suggested as well as the NFS junctions. > > > > I am on the latest centos/rhel 7.x line > > So you should have "browse_mode = no" in your installed configuration > unless you're nuked the default installed config and created a new one > from scratch without that option in a provisioning system or you retain > and old /etc/sysconfig/autofs that overrides it. > > So, supposing this isn't the case, unless you add a "browse" option > to mounts it should be disabled so that mount point directories aren't > pre-created for indirect mount map keys. > > In that case find(1) won't see the mount point directories and won't > be able to try and access them. > > > > > On Fri, Dec 6, 2019, 2:39 AM Ian Kent <ik...@redhat.com> wrote: > > > On Thu, 2019-12-05 at 23:09 -0500, Oguzhan Eris wrote: > > > > Thanks for the reply. > > > > My file based maps are as such: > > > > basically blank auto.master that includes auto.master.d > > > > /etc/auto.master.d/foo.autofs: > > > > /foo/x /etc/automap/x.txt > > > > /foo/x/dev /etc/automap/x_dev.txt > > > > foo/x/prod /etc/automap/x_prod.txt > > > > /foo/z /etc/automap/z.txt > > > > > > > > /etc/automap/x.txt: > > > > dir1 -fstype=nfs4 server1:/share_dir1 > > > > dir2 -fstype=nfs4 server2:/share_dir2 > > > > > > > > /etc/automap/x_dev.txt: > > > > dir3 -fstype=nfs4 devserver1:/share_dir3 > > > > dir4 -fstype=nfs4 devserver2:/share_dir4 > > > > > > > > /etc/automap/x_prod.txt: > > > > dir3 -fstype=nfs4 prodserver1:/share_dir3 > > > > dir4 -fstype=nfs4 prodserver2:/share_dir4 > > > > > > > > /etc/automap/z.txt: > > > > dir5 -fstype=nfs4 server5:/share_dir5 > > Following on from above this should be able to be done by using > autofs submounts so that offsets aren't used at all and together > with not enabling mount point browsing this should go a long way > to resolving the mount storm problem you see. > > Aside: stat family system calls aren't meant to trigger automounts > (really only applies to indirect mounts, direct mounts and offset > mounts always trigger mounts because of the way they behave) but > way back when this was done the statfs(2) system call was missed > and continued to trigger mounts. And IIRC find(1) uses statfs() > during tree traversal and we get mount storms. Now it's too late > to change that but there's also a case that statfs() should trigger > mounts (rather than requiring a trailing "/" as with making other > stat family calls trigger mounts) since a statfs() of an autofs fs > path is pretty much useless ... > > Now I haven't tested this at all but from the above this is what > I think might help. > > In the master map (auto.master or your master map fragment) use: > /foo /etc/automap/auto.foo > > where auto.foo is: > x --fstype=autofs /etc/automap/auto.x > z --fstype=autofs /etc/automap/auto.z > > and auto.x is: > dir1 -fstype=nfs4 server1:/share_dir1 > dir2 -fstype=nfs4 server2:/share_dir2 > dev --fstype=autofs /etc/automap/auto.x_dev > prod --fstype=autofs /etc/automap/auto.x_prod > > and auto.x_dev is: > dir3 -fstype=nfs4 devserver1:/share_dir3 > dir4 -fstype=nfs4 devserver2:/share_dir4 > > and auto.x_prod is: > dir3 -fstype=nfs4 prodserver1:/share_dir3 > dir4 -fstype=nfs4 prodserver2:/share_dir4 > > and finally auto.z is: > dir5 -fstype=nfs4 server5:/share_dir5 > > It's not as compact and simple as using offsets allone but it > should prevent the mount storm. > > Converting this to be stored in LDAP should be straight forward > too since you've already had to do that for some of the maps. > > Ian > > > > > > > Ok, thanks for that. > > > > > > > > > > > > > > > Such a map might not make a lot of sense if you were designing > > > from > > > > scratch, but what I'm actually doing is mapping our Windows DFS > > > > namespace "\\foo" in Active Directory where a lot of apps have > > > been > > > > written to use just one global namespace, to now be available > > > > under linux in the same way at /foo . This will greatly minimize > > > the > > > > effort when having these apps migrate and run under linux, or in > > > some > > > > situations, be completely cross platform apps with very minor > > > > settings to them to make them work under windows vs. linux > > > (mostly > > > > python apps) > > > > > > LOL, the why people do what they do is not really my place to > > > question. > > > The fact you feel you need to do it is enough for me. > > > But, for various reasons, it may need to be done a bit differently. > > > > > > > > > > > Anyway, so as I have it above, as long as my indirect map in > > > > foo.autofs is ordered the way I have above, then everything > > > works > > > > fine, and I will end up with: > > > > > > > > /foo/x > > > > /foo/x/dir1 > > > > /foo/x/dir2 > > > > /foo/x/dev/ > > > > /foo/x/dev/dir3 > > > > /foo/x/dev/dir4 > > > > /foo/x/prod/ > > > > /foo/x/prod/dir3 > > > > /foo/x/prod/dir4 > > > > /foo/z > > > > > > > > When I model this in LDAP (via sssd) however, I can see the maps > > > > perfectly fine in automap -m > > > > but the order in which they're returned results in some of the > > > parent > > > > paths masking their nested children. > > > > i.e. > > > > if /foo/x/dev gets mounted first, and then /foo/x/prod, > > > followed > > > > by /foo/x > > > > It will result in me seeing just > > > > /foo/x/dir1 > > > > /foo/x/dir2 > > > > > > > > The moment I umount /foo/x I will then see > > > > /foo/x/dev/... and /foo/x/prod/... > > > > > > > > Hopefully that makes sense. > > > > > > Yes, I got the idea of the problem you were having at the start. > > > > > > > > > > > I had previously done hierarchical maps with something like: > > > > > > > > auto.master > > > > /foo /etc/automount/foo.txt > > > > > > > > /etc/automount/foo.txt: > > > > x \ > > > > dir1 ... \ > > > > dir2 .... \ > > > > dev/dir3 ....\ > > > > dev/dir4 ... \ > > > > prod/dir3 ...\ > > > > prod/dir4 ... > > > > > > > > And that worked, but it had the huge burden that any traversal to > > > a > > > > path under /foo/x meant that ALL mounts got mounted at once, > > > which > > > > was quite undesirable. > > > > > > Yes, that can happen with this type of map entry. > > > > > > This sort of construct probably could be done using submounts > > > similar to the example I gave previously. > > > > > > I guess I should ask what version of autofs your using and on what > > > distribution? > > > > > > Do you set browse_mode in your autofs configuration? > > > > > > If browse_mode is set to "no" (if it's not present in the config > > > it's default is yes) then mount point directories in indirect > > > mounts won't be seen until mounted so they won't get mounted > > > when the tree is scanned ... but with that construct you have > > > above those offsets will behave like direct mounts which will > > > be present and will respond to scans. > > > > > > It may be possible to break this tree down to use submounts which > > > could prevent (at least some of the sensitivity) to the mount on > > > scan problem you have. > > > > > > I'll need to think about it. > > > > > > > > > > > Would you have a recommendation on how this might be better > > > authored > > > > in ldap? As I said, I must unfortunately follow the namespace > > > as > > > > is. > > > > > > LDAP, and NIS and NISPLUS all have no ordering guarantee. > > > > > > Strictly speaking I don't allow nesting in anything other than > > > those > > > hierarchical mount map entries but I don't explicitly check for it. > > > > > > And I don't really want to put that sort of a limitation into > > > autofs. > > > > > > But the problem happens all to often (with hierarchical map > > > entries) > > > that someone rips out an export in the midst of a tree that's in > > > use > > > and the mount ends up not behaving properly. Then, after the kernel > > > mounts list gets disjointed they demand to know why this happened > > > and I can't give an answer because something like "you did > > > something > > > you should have known not to do" doesn't go down all that well. > > > > > > Inevitably I can't reproduce these types of problem so I can't even > > > work out if I can improve things ... > > > > > > Also, while I'm tempted to just sort the master map list I'm pretty > > > sure that could break other functionality people might have come to > > > expect to behave a certain way. > > > > > > Consequently I would like to find another way to achieve this if we > > > can. I'll give that some thought. > > > > > > Your mounting NFS mounts, could you use the NFS cross-mount > > > automounting here? > > > > > > I'm pretty sure nfs-utils (but perhaps fairly recent releases) have > > > a utility to set basic mount junctions. Have you investigated this, > > > autofs should play well with the NFS automounting? > > > > > > I guess that takes away the benefit of being able to manage your > > > mounts in one place using one method ... > > > > > > I'll have a bit of a think about this one. > > > Ian > > > > > > > > > > > Thanks again > > > > > > > > > > > > > > > > > > > > On Wed, Dec 4, 2019 at 7:20 PM Ian Kent <ik...@redhat.com> wrote: > > > > > On Wed, 2019-12-04 at 11:36 +0100, Pavel Březina wrote: > > > > > > On 11/30/19 5:41 PM, Oguzhan Eris wrote: > > > > > > > Hi everyone. > > > > > > > > > > > > > > First off, thanks to everyone who's ever worked on SSSD. > > > It's > > > > > > > easily in my top 5 favorite FOSS projects out there. > > > > > > > > > > > > > > I am not sure if this is the right way to ask for an > > > > > enhancement, > > > > > > > or whether I should file an issue on GitHub, but I am > > > running > > > > > into > > > > > > > an issue that's described in this Red Hat page > > > > > > > https://access.redhat.com/solutions/3673501 (login > > > required) > > > > > > > > > > > > > > Basically for an automount map where I need nested top > > > level > > > > > paths: > > > > > > > > > > > > > > /mnt/foo > > > > > > > /mnt/foo/bar > > > > > > > > > > Those look like master map entries or perhaps direct mount map > > > > > entries. > > > > > > > > > > Which is it? > > > > > > > > > > In either case nesting mounts is not supported so sorting is > > > not > > > > > needed. > > > > > > > > > > Nesting of mounts can only be done by using mount map entries > > > > > with offsets. > > > > > > > > > > > > > > > > > > > each defined by their own map objects. SSSD does not > > > handle > > > > > this > > > > > > > (neither does LDAP directly from autofs) because the return > > > map > > > > > > > from LDAP is unsorted, and if the maps are provided to > > > autofs > > > > > > > ordered as: > > > > > > > > > > So each has their own map, but you haven't provided a complete > > > > > master map entry or map examples so I still can't tell what you > > > > > actually want to do. > > > > > > > > > > I'll assume these are master map entries ... and the maps are > > > > > called auto.foo and auto.bar, and they are indirect mount maps, > > > > > and I'll use file maps for the example. > > > > > > > > > > In the master map: > > > > > /mnt /etc/auto.foo > > > > > > > > > > In /etc/auto.foo: > > > > > > > > > > foo / foo.server.net:/path/to/foo > > > > > /bar -type=autofs /etc/auto.bar > > > > > > > > > > might be something like what you need to do. > > > > > > > > > > Allowing nesting in offset mounts this problematic enough, due > > > > > to the fact that mount dependency changes can (and do) cause > > > > > problems that are extremely hard to resolve, nesting won't be > > > > > supported in any other way. > > > > > > > > > > > > > > > > > > > /mnt/foo/bar > > > > > > > /mnt/foo > > > > > > > > > > > > > > the /mnt/foo map masks the previous /mnt/foo/bar map and > > > > > you'll > > > > > > > only get the entries from /mnt/foo > > > > > > > > > > > > > > Using file based mount maps, one can easily sort the top > > > level > > > > > > > maps, and get around this issue. > > > > > > > Would it be possible to have SSSD return the maps from LDAP > > > > > query > > > > > > > in a sorted way? There is an LDAP control that most LDAP > > > > > servers > > > > > > > support to return a sorted output, ( > > > > > > > https://ldapwiki.com/wiki/Server%20Side%20Sort%20Control ) > > > but > > > > > with > > > > > > > so many clients and a large list, this might be better left > > > to > > > > > the > > > > > > > client to do instead. > > > > > > > > > > > > > > I'm happy to help out if someone can point me in the right > > > > > > > direction in the code. > > > > > > > > > > > > SSSD is just a data provider, if some sorting is needed I do > > > not > > > > > > think > > > > > > it should be done in SSSD (unless autofs interface says so) > > > but > > > > > > rather > > > > > > in autofs itself. > > > > > > > > > > > > CCing Ian, do you have any thoughts on this? > > > > > > > > > > Thanks Pavel, some more specific information about the use case > > > > > might still be needed, we'll see. > > > > > > > > > > Ian > > > > > > > > > >
_______________________________________________ sssd-users mailing list -- sssd-users@lists.fedorahosted.org To unsubscribe send an email to sssd-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org
