On Sun, 2019-12-08 at 10:04 -0500, Oguzhan Eris wrote: > Hi Ian > > I must apologize as I forgot to mention that I do in fact use browse > mode. For the windows->Linux migration, it helps because we are > moving to a case sensitive world, and people need to know the exact > case of the mount points.
Right but with it disabled in the configuration you have the option to add or remove the "browse" option in your master maps as you need. Personally I think leaving the global setting disabled and using the option as needed is a better choice. Unfortunately, if the mount point directories exist find will see them an poke its nose into them but OTOH seeing those directories is what you expect from a normal file system ... cant win ... > > I will still try out submounts as you suggested to see if the mount > storm is any better than the hierarchial mounts. > > Thanks once again. My pleasure. > > > On Sat, Dec 7, 2019, 11:44 PM Ian Kent <ik...@redhat.com> wrote: > > On Fri, 2019-12-06 at 07:34 -0500, Oguzhan Eris wrote: > > > Thank you for thinking about this. I will investigate the > > submounts > > > you suggested as well as the NFS junctions. > > > > > > I am on the latest centos/rhel 7.x line > > > > So you should have "browse_mode = no" in your installed > > configuration > > unless you're nuked the default installed config and created a new > > one > > from scratch without that option in a provisioning system or you > > retain > > and old /etc/sysconfig/autofs that overrides it. > > > > So, supposing this isn't the case, unless you add a "browse" option > > to mounts it should be disabled so that mount point directories > > aren't > > pre-created for indirect mount map keys. > > > > In that case find(1) won't see the mount point directories and > > won't > > be able to try and access them. > > > > > > > > On Fri, Dec 6, 2019, 2:39 AM Ian Kent <ik...@redhat.com> wrote: > > > > On Thu, 2019-12-05 at 23:09 -0500, Oguzhan Eris wrote: > > > > > Thanks for the reply. > > > > > My file based maps are as such: > > > > > basically blank auto.master that includes auto.master.d > > > > > /etc/auto.master.d/foo.autofs: > > > > > /foo/x /etc/automap/x.txt > > > > > /foo/x/dev /etc/automap/x_dev.txt > > > > > foo/x/prod /etc/automap/x_prod.txt > > > > > /foo/z /etc/automap/z.txt > > > > > > > > > > /etc/automap/x.txt: > > > > > dir1 -fstype=nfs4 server1:/share_dir1 > > > > > dir2 -fstype=nfs4 server2:/share_dir2 > > > > > > > > > > /etc/automap/x_dev.txt: > > > > > dir3 -fstype=nfs4 devserver1:/share_dir3 > > > > > dir4 -fstype=nfs4 devserver2:/share_dir4 > > > > > > > > > > /etc/automap/x_prod.txt: > > > > > dir3 -fstype=nfs4 prodserver1:/share_dir3 > > > > > dir4 -fstype=nfs4 prodserver2:/share_dir4 > > > > > > > > > > /etc/automap/z.txt: > > > > > dir5 -fstype=nfs4 server5:/share_dir5 > > > > Following on from above this should be able to be done by using > > autofs submounts so that offsets aren't used at all and together > > with not enabling mount point browsing this should go a long way > > to resolving the mount storm problem you see. > > > > Aside: stat family system calls aren't meant to trigger automounts > > (really only applies to indirect mounts, direct mounts and offset > > mounts always trigger mounts because of the way they behave) but > > way back when this was done the statfs(2) system call was missed > > and continued to trigger mounts. And IIRC find(1) uses statfs() > > during tree traversal and we get mount storms. Now it's too late > > to change that but there's also a case that statfs() should trigger > > mounts (rather than requiring a trailing "/" as with making other > > stat family calls trigger mounts) since a statfs() of an autofs fs > > path is pretty much useless ... > > > > Now I haven't tested this at all but from the above this is what > > I think might help. > > > > In the master map (auto.master or your master map fragment) use: > > /foo /etc/automap/auto.foo > > > > where auto.foo is: > > x --fstype=autofs /etc/automap/auto.x > > z --fstype=autofs /etc/automap/auto.z > > > > and auto.x is: > > dir1 -fstype=nfs4 server1:/share_dir1 > > dir2 -fstype=nfs4 server2:/share_dir2 > > dev --fstype=autofs /etc/automap/auto.x_dev > > prod --fstype=autofs /etc/automap/auto.x_prod > > > > and auto.x_dev is: > > dir3 -fstype=nfs4 devserver1:/share_dir3 > > dir4 -fstype=nfs4 devserver2:/share_dir4 > > > > and auto.x_prod is: > > dir3 -fstype=nfs4 prodserver1:/share_dir3 > > dir4 -fstype=nfs4 prodserver2:/share_dir4 > > > > and finally auto.z is: > > dir5 -fstype=nfs4 server5:/share_dir5 > > > > It's not as compact and simple as using offsets allone but it > > should prevent the mount storm. > > > > Converting this to be stored in LDAP should be straight forward > > too since you've already had to do that for some of the maps. > > > > Ian > > > > > > > > > > Ok, thanks for that. > > > > > > > > > > > > > > > > > > > Such a map might not make a lot of sense if you were > > designing > > > > from > > > > > scratch, but what I'm actually doing is mapping our Windows > > DFS > > > > > namespace "\\foo" in Active Directory where a lot of apps > > have > > > > been > > > > > written to use just one global namespace, to now be available > > > > > under linux in the same way at /foo . This will greatly > > minimize > > > > the > > > > > effort when having these apps migrate and run under linux, or > > in > > > > some > > > > > situations, be completely cross platform apps with very minor > > > > > settings to them to make them work under windows vs. linux > > > > (mostly > > > > > python apps) > > > > > > > > LOL, the why people do what they do is not really my place to > > > > question. > > > > The fact you feel you need to do it is enough for me. > > > > But, for various reasons, it may need to be done a bit > > differently. > > > > > > > > > > > > > > Anyway, so as I have it above, as long as my indirect map in > > > > > foo.autofs is ordered the way I have above, then everything > > > > works > > > > > fine, and I will end up with: > > > > > > > > > > /foo/x > > > > > /foo/x/dir1 > > > > > /foo/x/dir2 > > > > > /foo/x/dev/ > > > > > /foo/x/dev/dir3 > > > > > /foo/x/dev/dir4 > > > > > /foo/x/prod/ > > > > > /foo/x/prod/dir3 > > > > > /foo/x/prod/dir4 > > > > > /foo/z > > > > > > > > > > When I model this in LDAP (via sssd) however, I can see the > > maps > > > > > perfectly fine in automap -m > > > > > but the order in which they're returned results in some of > > the > > > > parent > > > > > paths masking their nested children. > > > > > i.e. > > > > > if /foo/x/dev gets mounted first, and then /foo/x/prod, > > > > followed > > > > > by /foo/x > > > > > It will result in me seeing just > > > > > /foo/x/dir1 > > > > > /foo/x/dir2 > > > > > > > > > > The moment I umount /foo/x I will then see > > > > > /foo/x/dev/... and /foo/x/prod/... > > > > > > > > > > Hopefully that makes sense. > > > > > > > > Yes, I got the idea of the problem you were having at the > > start. > > > > > > > > > > > > > > I had previously done hierarchical maps with something like: > > > > > > > > > > auto.master > > > > > /foo /etc/automount/foo.txt > > > > > > > > > > /etc/automount/foo.txt: > > > > > x \ > > > > > dir1 ... \ > > > > > dir2 .... \ > > > > > dev/dir3 ....\ > > > > > dev/dir4 ... \ > > > > > prod/dir3 ...\ > > > > > prod/dir4 ... > > > > > > > > > > And that worked, but it had the huge burden that any > > traversal to > > > > a > > > > > path under /foo/x meant that ALL mounts got mounted at once, > > > > which > > > > > was quite undesirable. > > > > > > > > Yes, that can happen with this type of map entry. > > > > > > > > This sort of construct probably could be done using submounts > > > > similar to the example I gave previously. > > > > > > > > I guess I should ask what version of autofs your using and on > > what > > > > distribution? > > > > > > > > Do you set browse_mode in your autofs configuration? > > > > > > > > If browse_mode is set to "no" (if it's not present in the > > config > > > > it's default is yes) then mount point directories in indirect > > > > mounts won't be seen until mounted so they won't get mounted > > > > when the tree is scanned ... but with that construct you have > > > > above those offsets will behave like direct mounts which will > > > > be present and will respond to scans. > > > > > > > > It may be possible to break this tree down to use submounts > > which > > > > could prevent (at least some of the sensitivity) to the mount > > on > > > > scan problem you have. > > > > > > > > I'll need to think about it. > > > > > > > > > > > > > > Would you have a recommendation on how this might be better > > > > authored > > > > > in ldap? As I said, I must unfortunately follow the > > namespace > > > > as > > > > > is. > > > > > > > > LDAP, and NIS and NISPLUS all have no ordering guarantee. > > > > > > > > Strictly speaking I don't allow nesting in anything other than > > > > those > > > > hierarchical mount map entries but I don't explicitly check for > > it. > > > > > > > > And I don't really want to put that sort of a limitation into > > > > autofs. > > > > > > > > But the problem happens all to often (with hierarchical map > > > > entries) > > > > that someone rips out an export in the midst of a tree that's > > in > > > > use > > > > and the mount ends up not behaving properly. Then, after the > > kernel > > > > mounts list gets disjointed they demand to know why this > > happened > > > > and I can't give an answer because something like "you did > > > > something > > > > you should have known not to do" doesn't go down all that well. > > > > > > > > Inevitably I can't reproduce these types of problem so I can't > > even > > > > work out if I can improve things ... > > > > > > > > Also, while I'm tempted to just sort the master map list I'm > > pretty > > > > sure that could break other functionality people might have > > come to > > > > expect to behave a certain way. > > > > > > > > Consequently I would like to find another way to achieve this > > if we > > > > can. I'll give that some thought. > > > > > > > > Your mounting NFS mounts, could you use the NFS cross-mount > > > > automounting here? > > > > > > > > I'm pretty sure nfs-utils (but perhaps fairly recent releases) > > have > > > > a utility to set basic mount junctions. Have you investigated > > this, > > > > autofs should play well with the NFS automounting? > > > > > > > > I guess that takes away the benefit of being able to manage > > your > > > > mounts in one place using one method ... > > > > > > > > I'll have a bit of a think about this one. > > > > Ian > > > > > > > > > > > > > > Thanks again > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Dec 4, 2019 at 7:20 PM Ian Kent <ik...@redhat.com> > > wrote: > > > > > > On Wed, 2019-12-04 at 11:36 +0100, Pavel Březina wrote: > > > > > > > On 11/30/19 5:41 PM, Oguzhan Eris wrote: > > > > > > > > Hi everyone. > > > > > > > > > > > > > > > > First off, thanks to everyone who's ever worked on > > SSSD. > > > > It's > > > > > > > > easily in my top 5 favorite FOSS projects out there. > > > > > > > > > > > > > > > > I am not sure if this is the right way to ask for an > > > > > > enhancement, > > > > > > > > or whether I should file an issue on GitHub, but I am > > > > running > > > > > > into > > > > > > > > an issue that's described in this Red Hat page > > > > > > > > https://access.redhat.com/solutions/3673501 (login > > > > required) > > > > > > > > > > > > > > > > Basically for an automount map where I need nested top > > > > level > > > > > > paths: > > > > > > > > > > > > > > > > /mnt/foo > > > > > > > > /mnt/foo/bar > > > > > > > > > > > > Those look like master map entries or perhaps direct mount > > map > > > > > > entries. > > > > > > > > > > > > Which is it? > > > > > > > > > > > > In either case nesting mounts is not supported so sorting > > is > > > > not > > > > > > needed. > > > > > > > > > > > > Nesting of mounts can only be done by using mount map > > entries > > > > > > with offsets. > > > > > > > > > > > > > > > > > > > > > > each defined by their own map objects. SSSD does not > > > > handle > > > > > > this > > > > > > > > (neither does LDAP directly from autofs) because the > > return > > > > map > > > > > > > > from LDAP is unsorted, and if the maps are provided to > > > > autofs > > > > > > > > ordered as: > > > > > > > > > > > > So each has their own map, but you haven't provided a > > complete > > > > > > master map entry or map examples so I still can't tell what > > you > > > > > > actually want to do. > > > > > > > > > > > > I'll assume these are master map entries ... and the maps > > are > > > > > > called auto.foo and auto.bar, and they are indirect mount > > maps, > > > > > > and I'll use file maps for the example. > > > > > > > > > > > > In the master map: > > > > > > /mnt /etc/auto.foo > > > > > > > > > > > > In /etc/auto.foo: > > > > > > > > > > > > foo / foo.server.net:/path/to/foo > > > > > > /bar -type=autofs /etc/auto.bar > > > > > > > > > > > > might be something like what you need to do. > > > > > > > > > > > > Allowing nesting in offset mounts this problematic enough, > > due > > > > > > to the fact that mount dependency changes can (and do) > > cause > > > > > > problems that are extremely hard to resolve, nesting won't > > be > > > > > > supported in any other way. > > > > > > > > > > > > > > > > > > > > > > /mnt/foo/bar > > > > > > > > /mnt/foo > > > > > > > > > > > > > > > > the /mnt/foo map masks the previous /mnt/foo/bar map > > and > > > > > > you'll > > > > > > > > only get the entries from /mnt/foo > > > > > > > > > > > > > > > > Using file based mount maps, one can easily sort the > > top > > > > level > > > > > > > > maps, and get around this issue. > > > > > > > > Would it be possible to have SSSD return the maps from > > LDAP > > > > > > query > > > > > > > > in a sorted way? There is an LDAP control that most > > LDAP > > > > > > servers > > > > > > > > support to return a sorted output, ( > > > > > > > > > > https://ldapwiki.com/wiki/Server%20Side%20Sort%20Control ) > > > > but > > > > > > with > > > > > > > > so many clients and a large list, this might be better > > left > > > > to > > > > > > the > > > > > > > > client to do instead. > > > > > > > > > > > > > > > > I'm happy to help out if someone can point me in the > > right > > > > > > > > direction in the code. > > > > > > > > > > > > > > SSSD is just a data provider, if some sorting is needed I > > do > > > > not > > > > > > > think > > > > > > > it should be done in SSSD (unless autofs interface says > > so) > > > > but > > > > > > > rather > > > > > > > in autofs itself. > > > > > > > > > > > > > > CCing Ian, do you have any thoughts on this? > > > > > > > > > > > > Thanks Pavel, some more specific information about the use > > case > > > > > > might still be needed, we'll see. > > > > > > > > > > > > Ian > > > > > > > > > > > > _______________________________________________ sssd-users mailing list -- sssd-users@lists.fedorahosted.org To unsubscribe send an email to sssd-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org
