Re: [Puppet Users] Using file and mount more efficiently
On Tuesday, October 15, 2013 3:21:42 PM UTC-5, Forrie wrote: > > Thanks for the reference, John. > > We need to ensure that these remote mounts are owned/grouped by specific > UID/GID -- hence why I had ownership involved there. We could do this via > UID/GID only (not name) if that works better? I don't understand how > apply that ownership to /home/201301 would affect / or /home. > > Managing /home/201301 etc. does not itself affect /home, but the definition you presented attempted to manage /home separately along with each of its managed subdirectories. That's what caused your duplicate declaration problem when you attempted to declare multiple instances of that defined type. It's actually a pretty common breakage pattern -- when you define a type that must support multiple instances, it must declare only resources that are different for each defined type instance, but people seem to want to include declarations of common resources on which all the instances rely. Those common resources need to be factored out so that they are declared only once globally, instead of once for each defined type instance. If you want to manage the uid/gid/permissions of the mounted remote filesystem, then you should manage them on the host that exports them, not on the client machines. The remote uid/gid/permissions are the ones that will be presented on client machines. On the other hand, you do not need to be concerned with the properties of the underlying mount point directory because *they are invisible and do not matter at all when the remote file system is mounted*. > Then, Puppet would need to check that it's present, has the correct > permissions and owership, and ensure it's mounted -- or, in the case of > aged data, not mounted and not present. > > In the event that you want the remote mount to be present, you need to account for two basic cases: 1. The remote file system is initially unmounted (regardless of the presence or properties of its target mount point directory), and 2. The remote file system is initially mounted (and therefore its mount point directory is present). Because the mount point directory must be present in order for anything to be mounted on it, you must instruct Puppet to manage the corresponding File resource before it manages the Mount resource. But when you apply that File (e.g. File['/home/201301']) in case (1) you are managing the mount point directory, whereas when you apply it in case (2) you are managing the remote file system root. You cannot do both in one run with one resource. Moreover, the uid/gid/permissions do not matter in case (1) because, I repeat, *they are invisible and do not matter at all when the remote file system is mounted*. At the same time, NFS clients cannot modify uid/gid/permissions in case (2) if the NFS server performs root squashing, which it should. Thus, the File resources for the mount point directories should manage only their presence as directories, not uid/gid/permission. That will work in both cases, doesn't do any unnecessary work in case (1), and doesn't attempt anything that cannot reliably be done in case (2). The uid/gid/permissions need to be managed on the exporting host. The other direction, where you want the mount absent, is easier at least. You just ensure the Mount absent and ensure the File absent, but you must be certain to do it in that order (the reverse of the order needed when you want the mount present). John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Using file and mount more efficiently
Thanks for the reference, John. We need to ensure that these remote mounts are owned/grouped by specific UID/GID -- hence why I had ownership involved there. We could do this via UID/GID only (not name) if that works better? I don't understand how apply that ownership to /home/201301 would affect / or /home. Then, Puppet would need to check that it's present, has the correct permissions and owership, and ensure it's mounted -- or, in the case of aged data, not mounted and not present. Thanks. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Using file and mount more efficiently
On Monday, September 23, 2013 7:15:32 PM UTC-5, Forrie wrote:
>
> I've been playing around with this code and have encountered several
> errors. As noted below, there is going to be an issue with /home;
> however, I thought I could get around that by declaring that /first/, which
> won't work -- as it complains about duplicate declarations of /home.
>
>
As it should. More on that below.
>
> class nfs_mounts_prod {
>
> define nfs_mounts {
>
As a matter of style and good practice, do not lexically nest classes or
definitions inside classes. Put them in separate files.
Also, put all your classes and definitions into modules. Even if you have
a bunch of local one-offs that don't otherwise go together, either put them
in their own modules or create a grab-bag "site" module to house them.
Neither of those is the source of your errors, but structuring your
manifests well may help clarify the issues.
>
> $server = "ourserver.com"
> $options =
> "tcp,rw,hard,intr,vers=3,tcp,rsize=32768,wsize=32768,bg"
>
> # These needed to be defined here, it would not work
> outside of the class definition
> $prod_mounts = [
> '201301',
> '201301pod',
> ]
>
> file { "/home":
> ensure => directory,
> owner => "root",
> group => "root",
> mode => "0755",
> }
>
>
Why are all those variables and File['/home'] declared inside your
definition, when they do not depend in any way on the properties of any
instance of the defined type? They belong directly in the containing
class, instead. Although it's only a little redundant to put the variable
declarations in the definition, putting the File['/home'] there is what
causes your duplicate declaration errors, as you get one declaration of
that resource for every declared instance of the defined type in which it
resides.
> file { "/home/${name}":
> ensure => directory,
> owner => "16326",
> group => "90",
> mode => "0755",
> require => File["/home"],
> } # file
>
> mount { "/home/${name}":
> device => "${server}:/export/prod/${name}",
> atboot => yes,
> fstype => nfs,
> options => "${options}",
> name => "/home/${name}",
> ensure => mounted,
> remounts => true,
> pass => "0",
> require => File["/home/${name}"],
> } # mount
>
>
That's just broken. As I've been saying, it is seriously problematic to
manage a mount point directory, because what the target path means to the
OS depends on whether the filesystem is mounted on it. Moreover, if your
NFS setup is reasonably secure then you will have trouble getting Puppet to
manage anything about the remote filesystem. This is because the NFS
server will perform root squashing with respect to most or all clients, so
that local root on NFS client systems is mapped to a different,
unprivileged user for the NFS server's purposes.
I also think it's a poor plan to mount each user's home directory
separately. Why not just mount :/export/prod on local /home? That
will be a lot easier on you.
> } # nfs_mounts
>
> nfs_mounts { $prod_mounts: }
>
> } # class nfs_mounts_prod
>
>
> Can you tell me what's wrong -- or if this is even going to work :-)
>
>
Here's a better starting point:
modules/prod/manifests/params.pp:
class prod::params {
$nfs_server = "ourserver.com"
$nfs_options = "tcp,rw,hard,intr,vers=3,tcp,rsize=32768,wsize=32768,bg"
}
modules/prod/manifests/nfs_mounts.pp:
class prod::nfs_mounts {
file { "/home":
ensure => directory,
owner => "root",
group => "root",
mode => "0755",
}
prod::nfs_homedir { [
'201301',
'201301pod',
]:
}
}
modules/prod/manifests/nfs_homedir.pp:
define prod::nfs_homedir {
include 'prod::params'
file { "/home/${name}":
ensure => 'directory'
# Do not manage owner or permissions because this is a
# mount point / remote directory.
#
# We can rely on autorequires to make Puppet manage
# the parent directory first (if it is under management, which
# it is.
}
mount { "/home/${name}":
device => "${prod::params::nfs_server}:/export/prod/${name}",
atboot => yes,
fstype => nfs,
options => "${prod::params::nfs_options}",
ensure => mounted,
remounts => true,
pass => "0",
require => File["/home/${name}"],
} # mount
}
John
--
You received this mess
Re: [Puppet Users] Using file and mount more efficiently
I've been playing around with this code and have encountered several
errors. As noted below, there is going to be an issue with /home;
however, I thought I could get around that by declaring that /first/, which
won't work -- as it complains about duplicate declarations of /home.
class nfs_mounts_prod {
define nfs_mounts {
$server = "ourserver.com"
$options =
"tcp,rw,hard,intr,vers=3,tcp,rsize=32768,wsize=32768,bg"
# These needed to be defined here, it would not work
outside of the class definition
$prod_mounts = [
'201301',
'201301pod',
]
file { "/home":
ensure => directory,
owner => "root",
group => "root",
mode => "0755",
}
file { "/home/${name}":
ensure => directory,
owner => "16326",
group => "90",
mode => "0755",
require => File["/home"],
} # file
mount { "/home/${name}":
device => "${server}:/export/prod/${name}",
atboot => yes,
fstype => nfs,
options => "${options}",
name => "/home/${name}",
ensure => mounted,
remounts => true,
pass => "0",
require => File["/home/${name}"],
} # mount
} # nfs_mounts
nfs_mounts { $prod_mounts: }
} # class nfs_mounts_prod
Can you tell me what's wrong -- or if this is even going to work :-)
Thanks.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Using file and mount more efficiently
... and $name is the default variable given to you by puppet, so you don't
have to define it. Works like $_ in perl.
On Wed, Aug 21, 2013 at 7:18 PM, Peter Bukowinski wrote:
> You define an array-containing variable like this:
>
> $mounts = [ 'directory1', 'directory2', 'directory3' ]
>
> You can also put newlines after the commas for easier reading. The
> following code should be functional:
>
> class test_case {
> $mounts = [
> 'directory1',
> 'directory2',
> 'directory3',
> ]
> define my_mounts {
> file { "/home/${name}":
> ensure => directory,
> owner => "$name",
> mode => "0755",
> }
> mount { "/home/${name}":
> device => "our-thumper.domain.com:/export/${name}",
>
> atboot => yes,
> fstype => "nfs",
> options => "tcp,hard,intr,rw,bg",
> name => "/home/${name}",
>
> ensure => mounted,
> remounts => true,
> pass => "0",
> require => File["/home/${name}"],
> }
> }
> my_mounts { $mounts: }
> }
>
> Of course, instead of defining the $mounts variable in the manifest
> itself, you can get that array via a custom fact or from hiera.
>
> --
> Peter
>
> On Aug 21, 2013, at 2:55 PM, Forrie wrote:
>
> So I would need to define $mounts, presumably as:
>
> $mounts = "directory1 directory2 directory3"
>
> ?
>
> Where is $name being defined here.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Using file and mount more efficiently
You define an array-containing variable like this:
$mounts = [ 'directory1', 'directory2', 'directory3' ]
You can also put newlines after the commas for easier reading. The following
code should be functional:
class test_case {
$mounts = [
'directory1',
'directory2',
'directory3',
]
define my_mounts {
file { "/home/${name}":
ensure => directory,
owner => "$name",
mode => "0755",
}
mount { "/home/${name}":
device => "our-thumper.domain.com:/export/${name}",
atboot => yes,
fstype => "nfs",
options => "tcp,hard,intr,rw,bg",
name => "/home/${name}",
ensure => mounted,
remounts => true,
pass => "0",
require => File["/home/${name}"],
}
}
my_mounts { $mounts: }
}
Of course, instead of defining the $mounts variable in the manifest itself, you
can get that array via a custom fact or from hiera.
--
Peter
On Aug 21, 2013, at 2:55 PM, Forrie wrote:
> So I would need to define $mounts, presumably as:
>
> $mounts = "directory1 directory2 directory3"
>
> ?
>
> Where is $name being defined here.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Using file and mount more efficiently
So I would need to define $mounts, presumably as: $mounts = "directory1 directory2 directory3" ? Where is $name being defined here. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Using file and mount more efficiently
You can fake interation. "$name" is a free variable for whatever you're
passing in. I have NOT tested this, but it might look something like this:
define my_mounts {
mount { "/home/$name":
>
>device => "our-thumper.domain.com:/export/$name",
atboot => yes,
fstype => "nfs",
options => "tcp,hard,intr,rw,bg",
name => "/home/$name",
ensure => mounted,
remounts => true,
pass => "0",
require => File["/home/$name"],
}
>
>
}
Then call it with:
my_mounts { $mounts: }
On Wed, Aug 21, 2013 at 2:39 PM, Forrie wrote:
> I have several NFS mounts to manage, on many systems. On each system, I
> must ensure that the root directory and path exist and have the correct
> permissions beforehand, then ensure they are mounted in Puppet.
>
> For each, I would normally do:
>
> file { "/home/directory1":
>>
>>ensure => directory,
>>
>>owner => "user",
>>
>>group => "group",
>>
>>mode => "755",
>>
>>}
>>
>>
>>>mount { "/home/directory1":
>>
>>device => "our-thumper.domain.com:/export/directory1",
>>
>>atboot => yes,
>>
>>fstype => "nfs",
>>
>>options => "tcp,hard,intr,rw,bg",
>>
>>name => "/home/directory1",
>>
>>ensure => mounted,
>>
>>remounts => true,
>>
>>pass => "0",
>>
>>require => File["/home/directory1"],
>>
>>}
>>
>>
>
> which isn't very efficient when you have a ton of them to mange.
>
> It doesn't appear that Puppet can iterate through an array, but could I do
> something like:
>
> file { "/home/directory1", "/home/directory2", "/home/directory3":
>
> but requiring this File will be a problem in the "mount" pass.
>
> Perhaps a template?
>
> How are others solving this sort of problem? For me, it would be a lot
> easier (and more readable) if we could maintain an array at the top of the
> rules that contained either the full patch or the basename, then iterate
> through them.
>
>
>
> _F
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
