[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On May 15, 1:23 pm, Steve Roberts strob...@strobe.net wrote: I've contacted the person at puppetlabs we have been working with to get his take on things. may even be part of a professional services implementation for him at this point. I hope that your contact will be able to help you work this out to your satisfaction. Especially in light of the fact that you are obtaining assistance from that channel, I don't think any additional contributions from me here would be useful. Best, John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On May 14, 9:05 pm, Steve Roberts strob...@strobe.net wrote:
On Apr 30, 6:10 am, jcbollinger john.bollin...@stjude.org wrote:
It is conceivable that providers for Windows or OS X, which use
different primary keys for their user and group databases, could be
given special parameters that would enable the behavior similar to
what you're looking for, based on their own unique keys, but even then
you wouldn't be identifying groups by GID per se.
well, that depends on what you mean by unix or linux. at the
system call level
there is ONLY UID's, gids (see 'man 2 chown' for example).
The kernel level is irrelevant here, because there are no system calls
for managing the user and group databases, nor indeed any concept of
user and group names at that level at all. Your issue is not directed
there.
going to
have to dust off my POSIX spec's to see what it guarantees. my old
school Unix admin handbook (the red book since it is second edition)
says UIDs have to be unique...
That's excellent advice, but regardless of POSIX, there is a wide
variety of real-world systems that do not enforce such a constraint,
and there are many machines that have users and/or groups with
duplicate IDs. Therefore POSIX, too, is irrelevant to the question.
In fact is has a specific warning to
make sure you keep the UID the same for all machines for a gieven
login otherwise you break things like NFS.
Indeed you do, and that's an area Puppet is good at dealing with.
Note that already there is the concept that the login name is the key
identifier -- UIDs *for a given login* should be the same across
machines.
However, I fully understand it is possible to have two entries in /etc/
passwd have the same UID but different login strings.
And likewise /etc/group records. That's the point. You cannot have
duplicate user or group *names* (or if you do then the system only
recognizes the first, more or less).
what I am wondering is why not support that mapping coded into a
puppet data structure? specify I want uid 542 to have a 'foo' and a
'bar' entry and then using the API functions make it so.
You mean like this?
user {
'foo': uid = 542, allowdupe = true;
'bar': uid = 542, allowdupe = true;
}
It should work on systems that in fact do allow duplicate UIDs.
That's not the problem you first asked about.
From a quick scan, looking like as far as POSIX and UNIX specs are
concerned the UID,GID is unique.
so unix implementations are just allowing the duplication in how they
manage the user database.
Again, that's largely irrelevant. In practice, it is not safe to rely
on UIDs or GIDs to be unique, therefore Puppet cannot safely use them
to identify user or group records.
I get that since puppet uses the useradd utilitiy instead of an API to
handle the user database it is currently limited to the functionality
exposed by that userland utility.
User and group data other than ID numbers are all userland concepts in
the first place, so it is natural that that Puppet's data model for
these is matched to userland. The specific tools Puppet uses on
various systems are really not the point. Puppet goes to considerable
effort to provide a system abstraction that is internally consistent
and that is as suitable as possible across a wide variety of systems.
User and group names are the identifiers that work for that. UID and
GID just aren't suitable.
My preference is to have it set the state to how I want it without
having to cleanup the mess on a box.
And here we're heading back toward DWIM. Puppet provides means to
achieve each task you have brought up, just not the particular means
you would prefer it to have. You already acknowledged that you
weren't expecting a DWIM oracle. If you give Puppet a configuration
description that doesn't match what you really want, then a need for
some sort of cleanup is likely.
It is unfortunate that you didn't realize in advance that the
description you were giving didn't match what you really wanted, but
that sort of problem is commonplace for anyone learning a new system
or language. Puppet has good and valid reasons for its data model
choices, and those are part of the package deal.
As far as an 'authoritative' flag goes, you are free to create a
custom user provider that offers such a thing as an optional feature.
You would also need to modify the User type to know about that
feature. In the end, however, the fact remains that GID simply
*isn't* an authoritative identifier for UNIX groups, even if you
induce Puppet act as if it were.
I have been pondering doing that. considering all of the unix/linux
flavor specific attributes that are in the User Resource one more
doesn't seem like such a huge hurdle.
It should be doable. Good luck.
John
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On May 15, 9:19 am, jcbollinger john.bollin...@stjude.org wrote:
That's excellent advice, but regardless of POSIX, there is a wide
variety of real-world systems that do not enforce such a constraint,
and there are many machines that have users and/or groups with
duplicate IDs. Therefore POSIX, too, is irrelevant to the question.
Well, sorry but you said unix before. so I thought you wanted to
actually look at what UNIX specifies. enough talking about unix specs
then.
Indeed you do, and that's an area Puppet is good at dealing with.
Note that already there is the concept that the login name is the key
identifier -- UIDs *for a given login* should be the same across
machines.
not exactly. it can push out new users,groups but it can't out of the
box make a given machine match the user,group manifest that is desired
by a system administrator. you have said this yourself in that puppet
will error out and you have to go and fix it.
what I am wondering is why not support that mapping coded into a
puppet data structure? specify I want uid 542 to have a 'foo' and a
'bar' entry and then using the API functions make it so.
You mean like this?
user {
'foo': uid = 542, allowdupe = true;
'bar': uid = 542, allowdupe = true;
}
nope. the other way around :) using the uid as a key. but anyways
was just a quick thought. not sure it would be practical to make that
change. so we can ignore that thought.
User and group data other than ID numbers are all userland concepts in
the first place, so it is natural that that Puppet's data model for
these is matched to userland. The specific tools Puppet uses on
various systems are really not the point. Puppet goes to considerable
effort to provide a system abstraction that is internally consistent
and that is as suitable as possible across a wide variety of systems.
User and group names are the identifiers that work for that. UID and
GID just aren't suitable.
I get that puppet has decided on using the account name and group name
as keys.
and yes userland (aka not kernel). but there is a difference between
having a program using API calls and calling an external binary.
think of it like it you are writing perl code to update a mysql
database. do you exec the mysql command line binary and screen scrape
the output? or do you use DBD::mysql via the DBI module?
if you use an API you get a MUCH better view of the environment and
can handle errors far better.
for example right now. just delete a line for a user out of /etc/
shadow that puppet is controlling. will it put it back? no. it
errors out. this isn't a duplicate userid case. this is unique
usernames just a missing line and poof. puppet (out of the box) can't
make the system match the manifest.
to repair you have to manually delete that user out of /etc/passwd
and /etc/group as well and then re-apply puppet.
That just feels VERY fragile to me.
My preference is to have it set the state to how I want it without
having to cleanup the mess on a box.
And here we're heading back toward DWIM. Puppet provides means to
achieve each task you have brought up, just not the particular means
you would prefer it to have. You already acknowledged that you
weren't expecting a DWIM oracle. If you give Puppet a configuration
description that doesn't match what you really want, then a need for
some sort of cleanup is likely.
actually no it really doesn't. bombing out with an error and
requiring manual patch up on potentially thousands of systems seems be
be a really bad lack of feature. Having bad operators that sometimes
do things they shouldn't is reality. If I was in a small shop that
had 2-3 admins sure it would be fine.
puppet is billed as a tool to define a manifest and then have that
manifest applied to a given target system and puppet will make the
system look like the manifest that was specified.
It is unfortunate that you didn't realize in advance that the
description you were giving didn't match what you really wanted, but
that sort of problem is commonplace for anyone learning a new system
or language. Puppet has good and valid reasons for its data model
choices, and those are part of the package deal.
I am not sure where you are coming up with this comment from.
You are missing the point if you think I am concerned about initial
typos
and glitches learning a system.
I am concerned about the robustness of the system and how much custom
coding we might need to do to get puppet to be able to keep a system
at a defined manifest. I've contacted the person at puppetlabs we
have been working with to get his take on things. may even be part of
a professional services implementation for him at this point.
Where are those design and data model decisions documented and
discussed? if you could point me at those docs that might help shed
light on why the puppet Resources look the way they do. The generated
docs at puppetlabs.com don't really don't give
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 30, 6:10 am, jcbollinger john.bollin...@stjude.org wrote: On Apr 27, 6:21 pm, Steve Roberts strob...@strobe.net wrote: Ah. No. You're still missing a critical aspect of this situation: independent of the system tools used (useradd, etc.), the UNIX architecture provides username / groupname as the ONLY unique key to the user and group databases. UIDs and GIDs don't cut it because they are not required to be distinct. As such, Puppet will never change the name of an existing user or group, at least on UNIX or Linux, because that would change it into a *different* user / group than the one being managed. Additionally, that presents a sticky problem if the user having the target GID is also managed under its actual name. To put it a different way, Puppet provides no way to express the idea the group with GID 42 should have the name 'answer' because it cannot safely rely on GID 42 to uniquely identify one group. It is conceivable that providers for Windows or OS X, which use different primary keys for their user and group databases, could be given special parameters that would enable the behavior similar to what you're looking for, based on their own unique keys, but even then you wouldn't be identifying groups by GID per se. well, that depends on what you mean by unix or linux. at the system call level there is ONLY UID's, gids (see 'man 2 chown' for example). going to have to dust off my POSIX spec's to see what it guarantees. my old school Unix admin handbook (the red book since it is second edition) says UIDs have to be unique... In fact is has a specific warning to make sure you keep the UID the same for all machines for a gieven login otherwise you break things like NFS. However, I fully understand it is possible to have two entries in /etc/ passwd have the same UID but different login strings. what I am wondering is why not support that mapping coded into a puppet data structure? specify I want uid 542 to have a 'foo' and a 'bar' entry and then using the API functions make it so. From a quick scan, looking like as far as POSIX and UNIX specs are concerned the UID,GID is unique. so unix implementations are just allowing the duplication in how they manage the user database. I get that since puppet uses the useradd utilitiy instead of an API to handle the user database it is currently limited to the functionality exposed by that userland utility. I don't understand what you're looking for. You started off dissatisfied that Puppet goes boom immediately, yet you don't want the system to go boom later, either. You understandably don't want to create groups with duplicate gids. What behavior would you actually like to see? Like I mentioned above a 'authoritative' type flag. You've taken that out of context. The point I was making was that it is preferable for Puppet to report an error at runtime than to silently ignore problems or exercise behaviors that risk causing a failure later. I did not intend to. I was trying to clarify what I was looking for when I first looked into how puppet handles users. Which wasn't my first puppet usage. We kind of did it opposite of what most folks do at my work. we had a basic confman system handling system stuff but our applications needed help. so we started with the applications and then went to the system stuff later. And yes I agree splitting out an error is better than ignoring the problem. Although from a runtime effect it is basically ignoring the issue. the puppet run completes and you have to look for the error to see it occurred. My preference is to have it set the state to how I want it without having to cleanup the mess on a box. As far as an 'authoritative' flag goes, you are free to create a custom user provider that offers such a thing as an optional feature. You would also need to modify the User type to know about that feature. In the end, however, the fact remains that GID simply *isn't* an authoritative identifier for UNIX groups, even if you induce Puppet act as if it were. I have been pondering doing that. considering all of the unix/linux flavor specific attributes that are in the User Resource one more doesn't seem like such a huge hurdle. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 27, 6:21 pm, Steve Roberts strob...@strobe.net wrote: On Apr 18, 6:46 am, jcbollinger john.bollin...@stjude.org wrote: On Apr 17, 5:04 pm,SteveRobertsstrob...@strobe.net wrote: On Apr 17, 6:25 am, jcbollinger john.bollin...@stjude.org wrote: well, allowdupe doesn't fix the issue only masks it. I knew about taht attribute but it just adds a duped group instead of making right the user/group. Indeed, but that's what you asked about. actually I was asking if there was an attribute that would override the other group with the same GID with the new group name. would require a provider that didn't just try to use 'useradd' but it could be written. something like a 'authoritative' attribute that would check for an existing user/group witht he spec'd UID/GID and replace it if it exists. the allowdupe would still be around if you want to just have an additional name with the same UID/GID. Ah. No. You're still missing a critical aspect of this situation: independent of the system tools used (useradd, etc.), the UNIX architecture provides username / groupname as the ONLY unique key to the user and group databases. UIDs and GIDs don't cut it because they are not required to be distinct. As such, Puppet will never change the name of an existing user or group, at least on UNIX or Linux, because that would change it into a *different* user / group than the one being managed. Additionally, that presents a sticky problem if the user having the target GID is also managed under its actual name. To put it a different way, Puppet provides no way to express the idea the group with GID 42 should have the name 'answer' because it cannot safely rely on GID 42 to uniquely identify one group. It is conceivable that providers for Windows or OS X, which use different primary keys for their user and group databases, could be given special parameters that would enable the behavior similar to what you're looking for, based on their own unique keys, but even then you wouldn't be identifying groups by GID per se. I don't understand what you're looking for. You started off dissatisfied that Puppet goes boom immediately, yet you don't want the system to go boom later, either. You understandably don't want to create groups with duplicate gids. What behavior would you actually like to see? Like I mentioned above a 'authoritative' type flag. You've taken that out of context. The point I was making was that it is preferable for Puppet to report an error at runtime than to silently ignore problems or exercise behaviors that risk causing a failure later. As far as an 'authoritative' flag goes, you are free to create a custom user provider that offers such a thing as an optional feature. You would also need to modify the User type to know about that feature. In the end, however, the fact remains that GID simply *isn't* an authoritative identifier for UNIX groups, even if you induce Puppet act as if it were. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 18, 6:46 am, jcbollinger john.bollin...@stjude.org wrote:
On Apr 17, 5:04 pm,SteveRobertsstrob...@strobe.net wrote:
On Apr 17, 6:25 am, jcbollinger john.bollin...@stjude.org wrote:
well, allowdupe doesn't fix the issue only masks it. I knew about
taht attribute but
it just adds a duped group instead of making right the user/group.
Indeed, but that's what you asked about.
actually I was asking if there was an attribute that would override
the other group with the same GID with the new group name.
would require a provider that didn't just try to use 'useradd' but it
could be written.
something like a 'authoritative' attribute that would check for an
existing user/group witht he spec'd UID/GID and replace it if it
exists.
the allowdupe would still be around if you want to just have an
additional name with the same UID/GID.
Well, I'm not worried about when the human has been told they can make
changes but rather when a human (or bad tool) makes a change nad it
slips through the cracks initially and goes boom later.
I don't understand what you're looking for. You started off
dissatisfied that Puppet goes boom immediately, yet you don't want the
system to go boom later, either. You understandably don't want to
create groups with duplicate gids. What behavior would you actually
like to see?
Like I mentioned above a 'authoritative' type flag.
thinking the .pp would have something like this in it:
user { 'tuser':
ensure = 'present',
comment = 'Test User',
gid = '400',
home = '/home/tuser',
password = '!!',
password_max_age = '9',
password_min_age = '0',
shell= '/bin/sh',
uid = '400',
authoritative = 'true',
}
and then when applied on a machine would check for uid,gid 400 and if
either were not 'tuser' it would get replaced.
Puppet gives the appearance of a lot of intelligence, but it has a
long way to go before it can pass a Turing test. Until then, it's
unreasonable to expect DWIM. :)
No not looking for DWIM. just a User class (and more importantly an
underlying provider) that has the options to do what I would like it
to do.
Puppet also has a mechanism by which you can ensure that otherwise-
unmanaged resources of some types are all ensured absent. That's both
very powerful and very dangerous, and I advise you to avoid it at
least until you have more experience with Puppet. To that end, I'm
leaving it as an exercise to determine just what the mechanism is.
I may have to poke on how puppet does that. we actually do an
absolute /etc/passwd (and friends) in our current conf man system.
and yes it does have its pitfalls too.
You can do that with Puppet as well, if you prefer, but you cannot
safely mix that approach with using User and Group resources.
understood. we will be migrating from our current configuration
management to puppet. problem one area at a time. don't worry we
won't be trying to have two different confman systems trying to manage
the same resources. ick that would be a mess.
Steve
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 17, 5:04 pm, Steve Roberts strob...@strobe.net wrote: On Apr 17, 6:25 am, jcbollinger john.bollin...@stjude.org wrote: well, allowdupe doesn't fix the issue only masks it. I knew about taht attribute but it just adds a duped group instead of making right the user/group. Indeed, but that's what you asked about. That is possible. That general class of problems is one of many reasons to avoid sharing management responsibilities for any given set of resources among multiple agents (including humans). If it does happen, then the failure is probably good, because it signals admins that there is something they need to investigate. Well, I'm not worried about when the human has been told they can make changes but rather when a human (or bad tool) makes a change nad it slips through the cracks initially and goes boom later. I don't understand what you're looking for. You started off dissatisfied that Puppet goes boom immediately, yet you don't want the system to go boom later, either. You understandably don't want to create groups with duplicate gids. What behavior would you actually like to see? Puppet gives the appearance of a lot of intelligence, but it has a long way to go before it can pass a Turing test. Until then, it's unreasonable to expect DWIM. :) Puppet also has a mechanism by which you can ensure that otherwise- unmanaged resources of some types are all ensured absent. That's both very powerful and very dangerous, and I advise you to avoid it at least until you have more experience with Puppet. To that end, I'm leaving it as an exercise to determine just what the mechanism is. I may have to poke on how puppet does that. we actually do an absolute /etc/passwd (and friends) in our current conf man system. and yes it does have its pitfalls too. You can do that with Puppet as well, if you prefer, but you cannot safely mix that approach with using User and Group resources. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 16, 10:03 pm, Steve Roberts strob...@strobe.net wrote:
I'm hoping I'm just missing something simple.
I ran across this in my deployment setup and have replicated in a
simple set of manifests. command output and the manifests below.
basically start with a user/group name but it had the name was
misspelled 'txuser'
so correct it to 'tuser' but puppet errors out saying the gid is a
dupe.
I thought one of the key ideas for puppet was the ability to define a
manifest and puppet would make the machine look like that manifest.
That is correct. Another key idea, however, is that Puppet does not
manage resources that are not declared in the target node's
manifests. Yet another is that it avoids doing unusual things without
specific instruction to do so. Occasionally two or more of these
clash.
In this case, you need to recognize that Puppet identifies groups (and
users) by their names, just as the OS's tools do. Indeed, the only
other alternative, gid (uid) is unsuitable because the OS does not
require it to be unique. When you change the name of one of a Group
declaration in one of your manifests, therefore, that doesn't
correspond to changing the name field of an existing group record on
the client. Instead, it corresponds to managing an entirely different
group, leaving the other unmanaged.
I checked the group resource documentation and didn't see an option to
say 'force' or something.
It is spelled allowdupe, but it would be better to clean up the mess
than to make a bigger one. Add this to your manifest to do so:
group { 'txuser':
ensure = 'absent',
before = Group['tuser']
}
I'm thinking would also be a problem if someone locally edited /etc/
passwd and /etc/group the manifest would also fail.
That is possible. That general class of problems is one of many
reasons to avoid sharing management responsibilities for any given set
of resources among multiple agents (including humans). If it does
happen, then the failure is probably good, because it signals admins
that there is something they need to investigate.
Puppet also has a mechanism by which you can ensure that otherwise-
unmanaged resources of some types are all ensured absent. That's both
very powerful and very dangerous, and I advise you to avoid it at
least until you have more experience with Puppet. To that end, I'm
leaving it as an exercise to determine just what the mechanism is.
John
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 17, 6:25 am, jcbollinger john.bollin...@stjude.org wrote:
On Apr 16, 10:03 pm, Steve Roberts strob...@strobe.net wrote:
I thought one of the key ideas for puppet was the ability to define a
manifest and puppet would make the machine look like that manifest.
That is correct. Another key idea, however, is that Puppet does not
manage resources that are not declared in the target node's
manifests. Yet another is that it avoids doing unusual things without
specific instruction to do so. Occasionally two or more of these
clash.
I follow you and I get it can be a hard problem to get right
In this case, you need to recognize that Puppet identifies groups (and
users) by their names, just as the OS's tools do. Indeed, the only
other alternative, gid (uid) is unsuitable because the OS does not
require it to be unique. When you change the name of one of a Group
declaration in one of your manifests, therefore, that doesn't
correspond to changing the name field of an existing group record on
the client. Instead, it corresponds to managing an entirely different
group, leaving the other unmanaged.
I follow it is using the useradd utils to manage things andso refs by
name.
I checked the group resource documentation and didn't see an option to
say 'force' or something.
It is spelled allowdupe, but it would be better to clean up the mess
than to make a bigger one. Add this to your manifest to do so:
well, allowdupe doesn't fix the issue only masks it. I knew about
taht attribute but
it just adds a duped group instead of making right the user/group.
group { 'txuser':
ensure = 'absent',
before = Group['tuser']
}
Yes for this specific case it would do it. and I actually have a tiny
manifest to do just that as I was checking the behavior of this test
case.
I'm thinking would also be a problem if someone locally edited /etc/
passwd and /etc/group the manifest would also fail.
That is possible. That general class of problems is one of many
reasons to avoid sharing management responsibilities for any given set
of resources among multiple agents (including humans). If it does
happen, then the failure is probably good, because it signals admins
that there is something they need to investigate.
Well, I'm not worried about when the human has been told they can make
changes but rather when a human (or bad tool) makes a change nad it
slips through the cracks initially and goes boom later.
Puppet also has a mechanism by which you can ensure that otherwise-
unmanaged resources of some types are all ensured absent. That's both
very powerful and very dangerous, and I advise you to avoid it at
least until you have more experience with Puppet. To that end, I'm
leaving it as an exercise to determine just what the mechanism is.
I may have to poke on how puppet does that. we actually do an
absolute /etc/passwd (and friends) in our current conf man system.
and yes it does have its pitfalls too.
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
