Re: [gentoo-dev] Determenistic system group and user id

2015-12-14 Thread Alec Warner 
On Sun, Dec 13, 2015 at 10:39 PM, Robin H. Johnson 
wrote:

> On Sun, Dec 13, 2015 at 02:23:29PM -0800, Alec Warner wrote:
> > 1) Why do you need deterministic uid / gid's?
> > 2) If you do need deterministic uid / gid's, I would recommend storing
> them
> > all in the same place.
> They are ALL for system users and groups.
>

What does that mean though? What is a System user or a System group?
Why should you not store them in the user / group directory?


>
> TL;DR: if you're sharing data/config for system users/groups between
> multiple systems based on UID/GID (not username), you need consistent
> generation.
>

> Data on NFSv[23], with a shared apache/nginx user was one of the
> original examples. I agree since then, that the data should NOT be owned
> by apache/nginx anymore (NFSv4 also solves the problem).
>

This is certainly one workaround (you generate a shard user / role user
that is the same uid / gid on every machine and you make that own the data
/ run the service.) But you might as well just add apache to LDAP and call
it a day?


>
> A much newer example, is let's consider the system group 'plugdev'. It's
> one that is created dynamically at the moment.
>
> If I want to put my user in that group LDAP-wide, and have an LDAP
> environment, I need to make sure the plugdev GID is the same on all
> systems (actually it also varies slightly depending which LDAP group
> membership model you're using for NSS data).
>
> > For example, you typically want a deterministic UID for a user. To
> > accomplish this, you add that user to LDAP, give them a UID in LDAP, and
> > then either add LDAP to nssswitch or use something like nsscache to sync
> > the ldap UID's into the local system.
> >
> > 3) If you need deterministic GID's I would recommend storing them all in
> > LDAP and syncing the group memberships locally.
> So you want to define the group twice? Both in LDAP and locally?
>

Sorry, when I said sync I meant nsscache or similar. I understand why that
would be confusing ;)


>
> > I never understood why people would think the distro should handle unique
> > gid / uids. Plus you usually end up running:
> >
> > 1) More than one distro.
> > 2) More than one 'flavor' of a single distro where for whatever reason,
> uid
> > and gid decisions differed (they renumbered, etc.)
> Here's the work LSB did on it, with further references to what more
> distros do:
>
> https://github.com/LinuxStandardBase/lsb/blob/master/documents/wip/userNaming.txt
>
> Here's the debian central database for it:
> https://anonscm.debian.org/cgit/users/cjwatson/base-passwd.git/tree/README
>
>
I'm not opposed to this effort, simply that getting everyone to conform is
tricky and in complex environments you will inevitably end up in situations
outside of the scope of the effort. I prefer practicality ;)


> > So if you want a consistent GID for a group, store the group name and gid
> > in ldap and sync it; do not rely on your distro to do it. IMHO doing so
> is
> > a design error.
> Which is incompatible with NFSv3.
>

How so? (to be clear, most of my experience is with sec=krb5 nfsv3, and I
haven't had to futz with it in a number of years.)
But IIRC nfsv3 sends the uid / gid numbers in the protocol; you can't send
names. Which is why you need consistent uid / gids. But I do not quite
grasp how using LDAP to achieve this breaks nfsv3.


>
> > > [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/"
> $2}' |
> > > grep -v eclass | sort -u | wc -l
> > > 443
> > > So there not so much gid uids needed
> There are definitely entries like these, so be very careful in your
> counting.
> enewgroup $PN
> enewuser ${PN} -1 -1 /var/lib/${PN} ${PN}
>
> Based on counting unique tuples of
> ($CAT/$PN, $ARGS, I count 410+ of each enewgroup and enewuser calls.
>
> $ git grep -e 'enewuser ' -e 'enewgroup ' | \
>   sed -r -e 's,/[^/]+:[[:space:]]*,/: ,g' -e 's,#.*,,g' | \
>   grep -e ': enew' |sort |uniq
>
> Also watch out for packages that create MULTIPLE users/groups for privilege
> separation (qmail is notorious for this).
>
> --
> Robin Hugh Johnson
> Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
> E-Mail : robb...@gentoo.org
> GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
>
>

[gentoo-dev] Determenistic system group and user id

2015-12-13 Thread Alexey Shvetsov 

Hi all!

We trying to use ldap for users @work, many of our workstations running 
binary gentoo based distro called Calculate linux. However if we wanna 
have wide use of ldap there is a need for determenistic system group 
gids names and user uids.


Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next 
available parameter)[1]. However it will be much better to set distro 
wide deterministic uid and gid for system service name. So for example 
ldap users may have determenistic groups like video, audio, plugdev, 
etc..


[1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' 
| grep -v eclass | sort -u | wc -l

443
So there not so much gid uids needed

--
Best Regards,
Alexey 'Alexxy' Shvetsov
Best Regards,
Alexey 'Alexxy' Shvetsov, PhD
Department of Molecular and Radiation Biophysics
FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
Leningrad region, Gatchina, Russia
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexx...@gmail.com
mailto:ale...@gentoo.org
mailto:ale...@omrb.pnpi.spb.ru

Re: [gentoo-dev] Determenistic system group and user id

2015-12-13 Thread Mike Gilbert 
On Sun, Dec 13, 2015 at 1:03 PM, Alexey Shvetsov  wrote:
> We trying to use ldap for users @work, many of our workstations running
> binary gentoo based distro called Calculate linux. However if we wanna have
> wide use of ldap there is a need for determenistic system group gids names
> and user uids.

I think you are looking for GLEP 27 and the related bug 53269.

Unfortunately, it seems there has been no movement on that bug in
several years. Maybe you could help implement something?

https://wiki.gentoo.org/wiki/GLEP:27

https://bugs.gentoo.org/show_bug.cgi?id=53269

Re: [gentoo-dev] Determenistic system group and user id

2015-12-13 Thread Marc Schiffbauer 
* Alec Warner schrieb am 13.12.15 um 23:23 Uhr:
[...]
> I never understood why people would think the distro should handle unique
> gid / uids. Plus you usually end up running:
> 
> 1) More than one distro.

not in most places

> 2) More than one 'flavor' of a single distro where for whatever reason, uid
> and gid decisions differed (they renumbered, etc.)
> 
> So if you want a consistent GID for a group, store the group name and gid
> in ldap and sync it; do not rely on your distro to do it. IMHO doing so is
> a design error.

I disagree here. Most (enterprise) environments use just one distro. And 
its just very useful if you have sticky UIDs for daemon users for 
example.

One example: You build an apache two-node cluster using DRBD (and 
pacemaker...).  If you happen to install some daemons in random order on 
both nodes you might end up with apache having different UIDs which will 
break things. This is a PITA.

ANd you do not want another central LDAP-Cluster just to have apache 
UIDs in sync ;)

Red Hat for example has unique distro UIDs for many years now.

I would strongly vote for making GLEP27 reality. It makes life easier in 
many places.

-Marc

-- 
0x35A64134 - 8AAC 5F46 83B4 DB70 8317
 3723 296C 6CCA 35A6 4134


signature.asc
Description: Digital signature

Re: [gentoo-dev] Determenistic system group and user id

2015-12-13 Thread Alec Warner 
On Sun, Dec 13, 2015 at 10:03 AM, Alexey Shvetsov  wrote:

> Hi all!
>
> We trying to use ldap for users @work, many of our workstations running
> binary gentoo based distro called Calculate linux. However if we wanna have
> wide use of ldap there is a need for determenistic system group gids names
> and user uids.
>
> Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next
> available parameter)[1]. However it will be much better to set distro wide
> deterministic uid and gid for system service name. So for example ldap
> users may have determenistic groups like video, audio, plugdev, etc..
>

So the first question I normally ask here is:

1) Why do you need deterministic uid / gid's?
2) If you do need deterministic uid / gid's, I would recommend storing them
all in the same place.

For example, you typically want a deterministic UID for a user. To
accomplish this, you add that user to LDAP, give them a UID in LDAP, and
then either add LDAP to nssswitch or use something like nsscache to sync
the ldap UID's into the local system.

3) If you need deterministic GID's I would recommend storing them all in
LDAP and syncing the group memberships locally.

I never understood why people would think the distro should handle unique
gid / uids. Plus you usually end up running:

1) More than one distro.
2) More than one 'flavor' of a single distro where for whatever reason, uid
and gid decisions differed (they renumbered, etc.)

So if you want a consistent GID for a group, store the group name and gid
in ldap and sync it; do not rely on your distro to do it. IMHO doing so is
a design error.

-A


>
> [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' |
> grep -v eclass | sort -u | wc -l
> 443
> So there not so much gid uids needed
>
> --
> Best Regards,
> Alexey 'Alexxy' Shvetsov
> Best Regards,
> Alexey 'Alexxy' Shvetsov, PhD
> Department of Molecular and Radiation Biophysics
> FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
> Leningrad region, Gatchina, Russia
> Gentoo Team Ru
> Gentoo Linux Dev
> mailto:alexx...@gmail.com
> mailto:ale...@gentoo.org
> mailto:ale...@omrb.pnpi.spb.ru
>
>

Re: [gentoo-dev] Determenistic system group and user id

2015-12-13 Thread Alexey Shvetsov 

Hi Alec!

Alec Warner писал 14-12-2015 01:23:

On Sun, Dec 13, 2015 at 10:03 AM, Alexey Shvetsov 
wrote:


Hi all!

We trying to use ldap for users @work, many of our workstations
running binary gentoo based distro called Calculate linux. However
if we wanna have wide use of ldap there is a need for determenistic
system group gids names and user uids.

Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next
available parameter)[1]. However it will be much better to set
distro wide deterministic uid and gid for system service name. So
for example ldap users may have determenistic groups like video,
audio, plugdev, etc..


So the first question I normally ask here is:

1) Why do you need deterministic uid / gid's?


for exmaple plugdev group may have random gid from range 10-1000+ (i 
have some systems when it have gid >1000)
so if you're ldap user want to mount external drive on workstation you 
dont know what gid it should have..



2) If you do need deterministic uid / gid's, I would recommend storing
them all in the same place.

For example, you typically want a deterministic UID for a user. To
accomplish this, you add that user to LDAP, give them a UID in LDAP,
and then either add LDAP to nssswitch or use something like nsscache
to sync the ldap UID's into the local system.

3) If you need deterministic GID's I would recommend storing them all
in LDAP and syncing the group memberships locally.


Syncing groups localy is major design error if you have more then 10+ 
systems.




I never understood why people would think the distro should handle
unique gid / uids. Plus you usually end up running:

1) More than one distro.


Its not the case. Most time there only one 'supported' distro by local 
IT stuff.



2) More than one 'flavor' of a single distro where for whatever
reason, uid and gid decisions differed (they renumbered, etc.)

So if you want a consistent GID for a group, store the group name and
gid in ldap and sync it; do not rely on your distro to do it. IMHO
doing so is a design error.

-A


[1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/"
$2}' | grep -v eclass | sort -u | wc -l
443
So there not so much gid uids needed

--
Best Regards,
Alexey 'Alexxy' Shvetsov
Best Regards,
Alexey 'Alexxy' Shvetsov, PhD
Department of Molecular and Radiation Biophysics
FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
Leningrad region, Gatchina, Russia
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexx...@gmail.com
mailto:ale...@gentoo.org
mailto:ale...@omrb.pnpi.spb.ru


--
Best Regards,
Alexey 'Alexxy' Shvetsov
Best Regards,
Alexey 'Alexxy' Shvetsov, PhD
Department of Molecular and Radiation Biophysics
FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
Leningrad region, Gatchina, Russia
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexx...@gmail.com
mailto:ale...@gentoo.org
mailto:ale...@omrb.pnpi.spb.ru

Re: [gentoo-dev] Determenistic system group and user id

2015-12-13 Thread Robin H. Johnson 
On Sun, Dec 13, 2015 at 02:23:29PM -0800, Alec Warner wrote:
> 1) Why do you need deterministic uid / gid's?
> 2) If you do need deterministic uid / gid's, I would recommend storing them
> all in the same place.
They are ALL for system users and groups.

TL;DR: if you're sharing data/config for system users/groups between
multiple systems based on UID/GID (not username), you need consistent
generation.

Data on NFSv[23], with a shared apache/nginx user was one of the
original examples. I agree since then, that the data should NOT be owned
by apache/nginx anymore (NFSv4 also solves the problem).

A much newer example, is let's consider the system group 'plugdev'. It's
one that is created dynamically at the moment.

If I want to put my user in that group LDAP-wide, and have an LDAP
environment, I need to make sure the plugdev GID is the same on all
systems (actually it also varies slightly depending which LDAP group
membership model you're using for NSS data).

> For example, you typically want a deterministic UID for a user. To
> accomplish this, you add that user to LDAP, give them a UID in LDAP, and
> then either add LDAP to nssswitch or use something like nsscache to sync
> the ldap UID's into the local system.
> 
> 3) If you need deterministic GID's I would recommend storing them all in
> LDAP and syncing the group memberships locally.
So you want to define the group twice? Both in LDAP and locally?

> I never understood why people would think the distro should handle unique
> gid / uids. Plus you usually end up running:
> 
> 1) More than one distro.
> 2) More than one 'flavor' of a single distro where for whatever reason, uid
> and gid decisions differed (they renumbered, etc.)
Here's the work LSB did on it, with further references to what more
distros do:
https://github.com/LinuxStandardBase/lsb/blob/master/documents/wip/userNaming.txt

Here's the debian central database for it:
https://anonscm.debian.org/cgit/users/cjwatson/base-passwd.git/tree/README


> So if you want a consistent GID for a group, store the group name and gid
> in ldap and sync it; do not rely on your distro to do it. IMHO doing so is
> a design error.
Which is incompatible with NFSv3.

> > [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' |
> > grep -v eclass | sort -u | wc -l
> > 443
> > So there not so much gid uids needed
There are definitely entries like these, so be very careful in your counting.
enewgroup $PN
enewuser ${PN} -1 -1 /var/lib/${PN} ${PN}

Based on counting unique tuples of
($CAT/$PN, $ARGS, I count 410+ of each enewgroup and enewuser calls.

$ git grep -e 'enewuser ' -e 'enewgroup ' | \
  sed -r -e 's,/[^/]+:[[:space:]]*,/: ,g' -e 's,#.*,,g' | \
  grep -e ': enew' |sort |uniq

Also watch out for packages that create MULTIPLE users/groups for privilege
separation (qmail is notorious for this).

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85