Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On Thu, 2011-09-01 at 22:59 +0100, Richard Purdie wrote:
The latter sounds like what we'll need to do. I haven't looked at shadow
to see what kind of finessing is required though...
Fixing the immediate problem with shadow turned out to be rather
straightforward, see attached. However, with this done, I now encounter
two new issues.
1. the logic around $D in useradd.bbclass seems to be backwards to me
(and, empirically, isn't working because the supposedly-created users
are not showing up in my rootfs). Specifically, it does:
useradd_preinst () {
OPT=
SYSROOT=
if test x$D != x; then
# Installing into a sysroot
SYSROOT=${STAGING_DIR_TARGET}
OPT=--root ${STAGING_DIR_TARGET}
[...]
useradd_sysroot () {
export PSEUDO=${STAGING_DIR_NATIVE}/usr/bin/pseudo
export PSEUDO_LOCALSTATEDIR=${STAGING_DIR_TARGET}/var/pseudo
# Explicitly set $D since it isn't set to anything
# before do_install
D=${D}
useradd_preinst
}
It looks to me as though the code in useradd_preinst() should be using
SYSROOT=$D (and likewise for OPT), and useradd_sysroot() should be
setting D=${STAGING_DIR_TARGET}. But maybe there is some clever thing
going on here that I'm not properly understanding.
2. during rootfs construction, the script ordering is wrong. All the
preinsts run before all the postinsts, which has always been a bit wrong
but hasn't caused too much of a problem in the past. However,
crucially, this means that the useradd_preinst() runs before
base-passwd's postinst and hence /etc/passwd doesn't exist at the point
where useradd tries to modify it.
I can't think of any reasonable fix for (2) other than to teach
rootfs_ipk how to track package dependencies and run the scripts in the
right order. I guess that wouldn't be impossible by any means but
trying to do it in a shell script is not a prospect that fills me with a
lot of enthusiasm. Any better suggestions?
p.
diff --git a/meta/recipes-extended/shadow/shadow_4.1.4.3.bb b/meta/recipes-extended/shadow/shadow_4.1.4.3.bb
index 728b8e5..c5a6848 100644
--- a/meta/recipes-extended/shadow/shadow_4.1.4.3.bb
+++ b/meta/recipes-extended/shadow/shadow_4.1.4.3.bb
@@ -9,7 +9,7 @@ LIC_FILES_CHKSUM = file://COPYING;md5=08c553a87d4e51bbed50b20e0adcaede \
DEPENDS = ${@base_contains('DISTRO_FEATURES', 'pam', 'libpam', '', d)}
RDEPENDS_${PN} = ${@base_contains('DISTRO_FEATURES', 'pam', '${PAM_PLUGINS}', '', d)}
-PR = r3
+PR = r4
SRC_URI = http://pkg-shadow.alioth.debian.org/releases/${BPN}-${PV}.tar.bz2 \
file://login_defs_pam.sed \
@@ -122,12 +122,13 @@ pkg_postinst_${PN} () {
update-alternatives --install ${base_sbindir}/vipw vipw vipw.${PN} 200
update-alternatives --install ${base_sbindir}/vigr vigr vigr.${PN} 200
- if [ x$D != x ]; then
- exit 1
- fi
-
- pwconv
- grpconv
+ if [ -n $D ]; then
+ pwconv -R $D
+ grpconv -R $D
+ else
+ pwconv
+ grpconv
+ fi
}
pkg_prerm_${PN} () {
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On Fri, 2011-09-02 at 10:50 +0100, Phil Blundell wrote:
On Thu, 2011-09-01 at 22:59 +0100, Richard Purdie wrote:
The latter sounds like what we'll need to do. I haven't looked at shadow
to see what kind of finessing is required though...
Fixing the immediate problem with shadow turned out to be rather
straightforward, see attached. However, with this done, I now encounter
two new issues.
1. the logic around $D in useradd.bbclass seems to be backwards to me
(and, empirically, isn't working because the supposedly-created users
are not showing up in my rootfs). Specifically, it does:
useradd_preinst () {
OPT=
SYSROOT=
if test x$D != x; then
# Installing into a sysroot
SYSROOT=${STAGING_DIR_TARGET}
OPT=--root ${STAGING_DIR_TARGET}
[...]
useradd_sysroot () {
export PSEUDO=${STAGING_DIR_NATIVE}/usr/bin/pseudo
export PSEUDO_LOCALSTATEDIR=${STAGING_DIR_TARGET}/var/pseudo
# Explicitly set $D since it isn't set to anything
# before do_install
D=${D}
useradd_preinst
}
It looks to me as though the code in useradd_preinst() should be using
SYSROOT=$D (and likewise for OPT), and useradd_sysroot() should be
setting D=${STAGING_DIR_TARGET}. But maybe there is some clever thing
going on here that I'm not properly understanding.
It looks like the useradd_sysroot code will work since D is set to
'something' and then is set correctly in the preinst. What won't work
correctly is the rootfs user creation as you point out.
So I'd agree with your change.
2. during rootfs construction, the script ordering is wrong. All the
preinsts run before all the postinsts, which has always been a bit wrong
but hasn't caused too much of a problem in the past. However,
crucially, this means that the useradd_preinst() runs before
base-passwd's postinst and hence /etc/passwd doesn't exist at the point
where useradd tries to modify it.
I can't think of any reasonable fix for (2) other than to teach
rootfs_ipk how to track package dependencies and run the scripts in the
right order. I guess that wouldn't be impossible by any means but
trying to do it in a shell script is not a prospect that fills me with a
lot of enthusiasm. Any better suggestions?
Write it in python? ;-)
Seriously, does opkg have the logic in it to run this stuff? If so
perhaps we need to teach opkg about offline postinstalls since it should
already know about dependencies?
Cheers,
Richard
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
I just tried using useradd.bbclass for the first time (in an effort to
make dbus installable on a readonly-rootfs) and it doesn't seem to be
working very well for me.
The root of my problem seems to be the code below. As far as I can
tell, what's happening is that process_root_flag() consumes all the
command line arguments to useradd, which means that the subsequent call
to getopt() in process_flags() just returns immediately because there is
nothing left for it to do. The upshot of all this is that the switches
on the command line are simply ignored and useradd doesn't do what I
wanted.
Is anybody else using this code successfully in oe-core with a
nontrivial USERADD_PARAM?
p.
On Tue, 2011-05-31 at 12:53 -0700, Scott Garman wrote:
+ /*
++ * process_root_flag - chroot if given the --root option
++ *
++ * We do this outside of process_flags() because
++ * the is_shadow_pwd boolean needs to be set before
++ * process_flags(), and if we do need to chroot() we
++ * must do so before is_shadow_pwd gets set.
++ */
++static void process_root_flag (int argc, char **argv)
++{
++/*
++ * Parse the command line options.
++ */
++int flag;
++int option_index = 0;
++static struct option long_options[] = {
++{root, required_argument, NULL, 'Q'},
++{NULL, 0, NULL, '\0'}
++};
++
++while ((flag = getopt_long (argc, argv, a:A:d:gM:Q:rR, long_options,
option_index)) != -1) {
++switch (flag) {
++case 'Q':
++if ('/' != optarg[0]) {
++fprintf (stderr,
++ _(%s: invalid chroot path '%s'\n),
++ Prog, optarg);
++exit (E_BAD_ARG);
++}
++newroot = optarg;
++
++if (access (newroot, F_OK) != 0) {
++fprintf(stderr,
++_(%s: chroot directory %s does not
exist\n),
++Prog, newroot);
++exit (E_BAD_ARG);
++}
++if ( chroot(newroot) != 0 ) {
++fprintf(stderr,
++_(%s: unable to chroot to directory
%s\n),
++Prog, newroot);
++exit (E_BAD_ARG);
++}
++break;
++/* no-op on everything else - they will be hanled by
process_flags() */
++}
++}
++}
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On Thu, 2011-09-01 at 11:54 -0500, Mark Hatle wrote:
What is it depending on for the target? Is the shadow-utils or something now
required? That doesn't seem to make sense to me -- other then we need a
passwd/group/shadow/gshadow file to work with. As long as something can
provide
those, we should be ok.
I haven't investigated in detail, but the code from useradd.bbclass
says:
# base-passwd-cross provides the default passwd and group files in the
# target sysroot, and shadow -native and -sysroot provide the utilities
# and support files needed to add and modify user and group accounts
DEPENDS_append = base-passwd shadow-native shadow-sysroot
RDEPENDS_${USERADDPN}_append = base-passwd shadow
And, I guess, if you want to support online package management then it
does make some sense to have the shadow utils there. But I don't
need/want that in my configuration.
p.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On 9/1/11 11:58 AM, Phil Blundell wrote:
On Thu, 2011-09-01 at 11:54 -0500, Mark Hatle wrote:
What is it depending on for the target? Is the shadow-utils or something now
required? That doesn't seem to make sense to me -- other then we need a
passwd/group/shadow/gshadow file to work with. As long as something can
provide
those, we should be ok.
I haven't investigated in detail, but the code from useradd.bbclass
says:
# base-passwd-cross provides the default passwd and group files in the
# target sysroot, and shadow -native and -sysroot provide the utilities
# and support files needed to add and modify user and group accounts
DEPENDS_append = base-passwd shadow-native shadow-sysroot
RDEPENDS_${USERADDPN}_append = base-passwd shadow
Hmm, good point... I'd forgotten about that.
And, I guess, if you want to support online package management then it
does make some sense to have the shadow utils there. But I don't
need/want that in my configuration.
Does busybox or something else provide a compatible adduser? If so maybe a
virtual RDEPENDS is more reasonable in this case.
I think we're caught in the case of we build packages.. as such we need to cover
what the package needs at runtime, this includes install time. At least w/ a
virtual depend, we can likely fake it by providing it by something else.. but
I'm not sure..
--Mark
p.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On Thu, 2011-09-01 at 12:25 -0500, Mark Hatle wrote: On 9/1/11 11:58 AM, Phil Blundell wrote: And, I guess, if you want to support online package management then it does make some sense to have the shadow utils there. But I don't need/want that in my configuration. Does busybox or something else provide a compatible adduser? If so maybe a virtual RDEPENDS is more reasonable in this case. I'm not sure offhand (it's actually useradd, not adduser, for what that's worth) but, even if busybox does provide those applets, that probably isn't quite the point. The issue here is that I don't really want to have any implementation of useradd at all on the target system; using one from busybox would be a bit less bad than requiring standalone shadow, but still not really ideal. One workaround would be to weaken the RDEPENDS to an RRECOMMENDS, which would allow me to declare it as a BAD_RECOMMENDATION. Or I guess we could make it be a virtual and I could then provide a dummy-useradd package which satisfies the dependency but doesn't actually install any files. The approach we take with update-rc.d is to let it be installed and then have rootfs_ipk rip it back out again after image construction is done, but this won't work with shadow as it stands due to the postinst issue in that package. So a third option would be to find a way to finesse the postinst thing somehow and then use the same rootfs_ipk logic with shadow too. None of those really sound very appealing. Anybody have a better idea? p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
This adds a -native recipe for the shadow utilities.
The custom --root option allows the the following utilities to be
run within a chroot when invoked under pseudo:
* useradd
* groupadd
* usermod
* groupmod
* userdel
* groupdel
* passwd
* gpasswd
* pwconv
* pwunconv
* grpconv
* grpunconv
They can then be used to manipulate user and group account information
in target sysroots.
useradd was also modified to create home directories recursively when
necessary.
Signed-off-by: Scott Garman scott.a.gar...@intel.com
---
.../shadow/files/add_root_cmd_options.patch| 1296
.../shadow/shadow-native_4.1.4.3.bb| 66 +
2 files changed, 1362 insertions(+), 0 deletions(-)
create mode 100644
meta/recipes-extended/shadow/files/add_root_cmd_options.patch
create mode 100644 meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb
diff --git a/meta/recipes-extended/shadow/files/add_root_cmd_options.patch
b/meta/recipes-extended/shadow/files/add_root_cmd_options.patch
new file mode 100644
index 000..db969bb
--- /dev/null
+++ b/meta/recipes-extended/shadow/files/add_root_cmd_options.patch
@@ -0,0 +1,1296 @@
+Add a --root command option to the following utilties:
+
+* useradd
+* groupadd
+* usermod
+* groupmod
+* userdel
+* groupdel
+* passwd
+* gpasswd
+* pwconv
+* pwunconv
+* grpconv
+* grpunconv
+
+This option allows the utilities to be chrooted when run under pseudo.
+They can then be used to manipulate user and group account information
+in target sysroots.
+
+The useradd utility was also modified to create home directories
+recursively when necessary.
+
+Upstream-Status: Inappropriate [Other]
+Workaround is specific to our build system.
+
+Signed-off-by: Scott Garman scott.a.gar...@intel.com
+
+diff -urN shadow-4.1.4.3.orig//src/gpasswd.c shadow-4.1.4.3//src/gpasswd.c
+--- shadow-4.1.4.3.orig//src/gpasswd.c 2011-02-13 09:58:16.0 -0800
shadow-4.1.4.3//src/gpasswd.c 2011-05-28 17:09:52.346013331 -0700
+@@ -63,6 +63,7 @@
+ * (/etc/gshadow present) */
+ static bool is_shadowgrp;
+ #endif
++static const char *newroot = ;
+
+ /* Flags set by options */
+ static bool aflg = false;
+@@ -97,6 +98,7 @@
+ static void usage (void);
+ static RETSIGTYPE catch_signals (int killed);
+ static bool is_valid_user_list (const char *users);
++static void process_root_flag (int argc, char **argv);
+ static void process_flags (int argc, char **argv);
+ static void check_flags (int argc, int opt_index);
+ static void open_files (void);
+@@ -136,6 +138,7 @@
+ Options:\n
+-a, --add USERadd USER to GROUP\n
+-d, --delete USER remove USER from GROUP\n
++ -Q --root CHROOT_DIR directory to chroot into\n
+-r, --remove-password remove the GROUP's
password\n
+-R, --restrictrestrict access to GROUP to
its members\n
+-M, --members USER,...set the list of members of
GROUP\n
+@@ -226,6 +229,55 @@
+ }
+
+ /*
++ * process_root_flag - chroot if given the --root option
++ *
++ * We do this outside of process_flags() because
++ * the is_shadow_pwd boolean needs to be set before
++ * process_flags(), and if we do need to chroot() we
++ * must do so before is_shadow_pwd gets set.
++ */
++static void process_root_flag (int argc, char **argv)
++{
++ /*
++ * Parse the command line options.
++ */
++ int flag;
++ int option_index = 0;
++ static struct option long_options[] = {
++ {root, required_argument, NULL, 'Q'},
++ {NULL, 0, NULL, '\0'}
++ };
++
++ while ((flag = getopt_long (argc, argv, a:A:d:gM:Q:rR, long_options,
option_index)) != -1) {
++ switch (flag) {
++ case 'Q':
++ if ('/' != optarg[0]) {
++ fprintf (stderr,
++ _(%s: invalid chroot path '%s'\n),
++ Prog, optarg);
++ exit (E_BAD_ARG);
++ }
++ newroot = optarg;
++
++ if (access (newroot, F_OK) != 0) {
++ fprintf(stderr,
++ _(%s: chroot directory %s does not
exist\n),
++ Prog, newroot);
++ exit (E_BAD_ARG);
++ }
++ if ( chroot(newroot) != 0 ) {
++ fprintf(stderr,
++ _(%s: unable to chroot to directory
%s\n),
++ Prog, newroot);
++ exit (E_BAD_ARG);
++ }
++ break;
++ /* no-op on everything else - they will be hanled by
process_flags() */
++ }
++ }
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On Tue, 2011-05-31 at 12:53 -0700, Scott Garman wrote:
This adds a -native recipe for the shadow utilities.
The custom --root option allows the the following utilities to be
run within a chroot when invoked under pseudo:
Rather than patching the code for all these utilities, can't you just
wrap them in a call to chroot(8)? That is, make useradd.bbclass do:
eval $PSEUDO chroot ${STAGING_DIR_TARGET} useradd ...
rather than the existing
eval $PSEUDO useradd --root ${STAGING_DIR_TARGET} ...
?
p.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On 01/06/11 10:47, Phil Blundell wrote:
On Tue, 2011-05-31 at 12:53 -0700, Scott Garman wrote:
This adds a -native recipe for the shadow utilities.
The custom --root option allows the the following utilities to be
run within a chroot when invoked under pseudo:
Rather than patching the code for all these utilities, can't you just
wrap them in a call to chroot(8)? That is, make useradd.bbclass do:
eval $PSEUDO chroot ${STAGING_DIR_TARGET} useradd ...
rather than the existing
eval $PSEUDO useradd --root ${STAGING_DIR_TARGET} ...
?
You'd need root privileges to use chroot wouldn't you? I'm assuming you
wouldn't with the existing.
Martyn
--
Martyn Welch (Principal Software Engineer) | Registered in England and
GE Intelligent Platforms | Wales (3828642) at 100
T +44(0)127322748 | Barbirolli Square, Manchester,
E martyn.we...@ge.com | M2 3AB VAT:GB 927559189
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On Wed, 2011-06-01 at 13:34 +0100, Martyn Welch wrote:
On 01/06/11 10:47, Phil Blundell wrote:
On Tue, 2011-05-31 at 12:53 -0700, Scott Garman wrote:
This adds a -native recipe for the shadow utilities.
The custom --root option allows the the following utilities to be
run within a chroot when invoked under pseudo:
Rather than patching the code for all these utilities, can't you just
wrap them in a call to chroot(8)? That is, make useradd.bbclass do:
eval $PSEUDO chroot ${STAGING_DIR_TARGET} useradd ...
rather than the existing
eval $PSEUDO useradd --root ${STAGING_DIR_TARGET} ...
?
You'd need root privileges to use chroot wouldn't you? I'm assuming you
wouldn't with the existing.
Doesn't $PSEUDO sort that out? The --root option that Scott has patched
into all the utilities seems to just end up calling chroot(2) anyway so
you would need root privileges at that point. That said, I'm not
entirely au fait with pseudo so I might well be misunderstanding how it
works.
p.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On 06/01/2011 02:47 AM, Phil Blundell wrote:
On Tue, 2011-05-31 at 12:53 -0700, Scott Garman wrote:
This adds a -native recipe for the shadow utilities.
The custom --root option allows the the following utilities to be
run within a chroot when invoked under pseudo:
Rather than patching the code for all these utilities, can't you just
wrap them in a call to chroot(8)? That is, make useradd.bbclass do:
eval $PSEUDO chroot ${STAGING_DIR_TARGET} useradd ...
rather than the existing
eval $PSEUDO useradd --root ${STAGING_DIR_TARGET} ...
That's a reasonable suggestion. I haven't tried it yet, but I have found
that pseudo's chroot(2) implementation is not complete. One of the cases
where it does not work is when forking child processes, which breaks the
jail and the child processes are no longer chroot'ed.
My guess is that chroot(8) is going to call chroot(2) and then fork a
child process to run its additional arguments.
cc'ing Mark directly in case he has additional comments or needs to
correct me.
Scott
--
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] shadow: add a -native recipe with customized utilities
On 6/1/11 12:43 PM, Scott Garman wrote:
On 06/01/2011 02:47 AM, Phil Blundell wrote:
On Tue, 2011-05-31 at 12:53 -0700, Scott Garman wrote:
This adds a -native recipe for the shadow utilities.
The custom --root option allows the the following utilities to be
run within a chroot when invoked under pseudo:
Rather than patching the code for all these utilities, can't you just
wrap them in a call to chroot(8)? That is, make useradd.bbclass do:
eval $PSEUDO chroot ${STAGING_DIR_TARGET} useradd ...
rather than the existing
eval $PSEUDO useradd --root ${STAGING_DIR_TARGET} ...
That's a reasonable suggestion. I haven't tried it yet, but I have found
that pseudo's chroot(2) implementation is not complete. One of the cases
where it does not work is when forking child processes, which breaks the
jail and the child processes are no longer chroot'ed.
My guess is that chroot(8) is going to call chroot(2) and then fork a
child process to run its additional arguments.
chroot should be complete in pseudo, if you can reproduce any failures we should
pass them upstream.
The reason I suggested the --root option was primarily for the ease of people
who are NOT using the automated scripting, i.e. someone manually adding a
preinst (or similar) to their recipes.
The --root option is easier (to me at least) to understand that having to if-def
stuff around a chroot.. (but that is personal preference...)
--Mark
cc'ing Mark directly in case he has additional comments or needs to
correct me.
Scott
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
