Re: [Fedora-livecd-list] [PATCH] -fs.py-kickstart.py-chages-for-mulit-partitions
On Wed, 2008-05-28 at 14:36 -0400, Jeremy Katz wrote: The ability for a tool to create these partitioned disk images via the command like, livecd-creator, is very helpful in reproducing known state appliances, epically with the rising use of visualization. Going through a actual install in a vm limits you to VM container, xen, kvm, vmware and adds extra overhead of building on a machine with extra virt resources. Reproducing known state should work just as well with a kickstart config and doing an install via, eg, virt-install. Yes, it adds overhead, but there are also advantages in functionality and it also means *not* maintaining this sort of code in multiple places. And that's a pretty big win as partitioning code is not in the least bit trivial. Especially as you start trying to handle more and more of the cases. That pretty lame - you're trying to curtail what use cases people make imgcreate support purely because we haven't figured out how to do a better job of sharing code with anaconda. The very least we should be doing is allowing imgcreate to be extended externally in this manner ... but that doesn't fix the multiple copies of this code issue, it merely pushes it elsewhere and makes it less likely it'll ever be fixed. Clearly, one advantage of imgcreate tools over a full blown anaconda install in a VM is that it's far less complex to debug when it all goes wrong. That's a huge advantage to people repeatedly rebuilding images as they develop a LiveCD or an appliance. Cheers, Mark. -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] [PATCH] -fs.py-kickstart.py-chages-for-mulit-partitions
Mark McLoughlin wrote: On Wed, 2008-05-28 at 14:36 -0400, Jeremy Katz wrote: The ability for a tool to create these partitioned disk images via the command like, livecd-creator, is very helpful in reproducing known state appliances, epically with the rising use of visualization. Going through a actual install in a vm limits you to VM container, xen, kvm, vmware and adds extra overhead of building on a machine with extra virt resources. Reproducing known state should work just as well with a kickstart config and doing an install via, eg, virt-install. Yes, it adds overhead, but there are also advantages in functionality and it also means *not* maintaining this sort of code in multiple places. And that's a pretty big win as partitioning code is not in the least bit trivial. Especially as you start trying to handle more and more of the cases. That pretty lame - you're trying to curtail what use cases people make imgcreate support purely because we haven't figured out how to do a better job of sharing code with anaconda. If there were multiple people really clamoring for support to do partitioned images, it would be one thing. But right now, it's one tool. And to then carry the onus outside of that tool for keeping the non-trivial bits working? The very least we should be doing is allowing imgcreate to be extended externally in this manner ... but that doesn't fix the multiple copies of this code issue, it merely pushes it elsewhere and makes it less likely it'll ever be fixed. But we are doing the work to allow an external appliance-creator to carry the functionality if desired. That's why the refactoring patch is reasonable (modulo fixing up the API break). Clearly, one advantage of imgcreate tools over a full blown anaconda install in a VM is that it's far less complex to debug when it all goes wrong. That's a huge advantage to people repeatedly rebuilding images as they develop a LiveCD or an appliance. The reason that it's less complex to debug is the fact that the code is more limited in what it supports than anaconda. Adding more (complicated) pieces is directly in contradiction to keeping things simple to debug Jeremy -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] livecd-iso-to-disk broken in F9?
Todd N wrote: We are trying to use the livecd-iso-to-disk script in Fedora 9 to create a bootable USB drive. When running this script from a command prompt: # ./livecd-iso-to-disk all we get back is a line showing the command usage. However, the same command entered in Fedora 8, or even in Fedora 9 Beta, works fine. Has anyone else had this issue? Err... you need to give it more arguments than just that :) It's been working fine for me and I use it on a very regular basis Jeremy -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
[Fedora-livecd-list] Re: [RFC] Livecd-creator and selinux, we can play nice
seth vidal wrote: On Wed, 2008-05-28 at 16:11 -0400, Eric Paris wrote: My concern is this is a normal occurence (needing a chroot) Yes and no sure looks like we'd need to make sure: yum, mock and rpm all know how to set this up given how it would impact chroot creation. If we do so, then we should also make sure that all do things consistently wrt /dev for creating a chroot as well. And /proc and /sys. The reality is that the different applications do have a somewhat different idea of what they need/want out of their chroots and do things (or don't) accordingly. We may want to drop this back to the lowest level chroot creation. Which isn't to say that we might not decide down the road to push it down the stack, but I don't know that livecd-creator is a bad place in the short to medium term as Eric continues looking at SELinux and chroot interactions (Right Eric? :-) Jeremy -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
[Fedora-livecd-list] Re: [RFC] Livecd-creator and selinux, we can play nice
Eric Paris wrote:
So I've spent a fair bit of time the last 2 weeks trying to get
livecd-creator and an selinux enforcing machine to play nicely together.
It doesn't look like much, but from the point of view of the livecd
creator I think the following patch is all we need. Working with
rawhide as the host system I was able to build F8, F9 and rawhide
livecd's with an enforcing machine.
I wouldn't suggest jumping into enfocing builds just yet as there are
still some policy issues I need to work out with the selinux people but
I would like comments. Basically its quite simple, if selinux is on the
host we create a fake /selinux which tells the install chroot lies.
I've had to make some changes to some selinux libraries to support all
this, but I think we are just about there.
Very cool and definitely long needed. Thanks for taking the time to
really dive into this. And this is far simpler than the approach I had
started looking at once upon a time (... which involved fuse)
A few comments on the patch
diff -Naupr /usr/lib/python2.5/site-packages/imgcreate/creator.py
/root/imgcreate-5-28-08/creator.py
--- /usr/lib/python2.5/site-packages/imgcreate/creator.py 2008-05-06
12:16:08.0 -0400
+++ /root/imgcreate-5-28-08/creator.py 2008-05-28 15:48:30.0 -0400
@@ -460,6 +457,37 @@ class ImageCreator(object):
os.symlink('/proc/self/fd/2', self._instroot + /dev/stderr)
os.umask(origumask)
+# if selinux exists on the host we need to lie to the chroot
+if os.path.exists(/selinux/enforce):
+selinux_dir = self._instroot + /selinux
+
+# enforce=0 tells the chroot selinux is not enforcing
+# policyvers=99 tell the chroot to make the highest version of
policy it can
+files = [('/enforce', '0'),
+ ('/policyvers', '99')]
Does the kernel guarantee that 99 will be the highest version of policy?
Not that it likely matters much. Also, having this as a tuple rather
than a list makes it marginally faster since we're never going to modify it
+for (file, value) in files:
+fd = os.open(selinux_dir + file, os.O_WRONLY | os.O_TRUNC |
os.O_CREAT)
+os.write(fd, value)
+os.close(fd)
+
+# we steal mls from the host system for now, might be best to
always set it to 1
+files = [/selinux/mls]
+for file in files:
+shutil.copyfile(file, self._instroot + file)
+
+# make /load - /dev/null so chroot policy loads don't hurt
anything
+os.mknod(selinux_dir + /load, 0666 | stat.S_IFCHR, os.makedev(1,
3))
This being the big win :)
+# selinux is on whoo hooo
+if kickstart.selinux_enabled(self.ks):
+# label the fs like it is a root before the bind mounting
+cmd = /sbin/setfiles -F -r %s %s %s % (self._instroot,
selinux.selinux_file_context_path(), self._instroot)
+os.system(cmd)
+# these dumb things don't get magically fixed, so make the user
generic
+for f in [/proc, /sys, /selinux]:
+cmd = chcon -u system_u %s % (self._instroot + f)
+os.system(cmd)
os.system is generally not preferred -- using the subprocess module is a
lot safer.
Also, overall it might be nice to encapsulate the /selinux creation here
into its own __create_selinuxfs() method that gets called. /me makes a
note to do that to the /dev creation too.
@@ -853,6 +881,18 @@ class LoopImageCreator(ImageCreator):
(self._image, e))
def _unmount_instroot(self):
+# if the system was running selinux clean up our lies
+if os.path.exists(/selinux/enforce):
+files = ['/enforce',
+ '/policyvers',
+ '/mls',
+ '/load']
Again a tuple versus a list
+for file in files:
+try:
+os.unlink(self._instroot + /selinux + file)
+except OSError:
+pass
And again having it in a method is probably the nice thing to do. And I
know I said to stick it into _unmount_instroot, but seeing where you've
put the mount side, it probably makes more sense in unmount() instead
Jeremy
--
Fedora-livecd-list mailing list
Fedora-livecd-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-livecd-list
[Fedora-livecd-list] Re: [RFC] Livecd-creator and selinux, we can play nice
Daniel J Walsh ([EMAIL PROTECTED]) said: Well I think we need to do a couple of these to figure out the common requirements. I envision mock to be quite different then livecd. I think we need to full the mock chroot to think SELinux is disabled and to do no labeling in the chroot. This would allow us to confine the mock process to be able to write to the chroot and label the chroot mock_rw_t. We could then use SELinux to prevent mock environments from breaking out of the chroot, and stop mock environments from doing evil network things within the chroot. In livecd we need to be able to put down labels that the host machine does not understand. The problem is that mock can be used to do non-build things. (For example, creating the anaconda install images.) Bill -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] [PATCH] -fs.py-kickstart.py-chages-for-mulit-partitions
Jeremy Katz wrote: Mark McLoughlin wrote: On Wed, 2008-05-28 at 14:36 -0400, Jeremy Katz wrote: The ability for a tool to create these partitioned disk images via the command like, livecd-creator, is very helpful in reproducing known state appliances, epically with the rising use of visualization. Going through a actual install in a vm limits you to VM container, xen, kvm, vmware and adds extra overhead of building on a machine with extra virt resources. Reproducing known state should work just as well with a kickstart config and doing an install via, eg, virt-install. Yes, it adds overhead, but there are also advantages in functionality and it also means *not* maintaining this sort of code in multiple places. And that's a pretty big win as partitioning code is not in the least bit trivial. Especially as you start trying to handle more and more of the cases. That pretty lame - you're trying to curtail what use cases people make imgcreate support purely because we haven't figured out how to do a better job of sharing code with anaconda. If there were multiple people really clamoring for support to do partitioned images, it would be one thing. But right now, it's one tool. And to then carry the onus outside of that tool for keeping the non-trivial bits working? The very least we should be doing is allowing imgcreate to be extended externally in this manner ... but that doesn't fix the multiple copies of this code issue, it merely pushes it elsewhere and makes it less likely it'll ever be fixed. But we are doing the work to allow an external appliance-creator to carry the functionality if desired. That's why the refactoring patch is reasonable (modulo fixing up the API break). Clearly, one advantage of imgcreate tools over a full blown anaconda install in a VM is that it's far less complex to debug when it all goes wrong. That's a huge advantage to people repeatedly rebuilding images as they develop a LiveCD or an appliance. The reason that it's less complex to debug is the fact that the code is more limited in what it supports than anaconda. Adding more (complicated) pieces is directly in contradiction to keeping things simple to debug So.. to summarize.. as long as the partitioning code is in the appliance creator only.. we are fine. Is that correct? -- bk -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] [PATCH] -fs.py-kickstart.py-chages-for-mulit-partitions
I've been flat on my back for a while, so if I repeat anything previously sad, apologies. I've got a kickstart that is used to create a two partition solution on a 1G flash device. On the code side, I use a number of shell scripts. But, the main thing that makes is work is that I use a directory structure that looks like this project - files/ - partition_2/ - kickstart file files holds various odds ends (some custom rpms, some specific files, etc). partition_2 holds all the stuff I want on the second partion. I then have a simple script that backbones the livecd creation process and a second script that bundles up everything in the partition 2 directory as another iso. When I'm ready to burn the stuff to flash, I run a second script that handles the partitioning, writes the livecd to the first partition and the partition_2 iso's contents to the second partition. Mark, if this sounds helpful, I'd be happy to tar/zip the scripts in question with some basic directions. If the person who created the livecd creation tool (sorry ... I've forgotten your name and I seem to have accidently deleted the email) is interested, I could turn it into an rpm that could become part of that kickstart. Tim Mark McLoughlin wrote: On Wed, 2008-05-28 at 14:36 -0400, Jeremy Katz wrote: The ability for a tool to create these partitioned disk images via the command like, livecd-creator, is very helpful in reproducing known state appliances, epically with the rising use of visualization. Going through a actual install in a vm limits you to VM container, xen, kvm, vmware and adds extra overhead of building on a machine with extra virt resources. Reproducing known state should work just as well with a kickstart config and doing an install via, eg, virt-install. Yes, it adds overhead, but there are also advantages in functionality and it also means *not* maintaining this sort of code in multiple places. And that's a pretty big win as partitioning code is not in the least bit trivial. Especially as you start trying to handle more and more of the cases. That pretty lame - you're trying to curtail what use cases people make imgcreate support purely because we haven't figured out how to do a better job of sharing code with anaconda. The very least we should be doing is allowing imgcreate to be extended externally in this manner ... but that doesn't fix the multiple copies of this code issue, it merely pushes it elsewhere and makes it less likely it'll ever be fixed. Clearly, one advantage of imgcreate tools over a full blown anaconda install in a VM is that it's far less complex to debug when it all goes wrong. That's a huge advantage to people repeatedly rebuilding images as they develop a LiveCD or an appliance. Cheers, Mark. -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- _ Tim Wood, CLP, RHCT 719.338.7484 (tel) The Data Wranglers Web, Database more sincesince 1994 www.datawranglers.com -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] livecd-iso-to-disk broken in F9?
Correction: we also tried /dev/live as the input parameter (not /mnt/live). Also, this method worked in Fedora 9 Preview as well. Todd Todd N [EMAIL PROTECTED] wrote: Yes, we use the noverify and reset-mbr options along with /dev/sr0 as the input (and have also tried /mnt/live?) and /dev/sda1 as the output (no hard drive, so this corresponds to the USB). Worked fine previously, but now returns the usage() message. Jeremy Katz [EMAIL PROTECTED] wrote: Todd N wrote: We are trying to use the livecd-iso-to-disk script in Fedora 9 to create a bootable USB drive. When running this script from a command prompt: # ./livecd-iso-to-disk all we get back is a line showing the command usage. However, the same command entered in Fedora 8, or even in Fedora 9 Beta, works fine. Has anyone else had this issue? Err... you need to give it more arguments than just that :) It's been working fine for me and I use it on a very regular basis Jeremy -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] livecd-iso-to-disk broken in F9?
These are our results. [/dev/sr0 on /mnt/live type iso9660 (ro)] [EMAIL PROTECTED] LiveOS]# sh -x ./livecd-iso-to-disk --reset-mbr --noverify /dev/sr0 /dev/sda1 + export PATH=/sbin:/usr/sbin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin + PATH=/sbin:/usr/sbin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin ++ id -u + '[' 0 '!=' 0 ']' + '[' 4 -gt 2 ']' + case $1 in + resetmbr=1 + shift + '[' 3 -gt 2 ']' + case $1 in + noverify=1 + shift + '[' 2 -gt 2 ']' ++ readlink -f /dev/sr0 + ISO=/dev/sr0 + USBDEV=/dev/sda1 + '[' -z /dev/sr0 -o '!' -f /dev/sr0 ']' + usage + echo './livecd-iso-to-disk [--reset-mbr] [--noverify] [--overlay-size-mb size] isopath usbstick device' ./livecd-iso-to-disk [--reset-mbr] [--noverify] [--overlay-size-mb size] isopath usbstick device + exit 1 Jeremy Katz [EMAIL PROTECTED] wrote: Try running it with sh -x... that should at least give you an idea of where it's going wrong Jeremy Todd N wrote: Correction: we also tried /dev/live as the input parameter (not /mnt/live). Also, this method worked in Fedora 9 Preview as well. Todd */Todd N /* wrote: Yes, we use the noverify and reset-mbr options along with /dev/sr0 as the input (and have also tried /mnt/live?) and /dev/sda1 as the output (no hard drive, so this corresponds to the USB). Worked fine previously, but now returns the usage() message. */Jeremy Katz /* wrote: Todd N wrote: We are trying to use the livecd-iso-to-disk script in Fedora 9 to create a bootable USB drive. When running this script from a command prompt: # ./livecd-iso-to-disk all we get back is a line showing the command usage. However, the same command entered in Fedora 8, or even in Fedora 9 Beta, works fine. Has anyone else had this issue? Err... you need to give it more arguments than just that :) It's been working fine for me and I use it on a very regular basis Jeremy -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Re: [Fedora-livecd-list] livecd-iso-to-disk broken in F9?
Todd N wrote: These are our results. [/dev/sr0 on /mnt/live type iso9660 (ro)] [snip] + '[' -z /dev/sr0 -o '!' -f /dev/sr0 ']' Hmmm... I see what it is. We changed so that we ensure that you're not using like a directory, but that then doesn't work with using the mounted block device. The simple fix would be to change the -f there to a -e, but that's not the best fix. I'll try to kludge something in shell to work more completely Jeremy -- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
[Fedora-livecd-list] 2 commits - imgcreate/creator.py tools/livecd-iso-to-disk.sh
imgcreate/creator.py| 37 -
tools/livecd-iso-to-disk.sh |6 +-
2 files changed, 25 insertions(+), 18 deletions(-)
New commits:
commit fd5652933c0a49f8961d17bee5f0526bf1676a14
Author: Jeremy Katz [EMAIL PROTECTED]
Date: Thu May 29 17:04:41 2008 -0400
Allow isos to be blockdevs or regular files
diff --git a/tools/livecd-iso-to-disk.sh b/tools/livecd-iso-to-disk.sh
index 9676633..69e97e3 100644
--- a/tools/livecd-iso-to-disk.sh
+++ b/tools/livecd-iso-to-disk.sh
@@ -188,7 +188,11 @@ done
ISO=$(readlink -f $1)
USBDEV=$2
-if [ -z $ISO -o ! -f $ISO ]; then
+if [ -z $ISO ]; then
+usage
+fi
+
+if [ ! -b $ISO -a ! -f $ISO ]; then
usage
fi
commit 6e221ba27ed5a6c5a79ac1fbb7cfc5e11566e080
Author: Jeremy Katz [EMAIL PROTECTED]
Date: Thu May 29 08:53:25 2008 -0400
Move minimal /dev creation into a method
diff --git a/imgcreate/creator.py b/imgcreate/creator.py
index fc4298b..5d010a1 100644
--- a/imgcreate/creator.py
+++ b/imgcreate/creator.py
@@ -403,6 +403,24 @@ class ImageCreator(object):
fstab.write(self._get_fstab())
fstab.close()
+def __create_minimal_dev(self):
+Create a minimal /dev so that we don't corrupt the host /dev
+origumask = os.umask()
+devices = (('null', 1, 3, 0666),
+ ('urandom',1, 9, 0666),
+ ('random', 1, 8, 0666),
+ ('full', 1, 7, 0666),
+ ('ptmx', 5, 2, 0666),
+ ('tty',5, 0, 0666),
+ ('zero', 1, 5, 0666))
+for (node, major, minor, perm) in devices:
+os.mknod(self._instroot + /dev/ + node, perm | stat.S_IFCHR,
os.makedev(major,minor))
+os.symlink('/proc/self/fd', self._instroot + /dev/fd)
+os.symlink('/proc/self/fd/0', self._instroot + /dev/stdin)
+os.symlink('/proc/self/fd/1', self._instroot + /dev/stdout)
+os.symlink('/proc/self/fd/2', self._instroot + /dev/stderr)
+os.umask(origumask)
+
def mount(self, base_on = None, cachedir = None):
Setup the target filesystem in preparation for an install.
@@ -444,25 +462,10 @@ class ImageCreator(object):
if kickstart.selinux_enabled(self.ks):
self.__bindmounts.append(BindChrootMount(/selinux,
self._instroot, None))
-# Create minimum /dev
-origumask = os.umask()
-devices = [('null', 1, 3, 0666),
- ('urandom',1, 9, 0666),
- ('random', 1, 8, 0666),
- ('full', 1, 7, 0666),
- ('ptmx', 5, 2, 0666),
- ('tty',5, 0, 0666),
- ('zero', 1, 5, 0666)]
-for (node, major, minor, perm) in devices:
-os.mknod(self._instroot + /dev/ + node, perm | stat.S_IFCHR,
os.makedev(major,minor))
-os.symlink('/proc/self/fd', self._instroot + /dev/fd)
-os.symlink('/proc/self/fd/0', self._instroot + /dev/stdin)
-os.symlink('/proc/self/fd/1', self._instroot + /dev/stdout)
-os.symlink('/proc/self/fd/2', self._instroot + /dev/stderr)
-os.umask(origumask)
-
self._do_bindmounts()
+self.__create_minimal_dev()
+
os.symlink(../proc/mounts, self._instroot + /etc/mtab)
self.__write_fstab()
--
Fedora-livecd-list mailing list
Fedora-livecd-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-livecd-list
