Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
Hey Michael, tried this out on a saucy vm, and it looked good until it died with receiving incremental file list fedora-release-19-2.noarch.rpm sent 47 bytes received 33329 bytes 9536.00 bytes/sec total size is 32472 speedup is 0.97 warning: fedora-release-19-2.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID fb4b18e6: NOKEY Preparing... # [100%] Updating / installing... 1:fedora-release-19-2 # [100%] Loaded plugins: fastestmirror, langpacks Error: Cannot retrieve metalink for repository: fedora/19/x86_64. Please verify its path and try again mount: mount point proc does not exist chroot: failed to run command ‘yum’: No such file or directory Build of Installation RTE failed. Temp directory not removed so it can be investigated. Fedora Run Time Environment setup failed Failed to download 'fedora base' failed to install fedora lxc-create: container creation template for f1 failed lxc-create: Error creating container f1 Looks like unpacking didn't go right? shrug -serge -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
After perusing the entire script now, It seems to me that Michael's major contribution is all after the parts described in my previous post. In fact, it looks like Michael grafted his good work on to a Fedora setup script. If I'm right, then IMO substantial parts of what I posted about earlier may be removed. There are some good ideas in the later code which I think may be re-organized slightly to allow for modularization to possibly incorporate code that would run on other distros. It's a big read and analysis...Will post again when I've tested some modifications and will likely include some recommendations for restructuring slightly. In fact, I suspect that the error Serge just posted has to do with this pre-Container code and isn't really a result of Michael's main body of code so when that is removed to allow Michael's main script to run, it might be successful. Tony On Fri, Sep 20, 2013 at 7:57 AM, Serge Hallyn serge.hal...@ubuntu.com wrote: Hey Michael, tried this out on a saucy vm, and it looked good until it died with receiving incremental file list fedora-release-19-2.noarch.rpm sent 47 bytes received 33329 bytes 9536.00 bytes/sec total size is 32472 speedup is 0.97 warning: fedora-release-19-2.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID fb4b18e6: NOKEY Preparing... # [100%] Updating / installing... 1:fedora-release-19-2 # [100%] Loaded plugins: fastestmirror, langpacks Error: Cannot retrieve metalink for repository: fedora/19/x86_64. Please verify its path and try again mount: mount point proc does not exist chroot: failed to run command ‘yum’: No such file or directory Build of Installation RTE failed. Temp directory not removed so it can be investigated. Fedora Run Time Environment setup failed Failed to download 'fedora base' failed to install fedora lxc-create: container creation template for f1 failed lxc-create: Error creating container f1 Looks like unpacking didn't go right? shrug -serge -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Lxc-devel mailing list lxc-de...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
On Tue, 2013-09-17 at 10:26 -0700, Tony Su wrote:
Regarding
LXC and the use of Linux Bridge devices for configured networking,
At least on openSUSE, LXC is not configured with any /etc/lxc/* by
default, and possibly because I also have libvirt configured to
support LXC (although that is not working on my system).
From what I've seen though, I cannot see why configuring networking
should be in a general lxc configuration file, or even if it should
exist I see it as reaasonable to configure within a template(why not
point the build to a pre-existing virtual network complete with its
own configuration?).
In any case as I noted before, pointing the Container interface to a
pre-existing virtual Linux Bridge device instead of a physical
interface in the template should be nearly trivial. If you wanted to
support bridge creation and configuration, that'd be a big project
outside the scope of what I'm describing.
yum and chroot. H. From what I can see, at the point you invoke
yum, nothing has yet been downloaded so any functionality you invoke
must exist in the HostOS. Despite running in the chroot environment, I
can't see how yum can work unless it's also present in the HostOS.
Maybe if I test this on a distro that doesn't support yum natively
I'll find different, but based purely on perusing the code I can't see
how it would work otherwise.
line 155
chroot ${rootfs_path} yum --releasever=${release} -y install fedora-release
That line is in config_fedora, called at line 926 after the call to
install_fedora at line 920 where the run time install is downloaded and
run. It should be installed at the time in question or something
earlier has failed. Flow of control definitely needs to be cleaned up.
Regards,
Mike
Requirement for GPG keys
Again, based at this point purely on perusing and not actually testing
the code on an appropriately set up system, unless the repo is
configured without keys or the retrieval utility is configured not to
require key verification, I don't see how you've avoided this
requirement, particularly in this case you're using yum which is a
repo client utility. This is aside from verifying the downloaded image
whose integrity could be verified easily by invoking and comparing
checksum values if desired. I'd recommend no verification, though
because it would add maintenance issues. A superior solution might be
a file transfer transport that automatically does checksum (like
torrent but that opens up potential issues since some ISPs block
torrent indiscriminitately).
Regards,
Tony
On Tue, Sep 17, 2013 at 6:34 AM, Michael H. Warfield m...@wittsend.com
wrote:
On Sun, 2013-09-15 at 16:17 -0700, Tony Su wrote:
Hello Michael,
First a comment on problems with systemd you descrbe.
I probably have run into many of the things you itemized, but since my
time is usually focused on something I'm trying to use LXC and not LXC
itself, I usually just drop any further attempts and move on to find a
workaround(eg consoles) or use a different technology(x server issue).
Regarding many of the issues you describe though, I wonder if they
couldn't be addressed with more strict enforcement of using namespaces
(and less often cgroups). I've read how namespaces are supposed to be
an extremely powerful means of isolating processes and yet I don't see
any obvious indications it's being done consistently... by either
prepending to standard process or service names (if the goal is to
easily identify the namespace) or using a random string (if the goal
is better security so exploits can't anticipate commonly used
namespaces).
In fact, I think I see this namespace issue in various parts of the
template you created. If I understand what is happening, there are
numerous places where you create special nodes on the HostOS instead
of
(a) using the existing HostOS nodes but using namespaces to isolate
Container processes
(b) creating nodes entirely within the Container which would make the
Container entirely portable but lose the benefit perhaps of the better
ways nodes are created and mounted today(eg tmpfs in RAM).
Diving more into your template code, I applaud your effort, it's
significant and no minor effort.
As of this moment, I've mainly been perusing what I might call HostOS
Container Pre-Install, the part which precedes the actual
installation and relies on components running in the HostOS only. This
would be your script approx lines 0-410.
1. I like your method of identifying whether the OS is Fedora, and
additionally whether is ARM or not.
That was an effort working on my Raspberry Pi's.
2. It looks like you're configuring networking binding directly to
eth0. I would recommend instead supporting the use of Linux Bridge
devices,
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
Hey Serge, On Fri, 2013-09-20 at 09:57 -0500, Serge Hallyn wrote: Hey Michael, tried this out on a saucy vm, and it looked good until it died with receiving incremental file list fedora-release-19-2.noarch.rpm sent 47 bytes received 33329 bytes 9536.00 bytes/sec total size is 32472 speedup is 0.97 warning: fedora-release-19-2.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID fb4b18e6: NOKEY Preparing... # [100%] Updating / installing... 1:fedora-release-19-2 # [100%] Loaded plugins: fastestmirror, langpacks Error: Cannot retrieve metalink for repository: fedora/19/x86_64. Please verify its path and try again mount: mount point proc does not exist chroot: failed to run command ‘yum’: No such file or directory Build of Installation RTE failed. Temp directory not removed so it can be investigated. Fedora Run Time Environment setup failed Failed to download 'fedora base' failed to install fedora lxc-create: container creation template for f1 failed lxc-create: Error creating container f1 Looks like unpacking didn't go right? shrug That looks like the first one I sent out. I caught a couple of typos immediately after that and recent it with an opps. I'll double check it though. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
On Sun, 2013-09-15 at 16:17 -0700, Tony Su wrote: Hello Michael, First a comment on problems with systemd you descrbe. I probably have run into many of the things you itemized, but since my time is usually focused on something I'm trying to use LXC and not LXC itself, I usually just drop any further attempts and move on to find a workaround(eg consoles) or use a different technology(x server issue). Regarding many of the issues you describe though, I wonder if they couldn't be addressed with more strict enforcement of using namespaces (and less often cgroups). I've read how namespaces are supposed to be an extremely powerful means of isolating processes and yet I don't see any obvious indications it's being done consistently... by either prepending to standard process or service names (if the goal is to easily identify the namespace) or using a random string (if the goal is better security so exploits can't anticipate commonly used namespaces). In fact, I think I see this namespace issue in various parts of the template you created. If I understand what is happening, there are numerous places where you create special nodes on the HostOS instead of (a) using the existing HostOS nodes but using namespaces to isolate Container processes (b) creating nodes entirely within the Container which would make the Container entirely portable but lose the benefit perhaps of the better ways nodes are created and mounted today(eg tmpfs in RAM). Diving more into your template code, I applaud your effort, it's significant and no minor effort. As of this moment, I've mainly been perusing what I might call HostOS Container Pre-Install, the part which precedes the actual installation and relies on components running in the HostOS only. This would be your script approx lines 0-410. 1. I like your method of identifying whether the OS is Fedora, and additionally whether is ARM or not. That was an effort working on my Raspberry Pi's. 2. It looks like you're configuring networking binding directly to eth0. I would recommend instead supporting the use of Linux Bridge devices, make declaration of a bridge device name as one of the early Global Parameters, then if exists to bind to that device by name. Your code to bind to the physical interface is less flexible but can be a default option if no bridge device is specified. Original code. I haven't really looked it it. As far as bridges and all go, much of that is in the greater LXC setup and the default settings in /etc/lxc/default.conf and outside of our control. 3. Interesting that you include an option for nm controlled yet at least initially I don't see where your code might rely on this setting. My code does not. In the back of my mind, the setting for NetworkMangler control is something that is used internally to the container and we're merely setting a default value. 4. mknod I'll have to take a closer look whether and why you appear to be setting up various consoles, some /dev/ nodes, an explicit console path, more. I've generally been under the impression that a full install automatically creates these. Peeking a bit ahead of where I've been reading your script, I notice your install method uses a pre-built squashfs image, perhaps these are a special requirement because your chosen squashfs image doesn't include these by default or requires those nodes to already exist? 5. Your use of yum will work in the RH family plus various others like openSUSE but I don't think it's native to distros like the Debian family. IMO there is no special benefit to using a package manager specific to the HostOS to download bootstrap images and packages, they aren't too relevant to the overall apps running in the HostOS and I think we should avoid installing non-native packages in any OS. For that reason, I've been looking at pycurl, curl and wget which are generic apps common to all distros which can accomplish the simple task of retrieving the bootstrap objects. (See the template I included as an attachment which uses pycurl and finds fedora repos rather than installing a pre-built img) Because yum (and rpm) are no ONLY and EVER executed in a chrooted environment, they are executing in a Fedora context and independent of the host environment. That's the whole point, it never execute the distro-dependent components in the host environment and only execute distro-agnostic (independent) components in the host environment. A small FYI - Although the Fedora template distributed with openSUSE which I've included as an attachment to this message does not work as is it might be useful to see a different way of obtaining and installing Fedora bootstrap packages so I've included it as an attachment to this message. I've said this before. Take it to the bank. The template that is distributed with OpenSuse is OUT OF DATE AND WILL NOT WORK! In fact, the entire LXC package that is distributed with OpenSuse is
Re: [Lxc-users] [lxc-devel] Working LXC templates?
Quoting Michael H. Warfield (m...@wittsend.com): So, all of this has led Serge to list me on the roster for the LinuxPlumbers conference as the LXC systemd expert. I'll get even with him later next week for that one... Lol! Buy you a beer thu night? :) -serge -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
Hello Michael, First a comment on problems with systemd you descrbe. I probably have run into many of the things you itemized, but since my time is usually focused on something I'm trying to use LXC and not LXC itself, I usually just drop any further attempts and move on to find a workaround(eg consoles) or use a different technology(x server issue). Regarding many of the issues you describe though, I wonder if they couldn't be addressed with more strict enforcement of using namespaces (and less often cgroups). I've read how namespaces are supposed to be an extremely powerful means of isolating processes and yet I don't see any obvious indications it's being done consistently... by either prepending to standard process or service names (if the goal is to easily identify the namespace) or using a random string (if the goal is better security so exploits can't anticipate commonly used namespaces). In fact, I think I see this namespace issue in various parts of the template you created. If I understand what is happening, there are numerous places where you create special nodes on the HostOS instead of (a) using the existing HostOS nodes but using namespaces to isolate Container processes (b) creating nodes entirely within the Container which would make the Container entirely portable but lose the benefit perhaps of the better ways nodes are created and mounted today(eg tmpfs in RAM). Diving more into your template code, I applaud your effort, it's significant and no minor effort. As of this moment, I've mainly been perusing what I might call HostOS Container Pre-Install, the part which precedes the actual installation and relies on components running in the HostOS only. This would be your script approx lines 0-410. 1. I like your method of identifying whether the OS is Fedora, and additionally whether is ARM or not. 2. It looks like you're configuring networking binding directly to eth0. I would recommend instead supporting the use of Linux Bridge devices, make declaration of a bridge device name as one of the early Global Parameters, then if exists to bind to that device by name. Your code to bind to the physical interface is less flexible but can be a default option if no bridge device is specified. 3. Interesting that you include an option for nm controlled yet at least initially I don't see where your code might rely on this setting. 4. mknod I'll have to take a closer look whether and why you appear to be setting up various consoles, some /dev/ nodes, an explicit console path, more. I've generally been under the impression that a full install automatically creates these. Peeking a bit ahead of where I've been reading your script, I notice your install method uses a pre-built squashfs image, perhaps these are a special requirement because your chosen squashfs image doesn't include these by default or requires those nodes to already exist? 5. Your use of yum will work in the RH family plus various others like openSUSE but I don't think it's native to distros like the Debian family. IMO there is no special benefit to using a package manager specific to the HostOS to download bootstrap images and packages, they aren't too relevant to the overall apps running in the HostOS and I think we should avoid installing non-native packages in any OS. For that reason, I've been looking at pycurl, curl and wget which are generic apps common to all distros which can accomplish the simple task of retrieving the bootstrap objects. (See the template I included as an attachment which uses pycurl and finds fedora repos rather than installing a pre-built img) A small FYI - Although the Fedora template distributed with openSUSE which I've included as an attachment to this message does not work as is it might be useful to see a different way of obtaining and installing Fedora bootstrap packages so I've included it as an attachment to this message. I've made the two following modifications 1. line 33 - added release=18 The comments in this script describe passing the release number as an option to lxc-create but is not supported in openSUSE. Despite unable to pass as a command line option, passing within the template with this line works. BTW - If no release is specified, Fedora defaults to earliest release in the repos(which is 14) rather than latest. 2. Line 153 - The template hardcodes the RELEASE_URL string which is created by appending a hardcoded string to the MIRROR_URL string, but it appears that Fedora restructured their repos since this template was created. Now, an /f/ has to be inserted into the RELEASE_URL (initial letter of the word fedora). Additional - Especially when connecting to repos of a Distro different than the HostOS, GPG authentication keys are not yet installed. Have been investigating whether it's possible to simply download ahead of time and install into the default Key Ring or something more is required. If this approach is feasable, then this needs to be added early to the template
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
Cool. I'll block some significant time to look at what you built over the next 3 days. Tony -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. http://pubads.g.doubleclick.net/gampad/clk?id=64545871iu=/4140/ostg.clktrk___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!
On Thu, 2013-09-12 at 15:23 -0400, Michael H. Warfield wrote:
All - Especially Tony Su,
Couple of people where I work thought you couldn't do what I was trying
to do, that it was impossible. Oh well. Looks like they were
wrong. :-P It may not be efficient but it can be made to work.
Never fails, I swear, it never fails. Not minutes after I posted this
original message, one of my test runs blew an error that pointed out a
typo that pointed out six other spots where I had environment variables
out of sync and, after fixing them, I realized I had missed several
update paths. And then there were the initialization checks I missed.
Would work fine on fedora on fedora but not what my target was.
Ok... So... I fixed all that. The attached template should actually
work if you're not on a Fedora host (I hope).
It does dawn on me that all this effort really is a one trick pony
and, once it has built the very first container, is no longer needed
after the first run. Reason? Each container built can be a run time
environment to build new containers.
So... Before I submit an actual patch back to the upstream project,
I'll probably scope out the logic to scan the cached containers for the
most appropriate container and use it for the RTE and never use the
cached install RTE again.
This may also be appropriate for other templates. Once you've
bootstrapped the template process, you may not need the bootstrap again.
Just build and cache your best shot and use it in the future.
I have tested this template and built containers from F14 through F19.
F15 failed but, I already know it would not have run as a container
anyways because of the horribly broken and incompatible systemd in that
distro. Prior to F14, some versions will build and some won't.
Shouldn't be a problem with Fedora on Fedora and anything prior to F18
is past EOL anyways, so why should I care?
IAC... Please discard the previous template. Here is my current
template. And please test it.
Thanks!
Regards,
Mike
Way down below, in-line...
On Mon, 2013-09-09 at 07:28 -0400, Michael H. Warfield wrote:
On Mon, 2013-09-09 at 08:58 +0200, Natanael Copa wrote:
On Sun, 08 Sep 2013 20:33:16 -0400
Michael H. Warfield m...@wittsend.com wrote:
With all due respect...
On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote:
After putting some thought into this,
IMO LXC badly needs a universal tool with the following features
- A single script should be used to run on any HostOS, creating
any supported Container OS. Although this would make the script
extremely large, IMO it would actually be easier to maintain in the
long run.
Actually, no. From my experience (30+ years in software development),
it would turn into a morass.
The problem here is that the maintainer(s) would then need to understand
how each and every distribution is installed and how it would be
installed on each and every distribution. It would distill the worse of
all the problems we have now in the templates into one great big dung
pile. It would rapidly become unmaintainable. The extremely large is
the red letter warning that it will become unmaintainable as fewer and
fewer people understand what this great big huge blob does.
I tend to agree with this.
What I do think could be done is to use template APIs. Either by having
a script library with helper functions (for things like:
generate_mac_address etc) or let the template scripts be plugins that
must provide a set of pre-defined functions (eg. install_distro,
configure_distro, copy_configuration etc) or maybe a combination of
those two.
I like this idea, it's just that, in the short term, we have to get past
this one gotcha.
In the git-stage current Fedora template, the entire problem is embodied
in the download_fedora function starting around line 201... The
gotcha's are three commands around line 272 after we've identified and
downloaded the initial release rpm. We have this:
mkdir -p $INSTALL_ROOT/var/lib/rpm
rpm --root $INSTALL_ROOT --initdb
rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
$YUM install $PKG_LIST
Ok... Those are running in the host local run time environment.
Obviously, if the host does not have a (compatible) version of rpm
and/or yum in the host local environment, you're screwed. That's the
catch-22 situation and it's the same situation with zypper in the
OpenSuse template. That short space of code has to be recreated in a
totally distro-agnostic manner so it runs on any distro to create our
initial rootfs. After that, we can do what ever distro (Fedora)
specific commands we want by chrooting into the target container first
(including rebuilding the rpm database to the target version). That's
only even needed if you don't already have a cached rootfs for that
distro and version.
Re: [Lxc-users] [lxc-devel] Working LXC templates?
On Mon, 2013-09-09 at 08:58 +0200, Natanael Copa wrote:
On Sun, 08 Sep 2013 20:33:16 -0400
Michael H. Warfield m...@wittsend.com wrote:
With all due respect...
On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote:
After putting some thought into this,
IMO LXC badly needs a universal tool with the following features
- A single script should be used to run on any HostOS, creating
any supported Container OS. Although this would make the script
extremely large, IMO it would actually be easier to maintain in the
long run.
Actually, no. From my experience (30+ years in software development),
it would turn into a morass.
The problem here is that the maintainer(s) would then need to understand
how each and every distribution is installed and how it would be
installed on each and every distribution. It would distill the worse of
all the problems we have now in the templates into one great big dung
pile. It would rapidly become unmaintainable. The extremely large is
the red letter warning that it will become unmaintainable as fewer and
fewer people understand what this great big huge blob does.
I tend to agree with this.
What I do think could be done is to use template APIs. Either by having
a script library with helper functions (for things like:
generate_mac_address etc) or let the template scripts be plugins that
must provide a set of pre-defined functions (eg. install_distro,
configure_distro, copy_configuration etc) or maybe a combination of
those two.
I like this idea, it's just that, in the short term, we have to get past
this one gotcha.
In the git-stage current Fedora template, the entire problem is embodied
in the download_fedora function starting around line 201... The
gotcha's are three commands around line 272 after we've identified and
downloaded the initial release rpm. We have this:
mkdir -p $INSTALL_ROOT/var/lib/rpm
rpm --root $INSTALL_ROOT --initdb
rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
$YUM install $PKG_LIST
Ok... Those are running in the host local run time environment.
Obviously, if the host does not have a (compatible) version of rpm
and/or yum in the host local environment, you're screwed. That's the
catch-22 situation and it's the same situation with zypper in the
OpenSuse template. That short space of code has to be recreated in a
totally distro-agnostic manner so it runs on any distro to create our
initial rootfs. After that, we can do what ever distro (Fedora)
specific commands we want by chrooting into the target container first
(including rebuilding the rpm database to the target version). That's
only even needed if you don't already have a cached rootfs for that
distro and version.
I was S close last week working on this while on vacation. Recent
revs of Fedora have this downloadable LiveOS core that runs on the
netinst CD and others. That's the 200MB - 300MB blob I was referring
to. You just download it, mount it (requires squashfs) to a directory
and then mount the ext3 image it contains on another directory. Then
create and bind mount a few rw directories (etc var run), mount proc in
the image, and bind mount your rootfs to run/install in the image. Then
chroot into the image and voila, instant RTE. Yum and rpm are capable
of installing other versions of Fedora, so I wouldn't (shouldn't) even
need a version specific instantiation of the RTE, just one that can
install every version we might be interested in.
Except... It didn't work. Sigh... $#@$#@#@
Fedora came so close and then face planted for what I wanted to do.
Sure rpm is in there. But rpmdb is not. So, no --initdb and no
--rebuilddb. They've got yum in there, but no yummain.py so yum hurls
chunks immediately upon execution trying to import yummain. What the
hell good does it do to have yum in there but no yummain?!?! But,
something, fixed up, like that from all the distros would be a perfect
answer (albeit somewhat less than high performance thanks to that
download, but it's a single download that could be cached). The only
potential gotcha I see in there is requiring squashfs available for
mounting the image. Anyone have heartburn over that?
That squashfs image has to be able to do an install or it would be
useless on the netinst CD. I'm not sure if they still have anaconda on
there or not but it has to be able to do a kickstart install, so all I
need is a minimal.ks kickstart file to perform essentially an unattended
install of a minimal build into the target rootfs. That's where I'm at
now, trying to get that to work.
But it's all that work just to replace those few lines of code above
where you can not perform a host native install of the rootfs. That's
all that's standing in the way and it's frustratingly close for the
Fedora template.
We've got the following distro templates in the project, currently:
lxc-alpine
lxc-alt
lxc-arch
lxc-busybox (is this really a distro template
Re: [Lxc-users] [lxc-devel] Working LXC templates?
On Monday, September 09, 2013 07:28:43 AM Michael H. Warfield wrote:
In the git-stage current Fedora template, the entire problem is embodied
in the download_fedora function starting around line 201... The
gotcha's are three commands around line 272 after we've identified and
downloaded the initial release rpm. We have this:
mkdir -p $INSTALL_ROOT/var/lib/rpm
rpm --root $INSTALL_ROOT --initdb
rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
$YUM install $PKG_LIST
Ok... Those are running in the host local run time environment.
Obviously, if the host does not have a (compatible) version of rpm
and/or yum in the host local environment, you're screwed. That's the
catch-22 situation and it's the same situation with zypper in the
OpenSuse template. That short space of code has to be recreated in a
totally distro-agnostic manner so it runs on any distro to create our
initial rootfs. After that, we can do what ever distro (Fedora)
specific commands we want by chrooting into the target container first
(including rebuilding the rpm database to the target version). That's
only even needed if you don't already have a cached rootfs for that
distro and version.
Another approach could be to popularize the container downloads by the distro.
If each distro. could add a .tar.gz for a working container of a given
release, one could just download and configure them, no?
then the lxc project upstream, could just list those links or include them in
the a separate tool, that just downloads and untars the same?
That would completely sidestep the bootstrapping one-distro.-on-another
problem.
--
Regards
Shridhar
--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates?
On Sun, 08 Sep 2013 20:33:16 -0400 Michael H. Warfield m...@wittsend.com wrote: With all due respect... On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote: After putting some thought into this, IMO LXC badly needs a universal tool with the following features - A single script should be used to run on any HostOS, creating any supported Container OS. Although this would make the script extremely large, IMO it would actually be easier to maintain in the long run. Actually, no. From my experience (30+ years in software development), it would turn into a morass. The problem here is that the maintainer(s) would then need to understand how each and every distribution is installed and how it would be installed on each and every distribution. It would distill the worse of all the problems we have now in the templates into one great big dung pile. It would rapidly become unmaintainable. The extremely large is the red letter warning that it will become unmaintainable as fewer and fewer people understand what this great big huge blob does. I tend to agree with this. What I do think could be done is to use template APIs. Either by having a script library with helper functions (for things like: generate_mac_address etc) or let the template scripts be plugins that must provide a set of pre-defined functions (eg. install_distro, configure_distro, copy_configuration etc) or maybe a combination of those two. -nc signature.asc Description: PGP signature -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates?
On Mon, 2013-09-09 at 17:23 +0530, Shridhar Daithankar wrote:
On Monday, September 09, 2013 07:28:43 AM Michael H. Warfield wrote:
In the git-stage current Fedora template, the entire problem is embodied
in the download_fedora function starting around line 201... The
gotcha's are three commands around line 272 after we've identified and
downloaded the initial release rpm. We have this:
mkdir -p $INSTALL_ROOT/var/lib/rpm
rpm --root $INSTALL_ROOT --initdb
rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
$YUM install $PKG_LIST
Ok... Those are running in the host local run time environment.
Obviously, if the host does not have a (compatible) version of rpm
and/or yum in the host local environment, you're screwed. That's the
catch-22 situation and it's the same situation with zypper in the
OpenSuse template. That short space of code has to be recreated in a
totally distro-agnostic manner so it runs on any distro to create our
initial rootfs. After that, we can do what ever distro (Fedora)
specific commands we want by chrooting into the target container first
(including rebuilding the rpm database to the target version). That's
only even needed if you don't already have a cached rootfs for that
distro and version.
Another approach could be to popularize the container downloads by the
distro.
If each distro. could add a .tar.gz for a working container of a given
release, one could just download and configure them, no?
then the lxc project upstream, could just list those links or include them in
the a separate tool, that just downloads and untars the same?
That would completely sidestep the bootstrapping one-distro.-on-another
problem.
True, and that's been mentioned before in several different contexts.
The problem is that we have to get the cooperation of ALL the distros we
wish to support, which seems to be a bit of an intractable problem. In
Fedora, it would require someone to take that on as a project and be the
maintainer. It would also have to be architected into the build system
so it would be automatically build when a release it cut.
Since they already have the squashfs LiveOS system (which is 95% of what
we need), I don't think it would be a major leap for them to add a
parallel build to build an LXC LiveOS to live, say, in the same download
directory. In fact, if they fixed some of the deficiencies in the
LiveOS image, we could work directly from the squashfs image that's
already there.
The problem is in getting someone interested (I'm not a Fedora
maintainer) and getting them to do it. It would probably have to be
filled as an enhancement request and go through the months long vetting
process at best. We might have a better shot (in this case) of filing a
bug report in bugzilla for the busted components of the LiveOS image and
getting them to fix it. Even there, though, it's likely impossible to
get them to retroactively fix any of the previous images.
I agree it would work, I just don't think we can depend on getting
everyone else to agree to it and implement it in their distros.
Considering the direction Fedora has been taking in recent decisions
(removing sendmail / any MTA as well as the syslog server from the base
install) makes me seriously question if they would even care.
--
Regards
Shridhar
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates?
Hello Michael,
Yes, I would really appreciate your comments on what I'm attempting to do.
Here is my thought process...
In general, I've found that the problems you describe are all too
common exactly because no one seems to have sat down and taken a close
look at the flow involved in creating a Container, installing the OS
from scratch like any other install instead of using pre-built
bootstrap file systems.
The problem has been exaccerbated by using templates which are
literally spaghetti code with little organization. When code like that
works, no one can modify the code because no one can reasonably see
what the consequences are and things will break without seeming rhyme
or reason.
This is why after looking at each template script, I'm choosing to
build on the Fedora template. Although it's broken several ways, I
admire its modularity and organization.so it should be a good
foundation.
I had considered something similar to what you're describing you are
building but decided against it because as I described eariler... If
you require a whole new set of tools (even cross platform) then it's
something else that needs to be maintained which I'd think should be
unnecessary. But, if someone decided to go down this path, then I
strongly considered using a standardized, cross platform packaging
system like Ruby.
But, as I said eariler, given a preference I would rather not require
installing new applications which largely duplicate what is already
available and likely installed in each distro when the problem is
mainly one of configuration, not missing functionality.
Without the bits of early code, this(following, below) is the
organization and structure of my proposed script...
I'm also still looking at the best cross-platform/cross-packaging way
to download the initial bootstrap packages, the fedora template
invokes pycurl. Still investigating whether it will work everywhere
and if something else (like wget if http is supported everywhere) is
more appropriate. But, after the bootstrap packages are downloaded,
the ContainerOS' native packaging manager can be used without
installing into the HostOS.
Note that putting everything in one large script also emphasizes the
re-usability of a number of modules across all distros, both HostOS
and Container creation.
Global Variables
Variables which apply to any and all Containers regardless of HostOS
or ContainerOS
/Global Variables
Default Distro Container Parameters
Within this section there will be a section for each supported distro
Probably initially this may also be the main place to specify a
desired distro version for a Container but eventually making the
script interactive could be an eventual objective
These items will largely be determined by the content of the
lxc-create section
/Default Distro Container Parameters
HostOS pre-Container
All the code in this section utilizes packages from the HostOS repos
HostOS Identification
This is the first part that is specific to the HostOS, so unless you
want to distribute different versions of this script for each distro,
this is where the HostOS must be identified, likely by reading
/etc/os-release
/HostOS Identification
Container creation
lxc-create
Note that executing lxc-create is still in the context of the
HostOS, using the HostOS packages which means at this stage you
can't be for instance be using package managers not supported by the
HostOS. From what I can see lxc-create most importantly sets up the
chroot.
This section also most importantly configures and downloads the
required packages into
Container Install
Execute a standard network install for the target distro or
something similar
Any package managers specific to the target distro should run only
within this chroot environment
/Container Install
/lxc-create
/Container creation
On Mon, Sep 9, 2013 at 5:10 AM, Michael H. Warfield m...@wittsend.com wrote:
On Mon, 2013-09-09 at 17:23 +0530, Shridhar Daithankar wrote:
On Monday, September 09, 2013 07:28:43 AM Michael H. Warfield wrote:
In the git-stage current Fedora template, the entire problem is embodied
in the download_fedora function starting around line 201... The
gotcha's are three commands around line 272 after we've identified and
downloaded the initial release rpm. We have this:
mkdir -p $INSTALL_ROOT/var/lib/rpm
rpm --root $INSTALL_ROOT --initdb
rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
$YUM install $PKG_LIST
Ok... Those are running in the host local run time environment.
Obviously, if the host does not have a (compatible) version of rpm
and/or yum in the host local environment, you're screwed. That's the
catch-22 situation and it's the same situation with zypper in the
OpenSuse template. That short space of code has to be recreated in a
totally distro-agnostic manner so it runs on any distro to create our
initial rootfs. After
Re: [Lxc-users] [lxc-devel] Working LXC templates?
After putting some thought into this, IMO LXC badly needs a universal tool with the following features - A single script should be used to run on any HostOS, creating any supported Container OS. Although this would make the script extremely large, IMO it would actually be easier to maintain in the long run. Unknown at this point is how long it would take to load, but once loaded I would expect the script to run very fast. - Flexible enough to support any packaging system, and perhaps even non-package systems if someone should present a strong case or is willing to put in the work - Use the available HostOS distro packages whenever possible. Do not require packages from other distros or cross-platform packages. IMO this means less reliance on supporting non-mainline packages, keeps package and functionality support simple, understandable and responsibility clear. - The script should be clearly modular, with sections clearly marked for the functionality in that section. - The script as currently envisioned should be a single file to reduce missing script dependencies. Any calls should generally be only to internal functions or to mainstream packages provided by the distro. Separate custom scripts should be avoided because otherwise the dependency script(s) may be hard to locate. To this end, I've done the intial baby steps towards architecting a new universal lxc-create template script that should have all the above features and likely within a week or so post an early alpha on my github for anyonne to criticize/contribute/modify. In the meantime, I would be interested to know if anyone is successfully doing what the title of this message thread describes... Creating a Container with a different GuestOS than the HostOS, and if you could either provide a link or description to where the template script can be found (plus, as Nathaniel Copa described anything special and extraordinary you had to do). It would be done a lot faster except that this is being done in my spare time. Now, if someone who deals with LXC a lot wanted to hire me to do this and maybe more... :) Tony On Wed, Sep 4, 2013 at 10:52 AM, Natanael Copa nc...@alpinelinux.org wrote: On Wed, 04 Sep 2013 09:40:49 -0400 Michael H. Warfield m...@wittsend.com wrote: I do think it is an issue with the whole distribution agnostic template problem that may require some help from the distros or some innovative ideas of how we can bootstrap distros using distro agnostic tools (like stone knives and bear skins style install of the rootfs using nothing more than tar, gzip, gpg, and curl or wget). This would be very nice. I have not had success with any templates except the debian on Alpine Linux. Debian works because we build a debootstrap package. Ubuntu template did not work because it uses 'arch' command which we don't have. (ok, should be trivial to implement if we want it bad enough - and I haven't tested current git templates) However, the alpine template in current git should work on any distro. Here is what we do: * download static apk-tools (package manager) and the package with the public keys used for package signature checking. * unpack the the package manager and public keys package with tar. The package format is basically .tar.gz with some files in the beginning with metadata, so the .apk files can be extracted with tar -zx. * verify that the public keys are unmodified against a sha256 sum that is embedded in the template script. * verify that the static binary is unmodified using the public key and openssl. The apk-tools-static package includes a signature for the static binary. * use the verified static package manager to install a rootfs. The package manager will use the previously downloaded pub keys. This should work on any x86/x86_64 distro with tar, gzip, openssl and wget. -nc -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] Working LXC templates?
With all due respect... On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote: After putting some thought into this, IMO LXC badly needs a universal tool with the following features - A single script should be used to run on any HostOS, creating any supported Container OS. Although this would make the script extremely large, IMO it would actually be easier to maintain in the long run. Actually, no. From my experience (30+ years in software development), it would turn into a morass. The problem here is that the maintainer(s) would then need to understand how each and every distribution is installed and how it would be installed on each and every distribution. It would distill the worse of all the problems we have now in the templates into one great big dung pile. It would rapidly become unmaintainable. The extremely large is the red letter warning that it will become unmaintainable as fewer and fewer people understand what this great big huge blob does. I've written perl scripts that count 3K+ lines long and have to have over 50% of the lines be comments just so people can read it (ok - it's perl, what can I say). The template system with small, individually maintained, distribution templates are vastly more maintainable. I'm pretty good with the Fedora stuff. I'm not about to tell the maintainers of the Ubuntu or OpenSuse stuff how to do their thing. We can formulate rules and test each other's templates, but we can not write or maintain them. But, we can be good at maintaining our own templates. Unknown at this point is how long it would take to load, but once loaded I would expect the script to run very fast. Extremely doubtful, in my experience, and the run very fast is not the point and not the source of your problems. From a systems analysis basis, a function which is run rarely can run very inefficiently if it does the right thing every time and is superior to something that runs efficiently but missing way too many corner cases and is a complex gemish-gamash to maintain because it's a great big hairball. I don't care about run very fast. I care about does the right thing in this case. Plus, if it's the Fedora create that's busted, you're limited in the domain of script code you have to look at, even if you are unfamiliar with the code over all. - Flexible enough to support any packaging system, and perhaps even non-package systems if someone should present a strong case or is willing to put in the work It'll never happen. It would take a huge design team, conversant in all those packaging systems (I'm pretty damn good in yum and rpm, so-so in pkg and apt-get, and absolutely suck at yast and zypper) and then it would be design by committee. A committee is a life form with six or more legs AND NO BRAIN. - Use the available HostOS distro packages whenever possible. Do not require packages from other distros or cross-platform packages. IMO this means less reliance on supporting non-mainline packages, keeps package and functionality support simple, understandable and responsibility clear. Agreed. But that is an argument for a modularized system with small, distro specific, components melded with a small modularized system with distro independent components. BTW... You think trying to install Fedora on OpenSuse is bad? Try installing OpenSuse on Fedora (or Ubuntu). It's a disaster. That template won't even come off the dime (no offense to the author - we just don't have the tools in our distros). Here's what you get... -- [root@forest mhw]# lxc-create -n OpenSuse -t opensuse lxc-create: No config file specified, using the default config /etc/lxc/default.conf /usr/share/lxc/templates/lxc-opensuse: line 369: type: zypper: not found 'zypper' command is missing lxc-create: failed to execute template 'opensuse' lxc-create: aborted -- Epic fail. There is no zypper on Fedora. - The script should be clearly modular, with sections clearly marked for the functionality in that section. And that argues against one great big script (which would have to be collectively maintained) and for smaller components (which could be independently maintained). - The script as currently envisioned should be a single file to reduce missing script dependencies. Any calls should generally be only to internal functions or to mainstream packages provided by the distro. Separate custom scripts should be avoided because otherwise the dependency script(s) may be hard to locate. Totally disagree as my experience leads me to believe this ultimately leads to bit-rot and deterioration in the maintainability where nobody knows who wrote what line and why and nobody will touch it to avoid the blame of breaking someone else's code. Nope... Not going to happen (IMNSHO). I've run into this even with the Fedora template. Several times I've posted to the -devel list asking who wrote this or that line and what were you thinking only to get no answer. Now raise that to the N'th power
