Re: [openstack-dev] [Fuel] Separate master node provisioning and deployment
Dear colleagues, The spec for this feature is ready for review [0], so I'd like to encourage all the parties concerned in Fuel modularization to participate. Thanks [0] https://review.openstack.org/#/c/254270/ On Fri, Dec 11, 2015 at 9:45 PM, Vladimir Kozhukalov < vkozhuka...@mirantis.com> wrote: > Dear colleagues, > > At the moment part of the Fuel master deployment logic is located in ISO > kickstart file, which is bad. We'd better carefully split provisioning and > deployment stages so as to install base operating system during > provisioning stage and then everything else on the deployment stage. That > would make it possible to deploy Fuel on pre-installed vanilla Centos 7. > Besides, if we have deb packages for all Fuel components it will be easy to > support Fuel deployment on pre-installed Ubuntu and Debian. > > We (Fuel build team) are going to do this ASAP [0]. Right now we are on > the stage of writing design spec for the change [1]. > > Open questions are: > 1) Should fuel package have all other fuel packages like nailgun, astute, > etc. as its dependencies? Or maybe it should install only puppet modules > and deployment script that then could be used to deploy everything else? > > 2) bootstrap_admin_node.sh runs fuelmenu and then puppet to deploy Fuel > components. Should we run this script as post-install script or maybe we > should leave this up to a user to run this script later when fuel package > is already installed? > > Anyway, the final goal is to make ISO just one of possible delivery > schemes. Primary delivery approach should be rpm/deb repo, not ISO. > > [0] > https://blueprints.launchpad.net/fuel/+spec/separate-fuel-node-provisioning > [1] https://review.openstack.org/#/c/254270/ > > Vladimir Kozhukalov > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Regards, Vitaly Parakhin. Infra Build Engineer | Mirantis, Inc. | http://www.mirantis.com IRC: brain461 @ chat.freenode.net | Slack: vparakhin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Separate master node provisioning and deployment
> One of potential disadvantages is that it is harder to track package > dependencies, but I think > a deployment script should be a root of the package dependency tree. That's something I'd try to avoid. Let's be close to distro upstream practice. I never saw something like "fuel-deploy" that runs puppet that runs package installation. You usually install packages you want to use. If you want to use something beyond default distribution, you install some additional optional packages and that's it. On Mon, Dec 14, 2015 at 3:21 PM, Vladimir Kozhukalov wrote: >> Meantime we can provide fuel-menu which will become a configuration >> gate for different subprojects. Perhaps we could consider to use >> pluggable approach, so each component will export plugin for fuel-menu >> with own settings. > > fuel-menu could be a configuration gate for fuel deployment script > >> The wrong thing is that with such approach it would be impossible to >> setup Fuel with just something like > >>$ yum install fuel > > I see nothing wrong here. 'yum install fuel' would be appropriate approach > if fuel was a service, > not a bunch of services some of which are not even limited to be installed > on the master node. > > when you run > > # yum install fuel > # fuel-menu > > it the same as you run > > # yum install fuel > # fuel_deploy_script (which runs fuel-menu and then runs puppet which > installs everything else) > > I like the idea when fuel (let's rename it into fuel-deploy) package > provides just a deployment script. > It does not require a lot of changes and it corresponds to what we really > do. Besides, it is more flexible > because deployment could be modular (several stages). > > One of potential disadvantages is that it is harder to track package > dependencies, but I think > a deployment script should be a root of the package dependency tree. > > > > Vladimir Kozhukalov > > On Mon, Dec 14, 2015 at 12:53 PM, Igor Kalnitsky > wrote: >> >> Vladimir, >> >> Thanks for raising this question. I totally support idea of separating >> provisioning and deployment steps. I believe it'll simplify a lot of >> things. >> >> However I have some comments regarding this topic, see them inline. :) >> >> > For a package it is absolutely normal to throw a user dialog. >> >> It kills idea of fuel-menu, since each package will need to implement >> configuration menu of its own. Moreover, having such configuration >> menu in fuel-menu and in each package is too expensive, it will >> require more effort that I'd like to have. >> >> > Fuel package could install default astute.yaml (I'd like to rename it >> > into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the >> > file by default not running fuelmenu >> >> I don't like idea of having one common configuration file for Fuel >> components. I think it'd be better when each component (subproject) >> has its own configuration file, and knows nothing about external ones. >> >> Meantime we can provide fuel-menu which will become a configuration >> gate for different subprojects. Perhaps we could consider to use >> pluggable approach, so each component will export plugin for fuel-menu >> with own settings. >> >> > What is wrong with 'deployment script' approach? >> >> The wrong thing is that with such approach it would be impossible to >> setup Fuel with just something like >> >> $ yum install fuel >> >> In my opinion we should go into the following approach: >> >> * yum install fuel >> * fuel-menu >> >> The first command should install a basic Fuel setup, and everything >> should work when it's done. >> >> While the second one prompts a configuration menu where one might >> change default settings (reconfigure default installation). >> >> Thanks, >> Igor >> >> On Mon, Dec 14, 2015 at 9:30 AM, Vladimir Kozhukalov >> wrote: >> > Oleg, >> > >> > Thanks a lot for your opinion. Here are some more thoughts on this >> > topic. >> > >> > 1) For a package it is absolutely normal to throw a user dialog. But >> > probably there is kind of standard for the dialog that does not allow to >> > use >> > fuelmenu. AFAIK, for DEB packages it is debconf and there is a tutorial >> > [0] >> > how to get user input during post install. I don't know if there is such >> > a >> > standard for RPM packages. In some MLs it is written that any command >> > line >> > program could be run in %post section including those like fuel-menu. >> > >> > 2) Fuel package could install default astute.yaml (I'd like to rename it >> > into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the >> > file >> > by default not running fuelmenu. A user then is supposed to run fuelmenu >> > if >> > he/she needs to re-configure fuel installation. However, it is gonna be >> > quite intrusive. What if a user installs fuel and uses it for a while >> > with >> > default configuration. What if some clusters are already in use and then >> > the >> > user decides to re-configure the master node. Will it be ok? >> >
Re: [openstack-dev] [Fuel] Separate master node provisioning and deployment
> Meantime we can provide fuel-menu which will become a configuration > gate for different subprojects. Perhaps we could consider to use > pluggable approach, so each component will export plugin for fuel-menu > with own settings. fuel-menu could be a configuration gate for fuel deployment script > The wrong thing is that with such approach it would be impossible to > setup Fuel with just something like >$ yum install fuel I see nothing wrong here. 'yum install fuel' would be appropriate approach if fuel was a service, not a bunch of services some of which are not even limited to be installed on the master node. when you run # yum install fuel # fuel-menu it the same as you run # yum install fuel # fuel_deploy_script (which runs fuel-menu and then runs puppet which installs everything else) I like the idea when fuel (let's rename it into fuel-deploy) package provides just a deployment script. It does not require a lot of changes and it corresponds to what we really do. Besides, it is more flexible because deployment could be modular (several stages). One of potential disadvantages is that it is harder to track package dependencies, but I think a deployment script should be a root of the package dependency tree. Vladimir Kozhukalov On Mon, Dec 14, 2015 at 12:53 PM, Igor Kalnitsky wrote: > Vladimir, > > Thanks for raising this question. I totally support idea of separating > provisioning and deployment steps. I believe it'll simplify a lot of > things. > > However I have some comments regarding this topic, see them inline. :) > > > For a package it is absolutely normal to throw a user dialog. > > It kills idea of fuel-menu, since each package will need to implement > configuration menu of its own. Moreover, having such configuration > menu in fuel-menu and in each package is too expensive, it will > require more effort that I'd like to have. > > > Fuel package could install default astute.yaml (I'd like to rename it > > into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the > > file by default not running fuelmenu > > I don't like idea of having one common configuration file for Fuel > components. I think it'd be better when each component (subproject) > has its own configuration file, and knows nothing about external ones. > > Meantime we can provide fuel-menu which will become a configuration > gate for different subprojects. Perhaps we could consider to use > pluggable approach, so each component will export plugin for fuel-menu > with own settings. > > > What is wrong with 'deployment script' approach? > > The wrong thing is that with such approach it would be impossible to > setup Fuel with just something like > > $ yum install fuel > > In my opinion we should go into the following approach: > > * yum install fuel > * fuel-menu > > The first command should install a basic Fuel setup, and everything > should work when it's done. > > While the second one prompts a configuration menu where one might > change default settings (reconfigure default installation). > > Thanks, > Igor > > On Mon, Dec 14, 2015 at 9:30 AM, Vladimir Kozhukalov > wrote: > > Oleg, > > > > Thanks a lot for your opinion. Here are some more thoughts on this topic. > > > > 1) For a package it is absolutely normal to throw a user dialog. But > > probably there is kind of standard for the dialog that does not allow to > use > > fuelmenu. AFAIK, for DEB packages it is debconf and there is a tutorial > [0] > > how to get user input during post install. I don't know if there is such > a > > standard for RPM packages. In some MLs it is written that any command > line > > program could be run in %post section including those like fuel-menu. > > > > 2) Fuel package could install default astute.yaml (I'd like to rename it > > into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the > file > > by default not running fuelmenu. A user then is supposed to run fuelmenu > if > > he/she needs to re-configure fuel installation. However, it is gonna be > > quite intrusive. What if a user installs fuel and uses it for a while > with > > default configuration. What if some clusters are already in use and then > the > > user decides to re-configure the master node. Will it be ok? > > > > 3) What is wrong with 'deployment script' approach? Why can not fuel just > > install kind of deployment script? Fuel is not a service, it consists of > > many components. Moreover some of these components could be optional (not > > currently but who knows?), some of this components could be run on an > > external node (after all Fuel components use REST, AMQP, XMLRPC to > interact > > with each other). > > Imagine you want to install OpenStack. It also consists of many > components. > > Some components like database or AMQP service could be deployed using HA > > architecture. What if one needs Fuel to be run with external HA database, > > amqp? From this perspective I'd say Fuel package should not exist at all. > > Let's
Re: [openstack-dev] [Fuel] Separate master node provisioning and deployment
Vladimir, Thanks for raising this question. I totally support idea of separating provisioning and deployment steps. I believe it'll simplify a lot of things. However I have some comments regarding this topic, see them inline. :) > For a package it is absolutely normal to throw a user dialog. It kills idea of fuel-menu, since each package will need to implement configuration menu of its own. Moreover, having such configuration menu in fuel-menu and in each package is too expensive, it will require more effort that I'd like to have. > Fuel package could install default astute.yaml (I'd like to rename it > into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the > file by default not running fuelmenu I don't like idea of having one common configuration file for Fuel components. I think it'd be better when each component (subproject) has its own configuration file, and knows nothing about external ones. Meantime we can provide fuel-menu which will become a configuration gate for different subprojects. Perhaps we could consider to use pluggable approach, so each component will export plugin for fuel-menu with own settings. > What is wrong with 'deployment script' approach? The wrong thing is that with such approach it would be impossible to setup Fuel with just something like $ yum install fuel In my opinion we should go into the following approach: * yum install fuel * fuel-menu The first command should install a basic Fuel setup, and everything should work when it's done. While the second one prompts a configuration menu where one might change default settings (reconfigure default installation). Thanks, Igor On Mon, Dec 14, 2015 at 9:30 AM, Vladimir Kozhukalov wrote: > Oleg, > > Thanks a lot for your opinion. Here are some more thoughts on this topic. > > 1) For a package it is absolutely normal to throw a user dialog. But > probably there is kind of standard for the dialog that does not allow to use > fuelmenu. AFAIK, for DEB packages it is debconf and there is a tutorial [0] > how to get user input during post install. I don't know if there is such a > standard for RPM packages. In some MLs it is written that any command line > program could be run in %post section including those like fuel-menu. > > 2) Fuel package could install default astute.yaml (I'd like to rename it > into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the file > by default not running fuelmenu. A user then is supposed to run fuelmenu if > he/she needs to re-configure fuel installation. However, it is gonna be > quite intrusive. What if a user installs fuel and uses it for a while with > default configuration. What if some clusters are already in use and then the > user decides to re-configure the master node. Will it be ok? > > 3) What is wrong with 'deployment script' approach? Why can not fuel just > install kind of deployment script? Fuel is not a service, it consists of > many components. Moreover some of these components could be optional (not > currently but who knows?), some of this components could be run on an > external node (after all Fuel components use REST, AMQP, XMLRPC to interact > with each other). > Imagine you want to install OpenStack. It also consists of many components. > Some components like database or AMQP service could be deployed using HA > architecture. What if one needs Fuel to be run with external HA database, > amqp? From this perspective I'd say Fuel package should not exist at all. > Let's maybe think of Fuel package as a convenient way to deploy Fuel on a > single node, i.e single node deployment script. > > 4) If Fuel is just a deployment script, then I'd say we should not run any > post install dialog. Deployment script is to run this dialog (fuelmenu) and > then run puppet. IMO it sounds reasonable. > > > [0] http://www.fifi.org/doc/debconf-doc/tutorial.html > > Vladimir Kozhukalov > > On Fri, Dec 11, 2015 at 11:14 PM, Oleg Gelbukh > wrote: >> >> For the package-based deployment, we need to get rid of 'deployment >> script' whatsoever. All configuration stuff should be done in package specs, >> or by the user later on (maybe via some fuelmenu-like lightweight UI, or via >> WebUI). >> >> Thus, fuel package must install everything that is required for running >> base Fuel as it's dependencies (or dependencies of it's dependencies, as it >> could be more complicated with cross-deps between our components). >> >> -- >> Best regards, >> Oleg Gelbukh >> >> On Fri, Dec 11, 2015 at 10:45 PM, Vladimir Kozhukalov >> wrote: >>> >>> Dear colleagues, >>> >>> At the moment part of the Fuel master deployment logic is located in ISO >>> kickstart file, which is bad. We'd better carefully split provisioning and >>> deployment stages so as to install base operating system during provisioning >>> stage and then everything else on the deployment stage. That would make it >>> possible to deploy Fuel on pre-installed vanilla Centos 7. Besides, if we >>> have deb package
Re: [openstack-dev] [Fuel] Separate master node provisioning and deployment
Oleg, Thanks a lot for your opinion. Here are some more thoughts on this topic. 1) For a package it is absolutely normal to throw a user dialog. But probably there is kind of standard for the dialog that does not allow to use fuelmenu. AFAIK, for DEB packages it is debconf and there is a tutorial [0] how to get user input during post install. I don't know if there is such a standard for RPM packages. In some MLs it is written that any command line program could be run in %post section including those like fuel-menu. 2) Fuel package could install default astute.yaml (I'd like to rename it into /etc/fuel.yaml or /etc/fuel/config.yaml) and use values from the file by default not running fuelmenu. A user then is supposed to run fuelmenu if he/she needs to re-configure fuel installation. However, it is gonna be quite intrusive. What if a user installs fuel and uses it for a while with default configuration. What if some clusters are already in use and then the user decides to re-configure the master node. Will it be ok? 3) What is wrong with 'deployment script' approach? Why can not fuel just install kind of deployment script? Fuel is not a service, it consists of many components. Moreover some of these components could be optional (not currently but who knows?), some of this components could be run on an external node (after all Fuel components use REST, AMQP, XMLRPC to interact with each other). Imagine you want to install OpenStack. It also consists of many components. Some components like database or AMQP service could be deployed using HA architecture. What if one needs Fuel to be run with external HA database, amqp? From this perspective I'd say Fuel package should not exist at all. Let's maybe think of Fuel package as a convenient way to deploy Fuel on a single node, i.e single node deployment script. 4) If Fuel is just a deployment script, then I'd say we should not run any post install dialog. Deployment script is to run this dialog (fuelmenu) and then run puppet. IMO it sounds reasonable. [0] http://www.fifi.org/doc/debconf-doc/tutorial.html Vladimir Kozhukalov On Fri, Dec 11, 2015 at 11:14 PM, Oleg Gelbukh wrote: > For the package-based deployment, we need to get rid of 'deployment > script' whatsoever. All configuration stuff should be done in package > specs, or by the user later on (maybe via some fuelmenu-like lightweight > UI, or via WebUI). > > Thus, fuel package must install everything that is required for running > base Fuel as it's dependencies (or dependencies of it's dependencies, as it > could be more complicated with cross-deps between our components). > > -- > Best regards, > Oleg Gelbukh > > On Fri, Dec 11, 2015 at 10:45 PM, Vladimir Kozhukalov < > vkozhuka...@mirantis.com> wrote: > >> Dear colleagues, >> >> At the moment part of the Fuel master deployment logic is located in ISO >> kickstart file, which is bad. We'd better carefully split provisioning and >> deployment stages so as to install base operating system during >> provisioning stage and then everything else on the deployment stage. That >> would make it possible to deploy Fuel on pre-installed vanilla Centos 7. >> Besides, if we have deb packages for all Fuel components it will be easy to >> support Fuel deployment on pre-installed Ubuntu and Debian. >> >> We (Fuel build team) are going to do this ASAP [0]. Right now we are on >> the stage of writing design spec for the change [1]. >> >> Open questions are: >> 1) Should fuel package have all other fuel packages like nailgun, astute, >> etc. as its dependencies? Or maybe it should install only puppet modules >> and deployment script that then could be used to deploy everything else? >> >> 2) bootstrap_admin_node.sh runs fuelmenu and then puppet to deploy Fuel >> components. Should we run this script as post-install script or maybe we >> should leave this up to a user to run this script later when fuel package >> is already installed? >> >> Anyway, the final goal is to make ISO just one of possible delivery >> schemes. Primary delivery approach should be rpm/deb repo, not ISO. >> >> [0] >> https://blueprints.launchpad.net/fuel/+spec/separate-fuel-node-provisioning >> [1] https://review.openstack.org/#/c/254270/ >> >> Vladimir Kozhukalov >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: op
Re: [openstack-dev] [Fuel] Separate master node provisioning and deployment
For the package-based deployment, we need to get rid of 'deployment script' whatsoever. All configuration stuff should be done in package specs, or by the user later on (maybe via some fuelmenu-like lightweight UI, or via WebUI). Thus, fuel package must install everything that is required for running base Fuel as it's dependencies (or dependencies of it's dependencies, as it could be more complicated with cross-deps between our components). -- Best regards, Oleg Gelbukh On Fri, Dec 11, 2015 at 10:45 PM, Vladimir Kozhukalov < vkozhuka...@mirantis.com> wrote: > Dear colleagues, > > At the moment part of the Fuel master deployment logic is located in ISO > kickstart file, which is bad. We'd better carefully split provisioning and > deployment stages so as to install base operating system during > provisioning stage and then everything else on the deployment stage. That > would make it possible to deploy Fuel on pre-installed vanilla Centos 7. > Besides, if we have deb packages for all Fuel components it will be easy to > support Fuel deployment on pre-installed Ubuntu and Debian. > > We (Fuel build team) are going to do this ASAP [0]. Right now we are on > the stage of writing design spec for the change [1]. > > Open questions are: > 1) Should fuel package have all other fuel packages like nailgun, astute, > etc. as its dependencies? Or maybe it should install only puppet modules > and deployment script that then could be used to deploy everything else? > > 2) bootstrap_admin_node.sh runs fuelmenu and then puppet to deploy Fuel > components. Should we run this script as post-install script or maybe we > should leave this up to a user to run this script later when fuel package > is already installed? > > Anyway, the final goal is to make ISO just one of possible delivery > schemes. Primary delivery approach should be rpm/deb repo, not ISO. > > [0] > https://blueprints.launchpad.net/fuel/+spec/separate-fuel-node-provisioning > [1] https://review.openstack.org/#/c/254270/ > > Vladimir Kozhukalov > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Separate master node provisioning and deployment
Dear colleagues, At the moment part of the Fuel master deployment logic is located in ISO kickstart file, which is bad. We'd better carefully split provisioning and deployment stages so as to install base operating system during provisioning stage and then everything else on the deployment stage. That would make it possible to deploy Fuel on pre-installed vanilla Centos 7. Besides, if we have deb packages for all Fuel components it will be easy to support Fuel deployment on pre-installed Ubuntu and Debian. We (Fuel build team) are going to do this ASAP [0]. Right now we are on the stage of writing design spec for the change [1]. Open questions are: 1) Should fuel package have all other fuel packages like nailgun, astute, etc. as its dependencies? Or maybe it should install only puppet modules and deployment script that then could be used to deploy everything else? 2) bootstrap_admin_node.sh runs fuelmenu and then puppet to deploy Fuel components. Should we run this script as post-install script or maybe we should leave this up to a user to run this script later when fuel package is already installed? Anyway, the final goal is to make ISO just one of possible delivery schemes. Primary delivery approach should be rpm/deb repo, not ISO. [0] https://blueprints.launchpad.net/fuel/+spec/separate-fuel-node-provisioning [1] https://review.openstack.org/#/c/254270/ Vladimir Kozhukalov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
