Re: [openstack-dev] [Fuel] Modularization activity POC
Hey Evgeniy, > if we drop containers, those will be just rpm/deb packages and > dependencies between them Despite the fact I'm a big fan of Docker and the ideas it brings to us (build, ship & deploy fast), I'd prefer to go with RPM/DEB packages. Simply because it would be easy to support and update. > Regarding the spec our plan is to start writing spec after we have working > POC. Do you have some working draft, at least? Thanks, Igor On Wed, Oct 21, 2015 at 9:01 AM, Mike Scherbakov wrote: > Thanks Eugene. I'd recommend to start integration as early as possible... > > On Tue, Oct 20, 2015 at 2:11 AM Evgeniy L wrote: >> >> Hi Mike, >> >> 1. our plan is to have working partitioning/provisioning in a couple of >> weeks, >> networking is more complicated and it's better to ask Vladimir/Ryan. >> 2, 3. here we have just some general ideas, we should have independent >> releases on pypi, >> each component should have responsible person (team), the same way we >> do it for fuel-plugin-builder >> and fuelclient. The same applies to independent docker containers. >> Another thing is how >> we are going to combine it in ready to use tool, there are several >> ways, if we drop containers, >> those will be just rpm/deb packages and dependencies between them, if >> we continue using >> docker containers, the simplest and the most robust way is to build a >> single container which >> has everything user needs. >> >> Regarding the spec our plan is to start writing spec after we have working >> POC. >> >> Thanks, >> >> >> On Tue, Oct 20, 2015 at 4:43 AM, Mike Scherbakov >> wrote: >>> >>> This is great start, Evgeny. >>> I have a few questions: >>> >>> When do you expect to have POC to show? >>> Do you plan to put new services into separate repos? >>> How do you plan to package those - will you create RPM package for each, >>> as well as Docker container (as we have everything in containers on Fuel >>> master node) >>> >>> These questions, of course, should be covered in spec - so may be I >>> should just wait for you guys to create specs. I just think that it's very >>> important to think about integration from the very beginning. >>> >>> Thanks, >>> >>> On Mon, Oct 19, 2015 at 8:54 AM Igor Kalnitsky >>> wrote: Hey Evgeniy. This is awesome news1 I believe that microservices is way to go. Despite the fact that them bring a set of classical problems (e.g. duplication of domain entities) we will win more than loose. :) If there will be any specs or design meetings, please send me invite - I'd gladly join discussions. Thanks, Igor On Wed, Oct 14, 2015 at 5:30 PM, Evgeniy L wrote: > Hi, > > We are starting Fuel modularization POC activity which is basically in > one > sentence > can be explained as "Use Fuel without Fuel", it means that we want to > provide > for a user a way to use some good things from Fuel, without entire > master > node and > other parts which user doesn't need. > > Currently we have monolithic system which includes every aspect of > deployment > logic, those components tightly coupled together, and there are few > persons > who understand all the interactions between components. > > As a result of modularization we will get the next benefits: > 1. reusability of components > 1.1. it will lead to more components consumers (users) > 1.2. better integration with the community (some community projects > might >be interested in using some parts of Fuel) > 2. components decoupling will lead to clear interfaces between > components > 2.1. so it will be easier to replace some component > 2.2. it will be easier to split the work between teams and it will > help > scale teams in >a much more efficient way > > Here are some thing which naturally can be used separately: > * network configuration (is a part of POC) > * advanced partitioning/provisioning (is a part of POC) > * discovery mechanism (ironic-inspector?) > * power management (ironic?) > * network verification > * diagnostic snapshot > * etc > > The main purpose of POC is to try to make partly working system to > figure > out the > scope of work which we will have to do upstream in order to make it > production ready. > > Here are few basic component-specific ideas: > > Advanced partitioning/provisioning > * split provisioning into two independent actions partitioning and > provisioning > (currently we have a single call which does partitioning, > provisioning, >post provisioning configuration) > * figure out the data format (currently it's too Fuel and Cobbler > specific) > > Network configuration > * CRUD api on any e
Re: [openstack-dev] [Fuel] Modularization activity POC
Thanks Eugene. I'd recommend to start integration as early as possible... On Tue, Oct 20, 2015 at 2:11 AM Evgeniy L wrote: > Hi Mike, > > 1. our plan is to have working partitioning/provisioning in a couple of > weeks, > networking is more complicated and it's better to ask Vladimir/Ryan. > 2, 3. here we have just some general ideas, we should have independent > releases on pypi, > each component should have responsible person (team), the same way we > do it for fuel-plugin-builder > and fuelclient. The same applies to independent docker containers. > Another thing is how > we are going to combine it in ready to use tool, there are several > ways, if we drop containers, > those will be just rpm/deb packages and dependencies between them, if > we continue using > docker containers, the simplest and the most robust way is to build a > single container which > has everything user needs. > > Regarding the spec our plan is to start writing spec after we have working > POC. > > Thanks, > > > On Tue, Oct 20, 2015 at 4:43 AM, Mike Scherbakov > wrote: > >> This is great start, Evgeny. >> I have a few questions: >> >>1. When do you expect to have POC to show? >>2. Do you plan to put new services into separate repos? >>3. How do you plan to package those - will you create RPM package for >>each, as well as Docker container (as we have everything in containers on >>Fuel master node) >> >> These questions, of course, should be covered in spec - so may be I >> should just wait for you guys to create specs. I just think that it's very >> important to think about integration from the very beginning. >> >> Thanks, >> >> On Mon, Oct 19, 2015 at 8:54 AM Igor Kalnitsky >> wrote: >> >>> Hey Evgeniy. >>> >>> This is awesome news1 I believe that microservices is way to go. >>> Despite the fact that them bring a set of classical problems (e.g. >>> duplication of domain entities) we will win more than loose. :) >>> >>> If there will be any specs or design meetings, please send me invite - >>> I'd gladly join discussions. >>> >>> Thanks, >>> Igor >>> >>> >>> >>> >>> On Wed, Oct 14, 2015 at 5:30 PM, Evgeniy L wrote: >>> > Hi, >>> > >>> > We are starting Fuel modularization POC activity which is basically in >>> one >>> > sentence >>> > can be explained as "Use Fuel without Fuel", it means that we want to >>> > provide >>> > for a user a way to use some good things from Fuel, without entire >>> master >>> > node and >>> > other parts which user doesn't need. >>> > >>> > Currently we have monolithic system which includes every aspect of >>> > deployment >>> > logic, those components tightly coupled together, and there are few >>> persons >>> > who understand all the interactions between components. >>> > >>> > As a result of modularization we will get the next benefits: >>> > 1. reusability of components >>> > 1.1. it will lead to more components consumers (users) >>> > 1.2. better integration with the community (some community projects >>> might >>> >be interested in using some parts of Fuel) >>> > 2. components decoupling will lead to clear interfaces between >>> components >>> > 2.1. so it will be easier to replace some component >>> > 2.2. it will be easier to split the work between teams and it will help >>> > scale teams in >>> >a much more efficient way >>> > >>> > Here are some thing which naturally can be used separately: >>> > * network configuration (is a part of POC) >>> > * advanced partitioning/provisioning (is a part of POC) >>> > * discovery mechanism (ironic-inspector?) >>> > * power management (ironic?) >>> > * network verification >>> > * diagnostic snapshot >>> > * etc >>> > >>> > The main purpose of POC is to try to make partly working system to >>> figure >>> > out the >>> > scope of work which we will have to do upstream in order to make it >>> > production ready. >>> > >>> > Here are few basic component-specific ideas: >>> > >>> > Advanced partitioning/provisioning >>> > * split provisioning into two independent actions partitioning and >>> > provisioning >>> > (currently we have a single call which does partitioning, >>> provisioning, >>> >post provisioning configuration) >>> > * figure out the data format (currently it's too Fuel and Cobbler >>> specific) >>> > >>> > Network configuration >>> > * CRUD api on any entity in the system (in Fuel not all of the data are >>> > exposed via api, >>> > so user has to go and edit entries in db manually) >>> > * no constraints regarding to network topology (in Fuel there are a >>> lot of >>> > hardcoded >>> > assumptions) >>> > >>> > Node hardware discovery >>> > * should be able to support different source drivers at the same time >>> >(csv file, fuel-nailgun-agent, CMDB, dhcp, cobbler etc) >>> > * versioning of HW information (required for life cycle management) >>> > * notification about changes in hardware or about node statuses >>> > (required for life cycle management) >>> > >>>
Re: [openstack-dev] [Fuel] Modularization activity POC
Hi Mike, 1. our plan is to have working partitioning/provisioning in a couple of weeks, networking is more complicated and it's better to ask Vladimir/Ryan. 2, 3. here we have just some general ideas, we should have independent releases on pypi, each component should have responsible person (team), the same way we do it for fuel-plugin-builder and fuelclient. The same applies to independent docker containers. Another thing is how we are going to combine it in ready to use tool, there are several ways, if we drop containers, those will be just rpm/deb packages and dependencies between them, if we continue using docker containers, the simplest and the most robust way is to build a single container which has everything user needs. Regarding the spec our plan is to start writing spec after we have working POC. Thanks, On Tue, Oct 20, 2015 at 4:43 AM, Mike Scherbakov wrote: > This is great start, Evgeny. > I have a few questions: > >1. When do you expect to have POC to show? >2. Do you plan to put new services into separate repos? >3. How do you plan to package those - will you create RPM package for >each, as well as Docker container (as we have everything in containers on >Fuel master node) > > These questions, of course, should be covered in spec - so may be I should > just wait for you guys to create specs. I just think that it's very > important to think about integration from the very beginning. > > Thanks, > > On Mon, Oct 19, 2015 at 8:54 AM Igor Kalnitsky > wrote: > >> Hey Evgeniy. >> >> This is awesome news1 I believe that microservices is way to go. >> Despite the fact that them bring a set of classical problems (e.g. >> duplication of domain entities) we will win more than loose. :) >> >> If there will be any specs or design meetings, please send me invite - >> I'd gladly join discussions. >> >> Thanks, >> Igor >> >> >> >> >> On Wed, Oct 14, 2015 at 5:30 PM, Evgeniy L wrote: >> > Hi, >> > >> > We are starting Fuel modularization POC activity which is basically in >> one >> > sentence >> > can be explained as "Use Fuel without Fuel", it means that we want to >> > provide >> > for a user a way to use some good things from Fuel, without entire >> master >> > node and >> > other parts which user doesn't need. >> > >> > Currently we have monolithic system which includes every aspect of >> > deployment >> > logic, those components tightly coupled together, and there are few >> persons >> > who understand all the interactions between components. >> > >> > As a result of modularization we will get the next benefits: >> > 1. reusability of components >> > 1.1. it will lead to more components consumers (users) >> > 1.2. better integration with the community (some community projects >> might >> >be interested in using some parts of Fuel) >> > 2. components decoupling will lead to clear interfaces between >> components >> > 2.1. so it will be easier to replace some component >> > 2.2. it will be easier to split the work between teams and it will help >> > scale teams in >> >a much more efficient way >> > >> > Here are some thing which naturally can be used separately: >> > * network configuration (is a part of POC) >> > * advanced partitioning/provisioning (is a part of POC) >> > * discovery mechanism (ironic-inspector?) >> > * power management (ironic?) >> > * network verification >> > * diagnostic snapshot >> > * etc >> > >> > The main purpose of POC is to try to make partly working system to >> figure >> > out the >> > scope of work which we will have to do upstream in order to make it >> > production ready. >> > >> > Here are few basic component-specific ideas: >> > >> > Advanced partitioning/provisioning >> > * split provisioning into two independent actions partitioning and >> > provisioning >> > (currently we have a single call which does partitioning, >> provisioning, >> >post provisioning configuration) >> > * figure out the data format (currently it's too Fuel and Cobbler >> specific) >> > >> > Network configuration >> > * CRUD api on any entity in the system (in Fuel not all of the data are >> > exposed via api, >> > so user has to go and edit entries in db manually) >> > * no constraints regarding to network topology (in Fuel there are a lot >> of >> > hardcoded >> > assumptions) >> > >> > Node hardware discovery >> > * should be able to support different source drivers at the same time >> >(csv file, fuel-nailgun-agent, CMDB, dhcp, cobbler etc) >> > * versioning of HW information (required for life cycle management) >> > * notification about changes in hardware or about node statuses >> > (required for life cycle management) >> > >> > Common requirements for components: >> > * every component must follow OpenStack coding standards when >> > we start working upstream after POC (so there shouldn't be a question >> > what to use pecan of flask) >> > * Fuel specific interfaces should be implemented as drivers >>
Re: [openstack-dev] [Fuel] Modularization activity POC
This is great start, Evgeny. I have a few questions: 1. When do you expect to have POC to show? 2. Do you plan to put new services into separate repos? 3. How do you plan to package those - will you create RPM package for each, as well as Docker container (as we have everything in containers on Fuel master node) These questions, of course, should be covered in spec - so may be I should just wait for you guys to create specs. I just think that it's very important to think about integration from the very beginning. Thanks, On Mon, Oct 19, 2015 at 8:54 AM Igor Kalnitsky wrote: > Hey Evgeniy. > > This is awesome news1 I believe that microservices is way to go. > Despite the fact that them bring a set of classical problems (e.g. > duplication of domain entities) we will win more than loose. :) > > If there will be any specs or design meetings, please send me invite - > I'd gladly join discussions. > > Thanks, > Igor > > > > > On Wed, Oct 14, 2015 at 5:30 PM, Evgeniy L wrote: > > Hi, > > > > We are starting Fuel modularization POC activity which is basically in > one > > sentence > > can be explained as "Use Fuel without Fuel", it means that we want to > > provide > > for a user a way to use some good things from Fuel, without entire master > > node and > > other parts which user doesn't need. > > > > Currently we have monolithic system which includes every aspect of > > deployment > > logic, those components tightly coupled together, and there are few > persons > > who understand all the interactions between components. > > > > As a result of modularization we will get the next benefits: > > 1. reusability of components > > 1.1. it will lead to more components consumers (users) > > 1.2. better integration with the community (some community projects might > >be interested in using some parts of Fuel) > > 2. components decoupling will lead to clear interfaces between components > > 2.1. so it will be easier to replace some component > > 2.2. it will be easier to split the work between teams and it will help > > scale teams in > >a much more efficient way > > > > Here are some thing which naturally can be used separately: > > * network configuration (is a part of POC) > > * advanced partitioning/provisioning (is a part of POC) > > * discovery mechanism (ironic-inspector?) > > * power management (ironic?) > > * network verification > > * diagnostic snapshot > > * etc > > > > The main purpose of POC is to try to make partly working system to figure > > out the > > scope of work which we will have to do upstream in order to make it > > production ready. > > > > Here are few basic component-specific ideas: > > > > Advanced partitioning/provisioning > > * split provisioning into two independent actions partitioning and > > provisioning > > (currently we have a single call which does partitioning, provisioning, > >post provisioning configuration) > > * figure out the data format (currently it's too Fuel and Cobbler > specific) > > > > Network configuration > > * CRUD api on any entity in the system (in Fuel not all of the data are > > exposed via api, > > so user has to go and edit entries in db manually) > > * no constraints regarding to network topology (in Fuel there are a lot > of > > hardcoded > > assumptions) > > > > Node hardware discovery > > * should be able to support different source drivers at the same time > >(csv file, fuel-nailgun-agent, CMDB, dhcp, cobbler etc) > > * versioning of HW information (required for life cycle management) > > * notification about changes in hardware or about node statuses > > (required for life cycle management) > > > > Common requirements for components: > > * every component must follow OpenStack coding standards when > > we start working upstream after POC (so there shouldn't be a question > > what to use pecan of flask) > > * Fuel specific interfaces should be implemented as drivers > > * this one is obvious, but currently one of the most problematic parts in > > Fuel, > > HW gets changed, interface can be replaced, disk can be removed, > > component should have a way to handle it > > > > Technically speaking, we want to have separate services for every > component, > > it is one of the technical ways to force components to have clear > > interfaces. > > > > Other architecture decision which we want to try to solve is > extendability, > > currently we have a problem that all of the logic which is related to > > deployment > > data is hardcoded in Nailgun. Plugins should be able to retrieve any > data it > > needs from the system, in order to perform plugin deployment, these data > > should > > be retrieved using some interface, and we already have interface where we > > can > > provide stability and versioning, it's REST API. So basically deployment > > should > > consist of two steps first is to retrieve all required data from any > source > > it needs, > > and after that send data to the nodes and start deploym
Re: [openstack-dev] [Fuel] Modularization activity POC
Hey Evgeniy. This is awesome news1 I believe that microservices is way to go. Despite the fact that them bring a set of classical problems (e.g. duplication of domain entities) we will win more than loose. :) If there will be any specs or design meetings, please send me invite - I'd gladly join discussions. Thanks, Igor On Wed, Oct 14, 2015 at 5:30 PM, Evgeniy L wrote: > Hi, > > We are starting Fuel modularization POC activity which is basically in one > sentence > can be explained as "Use Fuel without Fuel", it means that we want to > provide > for a user a way to use some good things from Fuel, without entire master > node and > other parts which user doesn't need. > > Currently we have monolithic system which includes every aspect of > deployment > logic, those components tightly coupled together, and there are few persons > who understand all the interactions between components. > > As a result of modularization we will get the next benefits: > 1. reusability of components > 1.1. it will lead to more components consumers (users) > 1.2. better integration with the community (some community projects might >be interested in using some parts of Fuel) > 2. components decoupling will lead to clear interfaces between components > 2.1. so it will be easier to replace some component > 2.2. it will be easier to split the work between teams and it will help > scale teams in >a much more efficient way > > Here are some thing which naturally can be used separately: > * network configuration (is a part of POC) > * advanced partitioning/provisioning (is a part of POC) > * discovery mechanism (ironic-inspector?) > * power management (ironic?) > * network verification > * diagnostic snapshot > * etc > > The main purpose of POC is to try to make partly working system to figure > out the > scope of work which we will have to do upstream in order to make it > production ready. > > Here are few basic component-specific ideas: > > Advanced partitioning/provisioning > * split provisioning into two independent actions partitioning and > provisioning > (currently we have a single call which does partitioning, provisioning, >post provisioning configuration) > * figure out the data format (currently it's too Fuel and Cobbler specific) > > Network configuration > * CRUD api on any entity in the system (in Fuel not all of the data are > exposed via api, > so user has to go and edit entries in db manually) > * no constraints regarding to network topology (in Fuel there are a lot of > hardcoded > assumptions) > > Node hardware discovery > * should be able to support different source drivers at the same time >(csv file, fuel-nailgun-agent, CMDB, dhcp, cobbler etc) > * versioning of HW information (required for life cycle management) > * notification about changes in hardware or about node statuses > (required for life cycle management) > > Common requirements for components: > * every component must follow OpenStack coding standards when > we start working upstream after POC (so there shouldn't be a question > what to use pecan of flask) > * Fuel specific interfaces should be implemented as drivers > * this one is obvious, but currently one of the most problematic parts in > Fuel, > HW gets changed, interface can be replaced, disk can be removed, > component should have a way to handle it > > Technically speaking, we want to have separate services for every component, > it is one of the technical ways to force components to have clear > interfaces. > > Other architecture decision which we want to try to solve is extendability, > currently we have a problem that all of the logic which is related to > deployment > data is hardcoded in Nailgun. Plugins should be able to retrieve any data it > needs from the system, in order to perform plugin deployment, these data > should > be retrieved using some interface, and we already have interface where we > can > provide stability and versioning, it's REST API. So basically deployment > should > consist of two steps first is to retrieve all required data from any source > it needs, > and after that send data to the nodes and start deployment. > > If you have any questions/suggestion don't hesitate to ask. > > Thanks, > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Modularization activity POC
Hi, We are starting Fuel modularization POC activity which is basically in one sentence can be explained as "Use Fuel without Fuel", it means that we want to provide for a user a way to use some good things from Fuel, without entire master node and other parts which user doesn't need. Currently we have monolithic system which includes every aspect of deployment logic, those components tightly coupled together, and there are few persons who understand all the interactions between components. As a result of modularization we will get the next benefits: 1. reusability of components 1.1. it will lead to more components consumers (users) 1.2. better integration with the community (some community projects might be interested in using some parts of Fuel) 2. components decoupling will lead to clear interfaces between components 2.1. so it will be easier to replace some component 2.2. it will be easier to split the work between teams and it will help scale teams in a much more efficient way Here are some thing which naturally can be used separately: * network configuration (is a part of POC) * advanced partitioning/provisioning (is a part of POC) * discovery mechanism (ironic-inspector?) * power management (ironic?) * network verification * diagnostic snapshot * etc The main purpose of POC is to try to make partly working system to figure out the scope of work which we will have to do upstream in order to make it production ready. Here are few basic component-specific ideas: Advanced partitioning/provisioning * split provisioning into two independent actions partitioning and provisioning (currently we have a single call which does partitioning, provisioning, post provisioning configuration) * figure out the data format (currently it's too Fuel and Cobbler specific) Network configuration * CRUD api on any entity in the system (in Fuel not all of the data are exposed via api, so user has to go and edit entries in db manually) * no constraints regarding to network topology (in Fuel there are a lot of hardcoded assumptions) Node hardware discovery * should be able to support different source drivers at the same time (csv file, fuel-nailgun-agent, CMDB, dhcp, cobbler etc) * versioning of HW information (required for life cycle management) * notification about changes in hardware or about node statuses (required for life cycle management) Common requirements for components: * every component must follow OpenStack coding standards when we start working upstream after POC (so there shouldn't be a question what to use pecan of flask) * Fuel specific interfaces should be implemented as drivers * this one is obvious, but currently one of the most problematic parts in Fuel, HW gets changed, interface can be replaced, disk can be removed, component should have a way to handle it Technically speaking, we want to have separate services for every component, it is one of the technical ways to force components to have clear interfaces. Other architecture decision which we want to try to solve is extendability, currently we have a problem that all of the logic which is related to deployment data is hardcoded in Nailgun. Plugins should be able to retrieve any data it needs from the system, in order to perform plugin deployment, these data should be retrieved using some interface, and we already have interface where we can provide stability and versioning, it's REST API. So basically deployment should consist of two steps first is to retrieve all required data from any source it needs, and after that send data to the nodes and start deployment. If you have any questions/suggestion don't hesitate to ask. Thanks, __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
