On 21.12.2015 11:42, Oleg Gelbukh wrote: > The problem with this approach is that we can't manage the interfaces > between components without changing the code of 2+ components (i.e. one > that provides the data and all that consume it).
What code is supposed to be changed in an example LDAP data source? Otherwise, the only place I see shall require code changes is a data processor for it, if one may want to support new (or very old) version of the ldap with another database schema. > > I also don't like polling model for data processors. In my > understanding, components should push their changes through the > pipeline. Although this is pure implementation detail and is not really > important ATM. What do you think the example LDAP data source shall push? IIUC, for most auth cases, it shall be only pulled and processed. > > The point is that for Solar integration, we still need integration > points, and the less of them we have, the more simple the transition is > going to be.. Data processors may be the only integration point, as they are supposed to "know" how-to get and how-to shape data from sources > > -- > Best regards, > Oleg Gelbukh > > On Mon, Dec 21, 2015 at 11:32 AM, Evgeniy L <e...@mirantis.com > <mailto:e...@mirantis.com>> wrote: > > Hi Oleg, > > I understand the concern, but in case of integration specifically > with Solar, > I don't see any reasons to add ConfigDB, because Solar by itself is > a ConfigDB. > At the same time I would agree that there might be a case, when user > uses > Zookeeper/Puppet Master/CMDB as a data store, in this case we should > store > the data directly in those services, without keeping them in yet > another storage. > > So the flow will look like this: > Components -> > Data get polled by data processors -> > | Solar data processor puts the data into Solar in its format > | Zookeeper data processor puts the data into Zookeeper in its format > | Custom CMDB data processor puts the data into CMDB in its own format > > Thanks, > > On Fri, Dec 18, 2015 at 7:00 PM, Oleg Gelbukh <ogelb...@mirantis.com > <mailto:ogelb...@mirantis.com>> wrote: > > Hi, > > The idea behind configdb is that it is independent component > that defines data flows between other components of the system. > It has no knowledge about those components or specifics of their > data. Data formats are defined by components themselves via > schemas/templates and can be changed at any time (i.e. don't > require code changes). > > Important 'pro' having ConfigDB separate from Solar is that it > will simplify transition from current Fuel architecture by > breaking it into more definite stages and reducing the number of > components Solar have to be integrated with. > > -- > Best regards, > Oleg Gelbukh > > On Wed, Dec 16, 2015 at 4:38 PM, Evgeniy L <e...@mirantis.com > <mailto:e...@mirantis.com>> wrote: > > Hi Dmitry, > > I also don't think that we should duplicate the data in > configdb, > because in this case there will be +2 additional interfaces > which > will require to covert the data into configdb and after that > from > configdb to Solar, which seems redundant overhead. > > But we should be able to put the data directly to user's > CMDB/ZooKeeper/Puppet Master/etc. > > Thanks, > > On Wed, Dec 16, 2015 at 2:03 AM, Dmitriy Shulyak > <dshul...@mirantis.com <mailto:dshul...@mirantis.com>> wrote: > > Hello folks, > > This topic is about configuration storage which will > connect data sources (nailgun/bareon/others) and > orchestration. And right now we are developing two > projects that will overlap a bit. > > I understand there is not enough context to dive into > this thread right away, but i will appreciate if those > people, who participated in design, will add their > opinions/clarifications on this matter. > > Main disagreements > --------------------------- > 1. configdb should be passive, writing to configdb is > someone else responsibility > + simpler implementation, easier to use > - we will need another component that will do writing, > or split this responsibility somehow > > 2. can be used without other solar components > + clear inteface between solar components and storage layer > - additional work required to design/refactor > communication layer between modules in solar > - some data will be duplicated between solar > orchestrator layer and configdb > > 3. templates for output > technical detail, can be added on top of solardb if required > > Similar functionality > -------------------------- > 1. Hierachical storage > 2. Versioning of changes > 3. Possibility to overwrite config values > 4. Schema for inputs > > Overall it seems that we share same goals for both services, > the difference lies in organizational and technical > implementation details. > > Possible solutions > ------------------------ > 1. develop configdb and solar with duplicated functionality > - at least 2 additional components will be added to the > picture, > one is configdb, another one will need to sync data > between configdb and solar > - in some cases data in solar and configdb will be 100% > duplicated > - different teams will work on same functionality > - integration of additional component for fuel will > require integration with > configdb and with solar > + configdb will be independent from solar > orchestration/other components > > 2. make service out of solardb, allign with configdb use > cases > + solardb will be independent from solar > orchestration/other solar components > + integration of fuel component will be easier than in > 1st version > + clarity about components responsibility and new > architecture > - redesign/refactoring communication between components > in solar > > 3. do not use configdb/no extraction of solardb > - inproc communication, which can lead to coupled > components (not the case currently) > + faster implementation (no major changes required for > integration with fuel) > + clarity about components responsibility and new > architecture > > Summary > ------------- > For solar it makes no difference where data will come > from: configdb or > data sources, but in overall fuel architecture it will > lead to significant > complexity increase. > It would be the best to follow 2nd path, because in long > term we don't want tightly coupled components, but in > nearest future we need to concentrate > on: > - integration with fuel > - implementing policy engine > - polishing solar components > This is why i am not sure that we can spend time on 2nd > path right now, > or even before 9.0. > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Best regards, Bogdan Dobrelya, Irc #bogdando __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev