Ruslan, I'm absolutely agree with you, only one correction, I think murano-guestagent will better fit the repo content.
Thanks. On Tue, Feb 18, 2014 at 7:23 PM, Alexander Tivelkov <ativel...@mirantis.com>wrote: > Hi Ruslan, > > Thanks for your feedback. I completely agree with these arguments: > actually, these were the reasons why I've initiated this discussion. > > Team, let's discuss this on the IRC meeting today. > > -- > Regards, > Alexander Tivelkov > > > On Tue, Feb 18, 2014 at 5:55 PM, Ruslan Kamaldinov < > rkamaldi...@mirantis.com> wrote: > >> I'd suggest to reduce number of Murano repositories for several reasons: >> >> * All other OpenStack projects have a single repo per project. While this >> point might look like something not worth mentioning, it's really >> important: >> - unified project structure simplifies life for new developers. once they >> get familiar with one project, they can expect something similar from >> another project >> - unified project structure simplifies life for deployers. similar project >> structure simplifies packaging and deployment automation >> >> * Another important reason is to simplify gated testing. Just take a look >> at >> Solum layout [1], they have everything needed (contrib, functionaltests) >> to >> run dvsm job in a single repo. One simple job definition [2] allows to >> install Solum in DevStack and run Tempest tests for Solum. >> >> * As a side-effect, this approach will improve integrity of project >> components. Having murano-common in the same repo with other components >> will >> help to catch integration issues earlier. >> >> >> In an ideal world there will be only the following repos: >> - murano (api, common, conductor, docs, repository, tests) >> - python-muranoclient >> - murano-dashboard >> - murano-agent >> - puppet-murano (optional, but nice to have) >> >> >> [1] https://github.com/stackforge/solum >> [2] >> https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/solum.yaml >> >> >> Thanks, >> Ruslan >> >> >> On Thu, Feb 6, 2014 at 3:14 PM, Serg Melikyan <smelik...@mirantis.com>wrote: >> >>> Hi, Alexander, >>> >>> In general I am completely agree with Clint and Robert, and as one of >>> contributors of Murano I don't see any practical reasons for repositories >>> reorganization. And regarding of your proposal I have a few thoughts that I >>> would like to share below: >>> >>> >This enourmous amount of repositories adds too much infrustructural >>> complexity >>> Creating a new repository is a quick, easy and completely automated >>> procedure that requires only simple commit to Zuul configuration. All >>> infrastructure related to repositories is handled by Openstack CI and >>> supported by Openstack Infra Team, and actually don't require anything from >>> project development team. About what infrastructure complexity you are >>> talking about? >>> >>> >I actually think keeping them separate is a great way to make sure you >>> have ongoing API stability. (c) Clint >>> I would like to share a little statistic gathered by Stan Lagun >>> a little time ago regarding repositories count in different PaaS solution. >>> If you are concerned about large number of repositories used by Murano, you >>> will be quite amused: >>> >>> - https://github.com/heroku - 275 >>> - https://github.com/cloudfoundry - 132 >>> - https://github.com/openshift - 49 >>> - https://github.com/CloudifySource - 46 >>> >>> >First of all, I would suggest to have a single reposository for all >>> the three main components of Murano: main murano API (the contents of the >>> present), workflow execution engine (currently murano-conductor; also it >>> was suggested to rename the component itself to murano-engine for more >>> consistent naming) and metadata repository (currently murano-repository). >>> >>> *murano-api* and *murano-repository* have many things in common, they >>> are both present HTTP API to the user, and I hope would be rewritten to >>> common framework (Pecan?). But *murano-conductor* have only one thing >>> in common with other two components: code shared under *murano-common*. >>> That repository may be eventually eliminated by moving to Oslo (as it >>> should be done). >>> >>> >Also, it has been suggested to move our agents (both windows and >>> unified python) into the main repository as well - just to put them into a >>> separate subfolder. I don't see any reasons why they should be separated >>> from core Murano: I don't believe we are going to have any third-party >>> implementations of our "Unified agent" proposals, while this possibility >>> was the main reason for separatinng them. >>> >>> Main reason for murano-agent to have separate repository was not a >>> possibility to have another implementation, but that all sources that >>> should be able to be built as package, have tests and can be uploaded to >>> PyPI (or any other gate job) should be placed in different repository. >>> OpenStack CI have several rules regarding how repositories should be >>> organized to support running different gate jobs. For example, to run tests >>> *tox.ini* is need to be present in root directory, to build package >>> *setup.py* should be present in root directory. So we could not simply >>> move them to separate directories in main repository and have same >>> capabilities as in separate repository. >>> >>> >Next, deployment scripts and auto-generated docs: are there reasons >>> why they should be in their own repositories, instead of "docs" and >>> "tools/deployment" folders of the primary repo? I would prefer the latter: >>> docs and deployment scripts have no meaning without the sources which they >>> document/deploy - so it is better to have them consistent. >>> We have *developers documentation* alongside with all sources: >>> murano-conductor<https://github.com/stackforge/murano-conductor/tree/master/doc/source>, >>> murano-api<https://github.com/stackforge/murano-api/tree/master/doc/source> >>> and >>> so on. It is true that we have not so much documentation there, and not >>> much code is documented to add auto-generated documentation. Documentation >>> that is found in *murano-docs* repository actually is a docbook >>> documentation, that is presented in book manner, and follows documentation >>> patterns found in core projects itself: >>> openstack-manuals<https://github.com/openstack/openstack-manuals/tree/master/doc> >>> . >>> >>> *murano-deployment* contains scripts and other artefacts related to >>> deployment, but not necessary to source code. This repository don't use >>> much of CI capabilities, but raise it is logical place where we can place >>> different thing related to deployment: various scripts, specs, patches and >>> so on. Also with separate repository we can not to spam our deployment >>> engineers with software engineers related commits. >>> >>> >>> >>> On Tue, Jan 21, 2014 at 11:55 PM, Alexander Tivelkov < >>> ativel...@mirantis.com> wrote: >>> >>>> Hi folks, >>>> >>>> As we are moving towards incubation application, I took a closer look >>>> at what is going on with our repositories. >>>> An here is what I found. We currently have 11 repositories at >>>> stackforge: >>>> >>>> - murano-api >>>> - murano-conductor >>>> - murano-repository >>>> - murano-dashboard >>>> - murano-common >>>> - python-muranoclient >>>> - murano-metadataclient >>>> - murano-agent >>>> - murano-docs >>>> - murano-tests >>>> - murano-deployment >>>> >>>> This enourmous amount of repositories adds too much infrustructural >>>> complexity, and maintaining the changes in in consistent and reliable >>>> manner becomes a really tricky tasks. We often have changes which require >>>> modifing two or more repositories - and thus we have to make several >>>> changesets in gerrit, targeting different repositories. Quite often the >>>> dependencies between these changesets are not obvious, the patches get >>>> reviewed and approved on wrong order (yes, this also questions the quality >>>> of the code review, but that is a different topic), which causes in >>>> inconsostent state of the repositories. >>>> >>>> Well, anyway, this has to be changed in some way or another. >>>> I suggest to initiate the discussions on how to do all this. >>>> >>>> Below you may find my position. This is not final in any meaning, just >>>> a proposal. Please, feel free to discuss :) >>>> >>>> First of all, I would suggest to have a single reposository for all the >>>> three main components of Murano: main murano API (the contents of the >>>> present), workflow execution engine (currently murano-conductor; also it >>>> was suggested to rename the component itself to murano-engine for more >>>> consistent naming) and metadata repository (currently murano-repository). >>>> These should remain as independent modules, being able to run as >>>> different daemons, but stored within a single repository (similar to how >>>> heat has heat-api, heat-cfn and heat-engine under the same hood). The name >>>> of this repository is tentative: I think none of the existing match, so I >>>> would suggest to create a new repo (simple "murano" seems to fit the best), >>>> and then relocate all the content from other 3 repos and remove them >>>> aftwerwards. >>>> >>>> When the api, the repository and the engine are merged into a single >>>> repo, there will be no sense in having murano-common repo for storing their >>>> common classes: instead, there should be a "common" package inside the main >>>> murano repository. >>>> >>>> Also, it has been suggested to move our agents (both windows and >>>> unified python) into the main repository as well - just to put them into a >>>> separate subfolder. I don't see any reasons why they should be separated >>>> from core Murano: I don't believe we are going to have any third-party >>>> implementations of our "Unified agent" proposals, while this possibility >>>> was the main reason for separatinng them. >>>> >>>> Next, deployment scripts and auto-generated docs: are there reasons why >>>> they should be in their own repositories, instead of "docs" and >>>> "tools/deployment" folders of the primary repo? I would prefer the latter: >>>> docs and deployment scripts have no meaning without the sources which they >>>> document/deploy - so it is better to have them consistent. >>>> >>>> >>>> Then, the python bindings: "There can be only one" (c). Yes, the >>>> metadata API and the main murano API are different indeed, however there is >>>> no reason in having two repositories for their clients: let's have a single >>>> repo, containing two packages inside. Are there any technical reasons >>>> preventing us from doing that? >>>> CLI should be common as well - I think there should be a single >>>> command-line utility ("murano" should be the name), allowing to query both >>>> APIs. This CLI will eventually evolve into the developer's utility: it will >>>> get commands to package, sign and submit application packages. >>>> >>>> Openstack Dashboard plugin - aka Murano-dashboard - should remain in a >>>> separated repo, I have no objections here :) >>>> >>>> murano-tests may reamin independent as well - however, this repository >>>> is not likely to be transferred when we go to incubation: incubated >>>> projects should have tempest test in their repositories, shoudn't they? Our >>>> our test may remain on stackforge - this is irrelevant. >>>> >>>> And finally, we need some place to store sources of our metadata >>>> objects: the definition of core murano library, as well as example >>>> services, with all their stuff - metadata and ui definitions, heat >>>> templates, scripts etc. Here I propose to create a new repo, specially >>>> dedicated for this purpose. If we succeed in building the ecosystem for >>>> application developers and publishers, this will be the repo in which they >>>> should contribute, while the core murano repo's will remain relativele >>>> stable. >>>> >>>> >>>> So, this brings us to the following list of repos: >>>> >>>> - *murano* - main services, common, agents docs, deployments scripts >>>> - *python-muranoclient* - python bindings and CLI >>>> - *murano-dashboard* - OS Dashboard plugin >>>> - *murano-apps* - new repo for metadata, including core library and >>>> example apps. >>>> - *murano-tests* - existing test-repo, not going to be transferred >>>> when incubated. >>>> >>>> >>>> This leaves us with just 4 repositories (plus one additional which will >>>> remain on stackforge), with clear separation of concerns. >>>> >>>> There may be technical issues in doing this mergement (we do not want >>>> to loose revision history, do we?), but they should be solvable (I'll write >>>> to infra asking on what is possible and what is not), but in general this >>>> is the direction in which we should be moving, as it seems to me. >>>> >>>> -- >>>> Regards, >>>> Alexander Tivelkov >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> >>> -- >>> Serg Melikyan, Senior Software Engineer at Mirantis, Inc. >>> http://mirantis.com | smelik...@mirantis.com >>> >>> +7 (495) 640-4904, 0261 >>> +7 (903) 156-0836 >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc.
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev