Hi, Alexander, In general I am completely agree with Clint and Robert, and as one of contributors of Murano I don't see any practical reasons for repositories reorganization. And regarding of your proposal I have a few thoughts that I would like to share below:
>This enourmous amount of repositories adds too much infrustructural complexity Creating a new repository is a quick, easy and completely automated procedure that requires only simple commit to Zuul configuration. All infrastructure related to repositories is handled by Openstack CI and supported by Openstack Infra Team, and actually don't require anything from project development team. About what infrastructure complexity you are talking about? >I actually think keeping them separate is a great way to make sure you have ongoing API stability. (c) Clint I would like to share a little statistic gathered by Stan Lagun a little time ago regarding repositories count in different PaaS solution. If you are concerned about large number of repositories used by Murano, you will be quite amused: - https://github.com/heroku - 275 - https://github.com/cloudfoundry - 132 - https://github.com/openshift - 49 - https://github.com/CloudifySource - 46 >First of all, I would suggest to have a single reposository for all the three main components of Murano: main murano API (the contents of the present), workflow execution engine (currently murano-conductor; also it was suggested to rename the component itself to murano-engine for more consistent naming) and metadata repository (currently murano-repository). *murano-api* and *murano-repository* have many things in common, they are both present HTTP API to the user, and I hope would be rewritten to common framework (Pecan?). But *murano-conductor* have only one thing in common with other two components: code shared under *murano-common*. That repository may be eventually eliminated by moving to Oslo (as it should be done). >Also, it has been suggested to move our agents (both windows and unified python) into the main repository as well - just to put them into a separate subfolder. I don't see any reasons why they should be separated from core Murano: I don't believe we are going to have any third-party implementations of our "Unified agent" proposals, while this possibility was the main reason for separatinng them. Main reason for murano-agent to have separate repository was not a possibility to have another implementation, but that all sources that should be able to be built as package, have tests and can be uploaded to PyPI (or any other gate job) should be placed in different repository. OpenStack CI have several rules regarding how repositories should be organized to support running different gate jobs. For example, to run tests *tox.ini* is need to be present in root directory, to build package *setup.py* should be present in root directory. So we could not simply move them to separate directories in main repository and have same capabilities as in separate repository. >Next, deployment scripts and auto-generated docs: are there reasons why they should be in their own repositories, instead of "docs" and "tools/deployment" folders of the primary repo? I would prefer the latter: docs and deployment scripts have no meaning without the sources which they document/deploy - so it is better to have them consistent. We have *developers documentation* alongside with all sources: murano-conductor<https://github.com/stackforge/murano-conductor/tree/master/doc/source>, murano-api <https://github.com/stackforge/murano-api/tree/master/doc/source> and so on. It is true that we have not so much documentation there, and not much code is documented to add auto-generated documentation. Documentation that is found in *murano-docs* repository actually is a docbook documentation, that is presented in book manner, and follows documentation patterns found in core projects itself: openstack-manuals<https://github.com/openstack/openstack-manuals/tree/master/doc> . *murano-deployment* contains scripts and other artefacts related to deployment, but not necessary to source code. This repository don't use much of CI capabilities, but raise it is logical place where we can place different thing related to deployment: various scripts, specs, patches and so on. Also with separate repository we can not to spam our deployment engineers with software engineers related commits. On Tue, Jan 21, 2014 at 11:55 PM, Alexander Tivelkov <[email protected] > wrote: > Hi folks, > > As we are moving towards incubation application, I took a closer look at > what is going on with our repositories. > An here is what I found. We currently have 11 repositories at stackforge: > > - murano-api > - murano-conductor > - murano-repository > - murano-dashboard > - murano-common > - python-muranoclient > - murano-metadataclient > - murano-agent > - murano-docs > - murano-tests > - murano-deployment > > This enourmous amount of repositories adds too much infrustructural > complexity, and maintaining the changes in in consistent and reliable > manner becomes a really tricky tasks. We often have changes which require > modifing two or more repositories - and thus we have to make several > changesets in gerrit, targeting different repositories. Quite often the > dependencies between these changesets are not obvious, the patches get > reviewed and approved on wrong order (yes, this also questions the quality > of the code review, but that is a different topic), which causes in > inconsostent state of the repositories. > > Well, anyway, this has to be changed in some way or another. > I suggest to initiate the discussions on how to do all this. > > Below you may find my position. This is not final in any meaning, just a > proposal. Please, feel free to discuss :) > > First of all, I would suggest to have a single reposository for all the > three main components of Murano: main murano API (the contents of the > present), workflow execution engine (currently murano-conductor; also it > was suggested to rename the component itself to murano-engine for more > consistent naming) and metadata repository (currently murano-repository). > These should remain as independent modules, being able to run as different > daemons, but stored within a single repository (similar to how heat has > heat-api, heat-cfn and heat-engine under the same hood). The name of this > repository is tentative: I think none of the existing match, so I would > suggest to create a new repo (simple "murano" seems to fit the best), and > then relocate all the content from other 3 repos and remove them > aftwerwards. > > When the api, the repository and the engine are merged into a single repo, > there will be no sense in having murano-common repo for storing their > common classes: instead, there should be a "common" package inside the main > murano repository. > > Also, it has been suggested to move our agents (both windows and unified > python) into the main repository as well - just to put them into a separate > subfolder. I don't see any reasons why they should be separated from core > Murano: I don't believe we are going to have any third-party > implementations of our "Unified agent" proposals, while this possibility > was the main reason for separatinng them. > > Next, deployment scripts and auto-generated docs: are there reasons why > they should be in their own repositories, instead of "docs" and > "tools/deployment" folders of the primary repo? I would prefer the latter: > docs and deployment scripts have no meaning without the sources which they > document/deploy - so it is better to have them consistent. > > > Then, the python bindings: "There can be only one" (c). Yes, the metadata > API and the main murano API are different indeed, however there is no > reason in having two repositories for their clients: let's have a single > repo, containing two packages inside. Are there any technical reasons > preventing us from doing that? > CLI should be common as well - I think there should be a single > command-line utility ("murano" should be the name), allowing to query both > APIs. This CLI will eventually evolve into the developer's utility: it will > get commands to package, sign and submit application packages. > > Openstack Dashboard plugin - aka Murano-dashboard - should remain in a > separated repo, I have no objections here :) > > murano-tests may reamin independent as well - however, this repository is > not likely to be transferred when we go to incubation: incubated projects > should have tempest test in their repositories, shoudn't they? Our our test > may remain on stackforge - this is irrelevant. > > And finally, we need some place to store sources of our metadata objects: > the definition of core murano library, as well as example services, with > all their stuff - metadata and ui definitions, heat templates, scripts > etc. Here I propose to create a new repo, specially dedicated for this > purpose. If we succeed in building the ecosystem for application developers > and publishers, this will be the repo in which they should contribute, > while the core murano repo's will remain relativele stable. > > > So, this brings us to the following list of repos: > > - *murano* - main services, common, agents docs, deployments scripts > - *python-muranoclient* - python bindings and CLI > - *murano-dashboard* - OS Dashboard plugin > - *murano-apps* - new repo for metadata, including core library and > example apps. > - *murano-tests* - existing test-repo, not going to be transferred > when incubated. > > > This leaves us with just 4 repositories (plus one additional which will > remain on stackforge), with clear separation of concerns. > > There may be technical issues in doing this mergement (we do not want to > loose revision history, do we?), but they should be solvable (I'll write to > infra asking on what is possible and what is not), but in general this is > the direction in which we should be moving, as it seems to me. > > -- > Regards, > Alexander Tivelkov > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | [email protected] +7 (495) 640-4904, 0261 +7 (903) 156-0836
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
