On Fri, Jan 17, 2014 at 9:47 PM, Nirmal Fernando <nir...@wso2.com> wrote:
> Huge +1 for the proposed solution. One small question, when you say > components/features are those imply dependencies (in current terms) as > well? > With respect to orbit bundles, we will create them and immediately upload them to Nexus. With respected to forked code (dependencies such as Axis2/Synapse etc. we will need to maintain separate repos. In addition, forking should be a final resort. Such dependencies will also be uploaded to Nexus several times a day by Bamboo. > > > On Fri, Jan 17, 2014 at 9:26 PM, Afkham Azeez <az...@wso2.com> wrote: > >> [Sorry for the very long mail. I want to document all that I had in >> mind & the stuff we discussed. I would recommend all devs to take some time >> to read this] >> >> >> I would like to summarize he discussion we had a couple of days back. >> >> *The Problems* >> The problems we are trying to solve are as follows: >> >> 1. Trunk & branches structures being completely different >> 2. Branches containing directories with version numbers >> 3. It is impossible to move to GitHub with the current structure because >> of #2 >> 4. It is very easy to break the build by changing already released code. >> The room for human error is high. >> 5. Bamboo builds are eternally broken because the build fails at some >> point & Bamboo cannot continue any further >> 6. When we branch, the trunk quickly becomes obsolete, and remains in >> that broken state until the next major platform release. >> 7. Everybody has to build all components/features, even if those are not >> related to their products >> 8. Fixed versions in branches instead of using SNAPSHOT versions. This >> makes it impossible to upload build artifacts to Maven/Nexus repos. This >> leads to #7. >> 9. Impossible to integrate code quality tools such as EraInsight because >> of #5 >> >> *Proposed solution* >> We have come up with the following solution after much deliberation & >> thought. >> >> Rationale: >> We started looking at other open source projects out there. We took Axis2 >> as an example. Axis2 had many dependencies including Axiom, XmlSchema, >> Woden, WSS4J etc. Those 3rd party dependencies were also developed by some >> Axis2 contributors, but we never branched all of those together and brought >> them into the same code branch. We used to start what we used to call a >> release train, where the upstream code would have to be released first >> before the downstream code such as Axis2 & Synapse could be released. This >> way, we never had any of the problems outlined above. >> >> If you look at the WSO2 product releases, again, the scenario is not that >> much different from the Axis2/Synapse releases. Components & features are >> simply dependencies of the products. In order to get product releases out, >> we first need releases of those dependencies (components/features). So the >> proposal is to identify the top level components/features, and first >> release that code before doing product releases. Each of those components >> will have a GitHub repo. All active development will be done on the main >> branch of those components, and will be branched when they are close to the >> release. Instead of granting commit rights to all, we could only allow the >> primary developers of those components to commit, and others can send pull >> requests, which will be merged in by the primary owners after reviewing. >> These component teams could even have an internal Git repo or clone, and >> commit to that, run all tests, and when they are satisfied that the code is >> in a good state to be merged into the main branch, they can send a pull >> request & merge the changes. We would run Bamboo which would build the >> components several times a day & deploy them to the Nexus repo. That way, >> you would not have to build code that is not directly related to what you >> are working on, which would save 100s of valuable dev hours. >> >> In the majority of the cases, the P2 features correspond to one or more >> related components. So we would keep the feature close to the component. >> There are some cases where these components & features belong in the >> product itself. AppFactory components are a good example. >> >> Products will also have their own GitHub repos. We will have separate >> GitHub repos for platform integration tests. Products such as API Manager >> may opt to have the API Management feature in the product itself. We would >> have a separate P2-repo GitHub repo which would build & deploy the >> compatible set of P2 features. >> >> We will have Bamboo plans at multiple levels; components, products, >> platform, p2-repo and so on. >> >> We will not change any package structures at this time. We would defer >> that to Carbon 5 (if necessary). We will get rid of the chunk based release >> model. Continuous delivery is our ultimate aim & for that to happen, we >> have to release a compatible set of features. We are going to rely heavily >> on automation, automated builds & deploys to Maven. The developers will do >> the majority of the QA (just like the Apache Stratos team is doing now). >> >> We will use the Maven release plugin to create releases & upload them to >> the Maven repo. >> >> *Implications to WSO2 code developers* >> 1. Life becomes easier because you don't have to spend a lot of time >> building unrelated stuff >> 2. General development would happen in the main branch. >> 3. Close to a release, branches would be cut. >> 4. Once release branches are cut, fixes would be done in the main branch, >> and pull requests would be sent to the branch & merged in. This will be >> easy because we no longer have the trunk & branch structure being different. >> >> >> *Implications to users* >> Users will not see any difference in the components & features because we >> are not changing packages or binary structure. In fact, we would have a P2 >> repo with compatible features which all work together. >> >> Senaka, Isuruwan & Harshana have already started working on a PoC. >> >> Thoughts welcome. Those who were in these discussions, please add >> anything I may have missed. >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **az...@wso2.com* <az...@wso2.com> >> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > > Thanks & regards, > Nirmal > > Senior Software Engineer- Platform Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > _______________________________________________ > Dev mailing list > d...@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **az...@wso2.com* <az...@wso2.com> * cell: +94 77 3320919 blog: **http://blog.afkham.org*<http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> * linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
