> On 10 Sep 2015, at 12:18, Alexander Shorin <kxe...@gmail.com> wrote: > > On Thu, Sep 10, 2015 at 12:40 PM, Dale Harvey <d...@arandomurl.com> wrote: >> I dont think CI is a dream, It should take an afternoon at most to get >> CouchDB setup on travis on a single platform to ensure no major regressions >> come though. If anyone wants help doing that feel free to ping me on >> #pouchdb / #couchdb, we already test couchdb master in travis. > > It is a dream for now. > > We have travis enabled, but it has practical flaw: it cannot handle > changes that affected multiple repositories at the same time. Also, > for each repository we have to build full CouchDB to test what is a > bit overkill. > >> I do think multirepos is an issue and that solution is not so simple, we >> went through the same issues with pouchdb as we attempted to split out our >> repository. > > Well, there was a good theory that each of our repository will contain > an app which could be reused outside of CouchDB or being easily > replaced by alternative implementation. Suddenly, for now most of our > apps are heavy coupled
There are two distinct functions that git does for us here: 1. Separate code into different modules. 2. Make a module available to downstream users. In a Node/npm, Python/pip, Ruby/Gems world, 1. is solved by git, 2. is solved by npm/pip/Gems. Is there a way to publish an Erlang module for downstream users independently of git? If yes, we could move back to a single-git-tree model for CouchDB, but we can still release parts as independent modules. I know there have been a few attempts at Erlang package managers, but has anything taken off? * * * For Hoodie, we came up with this, based on our experience with using the full top level test suite on each module (it is indeed overkill): - separate modules - one module that is the top level module (like our couchdb.git) that only includes module dependencies and a top-level integration test suite. - two release trains: stable and next. - a service that attempts to create a new version of the top level module with every commit to a dependent module. E.g. we automatically bump the dependency in a checkout of the top level module, run the integration tests, if they fail, nothing happens and we report that back on the commit to the dependent module. If it succeeds, we automatically determine the next version number on the next release train (SemVer makes that easy) and release the new top level module with the updated dependency. - once we are comfortable that the next release train is stable enough for a release, we change switch out the release trains: next becomes stable, and we have a new empty next train (npm uses release tags for this, but the mechanics are irrelevant, although something like that needs to be supported in the package manager). This requires: - Strict SemVer on all modules (we use https://github.com/semantic-release/semantic-release) to make sure humans can’t screw this up. - Breaking change detection, so we don’t accidentally release a BC break in a non-major version (available as semantic-release plugins). - the service that does the auto-update dance described above. This is a lot of infrastructure (and some of it we are still developing), but once in place, it is really simple to not screw this up and it takes a huge cognitive load off of everyday development. Semantic-Release is language agnostic in theory, but afaik there are only Node and Python versions, so we’d have to find someone to make an Erlang variant. Also, the process uses npm heavily, so some sort of package management system would be nice. Eventually, I’d love to see this for CouchDB. PS: The person who invented semantic-release works for my company, and I’d be more than happy to have them have a crack at this. If anyone wants to sponsor an erlang-semantic-release, please get in touch. * * * Either way, this is a bit of a long shot. For now: We should do what Dale proposed in 2.: pin top level dependencies. I believe that was the plan all along, but with a fast moving 2.0 we just didn’t do it yet. Now is the time to pin versions and manually upgrade them after thorough testing (can still be in PRs and tested on Travis and everything). That means all of these modules (and their sub-deps) should get releases and release tags and those tags need to be filled into our top-level rebar.config.script: https://github.com/apache/couchdb/blob/master/rebar.config.script#L19-L53 Let’s get going :) Thanks Jan -- > >> 1. Dont split out into lots of repositories, if you put those components >> inside the CouchDB repo, then they will get the CouchDB tests run against >> them when changes are made and you wont break the CouchDB repo. > > It's good in retrospective, but useless now. We have to decide how our > CI will work: with multirepo layout or first fold all core repos back > to the one. I think we are ready to revise this too and probably this > should be done before CI work. > >> 2. Anything that does live outside the CouchDB repo, pin their version >> inside the CouchDB repo, dont have commits to subproject X automatically be >> applied to CouchDB, that means you can commit whatever you want to X but >> CouchDB will still be working, when you come to update the version of X you >> pinned, you can see that it breaks and not update until it is fixed. > > It's good strategy for milestones or when you don't have to manage 40+ repos > (: > >> 2.5 Another strategy that can help with this is to have the full CouchDB >> test suite run inside X, so changes to X will run the full CouchDB suite >> with an updated X > > This works, but until changes are belongs to a single repository. See > above about travis issue. > > > P.S. Jason, you are right again (: > > -- > ,,,^..^,,, -- Professional Support for Apache CouchDB: http://www.neighbourhood.ie/couchdb-support/
