Proposal for hosting our own set of modules
Hi, so following the discussion on puppet module, I would propose that we create a new infra-puppet-modules module on gerrit. The division between this module and the current one would be like this : infra-puppet would hold the manifests, the site, the Puppetfile , etc infra-puppet-modules would hold directories, 1 per module we develop (not the external one we use, they would still be on github or anywhere, pulled by librarian-puppet) . Since we are using librarian-puppet ( https://github.com/rodjek/librarian-puppet ) and r10k ( https://github.com/adrienthebo/r10k ) , it requires use to have the modules/ directory to be managed by librarian-puppet. In turn we need to have the modules in git, we can address them by path: mod "puppetlabs/apt", :git => "git://github.com/fake/puppet-modules.git", :path => "modules/apt" This wouldn't requires to change much, besides adding the module to Puppetfile and creating a git repository. If no one disagree, I will request the git repository. ( in the mean time, i did create a sample awstats repository for stats.ovirt.org, so we can have something to push ) -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Proposal for hosting our own set of modules
On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote: > Hi, > > so following the discussion on puppet module, I would propose that we > create a new infra-puppet-modules module on gerrit. > > The division between this module and the current one would be like > this : > > infra-puppet would hold the manifests, the site, the Puppetfile , etc > > infra-puppet-modules would hold directories, 1 per module we develop > (not the external one we use, they would still be on github or anywhere, > pulled by librarian-puppet) . > > Since we are using librarian-puppet > ( https://github.com/rodjek/librarian-puppet ) and r10k We're only using r10k, not librarian-puppet. > ( https://github.com/adrienthebo/r10k ) , it requires use to have the > modules/ directory to be managed by librarian-puppet. In turn we need to > have the modules in git, we can address them by path: > > mod "puppetlabs/apt", > :git => "git://github.com/fake/puppet-modules.git", > :path => "modules/apt" > > > This wouldn't requires to change much, besides adding the module to > Puppetfile and creating a git repository. > > If no one disagree, I will request the git repository. > > ( in the mean time, i did create a sample awstats repository for > stats.ovirt.org, so we can have something to push ) Unless we plan to make them reusable for other projects, I don't see the benefit. If we do plan to make them reusable, we should IMHO also publish them on the forge. Another potential issue is how we decide when to deploy. We could have a specific commit ID and update our Puppetfile every time, but again, little benefit over having them in one tree. In case you're unaware, we already load modules/* (which is managed by r10k) and site/* (tracked in git). That means we can host our modules in site and spit off when they are reusable. And if we do plan on creating modules repos, I'd be in favor of having one git repo per puppet module since that is what most people would expect. ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Proposal for hosting our own set of modules
Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a écrit : > On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote: > > Hi, > > > > so following the discussion on puppet module, I would propose that we > > create a new infra-puppet-modules module on gerrit. > > > > The division between this module and the current one would be like > > this : > > > > infra-puppet would hold the manifests, the site, the Puppetfile , etc > > > > infra-puppet-modules would hold directories, 1 per module we develop > > (not the external one we use, they would still be on github or anywhere, > > pulled by librarian-puppet) . > > > > Since we are using librarian-puppet > > ( https://github.com/rodjek/librarian-puppet ) and r10k > > We're only using r10k, not librarian-puppet. Indeed. But in the end, we still use Puppetfile nonetheless, no ? > > ( https://github.com/adrienthebo/r10k ) , it requires use to have the > > modules/ directory to be managed by librarian-puppet. In turn we need to > > have the modules in git, we can address them by path: > > > > mod "puppetlabs/apt", > > :git => "git://github.com/fake/puppet-modules.git", > > :path => "modules/apt" > > > > > > This wouldn't requires to change much, besides adding the module to > > Puppetfile and creating a git repository. > > > > If no one disagree, I will request the git repository. > > > > ( in the mean time, i did create a sample awstats repository for > > stats.ovirt.org, so we can have something to push ) > > Unless we plan to make them reusable for other projects, I don't see the > benefit. If we do plan to make them reusable, we should IMHO also > publish them on the forge. > > Another potential issue is how we decide when to deploy. We could have a > specific commit ID and update our Puppetfile every time, but again, > little benefit over having them in one tree. I was under the impression you could just give a tag and so use master ? ( and using branch for development ) > In case you're unaware, we already load modules/* (which is managed by > r10k) and site/* (tracked in git). That means we can host our modules in > site and spit off when they are reusable. It was not very obvious that site/* was for modules, indeed :) But my fault, I should also have read the doc in the git repository who clearly say that. I must also say that using r10k for different environments seems a bit overkill, as we seem to only have 1 single environment anyway ( but I am not using r10k usually, as I do not have enough systems for that and write all stuff myself ). > And if we do plan on creating modules repos, I'd be in favor of having > one git repo per puppet module since that is what most people would > expect. One git repo per module is a bit annoying when we are planning to write a lot of modules. But I guess it all depend on the time it take to create one. I would definitely prefer a approach of 1 big git as long as we are "growing" fast and maybe split later, since it permit faster growth ? -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Proposal for hosting our own set of modules
On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote: > Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a > écrit : > > On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote: > > > Hi, > > > > > > so following the discussion on puppet module, I would propose that we > > > create a new infra-puppet-modules module on gerrit. > > > > > > The division between this module and the current one would be like > > > this : > > > > > > infra-puppet would hold the manifests, the site, the Puppetfile , etc > > > > > > infra-puppet-modules would hold directories, 1 per module we develop > > > (not the external one we use, they would still be on github or anywhere, > > > pulled by librarian-puppet) . > > > > > > Since we are using librarian-puppet > > > ( https://github.com/rodjek/librarian-puppet ) and r10k > > > > We're only using r10k, not librarian-puppet. > > Indeed. > > But in the end, we still use Puppetfile nonetheless, no ? Correct, but librarian handles dependencies where r10k ignores them. That means you must manually specificy any dependency. > > > ( https://github.com/adrienthebo/r10k ) , it requires use to have the > > > modules/ directory to be managed by librarian-puppet. In turn we need to > > > have the modules in git, we can address them by path: > > > > > > mod "puppetlabs/apt", > > > :git => "git://github.com/fake/puppet-modules.git", > > > :path => "modules/apt" > > > > > > > > > This wouldn't requires to change much, besides adding the module to > > > Puppetfile and creating a git repository. > > > > > > If no one disagree, I will request the git repository. > > > > > > ( in the mean time, i did create a sample awstats repository for > > > stats.ovirt.org, so we can have something to push ) > > > > Unless we plan to make them reusable for other projects, I don't see the > > benefit. If we do plan to make them reusable, we should IMHO also > > publish them on the forge. > > > > Another potential issue is how we decide when to deploy. We could have a > > specific commit ID and update our Puppetfile every time, but again, > > little benefit over having them in one tree. > > I was under the impression you could just give a tag and so use master ? > ( and using branch for development ) You can, but how do we know we want to update and to which version? branches can easily break compatibility and it will bite you at some point. > > In case you're unaware, we already load modules/* (which is managed by > > r10k) and site/* (tracked in git). That means we can host our modules in > > site and spit off when they are reusable. > > It was not very obvious that site/* was for modules, indeed :) > > But my fault, I should also have read the doc in the git repository who > clearly say that. > > I must also say that using r10k for different environments seems a bit > overkill, as we seem to only have 1 single environment anyway ( but I am > not using r10k usually, as I do not have enough systems for that and > write all stuff myself ). The alternative is either include them or git submodules. I'm not a fan of the former and I've seen people have issues with the latter where people accidentally commit an older version. It's also hard to spot in reviews because you only see two git commit IDs with no idea if it's an upgrade or downgrade. That's why a Puppetfile-based solution was chosen. Minor nitpick: technically we have 2 environments and we should make an effort to remove master. This is done by changing the default branch to production and then remove master. The only way I know is logging into gerrit through SSH and changing infra-puppet.git/HEAD to point to production instead of master. Then you can use the gerrit UI to delete the master branch. > > And if we do plan on creating modules repos, I'd be in favor of having > > one git repo per puppet module since that is what most people would > > expect. > > One git repo per module is a bit annoying when we are planning to write > a lot of modules. But I guess it all depend on the time it take to > create one. I would definitely prefer a approach of 1 big git as long as > we are "growing" fast and maybe split later, since it permit faster > growth ? That's exactly what I was thinking. ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Proposal for hosting our own set of modules
Le mardi 24 juin 2014 à 09:59 +0200, Ewoud Kohl van Wijngaarden a écrit : > On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote: > > Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a > > écrit : > > > On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote: > > > > ( https://github.com/adrienthebo/r10k ) , it requires use to have the > > > > modules/ directory to be managed by librarian-puppet. In turn we need to > > > > have the modules in git, we can address them by path: > > > > > > > > mod "puppetlabs/apt", > > > > :git => "git://github.com/fake/puppet-modules.git", > > > > :path => "modules/apt" > > > > > > > > > > > > This wouldn't requires to change much, besides adding the module to > > > > Puppetfile and creating a git repository. > > > > > > > > If no one disagree, I will request the git repository. > > > > > > > > ( in the mean time, i did create a sample awstats repository for > > > > stats.ovirt.org, so we can have something to push ) > > > > > > Unless we plan to make them reusable for other projects, I don't see the > > > benefit. If we do plan to make them reusable, we should IMHO also > > > publish them on the forge. > > > > > > Another potential issue is how we decide when to deploy. We could have a > > > specific commit ID and update our Puppetfile every time, but again, > > > little benefit over having them in one tree. > > > > I was under the impression you could just give a tag and so use master ? > > ( and using branch for development ) > > You can, but how do we know we want to update and to which version? > branches can easily break compatibility and it will bite you at some > point. It depend on the release model of our own modules. We can have a policy of "master should be deployable and forever retro compatible" but once expressed like this, it doesn't sound sustainable (even if that's what we would do with a single git repository ). > > > In case you're unaware, we already load modules/* (which is managed by > > > r10k) and site/* (tracked in git). That means we can host our modules in > > > site and spit off when they are reusable. > > > > It was not very obvious that site/* was for modules, indeed :) > > > > But my fault, I should also have read the doc in the git repository who > > clearly say that. > > > > I must also say that using r10k for different environments seems a bit > > overkill, as we seem to only have 1 single environment anyway ( but I am > > not using r10k usually, as I do not have enough systems for that and > > write all stuff myself ). > > The alternative is either include them or git submodules. I'm not a fan > of the former and I've seen people have issues with the latter where > people accidentally commit an older version. It's also hard to spot in > reviews because you only see two git commit IDs with no idea if it's an > upgrade or downgrade. That's why a Puppetfile-based solution was chosen. I am not fan of git submodule either. But the Puppetfile file part is good, it just the multiple env that I found weird. Now, if that's unavoidable, no problem. > Minor nitpick: technically we have 2 environments and we should make an > effort to remove master. This is done by changing the default branch to > production and then remove master. The only way I know is logging into > gerrit through SSH and changing infra-puppet.git/HEAD to point to > production instead of master. Then you can use the gerrit UI to delete > the master branch. As we discussed on IRC later, I would be +1 for this, provided that this doesn't break too much assumption in docs ( especially gerrit doc around ). But as we discussed too, maybe we should just push to use git-review. > > > And if we do plan on creating modules repos, I'd be in favor of having > > > one git repo per puppet module since that is what most people would > > > expect. > > > > One git repo per module is a bit annoying when we are planning to write > > a lot of modules. But I guess it all depend on the time it take to > > create one. I would definitely prefer a approach of 1 big git as long as > > we are "growing" fast and maybe split later, since it permit faster > > growth ? > > That's exactly what I was thinking. This also bring the question of using external module vs using our own. I am more the kind of guy that write his own (while I am not the kind of guy who write his own code), mostly because I started to use puppet before the forge was up, and because I did have very precise ideas on what I wanted to achieve, but I am not against using stuff if that's the best practices :) I would however make sure we have proper guidelines on when we take a module, as I am not sure they are all equally good :/ -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/l