Proposal for hosting our own set of modules

2014-06-23 Thread Michael Scherer
Hi,

so following the discussion on puppet module, I would propose that we
create a new infra-puppet-modules module on gerrit. 

The division between this module and the current one would be like
this :

infra-puppet would hold the manifests, the site, the Puppetfile , etc

infra-puppet-modules would hold directories, 1 per module we develop
(not the external one we use, they would still be on github or anywhere,
pulled by librarian-puppet) .

Since we are using librarian-puppet
( https://github.com/rodjek/librarian-puppet ) and r10k
( https://github.com/adrienthebo/r10k ) , it requires use to have the
modules/ directory to be managed by librarian-puppet. In turn we need to
have the modules in git, we can address them by path: 

  mod "puppetlabs/apt",
:git => "git://github.com/fake/puppet-modules.git",
:path => "modules/apt"


This wouldn't requires to change much, besides adding the module to Puppetfile 
and creating a git repository.

If no one disagree, I will request the git repository.

( in the mean time, i did create a sample awstats repository for 
stats.ovirt.org, so we can have something to push )

-- 
Michael Scherer
Open Source and Standards, Sysadmin





signature.asc
Description: This is a digitally signed message part
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra


Re: Proposal for hosting our own set of modules

2014-06-23 Thread Ewoud Kohl van Wijngaarden
On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
> Hi,
> 
> so following the discussion on puppet module, I would propose that we
> create a new infra-puppet-modules module on gerrit. 
> 
> The division between this module and the current one would be like
> this :
> 
> infra-puppet would hold the manifests, the site, the Puppetfile , etc
> 
> infra-puppet-modules would hold directories, 1 per module we develop
> (not the external one we use, they would still be on github or anywhere,
> pulled by librarian-puppet) .
> 
> Since we are using librarian-puppet
> ( https://github.com/rodjek/librarian-puppet ) and r10k

We're only using r10k, not librarian-puppet.

> ( https://github.com/adrienthebo/r10k ) , it requires use to have the
> modules/ directory to be managed by librarian-puppet. In turn we need to
> have the modules in git, we can address them by path: 
> 
>   mod "puppetlabs/apt",
> :git => "git://github.com/fake/puppet-modules.git",
> :path => "modules/apt"
> 
> 
> This wouldn't requires to change much, besides adding the module to 
> Puppetfile and creating a git repository.
> 
> If no one disagree, I will request the git repository.
> 
> ( in the mean time, i did create a sample awstats repository for 
> stats.ovirt.org, so we can have something to push )

Unless we plan to make them reusable for other projects, I don't see the
benefit. If we do plan to make them reusable, we should IMHO also
publish them on the forge.

Another potential issue is how we decide when to deploy. We could have a
specific commit ID and update our Puppetfile every time, but again,
little benefit over having them in one tree.

In case you're unaware, we already load modules/* (which is managed by
r10k) and site/* (tracked in git). That means we can host our modules in
site and spit off when they are reusable.

And if we do plan on creating modules repos, I'd be in favor of having
one git repo per puppet module since that is what most people would
expect.
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra


Re: Proposal for hosting our own set of modules

2014-06-23 Thread Michael Scherer
Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a
écrit :
> On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
> > Hi,
> > 
> > so following the discussion on puppet module, I would propose that we
> > create a new infra-puppet-modules module on gerrit. 
> > 
> > The division between this module and the current one would be like
> > this :
> > 
> > infra-puppet would hold the manifests, the site, the Puppetfile , etc
> > 
> > infra-puppet-modules would hold directories, 1 per module we develop
> > (not the external one we use, they would still be on github or anywhere,
> > pulled by librarian-puppet) .
> > 
> > Since we are using librarian-puppet
> > ( https://github.com/rodjek/librarian-puppet ) and r10k
> 
> We're only using r10k, not librarian-puppet.

Indeed. 

But in the end, we still use Puppetfile nonetheless, no ?

> > ( https://github.com/adrienthebo/r10k ) , it requires use to have the
> > modules/ directory to be managed by librarian-puppet. In turn we need to
> > have the modules in git, we can address them by path: 
> > 
> >   mod "puppetlabs/apt",
> > :git => "git://github.com/fake/puppet-modules.git",
> > :path => "modules/apt"
> > 
> > 
> > This wouldn't requires to change much, besides adding the module to 
> > Puppetfile and creating a git repository.
> > 
> > If no one disagree, I will request the git repository.
> > 
> > ( in the mean time, i did create a sample awstats repository for 
> > stats.ovirt.org, so we can have something to push )
> 
> Unless we plan to make them reusable for other projects, I don't see the
> benefit. If we do plan to make them reusable, we should IMHO also
> publish them on the forge.
> 
> Another potential issue is how we decide when to deploy. We could have a
> specific commit ID and update our Puppetfile every time, but again,
> little benefit over having them in one tree.

I was under the impression you could just give a tag and so use master ?
( and using branch for development )

> In case you're unaware, we already load modules/* (which is managed by
> r10k) and site/* (tracked in git). That means we can host our modules in
> site and spit off when they are reusable.

It was not very obvious that site/* was for modules, indeed :)

But my fault, I should also have read the doc in the git repository who
clearly say that.

I must also say that using r10k for different environments seems a bit
overkill, as we seem to only have 1 single environment anyway ( but I am
not using r10k usually, as I do not have enough systems for that and
write all stuff myself ). 

> And if we do plan on creating modules repos, I'd be in favor of having
> one git repo per puppet module since that is what most people would
> expect.

One git repo per module is a bit annoying when we are planning to write
a lot of modules. But I guess it all depend on the time it take to
create one. I would definitely prefer a approach of 1 big git as long as
we are "growing" fast and maybe split later, since it permit faster
growth ?


-- 
Michael Scherer
Open Source and Standards, Sysadmin





signature.asc
Description: This is a digitally signed message part
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra


Re: Proposal for hosting our own set of modules

2014-06-24 Thread Ewoud Kohl van Wijngaarden
On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote:
> Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a
> écrit :
> > On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
> > > Hi,
> > > 
> > > so following the discussion on puppet module, I would propose that we
> > > create a new infra-puppet-modules module on gerrit. 
> > > 
> > > The division between this module and the current one would be like
> > > this :
> > > 
> > > infra-puppet would hold the manifests, the site, the Puppetfile , etc
> > > 
> > > infra-puppet-modules would hold directories, 1 per module we develop
> > > (not the external one we use, they would still be on github or anywhere,
> > > pulled by librarian-puppet) .
> > > 
> > > Since we are using librarian-puppet
> > > ( https://github.com/rodjek/librarian-puppet ) and r10k
> > 
> > We're only using r10k, not librarian-puppet.
> 
> Indeed. 
> 
> But in the end, we still use Puppetfile nonetheless, no ?

Correct, but librarian handles dependencies where r10k ignores them.
That means you must manually specificy any dependency.

> > > ( https://github.com/adrienthebo/r10k ) , it requires use to have the
> > > modules/ directory to be managed by librarian-puppet. In turn we need to
> > > have the modules in git, we can address them by path: 
> > > 
> > >   mod "puppetlabs/apt",
> > > :git => "git://github.com/fake/puppet-modules.git",
> > > :path => "modules/apt"
> > > 
> > > 
> > > This wouldn't requires to change much, besides adding the module to 
> > > Puppetfile and creating a git repository.
> > > 
> > > If no one disagree, I will request the git repository.
> > > 
> > > ( in the mean time, i did create a sample awstats repository for 
> > > stats.ovirt.org, so we can have something to push )
> > 
> > Unless we plan to make them reusable for other projects, I don't see the
> > benefit. If we do plan to make them reusable, we should IMHO also
> > publish them on the forge.
> > 
> > Another potential issue is how we decide when to deploy. We could have a
> > specific commit ID and update our Puppetfile every time, but again,
> > little benefit over having them in one tree.
> 
> I was under the impression you could just give a tag and so use master ?
> ( and using branch for development )

You can, but how do we know we want to update and to which version?
branches can easily break compatibility and it will bite you at some
point.

> > In case you're unaware, we already load modules/* (which is managed by
> > r10k) and site/* (tracked in git). That means we can host our modules in
> > site and spit off when they are reusable.
> 
> It was not very obvious that site/* was for modules, indeed :)
> 
> But my fault, I should also have read the doc in the git repository who
> clearly say that.
> 
> I must also say that using r10k for different environments seems a bit
> overkill, as we seem to only have 1 single environment anyway ( but I am
> not using r10k usually, as I do not have enough systems for that and
> write all stuff myself ). 

The alternative is either include them or git submodules. I'm not a fan
of the former and I've seen people have issues with the latter where
people accidentally commit an older version. It's also hard to spot in
reviews because you only see two git commit IDs with no idea if it's an
upgrade or downgrade. That's why a Puppetfile-based solution was chosen.

Minor nitpick: technically we have 2 environments and we should make an
effort to remove master. This is done by changing the default branch to
production and then remove master. The only way I know is logging into
gerrit through SSH and changing infra-puppet.git/HEAD to point to
production instead of master. Then you can use the gerrit UI to delete
the master branch.

> > And if we do plan on creating modules repos, I'd be in favor of having
> > one git repo per puppet module since that is what most people would
> > expect.
> 
> One git repo per module is a bit annoying when we are planning to write
> a lot of modules. But I guess it all depend on the time it take to
> create one. I would definitely prefer a approach of 1 big git as long as
> we are "growing" fast and maybe split later, since it permit faster
> growth ?

That's exactly what I was thinking.
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra


Re: Proposal for hosting our own set of modules

2014-06-24 Thread Michael Scherer
Le mardi 24 juin 2014 à 09:59 +0200, Ewoud Kohl van Wijngaarden a
écrit :
> On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote:
> > Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a
> > écrit :
> > > On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:

> > > > ( https://github.com/adrienthebo/r10k ) , it requires use to have the
> > > > modules/ directory to be managed by librarian-puppet. In turn we need to
> > > > have the modules in git, we can address them by path: 
> > > > 
> > > >   mod "puppetlabs/apt",
> > > > :git => "git://github.com/fake/puppet-modules.git",
> > > > :path => "modules/apt"
> > > > 
> > > > 
> > > > This wouldn't requires to change much, besides adding the module to 
> > > > Puppetfile and creating a git repository.
> > > > 
> > > > If no one disagree, I will request the git repository.
> > > > 
> > > > ( in the mean time, i did create a sample awstats repository for 
> > > > stats.ovirt.org, so we can have something to push )
> > > 
> > > Unless we plan to make them reusable for other projects, I don't see the
> > > benefit. If we do plan to make them reusable, we should IMHO also
> > > publish them on the forge.
> > > 
> > > Another potential issue is how we decide when to deploy. We could have a
> > > specific commit ID and update our Puppetfile every time, but again,
> > > little benefit over having them in one tree.
> > 
> > I was under the impression you could just give a tag and so use master ?
> > ( and using branch for development )
> 
> You can, but how do we know we want to update and to which version?
> branches can easily break compatibility and it will bite you at some
> point.

It depend on the release model of our own modules. We can have a policy
of "master should be deployable and forever retro compatible" but once
expressed like this, it doesn't sound sustainable (even if that's what
we would do with a single git repository ). 


> > > In case you're unaware, we already load modules/* (which is managed by
> > > r10k) and site/* (tracked in git). That means we can host our modules in
> > > site and spit off when they are reusable.
> > 
> > It was not very obvious that site/* was for modules, indeed :)
> > 
> > But my fault, I should also have read the doc in the git repository who
> > clearly say that.
> > 
> > I must also say that using r10k for different environments seems a bit
> > overkill, as we seem to only have 1 single environment anyway ( but I am
> > not using r10k usually, as I do not have enough systems for that and
> > write all stuff myself ). 
> 
> The alternative is either include them or git submodules. I'm not a fan
> of the former and I've seen people have issues with the latter where
> people accidentally commit an older version. It's also hard to spot in
> reviews because you only see two git commit IDs with no idea if it's an
> upgrade or downgrade. That's why a Puppetfile-based solution was chosen.

I am not fan of git submodule either. But the Puppetfile file part is
good, it just the multiple env that I found weird. Now, if that's
unavoidable, no problem.


> Minor nitpick: technically we have 2 environments and we should make an
> effort to remove master. This is done by changing the default branch to
> production and then remove master. The only way I know is logging into
> gerrit through SSH and changing infra-puppet.git/HEAD to point to
> production instead of master. Then you can use the gerrit UI to delete
> the master branch.

As we discussed on IRC later, I would be +1 for this, provided that this
doesn't break too much assumption in docs ( especially gerrit doc around
). But as we discussed too, maybe we should just push to use git-review.

> > > And if we do plan on creating modules repos, I'd be in favor of having
> > > one git repo per puppet module since that is what most people would
> > > expect.
> > 
> > One git repo per module is a bit annoying when we are planning to write
> > a lot of modules. But I guess it all depend on the time it take to
> > create one. I would definitely prefer a approach of 1 big git as long as
> > we are "growing" fast and maybe split later, since it permit faster
> > growth ?
> 
> That's exactly what I was thinking.

This also bring the question of using external module vs using our own.
I am more the kind of guy that write his own (while I am not the kind of
guy who write his own code), mostly because I started to use puppet
before the forge was up, and because I did have very precise ideas on
what I wanted to achieve, but I am not against using stuff if that's the
best practices :)

I would however make sure we have proper guidelines on when we take a
module, as I am not sure they are all equally good :/

-- 
Michael Scherer
Open Source and Standards, Sysadmin





signature.asc
Description: This is a digitally signed message part
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/l