Re: Deploying perl code
On 29 July 2014 06:11, Paul Johnson wrote: > On Mon, Jul 28, 2014 at 07:51:12PM +0100, Robert Rothenberg wrote: >> On Thu, Jul 24, 2014 at 4:25 PM, David Cantrell >> wrote: >> >> > I'm looking for tools that will make it easy to go from a bunch of code >> > in a release branch on github to an updated bunch of servers, with >> > minimal downtime. If it matters we're using Debian. >> >> At $previous, we deployed (mainly bespoke) Catalyst apps using the >> following toolchain at a small shop: > > If I were designing an environment from scratch I would take a serious > look at docker. I've been using it recently for cpancover.com and, > whilst it still has some rough edges, I really like the concept of > compartmentalising functionality. It's probably not the whole solution > (and in this case in particular it may not be any part of the solution) > but I predict a strong future for docker. (Ha! What do I know?) +1 for docker. You can host your own "docker registry" which then allows you to push/pull docker images to it. Images with shared history will only need to send and store the differences. So, build a company base image that includes your usual libraries and configuration. Then for each app, commit a "Dockerfile" to the repo, which is instructions on how it builds itself into a docker image, using the company base image as the starting point. Then setup your continuous integration server to run "docker build" and "docker push" for every revision that passes its tests. Over on your application servers, get them to reboot or restart the services frequently, pulling updates from the docker registry as they go. (Perhaps using different tags for staging and production though) Achievement unlocked: Continuous delivery
Re: Deploying perl code
On Mon, Jul 28, 2014 at 07:51:12PM +0100, Robert Rothenberg wrote: > On Thu, Jul 24, 2014 at 4:25 PM, David Cantrell > wrote: > > > I'm looking for tools that will make it easy to go from a bunch of code > > in a release branch on github to an updated bunch of servers, with > > minimal downtime. If it matters we're using Debian. > > At $previous, we deployed (mainly bespoke) Catalyst apps using the > following toolchain at a small shop: If I were designing an environment from scratch I would take a serious look at docker. I've been using it recently for cpancover.com and, whilst it still has some rough edges, I really like the concept of compartmentalising functionality. It's probably not the whole solution (and in this case in particular it may not be any part of the solution) but I predict a strong future for docker. (Ha! What do I know?) -- Paul Johnson - p...@pjcj.net http://www.pjcj.net
Re: Deploying perl code
On Thu, Jul 24, 2014 at 4:25 PM, David Cantrell wrote: > Our "deployment process" at work isn't so much a well-documented > dependable repeatable process as a modern dance interpretation of > lemonparty. It needs taking out and shooting. > > ... > > I'm looking for tools that will make it easy to go from a bunch of code > in a release branch on github to an updated bunch of servers, with > minimal downtime. If it matters we're using Debian. > At $previous, we deployed (mainly bespoke) Catalyst apps using the following toolchain at a small shop: - Gitolite to manage security for repos. - Git Flow to manage workflow in git, with a master, devel (staging) branch. - Mini-cpan repos, which existed in git and were mirrored on multiple servers. - The devel servers were basically identical to live servers - Apps were installed in /opt/$name as self-contained installations (with the exception of a shared Perl, but we had the ability to install an alternative Perl there). The installations were git repos with tags for each release, so we could roll back if needed. Each instance of an app ran under a under a unique username. The idea was to build on devel (staging) servers, using Carton, and push new versions to the repo. In reality, we never quite got there. We'd build the app in the user's $HOME/src module (which was the source git repo) and just installed the new version into the live servers and updated git accordingly. - Configuration lived in /etc/opt/$name, and was managed by custom Puppet modules. These also contained symlinked web server configs for sites. (If I knew what I know now, I'd have used a common configuration for the webserver and managed the differences in Plack::Middleware, alas.) Each app has a custom Puppet module that uses a base one but set up databases and installed cron jobs as needed. - Database versioning was handled using DBIC::Migration, which worked well for vanilla Postgres. It was (maybe still is) buggy for PostGIS, so we ended up manually updating the SQL for deployment, upgrades and downgrades in git and just used it for deploying upgrades/downgrades. Database upgrades were multi-step: 1. Add new columns/tables/etc to database with schema that used them; 2. Release code that uses new columns, tables etc.; 3. Release code that no longer used; 4. Update database to delete things no longer used; Generally this worked out ok. We did most of the steps manually (it was a small shop with a few apps), but had helper shell scripts to do this. In theory it could have been automated completely, but we never felt the need.
Re: Deploying perl code
On Fri, 2014-07-25 at 12:08 +0200, James Laver wrote: > On 25 Jul 2014, at 11:54, Andrew Beverley wrote: > > > The main problem is that it seems to be a victim of its own success: > > there is a huge backlog of merge requests. I'd like to provide some > > simple patches to a couple of modules to make them work better for me, > > but have little hope that they'd be merged this side of Christmas. I > > provided a really simple patch a while ago - it's not been touched and > > now no longer merges cleanly. > > I won’t even try and submit a patch after my experience reporting a > documentation bug. I was at the time regularly tweeting with their CTO, > so I let him know on twitter. He then pointed me at the bug tracker, > where I proceeded to file my bug. Immediately, ‘ansibot’ informed me > that my bug would be closed in a few days if I didn’t reformat my bug > to correspond with their designated format for bugs, which included > such things as “steps to reproduce” (because you know, we all have > difficulty reproducing documentation bugs). I think the only way I > actually got that doc bug fixed was by complaining loudly and > repetitively on twitter about how ridiculous their processes are, but > they showed absolutely no willing to make it easier to contribute > towards their projects, so I’m showing absolutely no willing to help > them grow their company. And therein proves my point. I'd love to fork it, but that's not going to help of course. They'd do themselves a massive favour by splitting out the modules à la CPAN.
Re: Deploying perl code
On 25 Jul 2014, at 11:54, Andrew Beverley wrote: > The main problem is that it seems to be a victim of its own success: > there is a huge backlog of merge requests. I'd like to provide some > simple patches to a couple of modules to make them work better for me, > but have little hope that they'd be merged this side of Christmas. I > provided a really simple patch a while ago - it's not been touched and > now no longer merges cleanly. I won’t even try and submit a patch after my experience reporting a documentation bug. I was at the time regularly tweeting with their CTO, so I let him know on twitter. He then pointed me at the bug tracker, where I proceeded to file my bug. Immediately, ‘ansibot’ informed me that my bug would be closed in a few days if I didn’t reformat my bug to correspond with their designated format for bugs, which included such things as “steps to reproduce” (because you know, we all have difficulty reproducing documentation bugs). I think the only way I actually got that doc bug fixed was by complaining loudly and repetitively on twitter about how ridiculous their processes are, but they showed absolutely no willing to make it easier to contribute towards their projects, so I’m showing absolutely no willing to help them grow their company. /j
Re: Deploying perl code
On Fri, 2014-07-25 at 10:11 +0200, James Laver wrote: > Ansible I do like for the most part I'm a fan of Ansible, and am in the process of using it to deploy code (although more by accident than design). The main problem is that it seems to be a victim of its own success: there is a huge backlog of merge requests. I'd like to provide some simple patches to a couple of modules to make them work better for me, but have little hope that they'd be merged this side of Christmas. I provided a really simple patch a while ago - it's not been touched and now no longer merges cleanly.
Re: Deploying perl code
Hey, We're in a similar boat to Leo for our use of Puppet - used to manage stuff in /etc/ and the general Debian packages we have installed - but mostly we build our own. We have an /opt/lokku/pkgs which contains our own Perl, Apache, Percona DB, node.js, etc. We manage /opt/lokku/bin with swpkg - old school technology but it works ;-) For distributing code we just push out tarballs, untar them, flip a symlink and restart services. It's shell scripts with the distribute being based on rdist. We build the tarball with the 'git-archive' command. And to throw some even more ancient tech in there we use wigwam framework for managing roles (environment variables and service lists per role) and services (stop, start, restart, etc.) I'm not proud of it, but we launch multiple times a day so clearly it works. - Alex On 25 July 2014 08:40, mascip wrote: > I've loved using Ansible on a personal project recently. Almost zero set up > and learning. > Compared to bash scripts, I love the reuse with Roles, the fact that many > tasks and roles exist (Ansible Galaxy is Ansible's CPAN), and the > idempotence: you can run a playbook as many times as you like, it should > have just the same effect as running it once (true for most Ansible > things). > > -- Pierre Masci > > > On 25 July 2014 07:52, Ben Tisdall wrote: > > > >> On 24 July 2014 22:31, Paul Makepeace wrote: > > >>> > > >>> capistrano is a (the?) winner for sure. > > >> > > > > I've used Capistrano a bit - it's ok but too much magic for my liking > > (and in general I'm a big fan of the Ruby ecosystem). Fabric is a more > > sensible alternative IMO (you might find > > http://www.slideshare.net/panopticdev/fabric-a-capistrano-alternative > > useful). > > > > However, I would urge you to spend a day each investigating Ansible & > > SaltStack, the latter in salt-ssh mode if you want to make a direct > > comparison. Both of the aforementioned tools do ad-hoc remote > > execution, task orchestration and configuration management. > > > > FWIW I spent 3 days last week evaluating orchestration tools for $WORK > > last week, looking at dsh, Capistrano, Fabric, MCollective (covered > > that in one sentence - it's way heavy), Ansible & SaltStack. I liked > > both Ansible & SaltStack but concluded that the former was the best > > for the project in question because it was easier to get started with > > and it came with a lot of modules that were useful to us out of the > > box. > > > > HTH. > > > > -Ben > > >
Re: Deploying perl code
On 25 Jul 2014, at 09:40, mascip wrote: > and the idempotence: you can run a playbook as many times as you like, it > should > have just the same effect as running it once (true for most Ansible things). That’s in stark contrast to my experiences. I found ansible requires you to think about the side effects of everything so that you can write idempotent code. Where the default puppet library and constraint resolution mode of thinking basically gives you the idempotence for free, ansible pushes the responsibility back to the developer. Chef is even worse at this. Without idempotence for free, the only compelling reason to use it over a shell script is the reusable roles. James
Re: Deploying perl code
On 25 Jul 2014, at 08:52, Ben Tisdall wrote: > However, I would urge you to spend a day each investigating Ansible & > SaltStack, the latter in salt-ssh mode if you want to make a direct > comparison. Both of the aforementioned tools do ad-hoc remote > execution, task orchestration and configuration management. Ansible I do like for the most part, but there are a few sharp edges. I don’t think it was usable up until they put out 1.3 (was it 1.3? it’s been a while since i’ve written a playbook) which added a bunch of useful stuff in the standard library — I was running out of git master for ages. I don’t think it’s the right solution for this case, though. Salt: just no. Really no. I’m really surprised to see a recommendation for it, because it’s a terrible piece of software. James
Re: Deploying perl code
I've loved using Ansible on a personal project recently. Almost zero set up and learning. Compared to bash scripts, I love the reuse with Roles, the fact that many tasks and roles exist (Ansible Galaxy is Ansible's CPAN), and the idempotence: you can run a playbook as many times as you like, it should have just the same effect as running it once (true for most Ansible things). -- Pierre Masci On 25 July 2014 07:52, Ben Tisdall wrote: > >> On 24 July 2014 22:31, Paul Makepeace wrote: > >>> > >>> capistrano is a (the?) winner for sure. > >> > > I've used Capistrano a bit - it's ok but too much magic for my liking > (and in general I'm a big fan of the Ruby ecosystem). Fabric is a more > sensible alternative IMO (you might find > http://www.slideshare.net/panopticdev/fabric-a-capistrano-alternative > useful). > > However, I would urge you to spend a day each investigating Ansible & > SaltStack, the latter in salt-ssh mode if you want to make a direct > comparison. Both of the aforementioned tools do ad-hoc remote > execution, task orchestration and configuration management. > > FWIW I spent 3 days last week evaluating orchestration tools for $WORK > last week, looking at dsh, Capistrano, Fabric, MCollective (covered > that in one sentence - it's way heavy), Ansible & SaltStack. I liked > both Ansible & SaltStack but concluded that the former was the best > for the project in question because it was easier to get started with > and it came with a lot of modules that were useful to us out of the > box. > > HTH. > > -Ben >
Re: Deploying perl code
>> On 24 July 2014 22:31, Paul Makepeace wrote: >>> >>> capistrano is a (the?) winner for sure. >> I've used Capistrano a bit - it's ok but too much magic for my liking (and in general I'm a big fan of the Ruby ecosystem). Fabric is a more sensible alternative IMO (you might find http://www.slideshare.net/panopticdev/fabric-a-capistrano-alternative useful). However, I would urge you to spend a day each investigating Ansible & SaltStack, the latter in salt-ssh mode if you want to make a direct comparison. Both of the aforementioned tools do ad-hoc remote execution, task orchestration and configuration management. FWIW I spent 3 days last week evaluating orchestration tools for $WORK last week, looking at dsh, Capistrano, Fabric, MCollective (covered that in one sentence - it's way heavy), Ansible & SaltStack. I liked both Ansible & SaltStack but concluded that the former was the best for the project in question because it was easier to get started with and it came with a lot of modules that were useful to us out of the box. HTH. -Ben
Re: Deploying perl code
On 24 Jul 2014, at 23:47, Schmoo wrote: > On 24 July 2014 22:31, Paul Makepeace wrote: >> >> capistrano is a (the?) winner for sure. > > Why do these new fangled things all have such off-putting names? Just wait til you find out that someone built a web frontend for it called ‘webistrano’. oops. /j
Re: Deploying perl code
On Thu, Jul 24, 2014 at 2:47 PM, Schmoo wrote: > On 24 July 2014 22:31, Paul Makepeace wrote: >> On Thu, Jul 24, 2014 at 2:06 PM, James Laver wrote: >>> >>> Then I’ll double down on my capistrano/tak recommendation. >> >> capistrano is a (the?) winner for sure. > > Why do these new fangled things all have such off-putting names? > I'm sure the Italians have a similar opinion of Huddersfield.
Re: Deploying perl code
On 24 July 2014 22:31, Paul Makepeace wrote: > On Thu, Jul 24, 2014 at 2:06 PM, James Laver wrote: >> >> Then I’ll double down on my capistrano/tak recommendation. > > capistrano is a (the?) winner for sure. Why do these new fangled things all have such off-putting names?
Re: Deploying perl code
On Thu, Jul 24, 2014 at 2:06 PM, James Laver wrote: > > Then I’ll double down on my capistrano/tak recommendation. capistrano is a (the?) winner for sure. Paul
Re: Deploying perl code
On 24 Jul 2014, at 18:59, David Cantrell wrote: > On Thu, Jul 24, 2014 at 06:34:08PM +0200, James Laver wrote: > >> And if your jenkins isn?t already singing the single-button deployment song, >> make it do so*. > > That does, of course, depend on our code being easily deployable. We're > a long way from that at the moment. It's something to consider once > I can type 'myappdeploy live' and be confident that it will Just Work. Then I’ll double down on my capistrano/tak recommendation. If that’s what you want, it seems like the right tool for the job. Cap is ruby, so perhaps Tak may be more what you want. Or not. I don’t know. James
Re: Deploying perl code
On Thu, Jul 24, 2014 at 06:34:08PM +0200, James Laver wrote: > And if your jenkins isn?t already singing the single-button deployment song, > make it do so*. That does, of course, depend on our code being easily deployable. We're a long way from that at the moment. It's something to consider once I can type 'myappdeploy live' and be confident that it will Just Work. -- David Cantrell | top google result for "topless karaoke murders" Immigration: making Britain great since AD43
Re: Deploying perl code
On 24 Jul 2014, at 17:25, David Cantrell wrote: > I'm looking for tools that will make it easy to go from a bunch of code > in a release branch on github to an updated bunch of servers, with > minimal downtime. If it matters we're using Debian. > > Is this the sort of thing that puppet and chef are for? If you've used > them are they as awesome as the hype makes out, or will they just push > me into the same murderous rage as our current bunch of incomplete shell > scripts do? Puppet/chef isn’t going to work terribly well for this model, imo. I suspect what you’re looking for is capistrano ( or Tak, the perl equivalent (and mst-ware!) ). Puppet is *shitty* if you’re wanting to deal with anything to do with versioning - it makes you go set up a repository server where you pin the versions by hand. I’m always going to recommend you build an app-specific perl and deploy that via rpm/deb, but whether you should build an os package of your code, well, depends on your code. I’m thinking from the sound of this, capistrano would be a better fit. And if your jenkins isn’t already singing the single-button deployment song, make it do so*. James * My company provides consultancy on such matters, if such a thing would interest you and your budget has increased since the last time we spoke about it ;)
Re: Deploying perl code
On 24 July 2014 16:25, David Cantrell wrote: > What tools do you use for: > > * deploying code to multiple servers; > * and multiple environments - eg live, staging, ... > so eg we have api1.live..., api2.live..., ... > We use a combination of: minicpan + CPAN::Mini::Inject All under version control in it's own repo Then build to: /opt/perl_VERSION_dev (under it's own version control) which is rsynced to 'dev' machines /opt/perl_VERSION_live (under it's own version control) is build by recording what was installed into dev and then mimicking that ( a yes/ no prompt for each branch, it works well but is complicated ). > * updated/added dependencies > * both CPANish modules and possibly OS packages such as libraries > that those modules wrap around > Puppet for OS packages, and.. well everything except perl and html/image deployment. > * database structure updates; > Thankfully don't do enough changes for this to be an issue. > I'm looking for tools that will make it easy to go from a bunch of code > in a release branch on github to an updated bunch of servers, with > minimal downtime. If it matters we're using Debian. > > Is this the sort of thing that puppet and chef are for? If you've used > them are they as awesome as the hype makes out, or will they just push > me into the same murderous rage as our current bunch of incomplete shell > scripts do? Puppet 3 with 'stdlib' is great, you can implement one feature at a time, so don't have to switch everything over at once, it IS software but then don't think you can escape that and it's WAY better than dozens of little shell scripts. We now have 99.9% of setup in there, we even configure Debian desktops with it at work. I'm just working on upgrading the metacpan puppet config here... https://github.com/CPAN-API/metacpan-puppet/tree/leo/puppet3 If I was starting work config again from scratch I might look at: A combination of: https://github.com/tokuhirom/Perl-Build https://stratopan.com/ - or Pinto ( https://metacpan.org/pod/Pinto ) https://metacpan.org/pod/Carton Heading to tech talk now TTFN! Leo
