Re: Marcus’s second bullet. Separating these into separate branches puts the burden on using CI/CD.
Marlon From: "Christie, Marcus Aaron" <machr...@iu.edu> Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org> Date: Friday, March 10, 2017 at 11:54 AM To: "dev@airavata.apache.org" <dev@airavata.apache.org>, "Shenoy, Gourav Ganesh" <goshe...@indiana.edu> Subject: Re: List of Devops Scripts for Apache Airavata Dev, Eric and I met this morning to discuss how to consolidate these various efforts. Here’s what we came up with: 1. (Marcus) Create an airavata-devops repo for holding all of the Ansible scripts. To start with, we’ll use the dev-tools/ansible scripts. 2. (Eric) Once the airavata-devops repo is in place, Eric will move his Jetstream provisioning scripts onto a branch there 3. (Eric) Eric is also going to start working on Jetstream scripts for other servers besides PGA A question for Gourav: are the modules/cloud/ansible-playbooks still being actively used? Some other things we discussed: * How to securely store secret information? We discussed using ansible-vault versus having a separate, private repo for Ansible inventories. We settled on using ansible-vault since it has the advantage of keeping everything in one repo. The disadvantage is in needing to share a password file out-of-band with the rest of the team. However, we felt the sharing of the password would be mitigated as we move toward more automated systems for running deployments (for example, users who only need to run a deploy won’t need the password file if we use Jenkins to run deploys; only Jenkins would really need to have the password file). * One advantage to NOT having a separate repo for devops scripts is that it keeps the devops scripts versioned together with the code those scripts can build and deploy. Having a separate repo means we will need to have some discipline when updating devops scripts. I think it makes sense to have the devops repo branch structure reflect the airavata branch structure. That is, the ‘develop’ branch of the devops repo should be good for deploying the ‘develop’ branch of airavata and likewise for ‘master’. If code is updated in airavata on a separate branch that needs corresponding devops script updates, that should likewise be done on a same-named separate branch in the devops repo. Eric, let me know if I missed anything. Thanks, Marcus On Mar 9, 2017, at 3:37 PM, Anuj Bhandar <bhandar.a...@gmail.com> wrote: Gourav, It is a great initiative, I vote for a separate repository for devops, the Airavata repository plays many roles already. A separate repository would aid active development and bring modularity. Thanks, Anuj Bhandar On 3/9/17 3:14 PM, Coulter, John Eric wrote: Thanks for starting this, Gourav! I'm inclined to vote for a separate DevOps repo, to keep things more modular. I know Marcus and I have done some work/testing starting from scripts that I think Shameera created in dev-tools/ansible. I've got a side-repo up which contains playbooks for provisioning and deploying the PGA from scratch as a proof of concept, using dynamic inventory instead of a static file (so, on an empty jetstream allocation, you get working instance running the PGA, with router, public ip with a single ansible-playbook command). Next step there is to add provisioning for VMs to run airavata, etc. Script locationBranchPurpose of Script modules/cloud/ansible-playbooks develop 1. Provisions instances on EC2 and OpenStack (Jetstream) 2. Deploys a Mesos/Marathon cluster on the provisioned instances dev-tools/ansible/develop1. Deploy/update airavata services on existing machines/instances 2. todo - include provisioning scripts for cloud resources https://github.com/ECoulter/airavata-vms1. POC for provisioning and deployment of PGA on Jetstream, with dynamic inventory. (intended to merge with dev-tools/ansible when complete/tested by others) --------------- Cheers, --------------------- Eric Coulter jecou...@iu.edu XSEDE Capabilities and Resource Integration Engineer IU Campus Bridging & Research Infrastructure RT/PTI/UITS 812-856-3250
smime.p7s
Description: S/MIME cryptographic signature
