Hi Harry, Even though I think that your request for having an OAE AMI makes a lot of sense, I'm going to resist executing on it ourselves to try and make a point.
It's important to understand that OAE is an open source project that requires a community of institutions and individuals to participate and contribute for the project to become sustainable. OAE is fortunate to have a number of institutions that are putting time, staff and money into the project, which makes it possible for a set of people to provide some continuity and help move things ahead, but this is not the same as being a vendor product. Just acting on requests from various places in the community and dealing with them ourselves is not helping the sustainability and is potentially distracting. The project is already providing a QA environment [1] that's available for everyone to use, but is redeployed every night. In the medium term future, it will be providing a demo environment that will be retaining data. There is also a repository [2] available that contains all of the puppet scripts required to set up a single box environment, a clustered environment, etc. On top of that, it's also possible to run OAE using Vagrant. I would say that that is already more than what most projects are offering out of the box. As I said, having an AMI makes a lot of sense. However, there will be some work involved in making an AMI available, as well as a lot of time required to maintain that AMI. Therefore, I'd prefer to see these things come in as contributions rather than requests, or at least having a different contribution (e.g. full Chinese translation) would help justify spending some time on the request. I just want to be clear that I'm not picking on the idea or you and I think it's absolutely fine for ideas like these to be floated and discussed on list, but I do think it's sometimes helpful to remind ourselves that we are an open source project and need to be building a community of contributors around that. [1] http://oae.oae-qa0.oaeproject.org [2] https://github.com/oaeproject/puppet-hilary Kind regards, Nicolaas On 10 Oct 2013, at 14:11, Harry Wang <harryjw...@gmail.com> wrote: > Hi Nicolaas and Branden, > > I have tried to configure a demo server using AWS by following the Readme but > ended up with tons of problems due to the various package dependencies and > etc. > > Given that the QA environment is on a single box, I wonder whether it is > possible for you guys to make it into an Amazon Machine Image > (https://aws.amazon.com/amis) so that it is much easier for the community to > setup a testing environment in the cloud? > > Thanks, > > Harry > > > > On Sep 5, 2013, at 5:35 AM, Nicolaas Matthijs > <nicolaas.matth...@caret.cam.ac.uk> wrote: > >> Hi Harry, >> >> As Branden indicated, there's a spectrum of ways in which a system can be >> deployed and configured, determined by the requirements around number of >> concurrent users you want to support, whether or not you want the components >> to be fully redundant, etc. >> >> For example, we have a QA environment that is automatically redeployed every >> night. As this environment is only used for testing and demo purposes, it >> uses a single box with (I believe) 3.75GB of memory and 1 vCPU, which is one >> of the standard Joyent Ubuntu machines, and that seems to be plenty for the >> limited use it is seeing. It does mean that if that box goes down for any >> reason, the environment will be unavailable as well. >> >> The existing production environment is currently driven by redundancy rather >> than number of concurrent users. It means that we deploy each component at >> least 2 times to make sure that there is no downtime associated to losing >> one of the servers. In order to do this, we are currently opting to use a >> larger number of small (a lot of these are single CPU 512MB VMs), rather >> than a smaller number of beefier machines. Because of what needs to be >> deployed to have a minimum fully redundant deployment, we are expecting that >> we won't be hitting the number of concurrent users this environment can >> support anytime soon (in which case we'd scale out and the system would be >> defined by concurrent usage). >> >> Obviously, it is possible to combine any number of components onto a single >> box, but this can cause problems for a highly loaded system where multiple >> components are fighting for the same physical resources. >> >> All of these different configurations can be found in the oae-provisioning >> repository. >> >> Hope that helps, >> Nicolaas >> >> >> On 5 Sep 2013, at 01:34, Branden Visser wrote: >> >>> Hi Harry, what machines you should run on dedicated hardware and how >>> many of them very much depends on your requirements, both performance >>> and SLA. The way the system is divided can be found in our technical >>> overview [1], slide 16 and onward -- this is probably the information >>> you've seen in status updates and presentations. >>> >>> Everything we use to setup our deployment can be found in our puppet >>> scripts [2], and our specific VM provisioning manifests for the Joyent >>> cloud can be found in our oae-provisioning [3] repository. These will >>> give a very detailed view of our configuration. In summary, you would >>> find that we run a fully scaled out environment in production and >>> staging, almost identical to what is on the technical overview slide >>> #16. However, such redundancy and scale might not be necessary for >>> most deployments. >>> >>> We run all Ubuntu machines, but we have had success wildly hammering >>> performance testing environments with smartos (for the app nodes, >>> redis, rabbitmq, nginx) and CentOS (for Cassandra and ElasticSearch). >>> We brought everything onto Ubuntu because puppetizing all the >>> components for multiple OS' (which was necessary for mixed >>> environments like QA) was a total nightmare. >>> >>> Hope that helps, >>> Branden >>> >>> [1] >>> http://www.slideshare.net/nicolaasmatthijs/apereo-oae-architectural-overview >>> [2] https://github.com/oaeproject/puppet-hilary >>> [3] https://github.com/oaeproject/oae-provisioning/blob/master/production >>> >>> On Wed, Sep 4, 2013 at 5:29 PM, Harry Wang <harryjw...@gmail.com> wrote: >>>> Hi All, >>>> >>>> I wonder whether there is any documentation on the recommended OAE >>>> production configuration, such as how many application servers, database >>>> servers, server for preview processing, recommended OS and hardware >>>> configuration, etc. >>>> >>>> Given the many systems used in OAE, this information could be of great >>>> help. >>>> >>>> Thanks! >>>> >>>> Harry >>>> _______________________________________________ >>>> oae-dev mailing list >>>> oae-dev@collab.sakaiproject.org >>>> http://collab.sakaiproject.org/mailman/listinfo/oae-dev >>> _______________________________________________ >>> oae-dev mailing list >>> oae-dev@collab.sakaiproject.org >>> http://collab.sakaiproject.org/mailman/listinfo/oae-dev >> > _______________________________________________ oae-dev mailing list oae-dev@collab.sakaiproject.org http://collab.sakaiproject.org/mailman/listinfo/oae-dev
