Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
Great thread! At WFU we used reserved AWS instances which lowered our overall costs but committed us to the amazon platform for a year. We also wound up grouping most of our services on a large server (~$87 per month after reservation fee) so that we could take advantage of all of that capacity. Our infrastructure included 3 servers and about 500 GB of storage (Large production server with 90% of library services, 1 small server for High density storage system, 1 small server for puppet/monitoring/documentation). The reservation fees for these servers were around $1380 per year and we paid approximately $275 total per month for computing costs and disk space. Data transfer and other costs were minimal and are included in the $275. A rough yearly cost for these services comes to $4680. A bit more than we were looking at for physical server costs (3 servers for $4000 each with 3 years paid support) but these costs meant that we had a lot more flexibility than we would have had with three servers sitting in our campus IT datacenter (without root access). As a side note, we found that storage space was more expensive than CPU time and wound up keeping our multi-TB storage array on site instead of in the cloud. If I was re-building this today I would explore some other options - RackSpace (Cheaper CPU time), RightScale (Automated server configuration/deployment), Heroku/Google Apps Engine (free PaaS levels) and focus on getting at least a more robust infrastructure at the same cost (if not with some savings). Rackspace for example offers their smallest server at $.015 per hour without reservation fees and RightScale offers a free support level that could work well for small/medium sized libraries. FWIW, when I was pulling together numbers for this email I noticed that Amazon has changed their reservation fees and pricing model. Depending on which reservation fee I selected I either saved about $600 per year or spent $300 more per year using the same infrastructure discussed above (http://aws.amazon.com/ec2/pricing/). If you are really interested in cost calculations and ROI, some helpful resources include Yan Han's recent work in ITAL comparing real-world cloud computing costs - http://ejournals.bc.edu/ojs/index.php/ital/article/view/1871/1709 and Chapter 3 of George Reese's book Cloud Computing Architectures in which he explores some approaches to calculating ROI for cloud services. Erik Erik Mitchell Assistant Professor College of Information Studies University of Maryland, College Park http://erikmitchell.info, http://ischool.umd.edu On Thu, Feb 23, 2012 at 12:38 AM, Tim Spalding t...@librarything.com wrote: We did some tests on it, but found it a very poor fit for a site dependent on huge amount of data which much be present to the basically the whole system all the time and up-to-date. In other words, we found it didn't match a site based on MySQL slaves replicating here and there, and with memcached needing to be spot-on. Under some circumstances we'd consider shuffling some image rendering and delivery tasks to it, but that's about it. Tim LibraryThing
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
On Feb 22, 2012, at 11:52 PM, Cary Gordon wrote: EC2 works for a lot of models, but one that it does not work for is small traffic apps that need to be available 24/7. If you have a small instance (AWS term) running full time with a fixed IP, it costs about $75 a month. If you turn it on for 2 hours a day, it costs about $15/month. A large instance is about $325. Now where it gets interesting is if your app needs a large instance, but only run a few hours a month, you might be able to run a micro instance that is set to start a large (or ???) instance on demand, and run the whole thing for peanuts. We've looked at something similar (not Amazon, NASA is working on its own cloud service) where we'd locally run a server, but at times of high demand, pass off to the cloud service. If you have applications that are cyclic, I could see it being an advantage to have something take over in the peak times. Eg, when I worked for a university, the system we used for class registration was okay ... not great, but okay ... but the incoming freshmen were brought in in 3 or 4 'orientation' periods over the summer, and they'd all hit the system on the same day, at the same hour (well, 1/3? 1/4 of the incoming class) The system performance went to complete crap. We're talking about throughputs worse than if we had metered the access. (and the DBAs refused to look at database tuning, insisting that it was a webserver problem ... it was of course, a database issue, but it was months before we got it straightened out) I could see conferences using something like this -- where almost all of their traffic is on the days of deadlines, or during the conference itself. If the load's pretty uniform, I don't think their pricing model is all that advantageous. (and I have no idea how they handle the loads over christmas, as the reason for the cloud is to make money back on their excess capacity they need for the christmas sales period.) -Joe
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
I have a single co-located host and I get ping, power, pipe, and air conditioned comfort for $75/month. I haven't seen nor touched my (Linux) server in more than four or five years, and I might have restarted it four times. -- Eric Lease Morgan
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
EC2 works for a lot of models, but one that it does not work for is small traffic apps that need to be available 24/7. If you have a small instance (AWS term) running full time with a fixed IP, it costs about $75 a month. If you turn it on for 2 hours a day, it costs about $15/month. A large instance is about $325. If you intend to run anything 24/7 for more than a couple months, a reserved instance is the only way to go as it will drastically reduce costs -- for example the small instance you mention above should be a bit above $25/mo including the reserved instance fee if you can commit for awhile. If you do a lot of experimentation, you can buy a reserved instance and create and destroy servers at will while paying really low rates (0.02/hr is the rack rate for small reserved instances). If you can build things to deal with a bit of uncertainty, you can pay rates like that without even reserving the instance buy purchasing CPU on the spot market. The micros are a bargain for low intensity apps that don't require much horsepower (e.g. proxy, backup, etc). A reserved instance goes for $0.005 per hour (roughly the same price on the spot market), and even full hourly rate is 0.02. One thing to be aware is that many organizations effectively shift substantial costs such as bandwidth onto other entities. Amazon is unlikely to line out on your balance sheets favorably if your application puts substantial demands on a resource that is cost shifted. Amazon is also less attractive if you have a need to manage a low value data that doesn't get much use on active disks. There are other providers, and it's really worth looking at what your actual needs are before deciding on an IAAS provider as the cost structures vary quite a bit. We looked at others, but Amazon was a better fit (and cheaper for meeting our specific needs). kyle
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
Erik, We did a study a few months ago to evaluate the Amazon EC2 as an alternative host to both physical and virtual server spaces managed in house. Won't go into too much detail on it (unless people are interested), but our benchmark tests showed the performance of the EC2 consistently beat the performance of our in-house servers. The only big issue we had was cost, where our estimation of the price of running our servers off the EC2 would make actually doing so prohibitive. There were also some confusing fees built in the payment model, the one off the top of my head being x cents per million I/O operations. As someone who went with the EC2 and is running one currently, could you comment quick on your monthly costs (though I understand though if you don't want to release that information.) Thanks. David K. Uspal Technology Development Specialist Falvey Memorial Library Phone: 610-519-8954 Email: david.us...@villanova.edu -Original Message- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Erik Mitchell Sent: Wednesday, February 22, 2012 6:22 AM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2? Hi Nate When I was at Wake Forest University we moved a large chunk of our web services to Amazon and it worked out well. We chose Amazon because at the time they were the clear leader in IaaS stuff but since then a number of providers (Linode and Rackspace are two) have emerged as alternatives. As for why we moved that is a long story :) Erik On Feb 21, 2012, at 10:40 PM, Nate Hill nathanielh...@gmail.com wrote: Apologies for cross-posting. If yes, I'd love to hear why you chose to and how that is working out for you. Thanks! -- Nate Hill nathanielh...@gmail.com http://www.natehill.net
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
For what its worth, I posted the details of a month of running http://dltj.org/ out of an EC2 instance after I converted last year. The details are at: http://dltj.org/article/aws-hosting-cost/ It is a WordPress site that gets about 20,000 page views a month. Peter On Feb 22, 2012, at 5:00 PM, David Uspal wrote: Erik, We did a study a few months ago to evaluate the Amazon EC2 as an alternative host to both physical and virtual server spaces managed in house. Won't go into too much detail on it (unless people are interested), but our benchmark tests showed the performance of the EC2 consistently beat the performance of our in-house servers. The only big issue we had was cost, where our estimation of the price of running our servers off the EC2 would make actually doing so prohibitive. There were also some confusing fees built in the payment model, the one off the top of my head being x cents per million I/O operations. As someone who went with the EC2 and is running one currently, could you comment quick on your monthly costs (though I understand though if you don't want to release that information.) Thanks. David K. Uspal Technology Development Specialist Falvey Memorial Library Phone: 610-519-8954 Email: david.us...@villanova.edu -- Peter Murray Assistant Director, Technology Services Development LYRASIS peter.mur...@lyrasis.org +1 678-235-2955 1438 West Peachtree Street NW Suite 200 Atlanta, GA 30309 Toll Free: 800.999.8558 Fax: 404.892.7879 www.lyrasis.org LYRASIS: Great Libraries. Strong Communities. Innovative Answers.
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
I'd also be interested in getting some real world cost information. I installed an app on EC2 that went mostly unused for a couple months but meanwhile racked up over $300 in charges. Color me surprised. Roy On Wed, Feb 22, 2012 at 2:00 PM, David Uspal david.us...@villanova.eduwrote: Erik, We did a study a few months ago to evaluate the Amazon EC2 as an alternative host to both physical and virtual server spaces managed in house. Won't go into too much detail on it (unless people are interested), but our benchmark tests showed the performance of the EC2 consistently beat the performance of our in-house servers. The only big issue we had was cost, where our estimation of the price of running our servers off the EC2 would make actually doing so prohibitive. There were also some confusing fees built in the payment model, the one off the top of my head being x cents per million I/O operations. As someone who went with the EC2 and is running one currently, could you comment quick on your monthly costs (though I understand though if you don't want to release that information.) Thanks. David K. Uspal Technology Development Specialist Falvey Memorial Library Phone: 610-519-8954 Email: david.us...@villanova.edu -Original Message- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Erik Mitchell Sent: Wednesday, February 22, 2012 6:22 AM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2? Hi Nate When I was at Wake Forest University we moved a large chunk of our web services to Amazon and it worked out well. We chose Amazon because at the time they were the clear leader in IaaS stuff but since then a number of providers (Linode and Rackspace are two) have emerged as alternatives. As for why we moved that is a long story :) Erik On Feb 21, 2012, at 10:40 PM, Nate Hill nathanielh...@gmail.com wrote: Apologies for cross-posting. If yes, I'd love to hear why you chose to and how that is working out for you. Thanks! -- Nate Hill nathanielh...@gmail.com http://www.natehill.net
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
Roy Tennant writes I'd also be interested in getting some real world cost information. I installed an app on EC2 that went mostly unused for a couple months but meanwhile racked up over $300 in charges. Color me surprised. I am not impressed by Amazon either. I have an instance given to me by a sponsor, and there I have been taken aback by the old Debian kernel version this puts me in. I rent three root servers with Hetzner.de. That's for large-scale work. To run a blog, a 3TB disk 16 Gig ram box from Hetzner is overkill. With Hetzner you have the exchange rate risk but the cost structure is much simpler. Cheers, Thomas Krichelhttp://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
At Wed, 22 Feb 2012 23:34:14 +0100, Thomas Krichel wrote: Roy Tennant writes I'd also be interested in getting some real world cost information. I installed an app on EC2 that went mostly unused for a couple months but meanwhile racked up over $300 in charges. Color me surprised. I am not impressed by Amazon either. I have an instance given to me by a sponsor, and there I have been taken aback by the old Debian kernel version this puts me in. I rent three root servers with Hetzner.de. That's for large-scale work. To run a blog, a 3TB disk 16 Gig ram box from Hetzner is overkill. With Hetzner you have the exchange rate risk but the cost structure is much simpler. Another satisfied customer. best, Erik Hetzner PS: But seriously, no relation. Sent from my free software system http://fsf.org/. pgpZvpL5tGJVN.pgp Description: PGP signature
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
On Wed, Feb 22, 2012 at 2:10 PM, Roy Tennant roytenn...@gmail.com wrote: I'd also be interested in getting some real world cost information. I installed an app on EC2 that went mostly unused for a couple months but meanwhile racked up over $300 in charges. Color me surprised. EC2 can be a bargain or a cash hog depending on what you do. Some aspects of their service are cheap, others are not so cheap. In all cases, you want to be very aware of what you're using and making sure you're not paying for things you don't need. For example, it's really important not to pay for excess capacity. In the regular world, you buy capacity for your highest potential use case. But if you do that with Amazon, you'll rack up charges quickly with such an approach. Set things up so you have what you need only when you actually need it. You have to pay attention to their pricing structures as doing the same thing on EC2 can cost wildly different amounts depending on how you do it. We've used EC2 for a few years, have been very happy with the experience, and are tending to shift more services in that direction. Provisioning what you need is a snap, changing what you have to meet your needs on the fly is easy, and it's been very cost effective for us. kyle -- -- Kyle Banerjee Digital Services Program Manager Orbis Cascade Alliance baner...@uoregon.edu / 503.999.9787
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
Erik Hetzner writes Another satisfied customer. Actually I did not write that I was/am satisfied. ;-) They once managed to disassemble my server and I lost all the data on it. They were so embarrassed that they gave my sponsor the box for free for a year. I was fine because I had a backup so not much of a problem. The lesson learnt is that in any case you always need a backup, and it better be a local one or something hosted with a different company. There is no substitute for system administration skills. PS: But seriously, no relation. Neither do I have with them, other than being a customer. Cheers, Thomas Krichelhttp://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
EC2 works for a lot of models, but one that it does not work for is small traffic apps that need to be available 24/7. If you have a small instance (AWS term) running full time with a fixed IP, it costs about $75 a month. If you turn it on for 2 hours a day, it costs about $15/month. A large instance is about $325. Now where it gets interesting is if your app needs a large instance, but only run a few hours a month, you might be able to run a micro instance that is set to start a large (or ???) instance on demand, and run the whole thing for peanuts. Cary On Wed, Feb 22, 2012 at 2:10 PM, Roy Tennant roytenn...@gmail.com wrote: I'd also be interested in getting some real world cost information. I installed an app on EC2 that went mostly unused for a couple months but meanwhile racked up over $300 in charges. Color me surprised. Roy On Wed, Feb 22, 2012 at 2:00 PM, David Uspal david.us...@villanova.eduwrote: Erik, We did a study a few months ago to evaluate the Amazon EC2 as an alternative host to both physical and virtual server spaces managed in house. Won't go into too much detail on it (unless people are interested), but our benchmark tests showed the performance of the EC2 consistently beat the performance of our in-house servers. The only big issue we had was cost, where our estimation of the price of running our servers off the EC2 would make actually doing so prohibitive. There were also some confusing fees built in the payment model, the one off the top of my head being x cents per million I/O operations. As someone who went with the EC2 and is running one currently, could you comment quick on your monthly costs (though I understand though if you don't want to release that information.) Thanks. David K. Uspal Technology Development Specialist Falvey Memorial Library Phone: 610-519-8954 Email: david.us...@villanova.edu -Original Message- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Erik Mitchell Sent: Wednesday, February 22, 2012 6:22 AM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2? Hi Nate When I was at Wake Forest University we moved a large chunk of our web services to Amazon and it worked out well. We chose Amazon because at the time they were the clear leader in IaaS stuff but since then a number of providers (Linode and Rackspace are two) have emerged as alternatives. As for why we moved that is a long story :) Erik On Feb 21, 2012, at 10:40 PM, Nate Hill nathanielh...@gmail.com wrote: Apologies for cross-posting. If yes, I'd love to hear why you chose to and how that is working out for you. Thanks! -- Nate Hill nathanielh...@gmail.com http://www.natehill.net -- Cary Gordon The Cherry Hill Company http://chillco.com
Re: [CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
We did some tests on it, but found it a very poor fit for a site dependent on huge amount of data which much be present to the basically the whole system all the time and up-to-date. In other words, we found it didn't match a site based on MySQL slaves replicating here and there, and with memcached needing to be spot-on. Under some circumstances we'd consider shuffling some image rendering and delivery tasks to it, but that's about it. Tim LibraryThing
[CODE4LIB] Any libraries have their sites hosted on Amazon EC2?
Apologies for cross-posting. If yes, I'd love to hear why you chose to and how that is working out for you. Thanks! -- Nate Hill nathanielh...@gmail.com http://www.natehill.net
