Some interesting developments in grid computing....
----- Forwarded by Ranga Nathan/AMERICA/BAX on 09/22/2006 04:29 PM -----
*Tom Metro <[EMAIL PROTECTED]>*
Sent by: [EMAIL PROTECTED]
09/22/2006 03:45 PM
To
L-boston-pm <boston-pm@mail.pm.org>
cc
Subject
[Boston.pm] Jeff Barr's talk: Amazon's web services
I wrote up my notes on Jeff Barr's talk to Boston.pm on 9/13/2006 for
some colleagues, and figured I might as well post a copy here for anyone
who wasn't able to attend.
Here are some of the highlights:
The Elastic Compute Cloud
http://www.amazon.com/b/ref=sc_fe_c_1_3435361_1/104-6093751-9591919?ie=UTF8&node=201590011&no=3435361&me=A36L942TSJ2AJA
A general purpose "grid" or cluster of machines. It uses Xen virtualized
machines and they'll rent you the equivalent of a 1.7 GHz Xenon CPU
w/1.7 GB of RAM, 160 GB HD, and 256 Mbps network (probably 1/4 of a
blade) for 10 cents/hr. You provide your application as a Xen image, and
load it up onto as many machines as you need. This can be used for a
variety of tasks - from compute intensive operations, to bulk data
processing, to web hosting. One example the speaker mentioned was a
telephone company using it to process end-of-the-month billing, where
the compute capacity is only needed for a few days out of the month.
The Elastic Compute Cloud is only 4 weeks old (and still in limited
beta), but it potentially will have a big impact on Internet startups.
Now it is no longer necessary to pay for substantial infrastructure up
front. You just expand to more machines almost transparently as your
business grows. (Though you could always rent machines, the ability to
expand and contract capacity wasn't as flexible.) The speaker told a
story of a friend who declined a VC investment because this service
allowed him to avoid the up-front expenses.
Alexa web services
http://www.amazon.com/b/ref=sc_fe_l_2/104-6093751-9591919?ie=UTF8&node=239513011&no=3435361&me=A36L942TSJ2AJA
It's basically the database and processing infrastructure of a search
engine that you rent, which you can then layer your specific search
engine design on top of. Their database includes a bunch of meta data on
the spidered sites, including a "web map," which is the linking
relationships between sites. They also have a thumbnail service that
returns an image for a specific site. And they have a custom processing
cluster that can execute custom data processing code (at a rate of
$1/CPU/hour; the speaker speculated the price may drop now that the ECC
service is priced so much lower).
Amazon's Simple Storage Service (S3)
http://www.amazon.com/S3-AWS-home-page-Money/b/ref=sc_fe_l_2/104-6093751-9591919?ie=UTF8&node=16427261&no=3435361&me=A36L942TSJ2AJA
Amazon's answer to online storage. This has been available for 6 months
or so, and already several startups (such as photo sharing services and
PC backup services) are using it as their back-end storage
infrastructure. It's been written about a lot, so I'll just mention that
the rates are 15 cents per GB per month, and 20 cents per GB for
transfers. It was also mentioned that S3 is on the same internal network
as The Elastic Compute Cloud, and that there are no transfer costs
between the two. Some related links:
JungleDisk (open standard/closed source/multi-platform, runs as a WebDAV
server, $0.15/GB)
http://jungledisk.com/
filicio.us (web-based file storage and sharing front-end for S3)
http://www.filicio.us/
Amazon Web Services Developer Connection:
s3sync -- a simple rsync look-alike
http://developer.amazonwebservices.com/connect/thread.jspa?messageID=37272&tstart=0#37272
S3Fox Organizer (Firefox extension)
http://www.rjonna.com/ext/s3fox.php
s3/fuse (Linux file system for S3)
http://dev.extensibleforge.net/wiki/s3/fuse
Windows-only commercial backup services using S3:
http://elephantdrive.com/
http://altexa.com/
Amazon Mechanical Turk
http://www.amazon.com/Mechanical-Turk-AWS-home-page/b/ref=sc_fe_l_2/104-6093751-9591919?ie=UTF8&node=15879911&no=3435361&me=A36L942TSJ2AJA
http://www.mturk.com/
Described as "Artificial Artificial Intelligence" or "people as a
service." This service provides the infrastructure for automating the
process of assigning small tasks to people - tasks that are too
difficult or impossible for computers to do. An example of a task is
image processing: does this image contain a picture of a human? This
task can be given to thousands of people, with multiple people
processing the same images to improve accuracy. Another example is
transcribing podcasts. It can also be used for marketing surveys. (Who
would have thought Amazon would end up dominating the "get paid to take
surveys" market.) One entertaining example use was for an art project,
where people were paid to draw sheep (http://www.thesheepmarket.com/).
HIT-Builder for Amazon's Mechanical Turk (a tool for creating tasks)
http://www.hit-builder.com/
CastingWords (podcast transcription services that uses AMT)
http://castingwords.com/
Amazon's E-Commerce Service
http://www.amazon.com/E-Commerce-Service-AWS-home-page/b/ref=sc_fe_l_2/104-6093751-9591919?ie=UTF8&node=12738641&no=3435361&me=A36L942TSJ2AJA
This was their first web service offering. It basically lets you create
your own Amazon clone, if that's what you want to do, or build any kind
of a supplementary service that depends on Amazon's product catalog. The
speaker demoed a couple of sites built on top of ECS that included some
interesting visualization technology. One, hivegroup.com, lets you
select products from the Amazon catalog based on parameters, while in
real time updating a graphical grid (Java applet) representing the
matching products. Another, liveplasma.com, was a music and movie
"discovery engine." You supply the name of a music group, and it
produces a slick connected graph representation of related groups, with
each node linking to a product page on Amazon.
Also briefly mentioned were:
Amazon Historical Pricing
http://www.amazon.com/Historical-Pricing-AWS-home-page/b/ref=sc_fe_l_2/104-6093751-9591919?ie=UTF8&node=15811391&no=3435361&me=A36L942TSJ2AJA
Amazon Simple Queue Service
http://www.amazon.com/Simple-Queue-Service-home-page/b/ref=sc_fe_l_2/104-6093751-9591919?ie=UTF8&node=13584001&no=3435361&me=A36L942TSJ2AJA
And general links:
http://aws.amazon.com/
The main site for Amazon's Web Services.
http://www.awszone.com/
A web front-end for interacting with Amazon's services.
http://aws.typepad.com/
Jeff Barr's blog
jbarr (at) amazon.com
Jeff Barr's email
-Tom
--
Tom Metro
Venture Logic, Newton, MA, USA
"Enterprise solutions through open source."
Professional Profile: http://tmetro.venturelogic.com/
_______________________________________________
Boston-pm mailing list
Boston-pm@mail.pm.org
http://mail.pm.org/mailman/listinfo/boston-pm
--
__________________
Ranga Nathan
Work: 714-442-7591
