We use an EC2 image onto which we install Java, Ant, Hadoop, etc. To make it simple, pull those from S3 buckets. That provides a flexible pattern for managing the frameworks involved, more so than needing to re-do an EC2 image whenever you want to add a patch to Hadoop.
Given that approach, you can add your Hadoop application code similarly. Just upload the current stable build out of SVN, Git, whatever, to an S3 bucket. We use a set of Python scripts to manage a daily, (mostly) automated launch of 100+ EC2 nodes for a Hadoop cluster. We also run a listener on a local server, so that the Hadoop job can send notification when it completes, and allow the local server to initiate download of results. Overall, that minimizes the need for having a sysadmin dedicated to the Hadoop jobs -- a small dev team can handle it, while focusing on algorithm development and testing. >> Or on EC2 and its competitors, just build a new image whenever you >>> need to update Hadoop itself.
