Hello Nipun, Please see my inline comments.
On Tue, Mar 4, 2014 at 10:32 AM, Nipun Udara <[email protected]> wrote: > Hi all, > > I was trying to understand GFac module and found different providers which > use to execute jobs like BESProvider and GRAMProvider which extends > AbstractProvider class like to know the use of those. > > And when search abouth EC2 instances there are three different kind of > instances such as reserved instances ,spot instances. i would like to know > out of those which kind is using in GFac-ec2 module. > As you might know, Amazon provides a java-sdk to communicate with it's AWS. EC2Provdier uses it to interact with AMIs and instances. When testing out or running a workflow, I usually used a general purpose instance as I did not have any requirement for running a workflow in compute optimized instance family. Having said that, you will be able to run a workflow in any kind of instance that you want. You might have to tweak the UI a bit to support this (ie. like adding an option to a drop down, etc and modifying the backend a bit). As for the sample workouts that you might be planning to run, you don't need to run them in compute optimized instances as your workflow might not be using the full potential of the resources. Keep in mind you are paying for the instance type as well. As you reserve an instance with higher compute power, you might be paying more (You might want to check the pricing of instances before running sample workflows in compute optimized instances). So, the short answer to your question might be, you can use which ever instance type you want as long as the AWS java-sdk api is exposing the capability of interacting with those instances (which they do). And is it need to configure to pass user-data scripts that runs the job > each time launch a job. I would be grateful if somebody can clarify > regarding these. > Let me tell you how the EC2 job submissions work in Airavata. First you should have a base image which has your application installed. This application can be a jar/shell script or other kind of a script. Try and wrap your application with a shell script and make it available for shell execution. The EC2 Provider will get the user information (secret key, access key, AMI id) and start up an instance from the base image (which has your application installed.) Once the instance is running, the EC2 provider establishes a SSH connection to the running instance and then runs the shell script (which wraps your application) with the user given input params. Provider will poll the standard out from the script and extract the result out. Then it will be returned to the provider. Then this will passed on to the workflow. AFAICR I used J2ssh for the ssh connections. Since you are getting the shell access to the instance, you might be able to run any script type that you want but please remember that if you are using any script that is dependent on the environment/version, you might run into some issues (eg. Running a python 3.0 script on a Python 2.5 system). These things are also minor issues and can be easily overcome. The reason why I told you this is that, then you would not get side tracked because of any environment issues. I didn't check Airavata mailing list for a long time. Therefore, didn't see this. :) Lahiru, pointed me to this thread. If you have any questions/doubts, please don't hesitate to ask. Thanks, Heshan. > > Regards > Nipun Udara > > *Undergraduate* > *Department of Computer Science & Engineering* > *University of Moratuwa* > *Sri Lanka* > -- Regards, Heshan Suriyaarachchi http://heshans.blogspot.com/ http://www.linkedin.com/in/heshan
