Hi saminda and Heshan, Thank you very much for the detailed explanation.
On Wed, Mar 5, 2014 at 11:04 PM, Heshan Suriyaarachchi < [email protected]> wrote: > Hello Nipun, > > Please see my inline comments. > > > On Tue, Mar 4, 2014 at 10:32 AM, Nipun Udara <[email protected]>wrote: > >> Hi all, >> >> I was trying to understand GFac module and found different providers >> which use to execute jobs like BESProvider and GRAMProvider which extends >> AbstractProvider class like to know the use of those. >> >> And when search abouth EC2 instances there are three different kind of >> instances such as reserved instances ,spot instances. i would like to know >> out of those which kind is using in GFac-ec2 module. >> > > As you might know, Amazon provides a java-sdk to communicate with it's > AWS. EC2Provdier uses it to interact with AMIs and instances. When testing > out or running a workflow, I usually used a general purpose instance as I > did not have any requirement for running a workflow in compute optimized > instance family. Having said that, you will be able to run a workflow in > any kind of instance that you want. You might have to tweak the UI a bit to > support this (ie. like adding an option to a drop down, etc and modifying > the backend a bit). > > As for the sample workouts that you might be planning to run, you don't > need to run them in compute optimized instances as your workflow might not > be using the full potential of the resources. Keep in mind you are paying > for the instance type as well. As you reserve an instance with higher > compute power, you might be paying more (You might want to check the > pricing of instances before running sample workflows in compute optimized > instances). > > So, the short answer to your question might be, you can use which ever > instance type you want as long as the AWS java-sdk api is exposing the > capability of interacting with those instances (which they do). > > And is it need to configure to pass user-data scripts that runs the job >> each time launch a job. I would be grateful if somebody can clarify >> regarding these. >> > > Let me tell you how the EC2 job submissions work in Airavata. > > First you should have a base image which has your application installed. > This application can be a jar/shell script or other kind of a script. Try > and wrap your application with a shell script and make it available for > shell execution. > > The EC2 Provider will get the user information (secret key, access key, > AMI id) and start up an instance from the base image (which has your > application installed.) Once the instance is running, the EC2 provider > establishes a SSH connection to the running instance and then runs the > shell script (which wraps your application) with the user given input > params. Provider will poll the standard out from the script and extract the > result out. Then it will be returned to the provider. Then this will passed > on to the workflow. > > AFAICR I used J2ssh for the ssh connections. Since you are getting the > shell access to the instance, you might be able to run any script type that > you want but please remember that if you are using any script that is > dependent on the environment/version, you might run into some issues (eg. > Running a python 3.0 script on a Python 2.5 system). These things are also > minor issues and can be easily overcome. The reason why I told you this is > that, then you would not get side tracked because of any environment > issues. > > I didn't check Airavata mailing list for a long time. Therefore, didn't > see this. :) Lahiru, pointed me to this thread. If you have any > questions/doubts, please don't hesitate to ask. > > Thanks, > Heshan. > >> >> Regards >> Nipun Udara >> >> *Undergraduate* >> *Department of Computer Science & Engineering* >> *University of Moratuwa* >> *Sri Lanka* >> > > > > -- > Regards, > Heshan Suriyaarachchi > > http://heshans.blogspot.com/ > http://www.linkedin.com/in/heshan > -- Nipun Udara *Undergraduate* *Department of Computer Science & Engineering* *University of Moratuwa* *Sri Lanka*
