Hi Suresh and Marlon I am trying to launch a hello world sample application from my local docker Airavata PGA. Can I get a demo admin role for few days so that I can create an application and test some sample experiments? It looks like without admin role I can not create applications.
Thanks Pankaj On Wed, Oct 28, 2015 at 1:55 PM, Pierce, Marlon <[email protected]> wrote: > But you could answer my implied question below with zero knowledge of > Airavata. > > From: Pankaj Saha <[email protected]> > Reply-To: Airavata Dev <[email protected]> > Date: Wednesday, October 28, 2015 at 1:48 PM > > To: Marlon Pierce <[email protected]> > Cc: Suresh Marru <[email protected]>, Airavata Dev < > [email protected]> > Subject: Re: launching a job through Airavata to Mesos cluster > > As Suresh said I think this is the time when I have to understand under > the hood details of Airavata Architecture, which I could not gain much > during GSoC. > > On Wed, Oct 28, 2015 at 1:00 PM, Pierce, Marlon <[email protected]> wrote: > >> The “hello world” for us is usually to just run some linux command on the >> remote queuing system. So that is something you could easily wrap via >> docker and submit via Mesos (I hope it is easy…). Related to this is >> determining that the job has gone through various states (queued, >> executing, completed) and communicating that back to Airavata. >> >> >> Marlon >> >> >> From: Pankaj Saha <[email protected]> >> Date: Wednesday, October 28, 2015 at 12:54 PM >> To: Marlon Pierce <[email protected]> >> Cc: Suresh Marru <[email protected]>, Airavata Dev < >> [email protected]> >> >> Subject: Re: launching a job through Airavata to Mesos cluster >> >> Hi Marlon, >> "we more typically use Airavata to run scientific applications on >> clusters" - Our plan is to run dockerized scientific application through >> Airavata. >> Definitely our target is not to run a Tomcat webserver, I just said that >> as an example. >> >> Give me some time, after I discuss with Prof. Madhu and Renan I will try >> to reply your queries in details. >> >> Thanks >> Pankaj >> >> On Wed, Oct 28, 2015 at 12:32 PM, Pierce, Marlon <[email protected]> wrote: >> >>> Hi Pankaj— >>> >>> I may not have the full background on what you want to do, but we more >>> typically use Airavata to run scientific applications on clusters. This >>> goes through a scheduling and queuing system (Torque/MOAB or SLURM). If >>> you want to do this, you would first need to say how you would submit a >>> dockerized application using Mesos as your queueing system. You could do >>> this for a Tomcat server, but that is a little different from the jobs we >>> normally run. >>> >>> We don’t use Airavata to bootstrap itself, although that may be an >>> interesting way to think of things. There it is interesting to use Mesos + >>> Marathon to run Airavata services (which may themselves by dockerized) but >>> that would be done today outside Airavata. >>> >>> Marlon >>> >>> >>> From: Suresh Marru <[email protected]> >>> Date: Wednesday, October 28, 2015 at 12:25 PM >>> To: Airavata Dev <[email protected]> >>> Cc: Marlon Pierce <[email protected]> >>> >>> Subject: Re: launching a job through Airavata to Mesos cluster >>> >>> Hi Pankaj, >>> >>> You cannot do this out of the box now, but with some guidance you can >>> easily write a task to run Dockerized application on a Mesos cluster, >>> please use 0.16 for this. As Shameera explained this is a improvised >>> architecture and any contributions you will make will carry forward. Note: >>> This will require you to understand under the hood details of Airavata >>> Architecture but I am sure you will motivate others to work with you to >>> speed it up. >>> >>> Suresh >>> >>> On Oct 28, 2015, at 12:22 PM, Pankaj Saha <[email protected]> wrote: >>> >>> “launch a dockerized job” means submitting a docker container for any >>> specific job (say a java webserver docker container) through Airavata. >>> What I remember Airavata php gateway can not submit any docker >>> container(correct me if I am wrong). What I want to figure out initially is >>> whether its feasible to submit a docker container with help of Airavata >>> into a Mesos cluster. >>> >>> Thanks >>> Pankaj >>> >>> On Wed, Oct 28, 2015 at 12:11 PM, Pierce, Marlon <[email protected]> >>> wrote: >>> >>>> I’ll add: if submitting a job to (for example) a SLURM queuing system, >>>> we need to create the correct SLURM submission script and submit it by >>>> executing the correct command line operation (sbatch). >>>> >>>> From: Marlon Pierce <[email protected]> >>>> Reply-To: Airavata Dev <[email protected]> >>>> Date: Wednesday, October 28, 2015 at 12:08 PM >>>> To: Airavata Dev <[email protected]>, Suresh Marru < >>>> [email protected]>, Pankaj Saha <[email protected]> >>>> >>>> Subject: Re: launching a job through Airavata to Mesos cluster >>>> >>>> Hi Pankaj, can you say more about what you mean by “launch a dockerized >>>> job”? >>>> >>>> Marlon >>>> >>>> >>>> From: Pankaj Saha <[email protected]> >>>> Reply-To: Airavata Dev <[email protected]> >>>> Date: Wednesday, October 28, 2015 at 11:56 AM >>>> To: Suresh Marru <[email protected]> >>>> Cc: Airavata Dev <[email protected]> >>>> Subject: Re: launching a job through Airavata to Mesos cluster >>>> >>>> Hi Suresh, >>>> >>>> My initial understanding is, I have to launch a dockerized job through >>>> Airavata which will be run in the Mesos cluster. I was looking for the >>>> code which submits jobs and wanted to make changes such a way that it can >>>> submit docker containers to Mesos/Marathon cluster. >>>> >>>> I can use 0.15 branch and I have no idea about data transfer protocol >>>> and job submission protocols that Shameera has mentioned. I may want to >>>> submit jobs by submitting a JSON through command line or any other way that >>>> you guys feel is more appropriate. >>>> >>>> I can talk to Prof. Madhu and let you know more on the requirement. >>>> >>>> Thanks >>>> Pankaj >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Oct 28, 2015 at 11:23 AM, Suresh Marru <[email protected]> >>>> wrote: >>>> >>>>> Pankaj can you clarify the following: >>>>> >>>>> Do you want an Airavata instance to run some dockerized applications >>>>> scheduled by Mesos? Or do you just need a client which will connect to >>>>> Airavata hosted and managed by Mesos/Marathon? >>>>> >>>>> Suresh >>>>> >>>>> On Oct 28, 2015, at 10:50 AM, Shameera Rathnayaka < >>>>> [email protected]> wrote: >>>>> >>>>> Hi Pankaj, >>>>> >>>>> Wich version of Airavata you are working on? what is the data >>>>> transfer protocol? What is the job submission protocol? >>>>> >>>>> Short answer: if you are using Airavata 0.15 then you need to write >>>>> new Provider implementation to submit the request to Mesos/Marathon >>>>> cluster. But if you are using Airavata 16.0 which is current master, then >>>>> you need to write JobSubmissionTask implementation. Either case you can go >>>>> through the existing implementations, for Provider implementation see >>>>> SSHProvider and JobSubmissionTask implementation see >>>>> SSHJobSubmissionTask. >>>>> >>>>> If I get the answers to my questions then i can provide exactly what >>>>> you need to do. BTW we have cleaned our internal architecture in Airavata >>>>> 16.0, as a developer you would find it easy to work with Airavata 16.0 >>>>> that >>>>> Airavata 15.0. But notice master is not yet stable as Airavata 15.0. >>>>> >>>>> Regards, >>>>> Shameera. >>>>> >>>>> >>>>> On Tue, Oct 27, 2015 at 1:53 PM Pankaj Saha <[email protected]> >>>>> wrote: >>>>> >>>>>> Hello Shameera, >>>>>> I am working on jet-stream project, where I have to find out a way to >>>>>> submit a job in mesos/marathon cluster through Airavata client. I don't >>>>>> have much idea from where to start looking into. Can you please give some >>>>>> clue so that I can start working and making changes to java code for the >>>>>> same. >>>>>> >>>>>> Thanks >>>>>> Pankaj >>>>>> >>>>>> -- >>>>> Shameera Rathnayaka >>>>> >>>>> >>>>> >>>> >>> >>> >> >
