Re: [External] [DISCUSS] New name for MFT

[Pierce, Marlon]

How about a ship theme? Here’s a list of types of ships: 
https://en.wikipedia.org/wiki/List_of_ship_types.

A couple of ancient types of ships: Kerkouros, Corbita.

Marlon


From: Suresh Marru 
Date: Monday, January 30, 2023 at 5:59 PM
To: Airavata Dev 
Subject: [External] [DISCUSS] New name for MFT
This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.

Hi All,

We chose MFT as a code name as we carved out all file transfer use cases from 
core Airavata. Now MFT has evolved significantly [1] and is most notably used 
for large scale data transfers between NASA and ESA HLS Data [2] Transfers 
between Openstack and AWS and IBM clouds.

If we choose the name Custos for the security spinoff [3] [4], it is probably 
time to choose a name for MFT. Please keep your creative naming thoughts 
flowing?

What would you name a more generic MFT without naming technology terms like 
Transfers?

Cheers,
Suresh

[1] - 
https://github.com/apache/airavata-mft
[2] - 
https://hls.gsfc.nasa.gov/
[3] - 
https://airavata.apache.org/custos/
[4] - 
https://github.com/apache/airavata-custos

Re: [External] Regarding Airavata Seagrid Rich Client

[Pierce, Marlon]

Hi Aishwarya, sorry for late reply, but this is really nice progress.

Marlon


From: Aishwarya Sinhasane 
Date: Tuesday, June 21, 2022 at 9:21 PM
To: dev@airavata.apache.org 
Subject: [External] Regarding Airavata Seagrid Rich Client
This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.

Hello Everyone,

I tried to make a login module in electronJS which is working properly for 
login with credentials as well as CILogon.

The electronJS application loads the django airavata portal so we can access 
all the modules that are already present in the django portal.

I discussed with sudhakar about including other modules such as Application 
Editors and Molecule Editors in application. We came to the conclusion that to 
include these modules in the main menu of the application. Users can access 
these before login and can create molecules and applications using editors and 
once it's ready to create an experiment the user needs to login to the system. 
Also users can login and can access editors.

Currently, I am developing the frontend for the molecule editor nanocad. Also I 
am trying to understand the logic of the molecule editor developed in JavaFX.

If anyone has other suggestions please let me know.

Screenshots are attached below for your reference.

Re: [External] github PRs for lib upgrades

[Pierce, Marlon]

Hi PJ, thanks for the nudge, we will review.

Marlon


From: PJ Fanning 
Date: Tuesday, April 5, 2022 at 7:11 AM
To: dev@airavata.apache.org 
Subject: [External] github PRs for lib upgrades
[You don't often get email from fannin...@apache.org. Learn why this is 
important at 
http://aka.ms/LearnAboutSenderIdentification.]

This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.
---

Hi,
I have a couple of PRs in github [1] to upgrade some lib dependencies
that have CVEs open against them.

Would any Airavata committer be in a position to review them?

Regards,
PJ

[1] 
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairavata%2Fpulls&data=04%7C01%7Cmarpierc%40iu.edu%7C8f168528bf114a7fb50208da16f4dcf2%7C1113be34aed14d00ab4bcdd02510be91%7C0%7C0%7C637847538624516035%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=KLTZQLQBDvzdZMNeBrd8J1nmVOGx4JpDCvcoIdlfWQg%3D&reserved=0

Re: [External] Data Catalog into its own repo

[Pierce, Marlon]

Schema.org has a lot of uptake in the NSF EarthCube community 
(https://www.earthcube.org/p418) and the DataONE activity 
(https://ui.adsabs.harvard.edu/abs/2018AGUFMIN31B..29M/abstract).   

 

Marlon

 

 

From: Suresh Marru 
Reply-To: dev 
Date: Tuesday, November 10, 2020 at 1:13 PM
To: dev 
Subject: Re: [External] Data Catalog into its own repo

 

Thank you all for weighing in. I bootstrapped the repo with some basic 
information, please contribute to set the goals for this refactored sub system 
- https://github.com/apache/airavata-data-lake

 

I was doing a literature and software survey on any open source metadata and 
provenance systems we can integrate with, I found this survey paper useful - 
https://www.researchgate.net/profile/Carlos_Saenz-Adan/publication/323242431_A_systematic_review_of_provenance_systems/links/5b34ae1caca2720785effb1a/A-systematic-review-of-provenance-systems.pdf

 

Seems like we can build fairly flexible and yet sophisticated capabilities 
using the schema.org JSON-LD schema - https://schema.org/ , thoughts?

 

Please contribute any other pointers we should brainstorm before proceeding. 

 

Cheers,

Suresh



On Nov 9, 2020, at 4:22 PM, Pierce, Marlon  wrote:

 

+1 for this refactoring. 

On 11/9/20, 9:47 AM, "Pamidighantam, Sudhakar"  wrote:

   This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.
   ---

   +2. 

   Thanks,
   Sudhakar.

   On 11/9/20, 9:37 AM, "Marru, Suresh"  wrote:

   Hi All,

   Airavata Experiment catalog evolved over time and though the replica 
catalog and data product models are stand alone, they are buried to use them 
outside the experiment context. Any objections to refactor experiment catalog 
and make data catalog a first class repo, in the lines of Custos and MFT?

   Cheers,
   Suresh

 



smime.p7s
Description: S/MIME cryptographic signature

Re: [External] Re: Data Catalog into its own repo

[Pierce, Marlon]

+1 for this refactoring. 

On 11/9/20, 9:47 AM, "Pamidighantam, Sudhakar"  wrote:

This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.
---

+2. 

Thanks,
Sudhakar.

On 11/9/20, 9:37 AM, "Marru, Suresh"  wrote:

Hi All,

Airavata Experiment catalog evolved over time and though the replica 
catalog and data product models are stand alone, they are buried to use them 
outside the experiment context. Any objections to refactor experiment catalog 
and make data catalog a first class repo, in the lines of Custos and MFT?

Cheers,
Suresh



smime.p7s
Description: S/MIME cryptographic signature

Re: [External] Re: Project Update and Questions

[Pierce, Marlon]

Hi Nick,

 

One specific observation about this particular client: the uploaded KML file 
needs to have a URL that can be consumed by your mapping API—it needs to have a 
URL. Also, you need to watch out for caching when displaying the file—if a user 
uploads a new version of the file with the same name, the map may display the 
old version still. This is an issue with Google Maps and KML, but I don’t know 
about leaflet.

 

Other things to look out for: file upload size limits and validation (is the 
file really a KML file?).  Also, beyond the immediate proof of concept, it may 
be good to see if this connects with the Airavata MFT work.

 

Marlon

 

 

From: "Mowery, Nicholas Benjamin" 
Reply-To: dev 
Date: Friday, June 5, 2020 at 6:04 PM
To: dev 
Subject: [External] Re: Project Update and Questions

 

This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.

 

Hi all, 

another update on the GeoGateway Django App. This week I've focused on state 
management with Vuex and developing a modular and lightweight method for adding 
KML map layers to the Leaflet map. I've opted to use Vuex to make communication 
between child components easier. I've also removed Vue as a CDN (in favor of 
Vue CLI + Webpack) and and fully separated the Vue frontend and Django backend. 
The boilerplate code is significant but I now have a stable development 
environment that can be deployed quickly. 

A question I'm considering:
What is the best way to implement file upload in an Airavata Django app.  
Django has a relatively straight-forward method but it might not be the best in 
our case 
https://simpleisbetterthancomplex.com/tutorial/2016/08/01/how-to-upload-files-with-django.html
Generally, I'm wondering if this problem should be handled locally by the 
application rather than communicating with Airavata.

Some things I've learned: 

If the application is going to have more than 10 Vue components, implement a 
Vuex store from the start
vue2-leaflet is not extensible enough for large GIS applications that require 
customized map layers
Webpack configuration can be painful and a boilerplate for future Django + Vue 
+ Webpack applications will be helpful
Please reply with any comments or questions. Any recommendation regarding file 
uploads would be appreciated. Thank you!

-Nick

 

Nicholas Mowery 

Student of Computer Science and Philosophy

Student Developer | CIRC | GSoC Apache

Indiana University Bloomington

 

From: Mowery, Nicholas Benjamin
Sent: Thursday, May 21, 2020 5:51 PM
To: dev@airavata.apache.org
Subject: Project Update and Questions 

 

 

Hi all,

This summer I am working on developing a GeoGateway Django app for the Airavata 
Django Portal. So far I have developed a skeleton of the app and integrated 
Vue.js for the frontend UI components. At this stage I'm looking for input on 
some of the choices I've made and if any of them should be adjusted. 

 
Leaflet and Vue.js (over Google Maps and React.js)
It seems at this scale that a modular app rather than Vue.js style SPA makes 
more sense (although some of the other Airavata GIS applications are mostly SPA)
There are a lot of style choices to be made and many I'm likely still unaware 
of, so any tips on structuring and organizing a project like this will be 
helpful. 

I also have a couple questions I've accumulated about Django for anyone willing 
to help:

When working with a custom Django app in the Airavata Portal, should I redefine 
the MEDIA_ROOT directory in the apps.py file? Is there a precedent for changing 
variables like this defined in the project-wide settings.py file on a per-app 
basis?
I'm finding that {% include .html %} works perfectly but {% extends 
html %} does not. I use the block/endblock syntax with unique names but the 
content doesn't show up. Is there a reason I should use extends over include?
Thanks for reading. 

-Nick

 

Nicholas Mowery 

Student of Computer Science and Philosophy

Student Developer | CIRC | GSoC Apache

Indiana University Bloomington



smime.p7s
Description: S/MIME cryptographic signature

Re: [External] Project Update and Questions

[Pierce, Marlon]

Hi Nick, 

 

Based on off-list feedback, I suggest you verify that the KML sources used in 
GeoGateway will work with Leaflet.  Give this part a chance to fail fast. 

 

Someone else will need to weigh in your third bullet, and your fourth bullet 
may need more details.

 

Marlon

 

 

From: "Mowery, Nicholas Benjamin" 
Reply-To: dev 
Date: Thursday, May 21, 2020 at 5:51 PM
To: dev 
Subject: [External] Project Update and Questions

 

This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.

 

 

Hi all,

This summer I am working on developing a GeoGateway Django app for the Airavata 
Django Portal. So far I have developed a skeleton of the app and integrated 
Vue.js for the frontend UI components. At this stage I'm looking for input on 
some of the choices I've made and if any of them should be adjusted. 

 
Leaflet and Vue.js (over Google Maps and React.js)
It seems at this scale that a modular app rather than Vue.js style SPA makes 
more sense (although some of the other Airavata GIS applications are mostly SPA)
There are a lot of style choices to be made and many I'm likely still unaware 
of, so any tips on structuring and organizing a project like this will be 
helpful. 

I also have a couple questions I've accumulated about Django for anyone willing 
to help:

When working with a custom Django app in the Airavata Portal, should I redefine 
the MEDIA_ROOT directory in the apps.py file? Is there a precedent for changing 
variables like this defined in the project-wide settings.py file on a per-app 
basis?
I'm finding that {% include .html %} works perfectly but {% extends 
html %} does not. I use the block/endblock syntax with unique names but the 
content doesn't show up. Is there a reason I should use extends over include?
Thanks for reading. 

-Nick

 

Nicholas Mowery 

Student of Computer Science and Philosophy

Student Developer | CIRC | GSoC Apache

Indiana University Bloomington



smime.p7s
Description: S/MIME cryptographic signature

Gateways 2020 Call for Participation

[Pierce, Marlon]

Hi all,

 

See below for the Gateways 2020 call for participation. This is a great venue 
for publicizing your work building science gateways and developing science 
gateway software.

 

Thanks,

 

Marlon

 

--

 

Call for Participation: Gateways 2020 (October 19–21, Bethesda, Maryland) is 
now accepting submissions.

What’s a gateway? Gateways are user-friendly interfaces to computing, data, 
instruments, and other domain-specific resources to support research and 
education. They may also be known as portals, virtual research environments, 
eScience, virtual labs, eResearch, digital repositories, or research 
cyberinfrastructure.

Deadlines:

Short Papers, Demos, and Panels: May 11 extended to May 18
Posters (open to all attendees): September 11
Read more details in the Call for Participation: 

http://sciencegateways.org/gateways2020/call

COVID-19 UPDATE:
Because the COVID-19 outbreak is changing rapidly, we are continuing to plan 
our October conference while monitoring the situation. In case we have to 
cancel, we will strive to offer an alternative format, and peer-reviewed, 
accepted submissions would still be published in the Gateways 2020 Proceedings. 
Questions, concerns, and suggestions may be sent to h...@sciencegateways.org.

 



smime.p7s
Description: S/MIME cryptographic signature

Gateways 2020 Call for Participation (Tutorials deadline extended to May 5)

[Pierce, Marlon]

Hi all, 

 

Please see the announcement below. If you are developing a science gateway, 
please consider submitting a paper, tutorial, or poster.

 

Thanks,


Marlon

 

---

 

Call for Participation: Gateways 2020 (October 19–21, Bethesda, Maryland) is 
now accepting submissions. 

What’s a gateway? Gateways are user-friendly interfaces to computing, data, 
instruments, and other domain-specific resources to support research and 
education. They may also be known as portals, virtual research environments, 
eScience, virtual labs, eResearch, digital repositories, or research 
cyberinfrastructure.

Deadlines:
Tutorials and Workshops: extended to May 5 
Short Papers, Demos, and Panels: May 11
Posters (open to all attendees): September 11
Read more details in the Call for Participation: 
http://sciencegateways.org/gateways2020/call 

COVID-19 UPDATE:
Because the COVID-19 outbreak is changing rapidly, we are continuing to plan 
our October conference while monitoring the situation. In case we have to 
cancel, we will strive to offer an alternative format, and peer-reviewed, 
accepted submissions would still be published in the Gateways 2020 Proceedings. 
Questions, concerns, and suggestions may be sent to h...@sciencegateways.org. 

 

 



smime.p7s
Description: S/MIME cryptographic signature

FW: [External] Today's webinar—Building Gateways with Apache Airavata Software & SciGaP

[Pierce, Marlon]

Hi all,

 

If you are interested, we have an Apache Airavata webinar today at 1 pm US ET. 
This will be recorded if you want to watch later.

 

Thanks,


Marlon

 

 

Building Science Gateways with Apache Airavata Software and SciGaP Platform 
Services
Presented by

Marlon Pierce, Director, Cyberinfrastructure Integration Research Center, 
Indiana University, and co-PI, SGCI

Suresh Marru, Cyberinfrastructure Integration Research Center, Indiana 
University, SGCI Staff

Marcus Christie, Principal Science Gateway Research Consultant, 
Cyberinfrastructure Integration Research Center, Indiana University, SGCI Staff

Part three of a series of webinars providing an introduction to a variety of 
gateway platforms that can be used for building new gateways. Part one, "Using 
Tapis," and part two, "HUBzero Platform for Scientific Collaboration" can be 
viewed in our webinar archive.

Building and operating an effective science gateway for the long term is harder 
than you think. This talk describes Apache Airavata and its growing ecosystem 
of software extensions that can be used to build science gateways. The Science 
Gateways Platform as a service (SciGaP.org) is a hosted version of Apache 
Airavata that is operated by the Cyberinfrastructure Integration Research 
Center (CIRC) at Indiana University. SciGaP makes building and operating 
gateways both easier and more sustainable. We review several example gateways 
in diverse research fields that are based on Apache Airavata and that use 
SciGaP.org services.

To join the meeting using Zoom from PC, Mac, Linux, iOS, or Android:

https://zoom.us/j/691002660

 

Meeting ID: 691 002 660

 

To join via phone:

1)  One tap mobile

+19294362866,,691002660# US (New York)
+16699006833,,691002660# US (San Jose)

2) Dial by your location

+1 929 436 2866 US (New York)

+1 669 900 6833 US (San Jose)

3) Find your local number if joining internationally: 
https://zoom.us/u/acX1cjA5iW

-- 

Nayiri Mullinix

Community Engagement and Exchange Coordinator 

Science Gateways Community Institute (SGCI)

Follow SGCI on Twitter

 

105 S. State Street

University of Michigan

School of Information

Ann Arbor, MI 48109-1285

T: 734.764.3952 

nay...@umich.edu



smime.p7s
Description: S/MIME cryptographic signature

Apache Airavata developer position

[Pierce, Marlon]

Hi all,

 

We have an open position in the Science Gateways Research Center at Indiana 
University for Airavata developers, solution experts, and frontend developers 
for Airavata using Django (https://github.com/apache/airavata-django-portal).

 

If you are interested, please see 
https://iujobs.peopleadmin.com/postings/72644. 

 

Thanks,

 

Marlon

 



smime.p7s
Description: S/MIME cryptographic signature

Re: [VOTE] Apache Airavata release 0.17 - RC1

[Pierce, Marlon]

+1

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: dev 
Date: Thursday, March 21, 2019 at 9:33 AM
To: dev 
Subject: Re: [VOTE] Apache Airavata release 0.17 - RC1

 

+1

 

On Thu, Mar 21, 2019 at 1:07 AM Suresh Marru  wrote:

Apache Airavata PMC is pleased to call for a vote on the following Apache 
Airavata 0.17 release candidate artifacts:
 
Detailed change log/release notes:
https://github.com/apache/airavata/blob/airavata-0.17/RELEASE_NOTES
All Release Artifacts:
https://dist.apache.org/repos/dist/dev/airavata/0.17/RC1/
PGP release keys (signed using 617DDBAD):
https://dist.apache.org/repos/dist/dev/airavata/KEYS
Specific URL’s:
GIT source tag (305cccab1d8eab8aff28e0fa06ebe9f01ffdde2e):
https://github.com/apache/airavata/tree/airavata-0.17
Source release:
https://dist.apache.org/repos/dist/dev/airavata/0.17/RC1/airavata-0.17-source-release.zip
Binary Artifacts:
https://dist.apache.org/repos/dist/dev/airavata/0.17/RC1/apache-airavata-server-0.17-bin.tar.gz
 
https://dist.apache.org/repos/dist/dev/airavata/0.17/RC1/apache-airavata-server-0.17-bin.zip
Maven staging repo:
https://repository.apache.org/content/repositories/orgapacheairavata-1008/
 
Please verify the artifacts and vote. The vote will be open for 72 hours.
 
[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)
 



smime.p7s
Description: S/MIME cryptographic signature

Re: [DISCUSS] Apache Airavata release 0.17 - RC1

[Pierce, Marlon]

I verified the release signatures and built the code on Mac OSX 10.14.x with 
Java 1.8 and Maven 3.6, so I’m +1 for the release, voted on the [VOTE) thread.

 

Marlon

 

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: dev 
Date: Thursday, March 21, 2019 at 1:55 AM
To: dev 
Subject: Re: [DISCUSS] Apache Airavata release 0.17 - RC1

 

Hi Suresh, 

 

Build succeeded in JDK 1.8 with Maven 3.5.2 on Mac OS. INSTALL file in the 
binary distribution looks little bit out dated. 

 

Thanks

Dimuthu

 

On Thu, Mar 21, 2019 at 1:07 AM Suresh Marru  wrote:

Discussion thread for vote on Apache Airavata 0.17 release candidate.
 
If you have any questions or feedback or to post results of validating the 
release, please reply to this thread. Once you verify the release, please post 
your vote to the VOTE thread.  
 
For reference, the Apache release guide  - 
http://www.apache.org/dev/release.html
 
Some tips to validate the release before you vote:
 
* Download the binary version and run the 5 minute or 10 minute tutorial as 
described in README and website.
* Download the source files from compressed files and release tag and build 
(which includes tests). 
* Verify the distribution for the required LICENSE and NOTICE files
* Verify if all the staged files are signed and the signature is verifiable. 
* Verify if the signing key in the project's KEYS file is hosted on a public 
server
 
Thanks for your time in validating the release and voting,
Suresh 
(On Behalf of Airavata PMC)
 



smime.p7s
Description: S/MIME cryptographic signature

Gateways 2019 Call for Participation

[Pierce, Marlon]

HI all

 

Please see the Gateways 2019 conference call for participation below.  

 

Thanks,

 

Marlon

 

===

 

GATEWAYS 2019 Call for Participation

 

September 23–25, 2019

San Diego, California

http://sciencegateways.org/gateways2019/call

 

We welcome submissions of papers, demos, tutorials, and panels (2–4 pages) on 
the topic of gateways for science, engineering, or other disciplines. Gateways 
are user-friendly interfaces to scientific computing, data, and other 
domain-specific resources to support research and education. 

 

Topics may include their design, use, impact, development processes, 
sustainability, best practices, or any other aspect that you think fellow 
gateway creators or users will find interesting to learn. We also welcome 
educational topics directed toward the next generation of gateway creators. 

 

In addition, this year Gateways 2019 is co-located with the eScience conference 
https://escience2019.sdsc.edu/. The two conferences will offer shared sessions 
and connected registration.

ABSTRACTS ARE DUE APRIL 29, and FULL SUBMISSIONS ARE DUE MAY 6, 2019. A poster 
session deadline (open to all attendees) will be August 15. Read more details 
in the Call for Participation: http://sciencegateways.org/gateways2019/call

 

 



smime.p7s
Description: S/MIME cryptographic signature

Re: Need to contribute to Airavata

[Pierce, Marlon]

Hi Asanka, you need to make a pull request. 

Marlon


On 2/21/19, 1:38 PM, "Asanka Sanjaya"  wrote:

Hi Suresh,

Could you please add me as a developer to the airavata-custos repository (
https://github.com/apache/airavata-custos)?
I have created a project with the profile service and credentials store
modules and I can push the initial version. I will integrate the sharing
service next.


On Wed, Feb 20, 2019 at 10:05 PM DImuthu Upeksha 

wrote:

> Sounds good.
>
> On Wed, Feb 20, 2019 at 8:05 AM Asanka Sanjaya  wrote:
>
>> Hi DImuthu,
>>
>> For the moment I will copy the airavata-common-utils module to the
>> custos repository (which is not the best approach) just to get this build
>> working. Later, once the 0.17 is released, I will remove it from custos
>> repo.
>>
>> On Wed, Feb 20, 2019 at 11:01 AM Asanka Sanjaya 
>> wrote:
>>
>>> Sure Dimuthu. I would do that.
>>>
>>> On Wed, Feb 20, 2019 at 2:30 AM DImuthu Upeksha <
>>> dimuthu.upeks...@gmail.com> wrote:
>>>
 Hi Asanka,

 Currently we are little bit behind our release schedule and we would
 add 0.17 dependencies to maven repository once we have completed with 
0.17
 release. In the meantime, can you try building
 airavata-common-utils 0.17-SNAPSHOT locally from the source?

 Thanks
 Dimuthu

 On Tue, Feb 19, 2019 at 1:31 AM Asanka Sanjaya 
 wrote:

> Hi Suresh,
>
> I started pulling out the profile service, sharing service and
> credential store out of Airavata to the new repository. There is a
> dependency in profile-service-commons to airavata-common-utils. The 
latest
> version I found for airavata-common-utils is 0.15 [1] and the current
> airavata codebase is depending on airavata-common-utils 0.17-SNAPSHOT.
> Appreciate if you can help me to clear out this blocker.
>
> 1.
> 
https://mvnrepository.com/artifact/org.apache.airavata/airavata-common-utils/0.15
>
>
>
>
>
>
> On Sat, Feb 16, 2019 at 9:43 AM Asanka Sanjaya 
> wrote:
>
>> Hi Suresh,
>>
>> Thanks for the details. This is great. I will work on it.
>>
>> On Sat, Feb 16, 2019 at 12:08 AM Suresh Marru 
>> wrote:
>>
>>> Hi Asanka,
>>>
>>> If you are interested to help, please see this thread for context -
>>> https://airavata.markmail.org/thread/lqez3hr75rxobs6e
>>>
>>> We would like to pull out profile service, sharing service and
>>> credential store out of Airavata into a new repo -
>>> https://github.com/apache/airavata-custos
>>>
>>> Suresh
>>>
>>> On Feb 7, 2019, at 1:30 PM, Asanka Sanjaya 
>>> wrote:
>>>
>>> Thank you Suresh. I will do that.
>>>
>>> On Thu, 7 Feb 2019, 11:51 pm Suresh Marru >>
 Hi Asanka,

 A good start will be trying to checkout the develop branch of the
 base and getting familiar with the code base. Last year GSoC 
students have
 provided some good starting documents you should be able to find 
them in
 the archives - https://airavata.markmail.org/

 Suresh

 On Feb 7, 2019, at 12:52 PM, Asanka Sanjaya 
 wrote:

 Hi All,

 I would like to contribute to apache Airavata as a developer. I
 really appreciate if somebody can give me ideas/guidance on this.

 --

 *Thanks,*
 Asanka Sanjaya Herath
 Senior Software Engineer | Zone24x7



>>>
>>
>> --
>>
>> *Thanks,*
>>
>> Asanka Sanjaya Herath
>>
>> Senior Software Engineer | Zone24x7
>>
>
>
> --
>
> *Thanks,*
>
> Asanka Sanjaya Herath
>
> Senior Software Engineer | Zone24x7
>

>>>
>>> --
>>>
>>> *Thanks,*
>>>
>>> Asanka Sanjaya Herath
>>>
>>> Senior Software Engineer | Zone24x7
>>>
>>
>>
>> --
>>
>> *Thanks,*
>>
>> Asanka Sanjaya Herath
>>
>> Senior Software Engineer | Zone24x7
>>
>

-- 

*Thanks,*

Asanka Sanjaya Herath

Senior Software Engineer | Zone24x7



smime.p7s
Description: S/MIME cryptographic signature

Re: Metascheduler work

[Pierce, Marlon]

Hi Dimuthu,

 

This is something we should re-evaluate. Mangirish Wangle looked at Mesos 
integration with Airavata back in 2016, but he ultimately ran into many 
difficulties, including getting MPI jobs to work, if I recall correctly.

 

Marlon

 

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: dev 
Date: Sunday, December 16, 2018 at 7:30 AM
To: dev 
Subject: Metascheduler work

 

Hi Folks, 

 

I found this [1] mail thread and the JIRA ticket [2] which have discussed about 
coming up with an Airavata specific job scheduler. At the end of the 
discussion, seems like an approach based on Mesos has been chosen to tryout. Is 
there any other discussion/ documents regarding this topic? Has anyone worked 
on this and if so, where are the code / design documents?

 

[1] 
https://markmail.org/message/tdae5y3togyq4duv#query:+page:1+mid:tdae5y3togyq4duv+state:results

[2] https://issues.apache.org/jira/browse/AIRAVATA-1436

 

Thanks

Dimuthu



smime.p7s
Description: S/MIME cryptographic signature

Re: Unused modules

[Pierce, Marlon]

Can you move these to an “archive” directory or make an “archive” branch?  It 
may be useful to easily find these for future reference.

 

Marlon

 

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: dev 
Date: Friday, November 30, 2018 at 8:29 AM
To: dev 
Subject: Re: Unused modules

 

Hi Suresh,

 

+1 for removing gfac modules as well 

 

Dimuthu

 

On Fri, Nov 30, 2018 at 6:32 PM Apache Airavata  wrote:

+1 to remove all of them. While you are at it, should we also remove gfac 
modules from develop and staging branches? 

 

Suresh

 


On Nov 30, 2018, at 6:44 AM, DImuthu Upeksha  wrote:

Hi Folks, 

 

I can see that some modules [1] are no longer being used or actively developed. 

 

allocation-manager

cloud

compute-account-provisioning

configuration

db-event-manager

integration-tests

monitoring

security

workflow

workflow-model

xbaya

xbaya-gui

 

I'm suggesting to remove these unused modules as they affect the build time and 
the clarity of the code. Any objections / suggestions?

 

[1] https://github.com/apache/airavata/tree/staging/modules

 

Thanks

Dimuthu

 



smime.p7s
Description: S/MIME cryptographic signature

Apache Airavata developer position

[Pierce, Marlon]

Hi all,

 

We have positions open for Apache Airavata developers, solution experts, and 
integrators open at the Science Gateways Research Center at Indiana University. 
Our generic position announcement is here: 
https://iujobs.peopleadmin.com/postings/67282. We are looking for people with a 
wide range of skills and of course a willingness to do exciting new things.   
International applications welcome. 

 

Thanks,

 

Marlon Pierce

Indiana University

Bloomington, IN, USA

 



smime.p7s
Description: S/MIME cryptographic signature

FW: [WSSSPE] BSSw Fellowship open for applications

[Pierce, Marlon]

Possibly of interest.

Marlon


On 9/10/18, 10:26 AM, "WSSSPE on behalf of David E. Bernholdt" 
 wrote:

Dear Colleague,

We would like to draw your attention to the Better Scientific Software 
(BSSw) Fellowship program, the main goal of which is to foster and 
promote practices, processes and tools to improve developer productivity 
and software sustainability of scientific codes. Each 2019 BSSw Fellow 
will receive up to $25,000 for an activity that promotes better 
scientific software.

Applications for the 2019 Fellowships will be accepted through Tuesday, 
16 October 2018. This is a firm deadline that will not be extended. 
Details of the program can be found at https://bssw.io/fellowship, 
including an FAQ list and information on prior Fellows.

Those interested in applying are encouraged to participate in an 
informational fellowship webinar and Q&A session, scheduled for 1-2pm 
EDT on Friday, 21 September 2018.  Please subscribe to our mailing list 
(https://bssw.io/pages/receive-our-email-digest) for teleconference 
details and other updates on the program. Note that the Fellowship is 
open to researchers with US institutional affiliations.

Please share this announcement with interested colleagues.

Thanks, David Bernholdt, Lois Curfman McInnes, and Mike Heroux
___
WSSSPE mailing list
wss...@lists.researchcomputing.org.uk

http://lists.researchcomputing.org.uk/listinfo.cgi/wssspe-researchcomputing.org.uk



smime.p7s
Description: S/MIME cryptographic signature

Gateways 2018 Conference Information

[Pierce, Marlon]

Hi all--

 

Please see below for information on Gateways 2018, including discounts for 
early registration.  Note also there is still time to submit a poster.

 

Thanks,

 

Marlon

 

 

--

GATEWAYS 2018

 

  Tuesday through Thursday, September 25–27, 2018
  Hosted at the University of Texas, Austin
  https://sciencegateways.org/gateways2018 

 

Science gateways connect components of advanced cyberinfrastructure behind 
streamlined, user-friendly interfaces. Join gateway creators and enthusiasts to 
learn, share, connect, and shape the future of gateways. 

 

Early-bird REGISTRATION is now open through Monday, August 6. (Regular 
registration closes Friday, Sept. 7.) Book now for the best rates: 
https://sciencegateways.org/web/gateways2018/attend/register 

 

The conference PROGRAM is already brimming with exciting keynotes, tutorials, 
and sessions:

https://sciencegateways.org/web/gateways2018/program

 

Anyone is welcome to participate in the POSTER SESSION. Submit an abstract for 
your poster by Wednesday, August 1:  
https://sciencegateways.org/web/gateways2018/program/cfp#poster

 

Do you or a colleague offer resources or services that would be of interest to 
the science gateways community? Consider reserving a space at the RESOURCE 
EXPO: https://sciencegateways.org/web/gateways2018/program/resource-expo


Questions? Email h...@sciencegateways.org



 



smime.p7s
Description: S/MIME cryptographic signature

Re: [GSoC] Re-architect Output Data Parsing into Airavata core

[Pierce, Marlon]

Yes, +1 on the detailed email summaries.

 

Marlon

 

 

From: Suresh Marru 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 22, 2018 at 8:46 AM
To: Airavata Dev 
Cc: Supun Nakandala 
Subject: Re: [GSoC] Re-architect Output Data Parsing into Airavata core

 

Hi Lahiru, 

 

Thank you for sharing the detailed summary. I do not have comments on your 
questions, may be Supun can weigh in. I have couple of meta requests though:

 

Can you consider adding few Molecular dynamics parsers in this order LAMMPS,  
Amber, and CHARMM. The cclib library you used for others do not cover these, 
but InterMol [1] provides a python library to parse these. We have to be 
careful here, InterMol itself is MIT licensed and we can have its dependency 
but it depends upon ParamEd[2] which is LGPL license. Its a TODO for me on how 
to deal wit this but please see if you can include adding these parsers into 
your timeline. 

 

Can you evaluate if we can provide export to Quantum Chemistry JSON Scheme [3]? 
Is this is trivial we can pursue it. 

 

Lastly, can you see if Apache Tikka will help with any of your efforts. 

 

I will say my kudos again for your mailing list communications,

Suresh 

 

[1] - https://github.com/shirtsgroup/InterMol

[2] - https://github.com/ParmEd/ParmEd 

[3] - https://github.com/MolSSI/QC_JSON_Schema 

 



On Jun 22, 2018, at 12:37 AM, Lahiru Jayathilake  
wrote:

 

Hi Everyone, 

 

In the last couple of days, I've been working on the data parsing tasks. To 
give an update about it, I have already converted the code-base of Gaussian, 
Molpro, Newchem, and Gamess parsers to python[1]. With compared to code-base of 
seagrid-data there won't be any codes related to experiments in the project(for 
example no JSON mappings). The main reason for doing this because to de-couple 
experiments with the data parsing tasks. 

 

While I was converting the codes of Gaussian, Molpro, Newchem, and Gamess I 
found some JSON key value-pairs in the data-catalog docker container have not 
been used in the seagrid-data to generate the final output file. I have 
commented unused key-value pairs in the code itself [2], [3], [4], [5]. I would 
like to know is there any specific reason for this, hope @Supun Nakandala can 
answer it. 

 

The next update about the data parsing architecture.

The new requirement is to come up with a framework which is capable of parsing 
any kind of document to a known type when the metadata is given. By this new 
design, data parsing will not be restricted only to experiments(Gaussian, 
Molpro, etc.)  

 

The following architecture is designed according to the requirements specified 
by @dimuthu in the last GSoC meeting.

 

The following diagram depicts the top level architecture.

 



​

Following are the key components.

 

Abstract Parser 

This is a basic template for the Parser which specifies the parameters required 
for parsing task. For example, input file type, output file type, experiment 
type( if this is related to an experiment), etc.

 

Parser Manager

Constructs the set of parsers considering the input file type, output file 
type, and the experiment type.

Parser Manager will construct a graph to find the shortest path between input 
file type and output file type. Then it will return the constructed set of 
Parsers.

 



​Catalog 

A mapping which has records to get a Docker container that can be used to parse 
from one file type to another file type. For example, if the requirement is to 
parse a Gaussian .out file to JSON then "app/gaussian .out to JSON" docker will 
be fetched

 

Parsers

There are two types of parsers (according to the suggested way) 

 

The first type is the parsers those will be directly coded into the project 
code-base. For example, parsing Text file to a JSON will be straightforward, 
then it is not necessarily required to maintain a separate docker container to 
convert text file to JSON. With the help of a library and putting an entry to 
the catalog will be enough to get the work done.

 

The second type is parsers which have a separate docker container. For example 
Gaussian .out file to JSON docker container

 

For the overall scenario consider the following examples to get an idea

 

Example 1

Suppose a PDF should be parsed to XML

Parser Manager will look the catalog and find the shortest path to get the XML 
output from PDF. The available parsers are(both the coded parsers in the 
project and the dockerized parsers),

• PDF to Text

• Text to JSON

• JSON to XML

• application/gaussian .out to JSON (This is a very specific parsing mechanism 
not similar  to parsing a simple .out file to a JSON)

and the rest which I have included in the diagram

 

Then Parser Manager will construct the graph and find the shortest path as 

PDF -> Text -> JSON -> XML from the available parsers. 

 



Then Parser Manager will return 3 Parsers. From the three parsers a DAG will be 
constructed as follows,

 



​

The reason for th

FW: [ctsc-announce-l] Trusted CI Webinar May 21st at 11am ET: The EU General Data Protection Regulation (GDPR)

[Pierce, Marlon]

Hi all,

 

I want to draw your attention to this seminar.

 

Marlon

 

 

From: "Dopheide, Jeannette M" 
Reply-To: "ctsc-announc...@list.iu.edu" 
Date: Monday, May 7, 2018 at 9:04 AM
To: "announceme...@trustedci.org" 
Subject: [ctsc-announce-l] Trusted CI Webinar May 21st at 11am ET: The EU 
General Data Protection Regulation (GDPR)

 

Trusted CI Webinar May 21st at 11am ET: The EU General Data Protection 
Regulation (GDPR)

CACR's Scott Russell is presenting the talk, "The EU General Data Protection 
Regulation (GDPR)" on May 21st at 11am (Eastern).

 

Please register here. Be sure to check spam/junk folder for registration 
confirmation email.

The European Union’s General Data Protection Regulation (GDPR) is slated to 
come into effect on May 25, 2018, and organizations around the world are 
struggling to determine whether they are covered, what is required, and what 
will happen if they don’t satisfy its requirements.

This webinar will provide an introduction to GDPR, including an overview of the 
law's requirements, an in-depth discussion of when and to whom the law may 
apply, and potential strategies for organizations that are unsure of whether 
they are covered. The webinar will also provide insight into the motivation 
behind the law, the legal and practical ramifications of its enforcement 
outside of the EU, and highlight current uncertainties relating to the scope 
and impact of the law. Attendees will leave with an improved understanding of 
how GDPR may impact their organization, and will be equipped with basic 
strategies to manage risks arising from the enforcement of the law.

This webinar is a product of the Trusted CI, the NSF Cybersecurity Center of 
Excellence. Trusted CI is supported by the National Science Foundation under 
Grant Number ACI-1547272. For more information about the Trusted CI please 
visit: http://trustedci.org/. Any opinions, findings, and conclusions or 
recommendations expressed in this material are those of the author(s) and do 
not necessarily reflect the views of the National Science Foundation.

Scott Russell is a Senior Policy Analyst at the Indiana University Center for 
Applied Cybersecurity Research (CACR), where his work focuses on privacy and 
cybersecurity policy. A lawyer and researcher, Scott received his B.A. in 
Computer Science and History from the University of Virginia, received his J.D. 
from Indiana University, interned at MITRE, and served as a postdoctoral fellow 
at CACR.

Join Trusted CI's announcements mailing list for information about upcoming 
events. To submit topics or requests to present, see our call for 
presentations. Archived presentations are available on our siteunder "Past 
Events."

 

 

--

Jeannette Dopheide

Sr. Education, Outreach, and Training Coordinator

National Center for Supercomputing Applications

University of Illinois at Urbana-Champaign



smime.p7s
Description: S/MIME cryptographic signature

FW: Gateways 2018 CFP: Extended to May 16!

[Pierce, Marlon]

Hi all,

 

I hope you will consider submitting to the Gateways 2018 conference.  Deadline 
extension below.

 

Thanks,

 

Marlon

 

 

From: Science Gateways Community Institute 
 on behalf of Science 
Gateways Community Institute 
Reply-To: Science Gateways Community Institute 
Date: Monday, April 30, 2018 at 9:52 AM
To: "Pierce, Marlon" 
Subject: Gateways 2018 CFP: Extended to May 16!

 

Submissions for Gateways 2018 are now due by Weds., May 16 
View this email in your browser
Forward this email to a friend 

  
 

Gateways 2018 CFP—Deadline Extended to May 16!
The deadline for the Gateways 2018 Call for Participation has been extended to 
May 16, 2018.Gateways 2018 (September 25-27, at the University of Texas at 
Austin) is accepting submissions of papers, demos, tutorials, and panels (2-4 
pages) on the topic of science or engineering gateways.Topics may include their 
design, use, impact, development processes, sustainability, best practices, or 
any other aspect that you think fellow gateway creators or users will find 
interesting to learn. We also welcome educational topics directed toward the 
next generation of gateway creators.Submissions of papers, demos, tutorials, 
and panels are due May 16, 2018. There will be no further extension of this 
deadline. The poster session deadline (open to all) is August 1, 2018.Full 
submission details can be found here: 
https://sciencegateways.org/gateways2018/call

 

 

 

Forward this email to a friend

Copyright © 2018 Science Gateways Community Institute, All rights reserved. 
You are receiving this email because you asked to be informed about the 
activities of the Science Gateways Community Institute or you have participated 
in an event associated with the Institute. These updates will not be sent too 
frequently, so we hope that you'll continue to stay in touch with us!  Please 
add i...@sciencegateways.org to your address book so that you get these notices 
in your inbox, not in a promotional folder.Our mailing address is:
Science Gateways Community Institute
University of California at San Diego
9500 Gilman Dr
San Diego, CA 92093-0505
  





This email was sent to marpi...@iu.edu 
why did I get this?unsubscribe from this listupdate subscription 
preferences 
Science Gateways Community Institute · University of California at San Diego · 
9500 Gilman Dr · San Diego, CA 92093-0505 · USA 





smime.p7s
Description: S/MIME cryptographic signature

Re: C++ Client SDK with Airavata 0.17

[Pierce, Marlon]

Thanks, Vasanth.  Do you have a proof of concept that demonstrates your 
solution, or is a pull request appropriate?  Note also I suggest you repost 
with the content of your google doc in the email message itself for future 
references. These emails are publically archived and are searchable.

 

Thanks,

 

Marlon

 

 

From: Vasanth Methkupalli 
Reply-To: "dev@airavata.apache.org" 
Date: Tuesday, April 24, 2018 at 11:02 PM
To: "dev@airavata.apache.org" 
Subject: C++ Client SDK with Airavata 0.17

 

Hello Dev, 

 

Please find the attached Doc to look at the implementation and related work to 
jira issue AIRAVATA-2731. Some of the details have been updated in the Jira 
issue as well.

 

 

https://docs.google.com/document/d/1JxldKB7P-QLsZJgwIDSGham94QzVEDvQsf8BcCLwF1g/edit?usp=sharing

 

Thanks,

Vasanth



smime.p7s
Description: S/MIME cryptographic signature

FW: 3/14 Webinar: Sustaining Science Gateways—Finding your "best fit" model

[Pierce, Marlon]

Hi all, 

 

See below and please consider calling in. As many of you know, Nancy Maron is 
one of the principal presenters at SGCI’s Incubator Bootcamp.

 

Thanks,

 

Marlon

 

 

From: Science Gateways Community Institute 
 on behalf of Science 
Gateways Community Institute 
Reply-To: Science Gateways Community Institute 
Date: Tuesday, March 13, 2018 at 10:00 AM
To: "Pierce, Marlon" 
Subject: 3/14 Webinar: Sustaining Science Gateways—Finding your "best fit" model

 

"Sustaining Science Gateways" webinar on Weds. 3/14 
View this email in your browser
Forward this email to a friend 

  
 

March SGCI Webinar
Sustaining Science Gateways—Finding your "best fit" model
Wednesday, March 14, 2018, at 1 pm Eastern/10 am Pacific.Presented by Nancy 
Maron, Principal and Founder, BlueSky to BluePrintScience Gateways can deliver 
a great deal of value to users, but they share something else in common, too: 
They are devilishly hard to run in a way that permits ongoing growth and 
expansion. This webinar will introduce participants to the key elements of 
sustainability planning—the building blocks for developing Science Gateways 
that have the best chance for ongoing growth. Nancy will share some key tactics 
for identifying the models that are most likely to work for your gateway and 
will touch upon funding models, the competitive environment, and audience 
assessment, to show how these need to be considered in tandem with any 
plan.Nancy will also offer a brief overview of the upcoming Science Gateways 
Bootcamp, running in May 2018, for which she is a lead instructor.Detailed 
upcoming and previous webinar descriptions, slides, and video are available on 
our website: https://sciencegateways.org/engage/webinars. Plus, add this to 
your calendar with our easy links.Webinar Coordinates:
To join the meeting using the downloadable app:
https://bluejeans.com/502679698 To join the meeting in your browser (not quite 
as good quality):
https://bluejeans.com/502679698/browser To join via phone: 
1)  Dial: +1.888.240.2560 (US Toll-Free; see all numbers - 
http://bluejeans.com/numbers)
2)  Enter Conference ID: 502679698

 

 

 

Forward this email to a friend

Copyright © 2018 Science Gateways Community Institute, All rights reserved. 
You are receiving this email because you asked to be informed about the 
activities of the Science Gateways Community Institute or you have participated 
in an event associated with the Institute. These updates will not be sent too 
frequently, so we hope that you'll continue to stay in touch with us!  Please 
add i...@sciencegateways.org to your address book so that you get these notices 
in your inbox, not in a promotional folder.Our mailing address is:
Science Gateways Community Institute
University of California at San Diego
9500 Gilman Dr
San Diego, CA 92093-0505
  





This email was sent to marpi...@iu.edu 
why did I get this?unsubscribe from this listupdate subscription 
preferences 
Science Gateways Community Institute · University of California at San Diego · 
9500 Gilman Dr · San Diego, CA 92093-0505 · USA 





smime.p7s
Description: S/MIME cryptographic signature

Gateways 2018 Call for Participation

[Pierce, Marlon]

Call for Participation: 

 

Gateways 2018 (September 25–27, at the University of Texas at Austin) is now 
accepting submissions of papers, demos, tutorials, and panels (2–4 pages) on 
the topic of science or engineering gateways. Gateways are user-friendly 
interfaces to scientific computing, data, and other domain-specific resources 
to support research and education. 

 

Topics may include their design, use, impact, development processes, 
sustainability, best practices, or any other aspect that you think fellow 
gateway creators or users will find interesting to learn. We also welcome 
educational topics directed toward the next generation of gateway creators. 

The primary submission deadline is May 7, 2018, and a poster session deadline 
(open to all) will be August 1. Read more details in the Call for 
Participation: http://sciencegateways.org/gateways2018/call 

 

 

Thank you,

 

Marlon

(On behalf of the conference organizers)





 



smime.p7s
Description: S/MIME cryptographic signature

Re: Install all Airavata server components in a single machine

[Pierce, Marlon]

Thanks, Dimuthu, very thorough.  If anyone finds any problems with this, you 
can leave a comment in the google doc. 

 

We should move this to some place like http://airavata.readthedocs.io.

 

Marlon

 

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: "dev@airavata.apache.org" 
Date: Tuesday, February 27, 2018 at 1:43 PM
To: Airavata Dev 
Subject: Install all Airavata server components in a single machine

 

Hi Folks, 

 

I'm writing this mail specially for the people who are trying to get started 
and try out in Apache Airavata. 

 

Usually Airavata comes as set of components that are needed to be installed and 
it could be painful for a beginner to manage all of them. We already had a set 
of ansible scripts that you can use to install these components and now we 
optimized them in order to install all the components in a single machine or a 
VM. This [1] document describes steps and configurations that you need to 
follow. However we made the document commentable so if you need any 
clarification, please feel free to comment on this and we will try our best to 
resolve the issues.

 

If you are a student who is willing to participate GSoC with Airavata, this 
could be a good starting point

 

[1] 
https://docs.google.com/document/d/1dFhajiYlPtNymIKbMvCXeLivLHujLjvE107QyUDEgdA/edit?usp=sharing

 

Thanks 

Dimuthu



smime.p7s
Description: S/MIME cryptographic signature

Re: Airavata unit tests - Mocking

[Pierce, Marlon]

?+1, very good suggestion.


Marlon


From: Sachin Kariyattin 
Sent: Tuesday, February 27, 2018 8:09 PM
To: dev@airavata.apache.org
Subject: Airavata unit tests - Mocking

Hello Dev,

I see that we don't use mocking in any of the Airavata unit tests. This 
sometimes makes it difficult because all the dependent objects need to be 
created before one particular piece can be tested.

I came across this when I tried to test the validator class 
(BatchQueueValidator, GroupResourceProfileValidator etc).

I am Planning to use JMockit http://jmockit.github.io and implement a unit test 
to start with. Any feedback/views on this is appreciated.


--
Regards,
Sachin Kariyattin

Re: [DISCUSS] SEAGrid Rich Client to Airavata

[Pierce, Marlon]

We have seen the interest. I’m wondering how much work needs to be done to 
generalize the current SEAGrid desktop tools so that they are easily deployable 
by new gateway tenants. 

 

Marlon

 

From: Suresh Marru 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, November 17, 2017 at 4:43 PM
To: Airavata Dev 
Subject: [DISCUSS] SEAGrid Rich Client to Airavata

 

Hi All, 

 

In discussing with some of Airavata users at Supercomputing conference, there 
seems to be an increasing interest in desktop application interrogations with 
Airavata. Any opinions on bringing seagrid rich client to Airavata 
(https://github.com/SciGaP/seagrid-rich-client), generalize it and make the 
third party interfaces into a plugin model? 

 

Thoughts? 

 

Suresh

 



smime.p7s
Description: S/MIME cryptographic signature

FW: Weds. 11/8 Webinar: An introduction to usability & user-centered design

[Pierce, Marlon]

Hi all, 

 

This is a reminder about today’s science gateways webinar: Prof. Paul Parsons 
will provide an introduction to usability and user-centered design for science 
gateways and other user interfaces. This has been one of the favorite (and best 
reviewed) sessions of the two science gateway incubator boot camps, so I 
encourage you to attend.

 

Marlon

 

 

 

From: Science Gateways Community Institute 
 on behalf of Science 
Gateways Community Institute 
Reply-To: Science Gateways Community Institute 
Date: Tuesday, November 7, 2017 at 10:00 AM
To: "Pierce, Marlon" 
Subject: Weds. 11/8 Webinar: An introduction to usability & user-centered design

 

Introduction to usability & user-centered design webinar on Weds. 11/8 
View this email in your browser 

  
 

November SGCI Webinar
Making your gateway easy and pleasant to use: An introduction to usability and 
user-centered design

Wednesday, November 8, 2017, at 1 pm Eastern/10 am Pacific.

Presented by Paul Parsons, Assistant Professor in Computer Graphics Technology 
at Purdue University

This webinar aims to convey the importance of usability and user-centered 
design, identify common usability problems, and offer strategies and fixes for 
common usability problems.

Detailed upcoming and previous webinar descriptions, slides, and video are 
available on our website: https://sciencegateways.org/engage/webinars. Plus, 
add this to your calendar with our easy links.

Registration no longer required! 

To join the meeting using the downloadable app:
https://bluejeans.com/502679698
 
To join the meeting in your browser (not quite as good quality):
https://bluejeans.com/502679698/browser
 
To join via phone:
1)  Dial: +1.888.240.2560 (US Toll-Free; see all numbers - 
http://bluejeans.com/numbers)
2)  Enter Conference ID: 502679698 

 

 

 


Copyright © 2016 Science Gateways Community Institute, All rights reserved. 
You are receiving this email because you asked to be informed about the 
activities of the Science Gateways Community Institute or you have participated 
in an event associated with the Institute. These updates will not be sent too 
frequently, so we hope that you'll continue to stay in touch with us!  Please 
add i...@sciencegateways.org to your address book so that you get these notices 
in your inbox, not in a promotional folder.Our mailing address is:
Science Gateways Community Institute
University of California at San Diego
9500 Gilman Dr
San Diego, CA 92093-0505

  





This email was sent to marpi...@iu.edu 
why did I get this?unsubscribe from this listupdate subscription 
preferences 
Science Gateways Community Institute · University of California at San Diego · 
9500 Gilman Dr · San Diego, CA 92093-0505 · USA 





smime.p7s
Description: S/MIME cryptographic signature

Re: Linked Container Services for Apache Airavata Components - Phase 2 - Initial Prototype

[Pierce, Marlon]

ps://www.youtube.com/watch?v=kZT8v2_b2XE&index=15&list=PLkz1SCf5iB4enAR00Z46JwY9GGkaS2NON

[2] https://github.com/DImuthuUpe/kafka-transactionality

[3] https://www.youtube.com/watch?v=j6bOVLUlyf4&feature=youtu.be

[4] 
https://www.youtube.com/watch?v=AshMNCxSp3c&list=PLkz1SCf5iB4enAR00Z46JwY9GGkaS2NON&index=17

[5] 
https://www.youtube.com/watch?v=pMDAcNRkWkE&index=10&list=PLkz1SCf5iB4enAR00Z46JwY9GGkaS2NON

[6] 
https://github.com/apache/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/api-server/src/main/java/org/apache/airavata/k8s/api/server

 

On Thu, Nov 2, 2017 at 2:52 AM, Pierce, Marlon  wrote:

Hi Dimuthu,

 

Thanks for sending this very thoughtful document. A couple of comments:

 

* Use of Kafka instead of RabbitMQ is interesting. Can you say more about how 
this approach can handle Kafka client failures?  For RabbitMQ, for example, 
there is the simple “Work Queue” approach in which the broker pushes a task to 
a worker. The task remains in queue until the worker sends an acknowledgement 
that the job has been handled, not just received. “Handled” may mean for 
example that the job has been submitted to an external batch scheduler over 
SSH, which may require some retries, etc.   If the worker crashes before the 
job has been submitted, then the broker can resend the message to another 
worker.   I’m wondering how your Kafka-based solution would handle the same 
issue. 

 

* A simpler but more common failure is communicating with external resources. A 
task executor may need to SSH to a remote resource, which can fail (the 
resource is slow to communicate, usually). How do you handle this case?

 

* Your design focuses on Airavata’s experiment execution handling. Airavata’s 
registry is another important component: this is where experiment objects get 
persistently stored. The registry stores metadata about both “live” experiments 
that are currently executing as well as archived experiments that have 
completed.

 

How would you extend your architecture to include the registry?

 

Marlon

 

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: "dev@airavata.apache.org" 
Date: Monday, October 30, 2017 at 10:45 AM
To: "dev@airavata.apache.org" 
Subject: Linked Container Services for Apache Airavata Components - Phase 2 - 
Initial Prototype

 

Hi All, 

 

Based on the analysis of Phase 1, within past two weeks I have been working on 
implementing a task execution workflow following the microservices deployment 
pattern and Kubernetes as the deployment platform. 

 

Please find attached design document that explains the components and messaging 
interactions between components. Based on that design, I have implemented 
following components

 

1. Set of microservices to compose the workflow

2. A simple Web Console to  deploy and monitor workflows on the framework

 

I used Kakfa as the primary messaging medium to communicate among the 
microservices due to its simplicity and powerful features like partitions and 
consumer groups.

 

I have attached a user guide so that you can install and try this in your local 
machine. And source code for each component can be found from [1]

 

Please share you ideas and suggestions.

 

Thanks

Dimuthu

 

[1] 
https://github.com/DImuthuUpe/airavata/tree/master/sandbox/airavata-kubernetes

[2] 
https://docs.google.com/document/d/1R1xrmuPldHiWVDn4xNVay9Vnxn9FODQZXtF55JxJpSY/edit?usp=sharing

[3] 
https://docs.google.com/document/d/1A5eRIZiuUj4ShZVMS0NdAxjAxtOTZXculaYDCZ7IMQ8/edit?usp=sharing

 



smime.p7s
Description: S/MIME cryptographic signature

Re: Linked Container Services for Apache Airavata Components - Phase 2 - Initial Prototype

[Pierce, Marlon]

Hi Dimuthu,

 

Thanks for sending this very thoughtful document. A couple of comments:

 

* Use of Kafka instead of RabbitMQ is interesting. Can you say more about how 
this approach can handle Kafka client failures?  For RabbitMQ, for example, 
there is the simple “Work Queue” approach in which the broker pushes a task to 
a worker. The task remains in queue until the worker sends an acknowledgement 
that the job has been handled, not just received. “Handled” may mean for 
example that the job has been submitted to an external batch scheduler over 
SSH, which may require some retries, etc.   If the worker crashes before the 
job has been submitted, then the broker can resend the message to another 
worker.   I’m wondering how your Kafka-based solution would handle the same 
issue. 

 

* A simpler but more common failure is communicating with external resources. A 
task executor may need to SSH to a remote resource, which can fail (the 
resource is slow to communicate, usually). How do you handle this case?

 

* Your design focuses on Airavata’s experiment execution handling. Airavata’s 
registry is another important component: this is where experiment objects get 
persistently stored. The registry stores metadata about both “live” experiments 
that are currently executing as well as archived experiments that have 
completed.

 

How would you extend your architecture to include the registry?

 

Marlon

 

 

From: "dimuthu.upeks...@gmail.com" 
Reply-To: "dev@airavata.apache.org" 
Date: Monday, October 30, 2017 at 10:45 AM
To: "dev@airavata.apache.org" 
Subject: Linked Container Services for Apache Airavata Components - Phase 2 - 
Initial Prototype

 

Hi All, 

 

Based on the analysis of Phase 1, within past two weeks I have been working on 
implementing a task execution workflow following the microservices deployment 
pattern and Kubernetes as the deployment platform. 

 

Please find attached design document that explains the components and messaging 
interactions between components. Based on that design, I have implemented 
following components

 

1. Set of microservices to compose the workflow

2. A simple Web Console to  deploy and monitor workflows on the framework

 

I used Kakfa as the primary messaging medium to communicate among the 
microservices due to its simplicity and powerful features like partitions and 
consumer groups.

 

I have attached a user guide so that you can install and try this in your local 
machine. And source code for each component can be found from [1]

 

Please share you ideas and suggestions.

 

Thanks

Dimuthu

 

[1] 
https://github.com/DImuthuUpe/airavata/tree/master/sandbox/airavata-kubernetes

[2] 
https://docs.google.com/document/d/1R1xrmuPldHiWVDn4xNVay9Vnxn9FODQZXtF55JxJpSY/edit?usp=sharing

[3] 
https://docs.google.com/document/d/1A5eRIZiuUj4ShZVMS0NdAxjAxtOTZXculaYDCZ7IMQ8/edit?usp=sharing



smime.p7s
Description: S/MIME cryptographic signature

Re: Linked Container Services for Apache Airavata Components - Phase 1 - Requirement identification

[Pierce, Marlon]

References

 

Marru, S., Gunathilake, L., Herath, C., Tangchaisin, P., Pierce, M., Mattmann, 
C., Singh, R., Gunarathne, T., Chinthaka, E., Gardler, R. and Slominski, A., 
2011, November. Apache airavata: a framework for distributed applications and 
computational workflows. In Proceedings of the 2011 ACM workshop on Gateway 
computing environments (pp. 21-28). ACM.

 

Nakandala, S., Pamidighantam, S., Yodage, S., Doshi, N., Abeysinghe, E., 
Kankanamalage, C.P., Marru, S. and Pierce, M., 2016, July. Anatomy of the 
SEAGrid Science Gateway. In Proceedings of the XSEDE16 Conference on Diversity, 
Big Data, and Science at Scale (p. 40). ACM.

 

Pierce, Marlon E., Suresh Marru, Lahiru Gunathilake, Don Kushan Wijeratne, 
Raminder Singh, Chathuri Wimalasena, Shameera Ratnayaka, and Sudhakar 
Pamidighantam. "Apache Airavata: design and directions of a science gateway 
framework." Concurrency and Computation: Practice and Experience 27, no. 16 
(2015): 4282-4291.

 

Pierce, Marlon, Suresh Marru, Borries Demeler, Raminderjeet Singh, and Gary 
Gorbet. "The apache airavata application programming interface: overview and 
evaluation with the UltraScan science gateway." In Proceedings of the 9th 
Gateway Computing Environments Workshop, pp. 25-29. IEEE Press, 2014.

 

Marru, Suresh, Marlon Pierce, Sudhakar Pamidighantam, and Chathuri Wimalasena. 
"Apache Airavata as a laboratory: architecture and case study for component- 
based gateway middleware." In Proceedings of the 1st Workshop on The Science of 
Cyberinfrastructure: Research, Experience, Applications and Models, pp. 19-26. 
ACM, 2015.

 

Thanks

Dimuthu



smime.p7s
Description: S/MIME cryptographic signature

Re: Lecture Slides-Fall 2017

[Pierce, Marlon]

Thanks for catching this. I updated.  The slides are in the website git repo, 
and the website should be up dated soon (automatically).

 

Here’s the repo URL: 
https://github.com/airavata-courses/airavata-courses.github.io/tree/master/slides

 

Marlon

 

 

From: Vasanth Methkupalli 
Reply-To: "dev@airavata.apache.org" 
Date: Tuesday, October 3, 2017 at 11:40 PM
To: "dev@airavata.apache.org" 
Subject: Lecture Slides-Fall 2017

 

Hi Dev, 

 

The link to PPT of Lecture 13 seems to be broken. Please verify.
 

-- 

 

Thanks,

Regards,

Methkupalli Vasanth

ᐧ



smime.p7s
Description: S/MIME cryptographic signature

Re: Running PGA locally on OS X

[Pierce, Marlon]

Check permissions and make sure you know who the user is.  You can always 
(ahem) chmod -R 777 to see if this solves the issue.


Marlon

 

 

From: Neeraj Lad 
Reply-To: "dev@airavata.apache.org" 
Date: Wednesday, October 4, 2017 at 10:56 AM
To: "dev@airavata.apache.org" 
Subject: Re: Running PGA locally on OS X

 

Hi all, 

 

I was able to run PGA locally, then I changed the Virtual host to point to a 
new directory to check out the new commits on PGA.

 

Now when I try to access localhost:8080, I get the following error: 

 

Forbidden

 

You don't have permission to access / on this server.

 

Here is my httpd-pga.conf file

 



 DocumentRoot /Users/student/git/airavata-php-gateway

 

   DirectoryIndex index.php

   AllowOverride All

   Require all granted

 



 

Any ideas about this?

 

On Fri, Sep 29, 2017 at 11:59 AM, Neeraj Lad  wrote:

That sounds good to me.

 

On Fri, Sep 29, 2017 at 11:56 AM, Marcus Christie  wrote:

Ok, we’ll probably have to debug this in person. I’ll be at the Science 
Gateways course office hours today. 

 

On Sep 29, 2017, at 11:44 AM, Neeraj Lad  wrote:

 

I do have the server certificate file path in my pga_config, Marcus.

 

On Fri, Sep 29, 2017 at 9:18 AM, Christie, Marcus Aaron  wrote:

Neeraj, 

 

Make sure in pga_config.php you have the following

 

/**

 * Path to the server certificate file

 */

'cafile-path' => app_path() . 
'/resources/security/incommon_rsa_server_ca.pem',

 

On Sep 28, 2017, at 3:34 PM, Neeraj Lad  wrote:

 

Marcus, 

 

Thank you for the change.

 

Now I can reach the CILogon page from localhost, but after selecting Indiana 
University as the Identity provider, I get this error:

 

curl_exec() failed. Error: SSL certificate problem: Invalid certificate chain



I tried to login in Incognito mode, but the error persists.

 

 

 

On Thu, Sep 28, 2017 at 3:02 PM, Christie, Marcus Aaron  wrote:

Neeraj, 

 

Try again. This required a change in Keycloak to allow http://localhost:8080 
and http://localhost:8080/callback-url as redirect URLs for the dev testdrive 
realm.

 

On Sep 28, 2017, at 1:01 PM, Neeraj Lad  wrote:

 

Marcus, 

 

Changing the oauth-callback-url gives me the Keycloak error 'Invalid parameter: 
redirect_uri'. It tried using https, 127.0.0.1 but both did not work. A Stack 
Overflow question mentions that Keycloak admin console needs to be configured. 
Here is the question:

https://stackoverflow.com/questions/45352880/keycloak-invalid-parameter-redirect-uri

 

 

There are three of files apart from pga_config which make use of the variable 
oauth-callback-url.

 

1. /app/config/pga_config.php.template: This sets the url to 
'https://dev.seagrid.org/callback-url'. I guess this file is not relevant here.

 

2. /app/libraries/Keycloak/KeycloakServiceProvider.php: This uses 
Config::get('pga_config.wsis') to create a variable 'identityServerConfig' 
which has the oauth-callback-url as a field. The file 'pga_config.wsis' is not 
on the codebase. Config is related to Laravel. I am not sure how Config returns 
back 'identityServerConfig'. Am I right in guessing that you are extracting the 
pga_config.wsis from some server? 

 

3. /app/libraries/Wsis/Wsis.php: This again makes use of 
Config::get('pga_config.wsis')  

 

 

Do you think there is a need to change some configuration on Keycloak. If yes, 
how can I do it?

 

 

 

 

On Thu, Sep 28, 2017 at 9:15 AM, Christie, Marcus Aaron  wrote:

 

On Sep 28, 2017, at 8:24 AM, Neeraj Lad  wrote:

 

Hi all, 

 

I've followed the instructions to install all the dependencies for the PGA. Now 
I need to run it locally.

 

I got the pga_config file from Eroma, but using it and logging in with CILogon 
takes me to the dev.testdrive.airavata.org, instead of the version at 
localhost:8080. How can I avoid this?

 

Neeraj,

 

change oauth-callback-url in pga_config.php to have the value 
‘http://localhost:8080/callback-url'



 

As another option, I know there are instructions to set PGA on CentOS using 
Ansible. But are there any similar instructions for OS X?

 

 

No. Closest thing would be to run VirtualBox on OS X and use Ansible to install 
PGA on the VirtualBox VM.  Eric Coulter did a POC of this: 
https://github.com/ECoulter/airavata/tree/develop/dev-tools/ansible/inventories/local-vbox.

 

 

 

 

 

 

 

 

 



smime.p7s
Description: S/MIME cryptographic signature

FW: Gateways 2017: Early-bird rates until Sept. 1; register by Th., Oct. 12

[Pierce, Marlon]

Hi all, 

 

Registration for the Gateways 2017 conference is now open!  I hope you are able 
to attend, and please also share this with others who may be interested.

 

Thanks,

 

Marlon

 

 

From: Science Gateways Community Institute 
 on behalf of Science 
Gateways Community Institute 
Reply-To: Science Gateways Community Institute 
Date: Thursday, August 17, 2017 at 9:29 AM
To: "Pierce, Marlon" 
Subject: Gateways 2017: Early-bird rates until Sept. 1; register by Th., Oct. 12

 

Registration open; Poster submissions due 9/8; Resource Expo seeking vendors 
and resource providers. 
View this email in your browser 

  
 

Gateways 2017: The 12th Gateway Computing Environments Conference
Monday through Wednesday, October 23–25, 2017
Hosted at the University of Michigan, Ann Arbor
https://sciencegateways.org/gateways2017
Science gateways allow science & engineering communities to access shared data, 
software, computing services, instruments, educational materials, and other 
resources specific to their disciplines. They are typically a web portal or a 
suite of desktop applications.

Gateway developers and users — regardless of their domain area — have a lot in 
common but have had few venues for exchanging experiences. The expanded 
Gateways annual conference is an opportunity for gateway creators and 
enthusiasts to learn, share, connect, and shape the future of gateways while 
forming a vibrant community with common interests.

Early-bird registration is now open through Friday, September 1. (Regular 
registration closes Thursday, October 12.) Book your hotel now for the best 
rates!The conference program is already brimming with exciting keynotes, 
tutorials, and sessions including:
Keynotes including Elyse Aurbach-Pruitt, founder of the Researchers Expanding 
Lay-Audience Teaching and Engagement (RELATE) program, about engaging your 
gateway’s audience
Eight hands-on tutorials featuring gateway building platforms, gateway tools 
and strategies for success
30 presentations and demos, plus a panel featuring lessons learned by gateway 
developers on their path to building successful and sustainable gateways
Opportunities to connect with like-minded colleagues and learn about resources 
and services that may benefit you at the Resource Expo, Poster Session, and 
Reception and during the “Open Space” session in which you create the agenda!
Please note that anyone is welcome to participate in the Poster Session. Submit 
an abstract for your poster by Friday, Sept. 8.Do you or a colleague offer 
resources or services that would be of interest to the science gateways 
community? Consider reserving a space at the Resource Expo, to be held on the 
evening of Tuesday, October 24. It’s an opportunity to showcase products and 
services, network with attendees, and recruit individuals eager to pursue 
careers in the field. For more information, visit 
https://sciencegateways.org/web/gateways2017/program/resource-expo.
Questions? Email h...@sciencegateways.org 

 

 

 


Copyright © 2016 Science Gateways Community Institute, All rights reserved. 
You are receiving this email because you asked to be informed about the 
activities of the Science Gateways Community Institute or you have participated 
in an event associated with the Institute. These updates will not be sent too 
frequently, so we hope that you'll continue to stay in touch with us!  Please 
add i...@sciencegateways.org to your address book so that you get these notices 
in your inbox, not in a promotional folder.Our mailing address is:
Science Gateways Community Institute
University of California at San Diego
9500 Gilman Dr
San Diego, CA 92093-0505

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list

  






This email was sent to marpi...@iu.edu 
why did I get this?unsubscribe from this listupdate subscription 
preferences 
Science Gateways Community Institute · University of California at San Diego · 
9500 Gilman Dr · San Diego, CA 92093-0505 · USA 





smime.p7s
Description: S/MIME cryptographic signature

Re: Using Redis or Zookeeper for Email Monitoring

[Pierce, Marlon]

Agreed. I think some architecture diagrams are in order.  

 

Marlon

 

From: "Shenoy, Gourav Ganesh" 
Reply-To: "dev@airavata.apache.org" 
Date: Thursday, July 20, 2017 at 1:35 PM
To: "dev@airavata.apache.org" 
Subject: Re: Using Redis or Zookeeper for Email Monitoring

 

Apoorv,

 

Good thought about making the email monitoring service scalable and reliable. 
But I was wondering if you considered the Helix angle? Helix inherently uses 
Zookeeper to manage state and the cluster as a whole. You might want to 
consider leveraging some of Helix’s data handling capabilities. Nothing against 
Redis, Zookeeper looks like a definite candidate to me just because of Helix. 

 

Also, could you chalk out and share an architecture of how this email 
monitoring service would integrate with Airavata (assume a Helix architecture). 
Drawing an architecture would help you better understand the problem and even 
find smaller issues which you didn’t consider.

 

Good work so far!

 

Thanks and Regards,

Gourav Shenoy 

 

From: Apoorv Palkar 
Reply-To: "dev@airavata.apache.org" 
Date: Wednesday, July 19, 2017 at 1:59 PM
To: "dev@airavata.apache.org" 
Subject: Using Redis or Zookeeper for Email Monitoring

 

Currently I'm trying to solve the problem of done emails coming before start 
emails and making the system scalable. Today, we have only one GFaC that 
handles the email monitoring system. As we are moving towards a microservices 
approach from the monolithic code we have, this email monitoring system also 
needs to adapt to these changes. Currently, in the gfac code, a concurrent 
hashmap is kept to keep track of start/end of emails using their respective 
experiment ID's. Instead of keeping the hashmap locally, we should keep it in a 
global state so in the future multiple GFaCs can handle the map. Supun has 
suggested to use Zookeeper for this as it has high avaliability and 
realibility. I was also thinking that since these experiment IDs are a key 
value pair, Redis would be a good option for such a use case. What do you guys 
think about each one. I understand airavata currently uses zookeeper, so 
development wise there seems to be an edge toward it. Would Redis be a good for 
such use case?



smime.p7s
Description: S/MIME cryptographic signature

Re: Adios

[Pierce, Marlon]

Thanks, Ajinkya, and good luck.  You are always welcome here on the dev list to 
share any insights.

 

Marlon

 

 

From: Ajinkya Dhamnaskar 
Date: Friday, June 16, 2017 at 2:12 PM
To: Airavata Dev 
Subject: Adios

 

Hello All, 

 

I am stepping out of Bloomington. I must take this opportunity to thank team 
Airavata. 

It has been a great experience which has definitely changed my way of thinking 
towards any computer science problem.

 

Special thanks to Marlon and Suresh for giving me this opportunity, I tried my 
level beast to stand up to your expectations. 

 

I hope to keep contributing as possible.

 

Let me be your host when you are in Bay Area :)

 

Again, thanks for everything!

 

-- 

Thanks and regards, 

 

Ajinkya Dhamnaskar

Student ID : 0003469679

Masters (CS)

+1 (812) 369- 5416



smime.p7s
Description: S/MIME cryptographic signature

Re: XML tips

[Pierce, Marlon]

Hi Apoorv, how are you using XML, and what do you want to do?

 

Marlon

 

 

From: Apoorv Palkar 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 2, 2017 at 10:51 AM
To: "dev@airavata.apache.org" 
Subject: XML tips

 

Hey Dev, 

 

I've completed my spouts/bolts for the Storm demo of the distributed workflow 
manager. I'm now putting together the pieces ( ie creating the mavin project, 
editing the xml, adding dependencies, getting config file to work.) Is anybody 
very familiar with XML? If so, what resources should I use?

 

Thanks



smime.p7s
Description: S/MIME cryptographic signature

Re: Apache Flink Execution

[Pierce, Marlon]

Thanks, Apoorv.  Note for everyone else: request access if you’d like to leave 
a comment or make a suggestion.

 

Marlon

 

From: Apoorv Palkar 
Reply-To: "dev@airavata.apache.org" 
Date: Wednesday, May 24, 2017 at 11:32 AM
To: "dev@airavata.apache.org" 
Subject: Apache Flink Execution

 

https://docs.google.com/document/d/1GDh8kEbAXVY9Gv1mmFvq__zLN_JP6m2_KbfN-9C0uO0/edit?usp=sharing
 

 

LINK for Flink Use/fundamental



smime.p7s
Description: S/MIME cryptographic signature

Re: Storm Analysis

[Pierce, Marlon]

Thank you, Apoorv, for the Storm and Spark analysis.  Can you move these docs 
to either the Airavata Confluence Wiki or to Google Docs and post the link?  

 

Marlon

 

 

From: Apoorv Palkar 
Reply-To: "dev@airavata.apache.org" 
Date: Monday, May 22, 2017 at 2:47 PM
To: "dev@airavata.apache.org" 
Subject: Storm Analysis

 

Uploaded basics of Apache Storm/ Uses for Airavata. Will now work on Flink and 
write similiar 1 pg paper on how it works/ pros/cons.  

 

 



smime.p7s
Description: S/MIME cryptographic signature

Re: [VOTE] enables writes to Airavata GitHub mirror

[Pierce, Marlon]

Thanks, Suresh, this looks like a nice summary.  I’m +1 for GitHub for the pros 
you list below. None of the cons look like blockers.

 

Marlon

 

 

From: Suresh Marru 
Reply-To: "dev@airavata.apache.org" 
Date: Monday, May 22, 2017 at 1:16 PM
To: Airavata Dev 
Subject: [VOTE] enables writes to Airavata GitHub mirror

 

Hi All. 

 

ASF has been experimentally allowing GitHub mirrors to enable write’s. 
Currently Airavata’s GitHub mirror is read-only and committers have to push the 
code to ASF’s canonical git. After discussing the feasibility and current 
status of this experiment with ASF Infrastructure team at apachecon 2017, I 
would like to propose we request Airavata mirrors also be write enabled. Here 
is a brief list of pros and cons:

 

Pros:

- Write enabled GitHub mirrors will allow us to use a simpler workflow of 
merging pull requests. Currently the steps are outlines at [1] and [2], Instead 
it will be [3]. 

- With more developers familiar with GitHub workflows, we can hopefully receive 
more contributions. 

- Accept contributions at a faster rate. 

- Potentially consider adopting every commits to go through pull requests (this 
a separate discussion topic). 

- Can setup Jenkins builds so all pull requests get automatically built and 
initiate the merge process only after CI succeeds. 

- In the future consider automated merges based on number of +1’s or based on 
number of tests passed and so forth. 

 

Cons:

- This is still an experimental capability and we will be one of early projects 
(certainly not the first) to get on board, there could be potentially bugs. 

- Technically speaking, both ASF and GitHub will accept writes, but we have to 
pick one as canonical for Airavata. This might be confusing, but with clear 
instructions we could clarify. 

- When GitHub has outage, we should manually switch to ASF canonical repos and 
push and hope the synchronization works. This could potentially lead to 
inconsistencies, but the downtimes of GitHub seems to be minimal.

- To merge the code to GitHub, Committers and PMC members have to enable 2 
factor authentication on their GitHub account [4] and ensure GitHub user name 
is specified in id.apache.org, verify if everything is setup correctly at [5].

 

I may be missing something’s, but lets discuss and vote on this thread. Please 
voice your opinions either way. 

 

Suresh

 

[1] - https://airavata.apache.org/community/how-to-contribute-code.html

[2] - https://airavata.apache.org/community/how-to-commit-contributed-code.html

[3] - https://help.github.com/articles/merging-a-pull-request/

[4] - 
https://help.github.com/articles/securing-your-account-with-two-factor-authentication-2fa/

[5] - https://gitbox.apache.org/setup/ 

 



smime.p7s
Description: S/MIME cryptographic signature

Improperly labeling output data staging failures

[Pierce, Marlon]

I’ve noticed jobs failing during the output data staging step that still get 
marked as “completed” rather than failed.  Failures were most likely caused by 
SCP timeouts to a very slow host. 

 

Looking at the code, outputDataStaging() [1] doesn’t call the checkFailures() 
[2] method like inputDataStaging and executeJobSubmission. It instead embeds 
the contents of checkFailures() directly in the method.  The problem is that 
while checkFailures() throws an exception if there was a task failure, it looks 
like outputDataStaging does not throw an exception.

 

Looks like a bug, but was this done for some reason?

 

Marlon

 

 

[1]  
https://github.com/apache/airavata/blob/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/impl/GFacEngineImpl.java#L749

 

[2] 
https://github.com/apache/airavata/blob/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/impl/GFacEngineImpl.java#L613

 

 



smime.p7s
Description: S/MIME cryptographic signature

Re: [GSoC] Number of Deliverables

[Pierce, Marlon]

Discussing those better ways on the dev list is also encouraged.  No plans 
every stay the same. The value of plans is the process of planning, not the 
plan itself, as someone else has said.

 

Marlon

 

 

From: Supun Nakandala 
Reply-To: "dev@airavata.apache.org" 
Date: Wednesday, March 29, 2017 at 6:12 PM
To: dev 
Subject: Re: [GSoC] Number of Deliverables

 

I think what is important here is what problem(s) you promise to solve. 

 

In the proposal you will have to show some plan on how you intend to solve 
those problem(s) mainly to convince the mentors that you have an idea on what 
you are planning to do.

 

While executing the project, finding better ways to solve the initial 
problem(s) is always encouraged. :)

 

On Wed, Mar 29, 2017 at 6:05 PM, Apoorv Palkar  wrote:

If you promise to do things a certain way, but you find a better solution when 
actually working the project, can you implement new ideas and scrap old ones? 



-Original Message-
From: Supun Nakandala 
To: dev 
Sent: Wed, Mar 29, 2017 4:43 pm
Subject: Re: [GSoC] Number of Deliverables

Hi Apoorv, 

 

As a sample project proposal, I would recommend you to refer this.

 

>From my experiences as a past GSoC student, I think having specific and 
>challenging goals should make your proposal more attractive and increase the 
>chances of getting accepted.

 

However trying to come up with a set of goals which you think is overly 
unrealistic will also hinder your success later because you will be judged 
based on what you promise(proposal). It is completely ok to not being able to 
achieve what you promise. But you will have to prove that you put significant 
effort in achieving your goals (which can get tricky).

 

-Supun

 

On Wed, Mar 29, 2017 at 5:23 PM, Apoorv Palkar  wrote:

How many goals should we aim to put in our proposal? Is it better to put in 
small goals and over-deliver? 

 

Thanks 



 

-- 

Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



 

-- 

Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



smime.p7s
Description: S/MIME cryptographic signature

Re: List of Devops Scripts for Apache Airavata

[Pierce, Marlon]

Re: Marcus’s second bullet. Separating these into separate branches puts the 
burden on using CI/CD.

 

Marlon

 

 

From: "Christie, Marcus Aaron" 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, March 10, 2017 at 11:54 AM
To: "dev@airavata.apache.org" , "Shenoy, Gourav 
Ganesh" 
Subject: Re: List of Devops Scripts for Apache Airavata

 

Dev, 

 

Eric and I met this morning to discuss how to consolidate these various 
efforts.  Here’s what we came up with:

 

1. (Marcus) Create an airavata-devops repo for holding all of the Ansible 
scripts.  To start with, we’ll use the dev-tools/ansible scripts.

2. (Eric) Once the airavata-devops repo is in place, Eric will move his 
Jetstream provisioning scripts onto a branch there

3. (Eric) Eric is also going to start working on Jetstream scripts for other 
servers besides PGA

 

A question for Gourav: are the modules/cloud/ansible-playbooks still being 
actively used?

 

Some other things we discussed:

* How to securely store secret information?  We discussed using ansible-vault 
versus having a separate, private repo for Ansible inventories.  We settled on 
using ansible-vault since it has the advantage of keeping everything in one 
repo. The disadvantage is in needing to share a password file out-of-band with 
the rest of the team.  However, we felt the sharing of the password would be 
mitigated as we move toward more automated systems for running deployments (for 
example, users who only need to run a deploy won’t need the password file if we 
use Jenkins to run deploys; only Jenkins would really need to have the password 
file).

* One advantage to NOT having a separate repo for devops scripts is that it 
keeps the devops scripts versioned together with the code those scripts can 
build and deploy.  Having a separate repo means we will need to have some 
discipline when updating devops scripts.  I think it makes sense to have the 
devops repo branch structure reflect the airavata branch structure.  That is, 
the ‘develop’ branch of the devops repo should be good for deploying the 
‘develop’ branch of airavata and likewise for ‘master’.  If code is updated in 
airavata on a separate branch that needs corresponding devops script updates, 
that should likewise be done on a same-named separate branch in the devops repo.

 

Eric, let me know if I missed anything.

 

 

Thanks,

 

Marcus

 

 

On Mar 9, 2017, at 3:37 PM, Anuj Bhandar  wrote:

 

Gourav,

 

It is a great initiative, I vote for a separate repository for devops, the 
Airavata repository plays many roles already.

 

A separate repository would aid active development and bring modularity.

 

Thanks,

Anuj Bhandar

 

On 3/9/17 3:14 PM, Coulter, John Eric wrote:

Thanks for starting this, Gourav! 

I'm inclined to vote for a separate DevOps repo, to keep things more modular. 

 

I know Marcus and I have done some work/testing starting from scripts that I 
think Shameera created in dev-tools/ansible. I've got a side-repo up which 
contains playbooks for provisioning and deploying the PGA from scratch as a 
proof of concept, using dynamic inventory instead of a static file (so, on an 
empty jetstream allocation, you get working instance running the PGA, with 
router, public ip with a single ansible-playbook command). Next step there is 
to add provisioning for VMs to run airavata, etc. 
 

Script locationBranchPurpose of Script
modules/cloud/ansible-playbooks​​​
develop
1. Provisions instances on EC2 and OpenStack (Jetstream)
2. Deploys a Mesos/Marathon cluster on the provisioned i​nstances
 ​
​dev-tools/ansible/​develop​1. Deploy/update airavata services on existing 
machines​/instances
2. todo - include provisioning scripts for cloud resources
https://github.com/ECoulter/airavata-vms​​1. POC for provisioning and 
deployment of PGA on Jetstream, with dynamic inventory. (intended to merge with 
dev-tools/ansible when complete/tested by others)
 

 

---

 

Cheers,

-
Eric Coulterjecou...@iu.edu
XSEDE Capabilities and Resource Integration Engineer 
IU Campus Bridging & Research Infrastructure
RT/PTI/UITS
812-856-3250

 

 



smime.p7s
Description: S/MIME cryptographic signature

FW: [sgci-mgmt] Webinar announcement to circulate to XSEDE, Campus Champions, etc.

[Pierce, Marlon]

Hi all, please see the Webinar announcement below.  This is open to anyone who 
is interested.

 

Marlon

--

 

 

For circulation:

 

The next SGCI Webinar will take place February 8, 2017, at 1 pm Eastern/10 am 
Pacific (1 hour). Register:  
http://sciencegateways.org/upcoming-events/webinar-registration/

 

Creating a developer pipeline by teaching gateway technologies
Presented by Marlon Pierce and Suresh Marru

Do you have trouble recruiting and retaining good gateway developers? Common 
challenges include the allure of the cutting-edge commercial sector, academic 
pay scales, and the specialized knowledge necessary for development. The 
Indiana University Science Gateways Research Center is attempting to turn this 
challenge into an opportunity by teaching Web-based cyberinfrastructure systems 
using concepts and technologies that will benefit students in their 
non-academic careers. Such distributed systems require knowledge of both 
classic and cutting-edge topics, such as microservices and “DevOps” practices. 
Meanwhile, the instructors can identify and connect with promising students 
while keeping themselves aware of important trends and technologies. This 
webinar will provide an overview of the material and concepts that we cover in 
the class, describe the student project assignments and class organization, and 
present outcomes and student feedback that we have gotten over the last two 
semesters. We will also discuss possibilities of making the course material and 
instruction available to other interested universities. Course information and 
material is available from http://courses.airavata.org/.

Our webinars are scheduled on the second Wednesday of each month at 1 pm 
Eastern/10 am Pacific. We’ll announce them several weeks before through our 
newsletter and on our website.



smime.p7s
Description: S/MIME cryptographic signature

FW: International Workshop on Science Gateways 2017 CFP, Poznan, Poland

[Pierce, Marlon]

Hi all, 

In case you are interested, here is a science gateways conference coming up.

Marlon


On 1/20/17, 12:25 PM, "XSEDE User News"  wrote:

A new message has been posted to XSEDE User News.

Categories: Science Gateways, Conferences

Start time: 19 Jun, 2017 09:00 CDT
End time: 21 Jun, 2017 17:00 CDT


Posted on 20 Jan, 2017 17:20 UTC by Nancy Wilkins-Diehr

***
* 9th International Workshop on Science Gateways
* IWSG 2017
* 19-21 June 2017 in Poznan, Poland
* http://iwsg2017.psnc.pl/
***

Science gateways are a community-specific portals offering tools,
applications, and data collections that are integrated together,
providing access to data sources and services of distributed computing
infrastructures (DCIs). Science gateways offer the potential to
facilitate data analysis on a large scale and open the utilisation of
DCIs to wider audiences by providing a customised and easy to use user
interface accessing large computational and data resources. The
complexity of the underlying infrastructure can be completely hidden
from the end-users by a suitably tailored interface. As interest in
science gateways has accelerated in the past few years, an increasing
number of new user communities can utilise grid or cloud computing
resources in a convenient manner.

The workshop aims at bringing together researchers and scientists from
different scientific domains, along with science gateways developers, to
discuss problems and solutions in the area, to identify new issues, to
shape future directions for research, foster the exchange of ideas,
standards and common requirements and push towards the wider adoption of
science gateways in e-Science.

We invite the submission of papers related to various aspects of Science
Gateways. Suggested topics include, but are not limited to:

* Science gateway enabling technologies and development frameworks
* Ready to use science gateways in different areas and disciplines
* Management of high-throughput data via science gateways
* Portal technology and portal construction methods
* Usage models and gateway tools in different disciplines
* Security aspects of science gateways
* Usability studies of science gateways

We in particular encourage the submissions of application scientists and
community members, reporting about their experiences with science
gateways. There are three alternatives for researchers to present their
work: talks, lightning talks, and demonstration sessions. Submission of
a full paper may result in a talk, submission of an abstract may result
in a lightning talk or a demonstration. All contributions will be
subject of a double blind peer review which will serve as basis for the
decision of acceptance. Further details of the call are available at
http://iwsg2017.psnc.pl/
Accepted full papers will be published in a fully indexed journal and
accepted abstracts will be published online on the IWSG webpage. As
negotiations with different publishers are pending please check the IWSG 
2017
website throughout the next weeks for more information.

Important dates:
• The deadline for paper/abstract submission: 20 March 2017
• Notification of acceptance: 17 April 2017
• Submission of camera ready papers/abstracts: 01 May, 2017
• Registration deadline for authors/presenters: 15 May, 2017
• Registration closing: 29 May 2017
• Date of the workshop: 19-21 June, 2017

For further details please visit the workshop website at
http://iwsg2017.psnc.pl/



Dr Krzysztof Kurowski
Application Division Director
Poznan Supercomputing and Networking Center

Dr Tamas Kiss
Centre for Parallel Computing
Department of Computer Science
Faculty of Science and Technology
University of Westminster



This message was generated by XSEDE User News v3.0.  You can view this 
message in the XSEDE User Portal at 
https://portal.xsede.org/web/xup/user-news/-/news/item/7817.

You can manage your XSEDE User News subscriptions at 
https://portal.xsede.org/group/xup/user-news/-/news/manage-subscription.



smime.p7s
Description: S/MIME cryptographic signature

Re: Improving the PGA Install

[Pierce, Marlon]

+1 in general for this “requirement”. I was thinking the same thing.

 

From: "Christie, Marcus Aaron" 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, January 6, 2017 at 2:20 PM
To: "dev@airavata.apache.org" 
Subject: Re: Improving the PGA Install

 

Eric, 

 

It’s great to have you working on this.

 

I’m not sure if this really impacts the decision making process much, but I had 
a thought. One need I have for an improved installation process for PGA is to 
use that as a much easier to use local development environment. However, the 
needs for a local development environment vs a deployed production instance of 
an application are somewhat different. For example, if I were using Docker for 
local development I would want to map my local filesystem into the Docker 
instance so that code changes I make are reflected in the app running in 
Docker.  But that sort of filesystem mounting wouldn’t be needed when running 
in production.

 

If we end up with a Docker configuration for development and one for production 
app deployments, it would be good if they could share as much configuration as 
possible.

 

Also, just to be clear, I’m just using Docker as an example above to illustrate 
the point.

 

Thanks,

 

Marcus

 

On Jan 5, 2017, at 2:22 PM, Eric Coulter  wrote:

 

Greetings, all!

As of the new year, I'm funded to help out more with different bits of
the Airavata project and the SGRC. One of the first things I'd like to look at 
is
improving the installation process for the default PGA. After a meeting
this morning, it seems there are a few options. I'd like to start moving
on one of them pretty soon, but we thought some discussion would be
helpful to see which way might be most useful.

Please take a look at these, and let me know any
thoughts/feedback/preferences, or any useful experience with the tools
mentioned! (They are in no particular order.)

1. Continue to use Ansible; develop a role that is as OS-agnostic as
possible (can detect OS, dynamic inventory, etc.) There is already some
work done in this direction, but is this the best place to spend my time?

2. Develop OS-specific packages - rpm, deb, something for mac
(homebrew/port?)
 - nice from a "getting other people to host this" viewpoint, but that
doesn't seem to be a huge sticking point.

3. Docker/Singularity? Puppet? Chef? Vagrant? Singularity might be
better for other people in the research space. This would still require
creating different container images for each OS. Does this cause any
problems for database backups? Worth the time to get used to a new tool?

4. Should we be leveraging the tools available from Amazon more heavily?
This might require a fundamental change in the PGA, but could possibly
save a lot of work on our side. We don't know how much it could save,
though.

Cheers,
Eric C.

 



smime.p7s
Description: S/MIME cryptographic signature

Re: Database Version Control for Airavata

[Pierce, Marlon]

+1 for looking into Liquibase. I’d be interested to hear other opinions.


Marlon

 

From: Anuj Bhandar 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, December 23, 2016 at 12:55 PM
To: "dev@airavata.apache.org" 
Subject: Database Version Control for Airavata

 

Hello Dev,

Merry Christmas ! Hope you guys are doing good.

As a part of the Airavata 0.17 release, we are faced with a industry standard 
problem of migrating our old master database to the new Development database, 
the effort needed to manually write migration SQL scripts is huge.
Hence for the current release, I'm relying on DBDiff 
(http://dbdiff.github.io/DBDiff/) for generating Delta SQL scripts for the 
entire database and manually write scripts for complex DB schema changes.

To address the issue in future releases, I propose to implement a database 
version control system like Liquibase (http://www.liquibase.org/) for Airavata, 
this will help automate the creation of delta scripts and also adding 
additional functionality like backward compatibility between releases.

Please feel to suggest changes and also comment about the products I'm using to 
accomplish these tasks.

Thanks and best regards,

Anuj Bhandar
MS Computer Science
Indiana University Bloomington
+1 812 361 9414



smime.p7s
Description: S/MIME cryptographic signature

Science Gateway Sustainability Workshop

[Pierce, Marlon]

Hi all,

 

I want to draw your attention to the opportunity below. This is not directly an 
Apache Airavata-related workshop, but we will be covering the Apache Way and 
general best practices for open source project management and governance.

 

Thanks,

 

Marlon

 



 

The Science Gateways Community Institute is offering an intensive workshop that 
will take place April 24-28, 2017 at the Purdue Research Park of Indianapolis, 
IN.

Science Gateways Bootcamp: Strategies for Developing, Operating, and Sustaining 
Your Gateway is designed for leaders of innovative digital offerings, sometimes 
called gateways, who are seeking to further develop and scale their work.

 

Participants will engage in hands-on activities to help them articulate the 
value of their work to key stakeholders and to create a strong development, 
operations, and sustainability plan. The workshop will include:

• Core business strategy skills 

• Technology best practices

• Long-term sustainability strategies

By the end of the workshop, participants will have developed a working 
hypothesis of their sustainability strategy and identified the key action steps 
to get there. 

For this session, we will be selecting 6-10 teams to participate, based on 
their potential to deliver significant value to the community. We expect that 
this session will fill quickly. There is no cost to participate in the workshop 
and most meals will be included. Attendees will need to pay for travel, hotel, 
and a couple of meals. 

 

APPLICATIONS WILL BE ACCEPTED THROUGH 2/22/2017.

 

The workshop will be led by:

Michael Zentner: Director of the SGCI Incubator, Senior Research Scientist and 
Director of HUBzero Platform for Science and Engineering, Entrepreneur in 
Residence at the Purdue Foundry

Betsy Hillery: SGCI Incubator, Manager, Research Support Operations, Research 
Computing, Purdue University

Juliana Casavan: Entrepreneurial Programs Manager, The Foundry, Purdue 
University

Sandra Gesing: Research assistant professor, Department of Computer Science and 
Engineering and computational scientist, Center for Research Computing, 
University of Notre Dame

Randy Heiland: Senior Systems Analyst/Programmer, Center for Applied 
Cybersecurity Research (CACR), Indiana University, Bloomington

Nancy Maron: Founder and Principal, BlueSky to BluePrint 

Suresh Marru: Deputy director of the Science Gateway Research Center at Indiana 
University and vice president of the Apache Airavata project.

Nayiri Mullinix: Community Engagement and Exchange Coordinator, SGCI, 
University of Michigan

Mihaela Vorvoreanu: Associate professor, Department of Computer Graphics 
Technology, Purdue University

 

For more details, or to apply, please visit this link: 

http://sciencegateways.org/upcoming-events/science-gateways-bootcamp/

 



smime.p7s
Description: S/MIME cryptographic signature

Apache Jenkins emails

[Pierce, Marlon]

How about directing the Jenkins messages to the “issues” or another list 
besides dev@airavata?

 

Marlon

 



smime.p7s
Description: S/MIME cryptographic signature

Re: Running MPI jobs on Mesos based clusters

[Pierce, Marlon]

BSD is ok: https://www.apache.org/legal/resolved. 

 

From: Mangirish Wagle 
Reply-To: "dev@airavata.apache.org" 
Date: Thursday, October 13, 2016 at 12:03 PM
To: "dev@airavata.apache.org" 
Subject: Re: Running MPI jobs on Mesos based clusters

 

Hello Devs,

I needed some advice on the license of the MPI libraries. The MPICH library 
that I have been trying claims to have a "BSD Like" license 
(http://git.mpich.org/mpich.git/blob/HEAD:/COPYRIGHT).

I am aware that OpenMPI which uses BSD license is currently used in our 
application. I had chosen to start investigating MPICH because it claims to be 
a highly portable and high quality implementation of latest MPI standard, 
suitable to cloud based clusters.

If anyone could please advise on the acceptance of the MPICH libraries MSD Like 
license for ASF, that would help.

Thank you.

Best Regards,

Mangirish Wagle

 

On Thu, Oct 6, 2016 at 1:48 AM, Mangirish Wagle  
wrote:

Hello Devs, 

 

The network issue mentioned above now stands resolved. The problem was with the 
iptables had some conflicting rules which blocked the traffic. It was resolved 
by simple iptables flush.

 

Here is the test MPI program running on multiple machines:-

 

[centos@mesos-slave-1 ~]$ mpiexec -f machinefile -n 2 ./mpitest

Hello world!  I am process number: 0 on host mesos-slave-1

Hello world!  I am process number: 1 on host mesos-slave-2

 

The next step is to try invoking this through framework like Marathon. However, 
the job submission still does not run through Marathon. It seems to gets stuck 
in the 'waiting' state forever (For example 
http://149.165.170.245:8080/ui/#/apps/%2Fmaw-try). Further, I notice that 
Marathon is listed under 'inactive frameworks' in mesos dashboard 
(http://149.165.171.33:5050/#/frameworks).

 

I am trying to get this working, though any help/ clues with this would be 
really helpful.

 

Thanks and Regards,

Mangirish Wagle



 

On Fri, Sep 30, 2016 at 9:21 PM, Mangirish Wagle  
wrote:

Hello Devs, 

 

I am currently running a sample MPI C program using 'mpiexec' provided by 
MPICH. I followed their installation guide to install the libraries on the 
master and slave nodes of the mesos cluster.

 

The approach that I am trying out here is that I am equipping the underlying 
nodes with MPI handling tools and then use the Mesos framework like Marathon/ 
Aurora to submit jobs to run MPI programs by invoking these tools.

 

You can potentially run an MPI program using mpiexec in the following manner:-

 

# mpiexec -f machinefile -n 2 ./mpitest

machinefile -> File which contains an inventory of machines to run the program 
on and number of processes on each machine.
mpitest -> MPI program compiled in C using mpicc compiler. The program returns 
the process number and he hostname of the machine running the process.
-n option indicates number of processes that it needs to spawn
Example of machinefile contents:-

 

# Entries in the format :

mesos-slave-1:1

mesos-slave-2:1

 

The reason for choosing slaves is that Mesos runs the jobs on slaves, managed 
by 'agents' pertaining to the slaves.

 

Output of the program with '-n 1':-

 

# mpiexec -f machinefile -n 1 ./mpitest

Hello world!  I am process number: 0 on host mesos-slave-1

 

But when I try for '-n 2', I am hitting the following error:-

 

# mpiexec -f machinefile -n 2 ./mpitest

[proxy:0:1@mesos-slave-2] HYDU_sock_connect 
(/home/centos/mpich-3.2/src/pm/hydra/utils/sock/sock.c:172): unable to connect 
from "mesos-slave-2" to "mesos-slave-1" (No route to host)

[proxy:0:1@mesos-slave-2] main 
(/home/centos/mpich-3.2/src/pm/hydra/pm/pmiserv/pmip.c:189): unable to connect 
to server mesos-slave-1 at port 44788 (check for firewalls!)

 

It seems to not allow the program execution due to network traffic being 
blocked. I checked security groups in scigap openstack for mesos-slave-1, 
mesos-slave-2 nodes and it is set to 'wideopen' policy. Furthermore, I tried 
adding explicit rules to the policies to allow all TCP and UDP (Currently I am 
not sure what protocol is used underneath), even then it continues throwing 
this error.

 

Any clues, suggestions, comments about the error or approach as a whole would 
be helpful.

 

Thanks and Regards,

Mangirish Wagle

 

Error! Filename not specified.

 

On Tue, Sep 27, 2016 at 11:23 AM, Mangirish Wagle  
wrote:

Hello Devs, 

 

Thanks Gourav and Shameera for all the work w.r.t. setting up the 
Mesos-Marathon cluster on Jetstream.

 

I am currently evaluating MPICH (http://www.mpich.org/about/overview/) to be 
used for launching MPI jobs on top of mesos. MPICH version 1.2 supports Mesos 
based MPI scheduling. I have been also trying to submit jobs to the cluster 
through Marathon. However, in either cases I am currently facing issues which I 
am working to get resolved.

 

I am compiling my notes into the following google doc. You may please review 
and let me know your comments, suggestions.

 

https://docs.google.com/document/d

Re: Ansible script to deploy Airavata on AWS

[Pierce, Marlon]

Nice, thanks. I hope this makes deployments a lot easier.

 

From: Shameera Rathnayaka 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, September 23, 2016 at 10:05 PM
To: dev 
Subject: Re: Ansible script to deploy Airavata on AWS

 

Hi Devs,  

 

I moved my local airavata ansible script from 
https://github.com/shamrath/airavata-ansible which already under Apache 2.0 
License header, to Airavata repository, Lets maintain the script here. 

 

 

Thanks, 

Shameera.

 

On Thu, Sep 1, 2016 at 7:24 PM Shameera Rathnayaka  
wrote:

Hi Amila,  

 

I would say we should publish this resources in Airavata web-site instead of 
packing this inside the distribution. I will maintain the scripts in my 
personal git repo until we decide to push this to Airavata repo. 

 

Thanks, 

Shameera.

 

On Thu, Sep 1, 2016 at 6:44 PM Amila Jayasekara  wrote:

Nice ! 

 

Shouldn't this be part of the distribution ?

 

Thanks

-AJ

 

On Thu, Sep 1, 2016 at 1:25 PM, Shameera Rathnayaka  
wrote:

Hi All,  

 

I wrote ansible scripts to deploy Apache Airavata and all its prerequisites on 
AWS instances (you can use this script to deploy any centos environment not 
specific to AWS). Plese find the code here 
https://github.com/shamrath/airavata-ansible , I am really happy to help if 
anyone wants to try it out. I am sure there more people have firsthand 
experience of writing ansible for complex systems. Let me know if there 
anything I can do to improve my scripts. 

 

Above repo has ansible play for following.

1, RabbitMQ

2. MariaDB

3 Zookeeper

4 Airavata components ( api -orch, gfac)

5 PGA

6 WSO2 IS

 

Thanks, 

Shameera.

-- 

Shameera Rathnayaka

 

-- 

Shameera Rathnayaka

-- 

Shameera Rathnayaka



smime.p7s
Description: S/MIME cryptographic signature

Re: Docker with Airavata

[Pierce, Marlon]

+1, go for it Colin.

Marlon

On 9/1/16, 11:02 AM, "Suresh Marru"  wrote:

Hi Colin,

That will be a wonderful contribution. I do not think there will be one 
right way of using docker instead multiple possibilities. It will be great if 
we can brainstorm on this thread the possibilities and critique each other 
approaches and come to a consensus on what will work better for Airavata use 
cases. 

We will appreciate your contributions (both discussions and code) and as 
with everything in apache, they will be counted as contributions. 

Thanks,
Suresh

> On Sep 1, 2016, at 10:58 AM, Roy-Ehri, Colin Josef  
wrote:
> 
> Hi Lahiru,
> 
> Are you investigating running Airavata with docker for local testing 
purposes, or in order to spin up multiple instances to handle changing loads?  
I did some work with Terraform (www.terraform.io) and spinning up multiple 
containers in AWS with one command.  I'd be happy try that with Airavata, if 
you think it could be useful.
> 
> Thanks,
> Colin Roy-Ehri, MS.Ed.
> Senior Software Engineer
> Enterprise Decision Support Services (EDSS) Indiana University
> 
> 
>> Hi Lahiru,
>> That was my 2015 GSoC project. It was ended well and I did not work on 
>> this  after the demo to Suresh and Marlon last year. You can find the  
>> instructions how to create and run dockerized Airavata here  
>> > fwJiNtkvM/edit?usp=sharing>
>> .
>> 
>> I am sharing the docker files and required scripts attached. Please 
>> let me know if you find trouble accessing these files and the link.
>> 
>> Thanks
>> Pankaj
>> 
>> On Fri, Aug 19, 2016 at 10:23 PM, Lahiru Ginnaliya Gamathige < 
>> glah...@gmail.com> wrote:
>> 
>>> Hi Devs,
>>> 
>>> I remember a Gsoc student did some docker work, I am curious where 
>>> did it ended, I see that no Dockerfile is in the repo.
>>> 
>>> Do we have those separate ? Please give me some insight on that 
>>> project, I would like to look in to it.
>>> 
>>> Regards
>>> Lahiru
>>> 
> 

Apache Airaveata-Related job openings

[Pierce, Marlon]

Hi all,

The Indiana University Science Gateway Group has two job opens related to 
Apache Airavata, one for an experienced java developer and the other for a 
science gateway consultant. See [1] and [2]. You can contact me by email if you 
would like more information.

Thanks,

Marlon Pierce
Indiana University

[1] https://iujobs.peopleadmin.com/postings/26209
[2] https://iujobs.peopleadmin.com/postings/26195

Re: python 3 support

[Pierce, Marlon]

Hi Prady, have you gotten errors trying to use it?

From: Prady 
Reply-To: "dev@airavata.apache.org" 
Date: Wednesday, July 6, 2016 at 11:13 PM
To: "dev@airavata.apache.org" 
Subject: python 3 support

Hi Devs,

Does the apache thrift support python 3?

Thanks
Pradyut

Re: [GSoC] Midterm Evaluations

[Pierce, Marlon]

Google doc is ok for now if you don’t have access to the wiki.  Don’t let this 
be a blocker. We can migrate.

From: Bikramjeet Singh 
Reply-To: "dev@airavata.apache.org" 
Date: Sunday, June 26, 2016 at 1:10 AM
To: Airavata Dev 
Subject: Re: [GSoC] Midterm Evaluations

Hi,
Do we need to make it on cwiki only? As, I am not getting any option to create 
any page there.

Thanks

On Sat, Jun 25, 2016 at 8:03 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:
GSoC Students,

We (mentors from Airavata) need to turn in our evaluations into the GSOC system 
by Sunday Night. Unlike previous years, the new GSOC system does not allow edit 
of evaluations once submitted. Once we submit, GSoC admins within apache will 
review the code commits and then randomly admins from google. Airavata has been 
lucky to receive good number of gsoc slots over the years. To maintain this, we 
will have to follow the rules and strictly measure code contributions.

To help us do the evaluation, before sunday night please submit the following:
* A detailed modified original gsoc plan. This typically changes during 
community bonding period as you understand Airavata better since the proposal 
time.
* A detailed progress report with pointers to pull request on where you stand 
with the full summer plan.
* Detailed next steps on how you will plan to complete the program with test 
cases and documentation.

You can use [1] as reference.

Suresh

[1] - 
https://cwiki.apache.org/confluence/display/AIRAVATA/Update+for+the+mid+evaluation

Re: XBaya UI

[Pierce, Marlon]

To echo Suresh, it is great to see the progress you have made from the 
screenshots, but you need to share the code.

On 6/10/16, 2:12 PM, "Suresh Marru"  wrote:

>Hi Bikramjeet,
>
>Very nice. Can you submit a pull request with the code you have written so 
>far? 
>
>Suresh
>
>> On Jun 10, 2016, at 1:29 PM, Bikramjeet Singh  
>> wrote:
>> 
>> Hi,
>> 
>> My first milestone was to develop non-functional UI for XBaya in either 
>> Apache Pivot or JavaFX, earlier I started building in Pivot but I was not 
>> able to get quite a lot of things cleared from anywhere so I switched to 
>> JavaFX and finished developing it.
>> 
>> I designed it in the exact same way as it was designed for 0.14 version 
>> using Java Swing library, all the menus and toolbar are kept same.
>> 
>> In case any option is to be added or removed, kindly advise me on that. I am 
>> attaching all the screenshots with this mail.
>> 
>> P.S. I've left the part where Parameter and Monitoring is displayed 
>> intentionally for later as I will allocate the required space to both 
>> dynamically.
>> 
>> Thanks 
>> > 10.24.28 PM.png>> 2016-06-10 at 10.24.47 PM.png>> PM.png>> 10.25.27 PM.png>> 2016-06-10 at 10.26.07 PM.png>
>

Re: Airavata with Ansible

[Pierce, Marlon]

+1, I encourage everyone to try this out. Eric says next step is the PGA.

From: Suresh Marru 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 10, 2016 at 2:14 PM
To: Airavata Dev 
Subject: Re: Airavata with Ansible

Hi Eric,

This is indeed very exciting to see. Thanks for your contribution. I will play 
with it and get some hands on experience before I comment.

Suresh

On Jun 10, 2016, at 12:19 PM, Eric Coulter 
mailto:coulter...@gmail.com>> wrote:

Hi All,

I've been working (sporadically) for the last few weeks to create a unified way 
of installing airavata (next step is the PGA) - I've got a working (I *think* - 
from a bare CentOS 7 minimal VM) version up now at

https://github.com/ECoulter/Airavata_deploy

Would ya'll mind taking a look and letting me know how it goes?

If it seems workable, I can create a pull request to add this to the project in 
a separate folder like scripts/ or install/.

Cheers,
Eric Coulter.

Re: Apache Aurora-Mesos Integration (Job Submission code)

[Pierce, Marlon]

+1 for investigating a native client instead of wrapping the command line.

From: Suresh Marru 
Reply-To: "dev@airavata.apache.org" 
Date: Tuesday, June 7, 2016 at 4:08 PM
To: Airavata Dev 
Subject: Re: Apache Aurora-Mesos Integration (Job Submission code)

Hi Gourav,

Very nice. Can you create a JIRA to track this project and I will create a 
placeholder module to push this code to?

Also, have you or Abhishek been looking into the native Java Client? -Bill 
Farner’s suggestion in this thread?  http://markmail.org/thread/yfq7nj744kqk4pf5

Suresh


On Jun 7, 2016, at 12:07 AM, Gourav Rattihalli 
mailto:gratt...@binghamton.edu>> wrote:

Hi Dev's,

I have started on writing the code for the submission of a job to Apache 
Aurora. It's a beginning and currently, it can create .aurora config files, 
launch a job and kill a specified job.

Options:
-o = create/kill
-n = name of the job
-r = amount of RAM
-c = CPU count
-d = disk space
-k = name of the task to be killed

Example:
To create and launch: auroraCreate -o create -r 1024 -n batik -c 2.0 -d 1000
To kill: auroraCreate -o kill -k batik

I have tested the code on my local setup and will be improving the code and 
also be adding more functionality.

Any feedback will be very helpful.

--
Regards,
Gourav Rattihalli

Re: Git Develop branch for Apache Aurora Integration

[Pierce, Marlon]

Hi Gourav, please share the script on the list and tell us more about what you 
are doing.

From: Gourav Rattihalli 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 3, 2016 at 5:17 PM
To: "dev@airavata.apache.org" 
Subject: Git Develop branch for Apache Aurora Integration

Hi Dev's,
I'm developing a aurora submit script and I would like to know where I should 
place them in the develop branch.

--
Regards,
Gourav Rattihalli

Re: "Something went wrong message" during account confirmation on locally installed PGA

[Pierce, Marlon]

Thanks, Abhishek. Do we need these instructions in 
http://airavata.readthedocs.io/en/latest/WSO2-IS-Configuration/ ?

From: Abhishek Jain 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 3, 2016 at 4:51 PM
To: dev 
Subject: Re: "Something went wrong message" during account confirmation on 
locally installed PGA

Hi Marlon,
So Supun pointed out that I had forgotten to create the OAuth client key and 
client secret for the tenant. I then created the OAuth client key and secret.
This enabled me to log-in to the PGA gateway with the tenant credentials which 
was failing initially.
I also tried changing the confirmation link as mentioned by Supun, which is 
still giving me the same message.
So the issue of authentication failing with tenant credentials got fixed.
Now, I will review the PGA app/controller code to find out what the actual 
error is, which is being generated from the PGA side.

On Fri, Jun 3, 2016 at 4:02 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Did you find the problem?

From: Abhishek Jain mailto:ajai...@binghamton.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Friday, June 3, 2016 at 11:28 AM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>

Subject: Re: "Something went wrong message" during account confirmation on 
locally installed PGA

Sure will surely do that.

On Friday, June 3, 2016, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Please be sure to post the problem and solution.

Marlon

From: Supun Nakandala 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 3, 2016 at 11:23 AM
To: dev 
Subject: Re: "Something went wrong message" during account confirmation on 
locally installed PGA

Yes that works for me.

On Fri, Jun 3, 2016 at 11:21 AM, Abhishek Jain  wrote:
Sure Supun, that would be great !
How about we have a hangout after exactly 2 hours exact timing being 1:20 pm 
(US time). Does that sound good to you ?


On Friday, June 3, 2016, Supun Nakandala  wrote:
Hi Abhishek,

If you still have this issue we can a hangout and try to fix this issue today. 
Let me know the time that you are free.

On Thu, Jun 2, 2016 at 1:42 PM, Supun Nakandala  
wrote:
I would suggest that you debug and find out the actual error message is thrown 
from PGA side. Failed Sending Account Confirmation Email after Account Creation 
with local PGA - WSO2 IS set up is a user friendly message shown for users. If 
I can know actual error message I can help you.


On Thu, Jun 2, 2016 at 1:40 PM, Abhishek Jain  wrote:
Hi Supun,
So is there a problem in the account confirmation link which is creating the 
issue ? As fixing the link the way you mentioned gave me the same message.
Should I share my email-admin-config.xml and pga_config.php file with you ?

On Thu, Jun 2, 2016 at 11:33 AM, Abhishek Jain  wrote:
I changed it. I am getting the same error message.

On Thu, Jun 2, 2016 at 11:03 AM, Supun Nakandala  
wrote:
Please change the account confirmation email template in your wso2 IS to 
contain this email link
http://localhost/airavata-php-gateway/public/<http://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation=87348d48-c6c7-44a3-87d2-39ed3bb7e439&userstoredomain=PRIMARY&username=jainabhishek99&tenantdomain=test.bu.com>confirm-user-registration?confirmation={confirmation-code}&username={user-name}

On Thu, Jun 2, 2016 at 10:56 AM, Abhishek Jain  wrote:
Hi Supun,
I do not see any error log in the wso2 IS. The last update in the log is as 
follows :

[2016-06-02 10:51:59,306] @test.bu.com<http://test.bu.com> [1] [IS] INFO 
{org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule} -  Sending user 
credentials configuration mail to jainabhishe...@gmail.com
[2016-06-02 10:52:01,322] @test.bu.com<http://test.bu.com> [1] [IS] INFO 
{org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule} -  User 
credentials configuration mail has been sent to jainabhishe...@gmail.com

I did get a confirmation link. I get the above mentioned "something went wrong" 
message when I visit the link and there are no updates in the wso2 IS log.

On Wed, Jun 1, 2016 at 9:40 PM, Supun Nakandala  
wrote:
Can you send the error log that you get wso2 IS.

On Wed, Jun 1, 2016 at 5:27 PM, Abhishek Jain  wrote:
Hi Devs,
I have locally installed PGA and WSO2 IS server. Everything is up and running. 
Once I create an account, I do get a account confirmation link back from the 
locally installed WSO2 server.

The link looks like :

http://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation=87348d48-c6c7-44a3-87d2-39ed3bb7e439&userstoredomain=PRIMARY&username=jainabhishek99&tenantdomain=test.bu.com

But now, when I visit the link, I am getting the following message :

"Looks like something 

Re: "Something went wrong message" during account confirmation on locally installed PGA

[Pierce, Marlon]

Did you find the problem?

From: Abhishek Jain 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 3, 2016 at 11:28 AM
To: "dev@airavata.apache.org" 
Subject: Re: "Something went wrong message" during account confirmation on 
locally installed PGA

Sure will surely do that.

On Friday, June 3, 2016, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Please be sure to post the problem and solution.

Marlon

From: Supun Nakandala 
>
Reply-To: 
"dev@airavata.apache.org"
 
>
Date: Friday, June 3, 2016 at 11:23 AM
To: dev 
>
Subject: Re: "Something went wrong message" during account confirmation on 
locally installed PGA

Yes that works for me.

On Fri, Jun 3, 2016 at 11:21 AM, Abhishek Jain 
>
 wrote:
Sure Supun, that would be great !
How about we have a hangout after exactly 2 hours exact timing being 1:20 pm 
(US time). Does that sound good to you ?


On Friday, June 3, 2016, Supun Nakandala 
>
 wrote:
Hi Abhishek,

If you still have this issue we can a hangout and try to fix this issue today. 
Let me know the time that you are free.

On Thu, Jun 2, 2016 at 1:42 PM, Supun Nakandala 
>
 wrote:
I would suggest that you debug and find out the actual error message is thrown 
from PGA side. Failed Sending Account Confirmation Email after Account Creation 
with local PGA - WSO2 IS set up is a user friendly message shown for users. If 
I can know actual error message I can help you.


On Thu, Jun 2, 2016 at 1:40 PM, Abhishek Jain 
>
 wrote:
Hi Supun,
So is there a problem in the account confirmation link which is creating the 
issue ? As fixing the link the way you mentioned gave me the same message.
Should I share my email-admin-config.xml and pga_config.php file with you ?

On Thu, Jun 2, 2016 at 11:33 AM, Abhishek Jain 
>
 wrote:
I changed it. I am getting the same error message.

On Thu, Jun 2, 2016 at 11:03 AM, Supun Nakandala 
>
 wrote:
Please change the account confirmation email template in your wso2 IS to 
contain this email link
http://localhost/airavata-php-gateway/public/<http://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation=87348d48-c6c7-44a3-87d2-39ed3bb7e439&userstoredomain=PRIMARY&username=jainabhishek99&tenantdomain=test.bu.com>confirm-user-registration?confirmation={confirmation-code}&username={user-name}

On Thu, Jun 2, 2016 at 10:56 AM, Abhishek Jain 
>
 wrote:
Hi Supun,
I do not see any error log in the wso2 IS. The last update in the log is as 
follows :

[2016-06-02 10:51:59,306] @test.bu.com<http://test.bu.com> [1] [IS] INFO 
{org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule} -  Sending user 
credentials configuration mail to 
jainabhishe...@gmail.com
[2016-06-02 10:52:01,322] @test.bu.com<http://test.bu.com> [1] [IS] INFO 
{org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule} -  User 
credentials configuration mail has been sent to 
jainabhishe...@gmail.com

I did get a confirmation link. I get the above mentioned "something went wrong" 
message when I visit the link and there are no updates in the wso2 IS log.

On Wed, Jun 1, 2016 at 9:40 PM, Supun Nakandala 
>
 wrote:
Can you send the error log that you get wso2 IS.

On Wed, Jun 1, 2016 at 5:27 PM, Abhishek Jain 
>
 wrote:
Hi Devs,
I have locally installed PGA and WSO2 IS server. Everything is up and running. 
Once I create an account, I do get a account confirmation link back from the 
locally installed WSO2 server.

The link looks like :

http://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation=87348d48-c6c7-44a3-87d2-39ed3bb7e439&userstoredomain=PRIMARY&username=jainabhishek99&tenantdomain=test.bu.com

But now, when I visit the link, I am getting the following message :

"Looks like something went wrong. Please go back and try again or report the 
problem using help feature."

This problem is mentioned in the previous thread with subject :

Failed Sending Account Confirmation Email after Account Creation with local PGA 
- WSO2 IS set up

I wanted to start a new thread with this problem.

I have attached the screenshot below.
--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University - State University of New York (SUNY)





--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University - State University of New York (SUNY)





--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University -

Re: "Something went wrong message" during account confirmation on locally installed PGA

[Pierce, Marlon]

Please be sure to post the problem and solution.

Marlon

From: Supun Nakandala 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 3, 2016 at 11:23 AM
To: dev 
Subject: Re: "Something went wrong message" during account confirmation on 
locally installed PGA

Yes that works for me.

On Fri, Jun 3, 2016 at 11:21 AM, Abhishek Jain 
mailto:ajai...@binghamton.edu>> wrote:
Sure Supun, that would be great !
How about we have a hangout after exactly 2 hours exact timing being 1:20 pm 
(US time). Does that sound good to you ?


On Friday, June 3, 2016, Supun Nakandala 
mailto:supun.nakand...@gmail.com>> wrote:
Hi Abhishek,

If you still have this issue we can a hangout and try to fix this issue today. 
Let me know the time that you are free.

On Thu, Jun 2, 2016 at 1:42 PM, Supun Nakandala  
wrote:
I would suggest that you debug and find out the actual error message is thrown 
from PGA side. Failed Sending Account Confirmation Email after Account Creation 
with local PGA - WSO2 IS set up is a user friendly message shown for users. If 
I can know actual error message I can help you.


On Thu, Jun 2, 2016 at 1:40 PM, Abhishek Jain  wrote:
Hi Supun,
So is there a problem in the account confirmation link which is creating the 
issue ? As fixing the link the way you mentioned gave me the same message.
Should I share my email-admin-config.xml and pga_config.php file with you ?

On Thu, Jun 2, 2016 at 11:33 AM, Abhishek Jain  wrote:
I changed it. I am getting the same error message.

On Thu, Jun 2, 2016 at 11:03 AM, Supun Nakandala  
wrote:
Please change the account confirmation email template in your wso2 IS to 
contain this email link
http://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation={confirmation-code}&username={user-name}

On Thu, Jun 2, 2016 at 10:56 AM, Abhishek Jain  wrote:
Hi Supun,
I do not see any error log in the wso2 IS. The last update in the log is as 
follows :

[2016-06-02 10:51:59,306] @test.bu.com [1] [IS] INFO 
{org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule} -  Sending user 
credentials configuration mail to jainabhishe...@gmail.com
[2016-06-02 10:52:01,322] @test.bu.com [1] [IS] INFO 
{org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule} -  User 
credentials configuration mail has been sent to jainabhishe...@gmail.com

I did get a confirmation link. I get the above mentioned "something went wrong" 
message when I visit the link and there are no updates in the wso2 IS log.

On Wed, Jun 1, 2016 at 9:40 PM, Supun Nakandala  
wrote:
Can you send the error log that you get wso2 IS.

On Wed, Jun 1, 2016 at 5:27 PM, Abhishek Jain  wrote:
Hi Devs,
I have locally installed PGA and WSO2 IS server. Everything is up and running. 
Once I create an account, I do get a account confirmation link back from the 
locally installed WSO2 server.

The link looks like :

http://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation=87348d48-c6c7-44a3-87d2-39ed3bb7e439&userstoredomain=PRIMARY&username=jainabhishek99&tenantdomain=test.bu.com

But now, when I visit the link, I am getting the following message :

"Looks like something went wrong. Please go back and try again or report the 
problem using help feature."

This problem is mentioned in the previous thread with subject :

Failed Sending Account Confirmation Email after Account Creation with local PGA 
- WSO2 IS set up

I wanted to start a new thread with this problem.

I have attached the screenshot below.
--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University - State University of New York (SUNY)





--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University - State University of New York (SUNY)





--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University - State University of New York (SUNY)





--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Computer Science
Thomas J. Watson School of Engineering & Applied Science
Binghamton University - State University of New York (SUNY)





--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa


--
Thanks & Regards,
Abhishek Jain

Graduate Student
Department of Compute

Re: Airavata 0.16 Release Planning

[Pierce, Marlon]

+ 1 from me also

From: "Pamidighantam, Sudhakar V" 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, June 3, 2016 at 9:10 AM
To: "dev@airavata.apache.org" 
Subject: Re: Airavata 0.16 Release Planning

None. +1 for the release.

Thanks,
Sudhakar.
On Jun 3, 2016, at 6:55 AM, Suresh Marru 
mailto:sma...@apache.org>> wrote:


I gated the release earlier but this task was long done. Any objections to move 
forward with 0.16 release?

Suresh

On Mar 30, 2016, at 2:53 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:

To contradict my proposal, I would like to work on 
https://issues.apache.org/jira/browse/AIRAVATA-1945<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_AIRAVATA-2D1945&d=CwMFAg&c=8hUWFZcy2Z-Za5rBPlktOQ&r=7_-LbDwTKOoIiO4P4OLfUTX6lSdjys9jh2AJ7sBl9ag&m=R2ZVpIpwPyZsEiWW5Vb2qO6Fpy2bxRx2eeUVc32o0ws&s=HXTGQcfXi-0_JFcA-kExFEiEWO-XBIRLvYpHzs7ZFnQ&e=>
 before requesting feature freeze. Should not take long.

Suresh

On Mar 28, 2016, at 11:29 AM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:

Do we have any outstanding tasks that need to be wrapped up and committed to 
dev?

From: Shameera Rathnayaka 
mailto:shameerai...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Monday, March 28, 2016 at 11:20 AM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Airavata 0.16 Release Planning

+1

On Mon, Mar 28, 2016 at 10:50 AM Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi All,

Before we go too far, how about we call a feature freeze and stat working on 
0.16 release? Unless any one is in the middle of a development activity, how 
about we target end of the week to start working on it?

Suresh
--
Shameera Rathnayaka

Re: Failed Sending Account Confirmation Email after Account Creation with local PGA - WSO2 IS set up

[Pierce, Marlon]

+1, thanks, Jeff!

From: Abhishek Jain 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, May 27, 2016 at 8:35 PM
To: dev 
Subject: Re: Failed Sending Account Confirmation Email after Account Creation 
with local PGA - WSO2 IS set up

Yes. I think I verified it with my set-up and then added them which turned out 
to be similar. I will double check it.
Thanks a lot for the help !

On Fri, May 27, 2016 at 8:31 PM, Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>> wrote:
So those addresses need to be directed to your public folder in your PGA 
install--the actual address will depend on how you have set up Apache server, I 
just sent them as an example.

On Fri, May 27, 2016 at 7:32 PM, Abhishek Jain 
mailto:ajai...@binghamton.edu>> wrote:
Yes. Google was blocking the sign-in attempt. I allowed less secure apps to 
sign-in. I finally could receive a confirmation link which is as follows :

https://localhost/airavata-php-gateway/public/confirm-user-registration?confirmation=a507f404-858f-44f2-81d4-c8b2a9a7be3f&userstoredomain=PRIMARY&username=jainabhishek96&tenantdomain=test.bu.com
But, when I visit the link , I get an error stating that the site can't be 
reached. I believe the link is not correct.

On Fri, May 27, 2016 at 7:12 PM, Jeff mailto:jkinn...@nd.edu>> 
wrote:
Has abhi.airavat...@gmail.com received any 
emails from Google about a blocked sign in attempt? If so, try following the 
instructions at https://support.google.com/accounts/answer/6010255?hl=en


On 05/27/2016 07:01 PM, Abhishek Jain wrote:
This is how my identiy-mgt.properties file looks like :

# Enable authentication to the service using super tenant credentials and 
invoke operations on tenants.

UserInfoRecovery.SaaSEnable=false

# Enable email sending function when recovering the account and verifying the 
user creation

Notification.Sending.Enable=true
Notification.Expire.Time=7200

# Enable internal email sending module. If "false", email sending data would be 
available to application via webservice. Then application can send the email 
using its own email sender

Notification.Sending.Internally.Managed=true

# Enable verification of account creation. When self registration is done,  
user would be verified by sending email (confirmation link) to user's email 
account

UserAccount.Verification.Enable=false

UserAccount.Recovery.Enable=false


# Whether user can do the verification or not. If value is defined for role, 
Then the email is sent to users that have been assigned to this role. Not for 
the user who is registered.

UserAccount.Verification.Role=

# Whether captcha verification is done by identity server or application level

Captcha.Verification.Internally.Managed=false

# Whether temporary passwords are used when creating users or recovering 
accounts.

Temporary.Password.Enable=false
Temporary.Password.OneTime=false
Temporary.Password.Expire.Time=0

# Enable authentication rules other than password check

Authentication.Policy.Enable=true

# Enable following checks during authentication.  By default account lock check 
is enabled.

Authentication.Policy.Check.Account.Exist=false
Authentication.Policy.Check.Account.Lock=false
Authentication.Policy.Check.OneTime.Password=false
Authentication.Policy.Check.Password.Expire=false

# Configuration to build an authentication policy

Authentication.Policy.Password.Expire.Time=0
# If account verification is not enabled, following property will decide where 
user must be lock or not after user is created
Authentication.Policy.Account.Lock.On.Creation=true
Authentication.Policy.Account.Lock.Time=0
Authentication.Policy.Account.Lock.On.Failure=false
Authentication.Policy.Account.Lock.On.Failure.Max.Attempts=0

# Define pluggable extension points

Identity.Mgt.Random.Password.Grenerator=org.wso2.carbon.identity.mgt.password.DefaultPasswordGenerator
#Identity.Mgt.User.Data.Store=org.wso2.carbon.identity.mgt.store.UserStoreBasedIdentityDataStore
Identity.Mgt.User.Data.Store=org.wso2.carbon.identity.mgt.store.JDBCIdentityDataStore
Identity.Mgt.Notification.Sending.Module.1=org.wso2.carbon.identity.mgt.mail.DefaultEmailSendingModule
Identity.Mgt.User.Recovery.Data.Store=org.wso2.carbon.identity.mgt.store.RegistryRecoveryDataStore
Identity.Mgt.Registry.CleanUpPeriod=1440


# Define password policy enforce extensions

#Password.policy.extensions.1=org.wso2.carbon.identity.mgt.policy.password.DefaultPasswordLengthPolicy
#Password.policy.extensions.1.min.length=6
#Password.policy.extensions.1.max.length=12
#Password.policy.extensions.2=org.wso2.carbon.identity.mgt.policy.password.DefaultPasswordNamePolicy
#Password.policy.extensions.3=org.wso2.carbon.identity.mgt.policy.password.DefaultPasswordPatternPolicy
#Password.policy.extensions.3.pattern=^((?=.*\\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%&*])).{0,100}$
#Password.policy.extensions.3.errorMsg='Password pattern policy violated. 
Password should contain a digit[0-9], a lower case letter[a-z], an upper case 
le

Re: Trying to Set Up Molecular Dynamics Workflow

[Pierce, Marlon]

b170243d0bf, taskId: 
>TASK_f5028495-bc51-4400-90d3-f56ca06acbde, type: DATA_STAGING:- Task 
>status changed CREATED -> EXECUTING
>2016-05-23 11:15:01,095 [pool-8-thread-6] INFO 
>org.apache.airavata.orchestrator.server.OrchestratorServerHandler  - 
>expId: juhygf_ff4b4196-aaa7-4e05-9dfd-74978e9cfa19, processId: 
>PROCESS_71118aff-fa37-449f-ae7f-fb170243d0bf :- Process status changed 
>event received for status INPUT_DATA_STAGING
>
>
>On 05/23/2016 11:55 AM, Pierce, Marlon wrote:
>> Hi Jeff,
>>
>> What is the script that you are trying to run (see below)?  Can you add some 
>> debugging messages there?
>>
>> Marlon
>>
>>
>> On 5/23/16, 11:47 AM, "Jeff"  wrote:
>>   With my
>>> current setup, I can run jobs that do not require external scripts
>>> (e.g., echo ), but when I try to run any kind of script the
>>> experiment never completes.
>>>
>>
>

Re: Trying to Set Up Molecular Dynamics Workflow

[Pierce, Marlon]

Hi Jeff, 

What is the script that you are trying to run (see below)?  Can you add some 
debugging messages there?

Marlon


On 5/23/16, 11:47 AM, "Jeff"  wrote:
 With my 
>current setup, I can run jobs that do not require external scripts 
>(e.g., echo ), but when I try to run any kind of script the 
>experiment never completes.
>

Re: java.lang.RuntimeException: Failed to create database connection pool

[Pierce, Marlon]

Pratik, can you confirm that the mysql.jar is correct, uncorrupted, etc?  Use 
jar –tf or a similar command to list the contents.

Marlon

From: Eroma Abeysinghe 
Reply-To: "dev@airavata.apache.org" 
Date: Thursday, May 19, 2016 at 4:51 PM
To: "dev@airavata.apache.org" 
Subject: Re: java.lang.RuntimeException: Failed to create database connection 
pool

Hi Pratik,

What is the version you have copied? I copied version 
mysql-connector-java-5.1.38-bin.jar and it worked fine.
Could you try this version?

Thanks,
Eroma

On Thu, May 19, 2016 at 4:34 PM, Pratik Patel 
mailto:patel...@umail.iu.edu>> wrote:
Hi Suresh,

Thanks for prompt response.
I have already added mysql.jar in 'lib' folder.
[nline image 1]

I have downloaded the jar file from 
'http://dev.mysql.com/downloads/connector/j/'. This link was provided in 
instructions. Does it automatically look for this jar file or I have to provide 
the complete path of the jar with file name in airavata-server.properties file?

Thanks,
Pratik Patel

On Thu, May 19, 2016 at 4:29 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi Pratik,

As you can see from “ Can't find class for driver:” it is looking for the mysql 
jar which should be Step 8 in instructions - 
http://airavata.readthedocs.io/en/latest/Airavata-Installation/

Suresh

On May 19, 2016, at 4:26 PM, Pratik Patel 
mailto:patel...@umail.iu.edu>> wrote:

Hi all,

I am trying to start airavata-server. But I am getting following error while 
doing so:

Command Used:
$ ./airavata-server.sh start

Error:


2016-05-19 16:17:08,225 [main] ERROR org.apache.airavata.server.ServerMain  - 
Server Start Error:
java.lang.RuntimeException: Failed to create database connection pool.
at org.apache.airavata.api.server.util.JdbcStorage.(JdbcStorage.java:47)
at 
org.apache.airavata.api.server.util.CredentialStoreInitUtil.initializeDB(CredentialStoreInitUtil.java:64)
at 
org.apache.airavata.api.server.AiravataAPIServer.startAiravataServer(AiravataAPIServer.java:74)
at 
org.apache.airavata.api.server.AiravataAPIServer.start(AiravataAPIServer.java:202)
at org.apache.airavata.server.ServerMain.startAllServers(ServerMain.java:292)
at org.apache.airavata.server.ServerMain.performServerStart(ServerMain.java:141)
at org.apache.airavata.server.ServerMain.main(ServerMain.java:124)
Caused by: java.sql.SQLException: Can't find class for driver:
at 
org.apache.airavata.api.server.util.ConnectionPool.makeNewConnection(ConnectionPool.java:200)
at 
org.apache.airavata.api.server.util.ConnectionPool.initialize(ConnectionPool.java:127)
at 
org.apache.airavata.api.server.util.ConnectionPool.(ConnectionPool.java:90)
at org.apache.airavata.api.server.util.JdbcStorage.(JdbcStorage.java:41)
... 6 more


--



I have attached log file in this email. Could someone point out where I am 
going wrong?







--
Thank You,
Best Regards,
Eroma

Re: Problem in running airavata server in demon mode

[Pierce, Marlon]

See what your classpath is. Note we don’t normally test and run on Windows, so 
you may want to get a linux environment, virtual or otherwise.

Marlon


From: Ritesh Agarwal 
Reply-To: "dev@airavata.apache.org" 
Date: Tuesday, May 17, 2016 at 5:03 PM
To: "dev@airavata.apache.org" 
Subject: Re: Problem in running airavata server in demon mode

Since it is looking for the jar file inside the lib directory of 
'apache-airavata-server-0.16-SNAPSHOT', I don't think it has the jar file it 
needs.

On Tue, May 17, 2016 at 4:52 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi Ritesh,

As you can see from the error message, it complains of not finding the main 
class. Do you see all the jars in the lib directory where the script is looking 
for them?

Suresh

> On May 17, 2016, at 4:42 PM, Ritesh Agarwal 
> mailto:ritea...@umail.iu.edu>> wrote:
>
> Hi all,
>
> I am not able to start the airavata server in the background in demon mode 
> (windows) which is step 13 in Airavata installation. I have attached the 
> error below.
>
> Regards,
> Ritesh Agarwal.
> 

Re: Unable to start Airavata server

[Pierce, Marlon]

Are you following the steps in http://airavata.readthedocs.io/en/latest/?

From: Gowtham A R 
Reply-To: "dev@airavata.apache.org" 
Date: Friday, May 13, 2016 at 10:13 PM
To: "dev@airavata.apache.org" 
Subject: Unable to start Airavata server

Hi All,

I am getting the below error while trying to start Apache Airavata server.

2016-05-13 21:22:24,128 [main] ERROR 
org.apache.airavata.api.server.util.ExperimentCatalogInitUtil  - Unable to read 
airavata server properties
2016-05-13 21:22:24,129 [main] ERROR 
org.apache.airavata.api.server.util.ExperimentCatalogInitUtil  -
java.lang.NullPointerException
at 
org.apache.airavata.api.server.util.ExperimentCatalogInitUtil.getDBType(ExperimentCatalogInitUtil.java:143)
at 
org.apache.airavata.api.server.util.ExperimentCatalogInitUtil.initializeDB(ExperimentCatalogInitUtil.java:72)
at 
org.apache.airavata.api.server.AiravataAPIServer.startAiravataServer(AiravataAPIServer.java:68)
at 
org.apache.airavata.api.server.AiravataAPIServer.start(AiravataAPIServer.java:202)
at 
org.apache.airavata.server.ServerMain.startAllServers(ServerMain.java:292)
at 
org.apache.airavata.server.ServerMain.performServerStart(ServerMain.java:141)
at org.apache.airavata.server.ServerMain.main(ServerMain.java:124)

Can anyone let me know how to fix this issue.

Thanks in advance.

Regards,
Gowtham A R.

Re: Broken link for Build Instruction

[Pierce, Marlon]

Hi Pratek,

Thanks for finding that broken link. The full instructions can be found at 
http://airavata.readthedocs.io/en/latest/, which is for a full production 
deployment. For simpler testing of just Airavata with embedded DB, see for 
example 
https://svn.apache.org/repos/asf/airavata/site/trunk/content/development/build.mdtext.

Marlon


From: Pratik Patel mailto:patel...@umail.iu.edu>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Thursday, May 12, 2016 at 2:36 PM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: Broken link for Build Instruction

Hi all,

I was looking for building instruction for airavata on airavata's website, I 
found Building instruction link on 
'https://airavata.apache.org/development.html'
[Inline image 1]

But the link provided there is broken and responds with status '404'
[Inline image 2]

Thanks,
Pratik Patel

Connection timeout settings

[Pierce, Marlon]

We have an occasional issue of connection timeouts when performing remote SSH 
operations. This has a potentially bad side effect of successfully launching a 
large job but not getting back the Job ID. One straightforward fix is to use 
longer than the default connection timeout in the Jsch clients.

Looking through the code, I don’t see that we are doing this. Is this correct?  
 Would there be some unintended consequences for using something longer, like 
60 seconds? The default is 20 seconds.

There is also a longer discussion about the right way to handle these events in 
the first place. We may not want to depend on the standard output at all. 
Increasing the timeouts would at least put a bandaid on the current issue.

Marlon

Re: Planning for In-Situ Application and Resource Monitoring [GSoC Project]

[Pierce, Marlon]

+1 for publishing to RabbitMQ. Don’t worry about XBaya as it is obsolete; 
upgrading it is another GFAC project. I suggest you focus on the SimStream to 
RabbitMQ parts first. The API changes will need additional discussion, probably 
over a hangout.

Marlon


From: Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, May 4, 2016 at 6:41 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Planning for In-Situ Application and Resource Monitoring [GSoC 
Project]

The more I look into it, the more I like using RabbitMQ within the SimStream 
program to communicate with Airavata server. This is what I have so far for 
practical steps to take in implementing the project:

SimStream:

  *   Refactor to use RabbitMQ queues instead of Tornado Web Server.
  *   Define a config file to send with each job that contains information 
about how to contact Airavata server, which scripts to run to collect the 
simulation and resource data, any arguments to pass to the scripts. This will 
decouple data collection logic from SimStream and hopefully eliminate the need 
for long-running data collection scripts (i.e., one data point is collected per 
run of the collection script).

Within the Airavata API Server:

  *   Extend the existing org.apache.airavata.model.job.JobModel to include 
information about contacting SimStream (queue name, valid data stream names,
  *   Add classes RabbitMQJobDataPublisher and RabbitMQJobDataConsumer 
(analogous to 
org.apache.airavata.messaging.core.impl.RabbitMQProcessLaunchPublisher and 
org.apache.airavata.messaging.core.impl.RabbitMQProcessLaunchConsumer)
  *   Extend the API Server to listen for requests for job data (requires 
identification of which job, which data from the job, in addition to 
verification that the requester should be allowed to perform this operation)
  *   Extend the API Server to send requested job data back to the gateway and 
user that issued the request.

Within Airavata's XBaya :

  *   Create default services for the data collection and event 
monitoring/handling aspects of the project that can be added into the workflow 
composer.
  *   Create a service that accepts custom data collection scripts to send 
along with the job.

Within the PGA:

  *   Add blades and controllers that allow users to view requested data from a 
job.
  *   Extend the experiment-related app functionality to allow users to 
retrieve data from a running job through the gateway.

I saw that there are some existing but empty or commented-out classes 
(org.apache.airavata.model.util.ComputeResourceUtil, 
org.apache.airavata.monitoring.Main) that suggest there was work toward similar 
functionality as I am suggesting. Searching JIRA for these didn't turn up any 
information, and I'm curious about why these plans were abandoned, or if they 
were even related to my project.

I'd appreciate any comments on the above!

Best,

Jeff K.

On Wed, Apr 27, 2016 at 4:43 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
RabbitMQ has first class support for Python, so that should not be a problem.  
Suresh already included the link.  Suresh covered most of the bases already, so 
I’ll just reiterate that Airavata’s use of AMQP/RabbitMQ and Thrift should make 
it programming language independent. You can see how well this holds up in 
reality.

Marlon

From: Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, April 27, 2016 at 4:35 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Planning for In-Situ Application and Resource Monitoring [GSoC 
Project]

Thanks Suresh,

I was hoping that I could stick with Python for the meat of the project, not 
just because it's the language I'm most comfortable with, but also thanks to 
the fact that it's fairly ubiquitous on HPC systems.

I'll take a look at either interfacing the POC with RabbitMQ or converting it 
entirely to their Python bindings. If anyone has any alternative suggestions, 
they would be much appeciated.

Jeff K.

On Wed, Apr 27, 2016 at 4:20 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi Jeff,

On Apr 27, 2016, at 4:08 PM, Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>> wrote:

Hi Dev Team,

I'd like to develop a plan for implementing my GSoC project in conjunction to 
getting my development environment up and running. This is my first substantial 
experience with Open Source software development on this scale, so thank you in 
advance for bearing with me.

You did great during proposal (hence you have a p

Re: New version of Xbaya

[Pierce, Marlon]

You may want to look at https://github.com/SciGaP/seagrid-rich-client. This is 
a specific end application and outside Airavata, but it uses the API.




On 4/28/16, 1:32 PM, "Bikramjeet Singh"  wrote:

>Hi all,
>I will be working on developing XBaya which will be portable with latest 
>airavata server as part of my GSoc project. Since, the development of UI will 
>be from start any suggestions regarding a new layout or any changes in the 
>design of user interface will be very helpful. 
>
>Thanks

Re: Planning for In-Situ Application and Resource Monitoring [GSoC Project]

[Pierce, Marlon]

RabbitMQ has first class support for Python, so that should not be a problem.  
Suresh already included the link.  Suresh covered most of the bases already, so 
I’ll just reiterate that Airavata’s use of AMQP/RabbitMQ and Thrift should make 
it programming language independent. You can see how well this holds up in 
reality.

Marlon

From: Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, April 27, 2016 at 4:35 PM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: Re: Planning for In-Situ Application and Resource Monitoring [GSoC 
Project]

Thanks Suresh,

I was hoping that I could stick with Python for the meat of the project, not 
just because it's the language I'm most comfortable with, but also thanks to 
the fact that it's fairly ubiquitous on HPC systems.

I'll take a look at either interfacing the POC with RabbitMQ or converting it 
entirely to their Python bindings. If anyone has any alternative suggestions, 
they would be much appeciated.

Jeff K.

On Wed, Apr 27, 2016 at 4:20 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi Jeff,

On Apr 27, 2016, at 4:08 PM, Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>> wrote:

Hi Dev Team,

I'd like to develop a plan for implementing my GSoC project in conjunction to 
getting my development environment up and running. This is my first substantial 
experience with Open Source software development on this scale, so thank you in 
advance for bearing with me.

You did great during proposal (hence you have a project), just continue the 
same. At worse you will hear a lot of RTFM which is a common encounter in open 
source. I will let you google for it.

The full project proposal can be found at 
https://cwiki.apache.org/confluence/display/AIRAVATA/GSoC+Proposal+-+In+Situ+Simulation+Analysis+Using+Airavata

The idea is to allow Airavata users to look behind the curtain at jobs they are 
running and enable automatic response to conditions encountered as jobs run, 
both at the system and application level. This will likely require a 
lightweight server to run alongside each job, which will communicate with the 
Airavata server.

I have a prototype for the lightweight server 
(https://github.com/jeffkinnison/simstream) written in Python, however I know 
that Apache software is typically Java-based. The question here is one of 
whether or not the prototype can be rolled into Airavata, or if I need to begin 
looking into Java-based solutions.

No, you do not need to port your simstream to Java, infact. Since your 
application demeon will need to run on HPC compute nodes, Java will not be a 
good fit there. I think you should stick to python. For the communication with 
Airavata, one suggestion will be to send a AMQP message which Airavata listens 
to. You can follow this tutorial as a start - 
https://www.rabbitmq.com/tutorials/tutorial-one-python.html. Others may have 
different suggestions.

The other initial question I have is one of how the Airavata server submits 
jobs. From what I can tell, Airavata sends batch scripts to connected computing 
resources, and my thinking right now about how to deploy the lightweight server 
is to add its startup logic to the submit scripts. Is this the correct thinking?

Yes thats exactly right. As you might see from other discussions, the cloud 
based submissions might not have a batch script, but its fair to assume your 
server will be launched one way or another.


Again, thank you for answering these questions, and I'm looking forward to 
working with everyone this summer.

Keep them coming.

Suresh


Best,
Jeff K.

Re: Design of Mesos/Aurora integration with Airavata on Jetstream

[Pierce, Marlon]

Hi Emre, are you looking at containerizing any of your applications? 



On 4/27/16, 1:21 PM, "Emre Brookes"  wrote:

>Pierce, Marlon wrote:
>> Thanks, Gourav, this is great. How will the design differ in the two 
>> cases?
>>
>> Since installing the applications is not a simple task and there 
>> aren’t good docker libraries (for example) for many of them, I would 
>> assume that we would have a collection of VMs that have the codes 
>> already installed.
>FWIW, that model would work well with GenApp usage also.  Although, I 
>can imagine a scenario where a part of job initialization script 
>(running under the already "code installed" VM) checks for updates to 
>the code and installs them, but that can be independent of 
>Airavata/Aurora/Mesos's knowledge.
>
>-Emre
>
>>
>> Marlon
>>
>>
>> From: Gourav Rattihalli > <mailto:gratt...@binghamton.edu>>
>> Reply-To: "dev@airavata.apache.org <mailto:dev@airavata.apache.org>" 
>> mailto:dev@airavata.apache.org>>
>> Date: Wednesday, April 27, 2016 at 1:01 PM
>> To: "dev@airavata.apache.org <mailto:dev@airavata.apache.org>" 
>> mailto:dev@airavata.apache.org>>
>> Subject: Design of Mesos/Aurora integration with Airavata on Jetstream
>>
>> Hi Dev's,
>>
>> I have been working on the integration of Apache Aurora and Mesos with 
>> Airavata and Jetstream. Using Mangirish's latest code, VMs can now be 
>> created using on Jetstream. I want to understand how Airavata is 
>> likely to use the VMs for applications. Will airavata re-use existing 
>> VMs for successive applications for a given community, or will new VMs 
>> be created for each application? This will decide how we design the 
>> automated creation of a Mesos cluster using the new VMs that are 
>> created. I understand that we will have a Mesos master VM that will 
>> run all the time on Jetstream.
>>
>> Here is the Job submission flow that I am assuming:
>>
>> Airavata Client--> Airavata 
>> Server-->(Orchestrator-->GFAC)-->My-Apache-Aurora-module-->(A script 
>> will create .aurora configuration file that will be used to launch the 
>> job)-->Aurora/Mesos-->VMs
>>
>> So, I will design a module that will work with Orchestrator-->GFAC to 
>> generate the appropriate Apache Aurora script to be submitted to the 
>> Aurora master.
>>
>> Please let me know if there are any comments, suggestions on this plan.
>>
>> -- 
>> Regards,
>> Gourav Rattihalli
>

Re: Design of Mesos/Aurora integration with Airavata on Jetstream

[Pierce, Marlon]

Thanks, Gourav, this is great. How will the design differ in the two cases?

Since installing the applications is not a simple task and there aren’t good 
docker libraries (for example) for many of them, I would assume that we would 
have a collection of VMs that have the codes already installed.

Marlon


From: Gourav Rattihalli 
mailto:gratt...@binghamton.edu>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, April 27, 2016 at 1:01 PM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: Design of Mesos/Aurora integration with Airavata on Jetstream

Hi Dev's,

I have been working on the integration of Apache Aurora and Mesos with Airavata 
and Jetstream. Using Mangirish's latest code, VMs can now be created using on 
Jetstream. I want to understand how Airavata is likely to use the VMs for 
applications. Will airavata re-use existing VMs for successive applications for 
a given community, or will new VMs be created for each application? This will 
decide how we design the automated creation of a Mesos cluster using the new 
VMs that are created. I understand that we will have a Mesos master VM that 
will run all the time on Jetstream.

Here is the Job submission flow that I am assuming:

Airavata Client--> Airavata 
Server-->(Orchestrator-->GFAC)-->My-Apache-Aurora-module-->(A script will 
create .aurora configuration file that will be used to launch the 
job)-->Aurora/Mesos-->VMs

So, I will design a module that will work with Orchestrator-->GFAC to generate 
the appropriate Apache Aurora script to be submitted to the Aurora master.

Please let me know if there are any comments, suggestions on this plan.

--
Regards,
Gourav Rattihalli

Re: Jetstream VM creation through Airavata

[Pierce, Marlon]

Hi Emre,

The current case is the first: manage VMs to run jobs for Airavata, coupled 
with an Apache Mesos-based scheduling of jobs within the VMs.

We have a Jetstream VM with the PGA and Airavata installed there also.  


Marlon

On 4/20/16, 6:34 PM, "Emre Brookes"  wrote:

>Or are you working on creating dynamic VM's to run jobs via Airavata ?
>both that and getting an Airavata server VM or docker version running 
>there would be of benefit.
>
>-e.
>
>
>Emre Brookes wrote:
>> Hi Suresh,
>>
>> Just FYI - Glad to hear you are getting Airavata easy to deploy on 
>> Jetstream.
>> I am currently working on getting GenApp easy to deploy on the 
>> standard Ubuntu & CentOS images there.
>> I should be ready in the next month or two to revisit Airavata with a 
>> focus on Jetstream  ...
>>
>> Cheers,
>> Emre
>>
>>
>> Suresh Marru wrote:
>>> Pankaj,
>>>
>>> We need to fix this NPE. Meanwhile, I just created a testKey on the 
>>> keystone. Can you try again?
>>>
>>> Suresh
>>>
 On Apr 20, 2016, at 3:42 PM, Pankaj Saha >>> > wrote:

 I have just use maven to build the same project. here is thedetails
 # Common properties
 publicKeyFile=/home/psaha4/.ssh/id_rsa.pub

 # Jetstream properties
 jetstream_imageId=448636b0-450a-4b49-9bfe-b3135fc443e8
 jetstream_flavorId=3

 OS_PROJECT_DOMAIN_NAME=tacc
 OS_USER_DOMAIN_NAME=tacc
 OS_PROJECT_NAME=scigap
 OS_TENANT_NAME=scigap
 OS_USERNAME=scigap
 OS_PASSWORD=
 OS_AUTH_URL=https://jblb.jetstream-cloud.org:35357/v3
 OS_IDENTITY_API_VERSION=3

 # Network Properties
 OS_FLOATING_IP_POOL=public
 OS_SUBNET_NAME=airavata

 The test case error that I am getting for the Test case is al below:
 ---
  T E S T S
 ---
 Running org.apache.airavata.cloud.test.CloudIntfTest
 AuthenticationException{message=Unauthorized, status=401}
 at 
 org.openstack4j.core.transport.HttpExceptionHandler.mapException(HttpExceptionHandler.java:36)
 at 
 org.openstack4j.core.transport.HttpExceptionHandler.mapException(HttpExceptionHandler.java:23)
 at 
 org.openstack4j.openstack.internal.OSAuthenticator.authenticateV3(OSAuthenticator.java:161)
 at 
 org.openstack4j.openstack.internal.OSAuthenticator.invoke(OSAuthenticator.java:78)
 at 
 org.openstack4j.openstack.client.OSClientBuilder$ClientV3.authenticate(OSClientBuilder.java:163)
 at 
 org.openstack4j.openstack.client.OSClientBuilder$ClientV3.authenticate(OSClientBuilder.java:127)
 at 
 org.apache.airavata.cloud.openstack.OS4JClientProvider.getOSClientV3(OS4JClientProvider.java:86)
 at 
 org.apache.airavata.cloud.openstack.OS4JClientProvider.getOSClient(OS4JClientProvider.java:49)
 at 
 org.apache.airavata.cloud.intf.impl.OpenstackIntfImpl.(OpenstackIntfImpl.java:78)
 at 
 org.apache.airavata.cloud.test.CloudIntfTest.jetstreamCreateDeleteServerTest(CloudIntfTest.java:77)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
 at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
 at 
 org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
 at 
 org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
 at 
 org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
 at 
 org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
 at 
 org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.ja

Re: [VOTE] New website and logo

[Pierce, Marlon]

I’m +1 for this.

The content isn’t all there, but the new site repository is in Git and the 
content is much easier to manage than the old site.

From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, April 13, 2016 at 4:17 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: [VOTE] New website and logo

Hi All,

Any further thoughts on the website and logo redesign - 
http://gw120.iu.xsede.org/airavata-new-site/content/

Logo discussion is on the JIRA - 
https://issues.apache.org/jira/browse/AIRAVATA-1847

Nipurn has explained his thought process on the current proposed logo at - 
http://gw120.iu.xsede.org/airavata-new-site/content/logo.html

If we all agree to move forward wit this one, then we can replace the current 
website with new one and start iterating on the content.

Suresh


On Apr 7, 2016, at 3:48 PM, Nipurn Doshi 
mailto:nido...@umail.iu.edu>> wrote:


Thanks Marlon. This will help everyone to add their suggestions to the issue.

On Apr 7, 2016 2:39 PM, "Pierce, Marlon" 
mailto:marpi...@iu.edu>> wrote:
Also, I didn’t see a Jira so I created one, 
https://issues.apache.org/jira/browse/AIRAVATA-1950. If I missed the 
appropriate Jira, please point me to it.

Thanks,

Marlon


From: marpierc mailto:marpi...@iu.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:36 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

So from scratch use

git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git -b 
"asf-site"

Layout is looking good but the placeholder text needs to be replaced with real 
text.  Nipurn, if you can get this started from the old site, I can iterate it.

Marlon

From: Suresh Marru mailto:smarru.apa...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:17 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update


This is an expected warning since the default (and only) branch is asf-site. 
Git is looking for non existant master branch.

Just do a git checkout asf-site and you should be good.

Suresh


On Apr 7, 2016, at 2:12 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:

I am not able to check out:

 git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git
Cloning into 'airavata-site'...
remote: Counting objects: 158, done.
remote: Compressing objects: 100% (146/146), done.
remote: Total 158 (delta 28), reused 104 (delta 9)
Receiving objects: 100% (158/158), 5.61 MiB | 3.07 MiB/s, done.
Resolving deltas: 100% (28/28), done.
Checking connectivity... done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.


From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 1:16 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

Hi Nipurn,

I converted the website into a Jekyll based site. I still retained all your 
html as it is, but plan to move content into mark down format as much as 
possible so it will be easier to edit and not worry about formatting.

Here is an important change and I documented in the README - 
https://github.com/apache/airavata-site/blob/asf-site/README.md

* Now the sources are in the parent folder of the git repo. You will see that, 
they are the same HTML content with just the header and footer moved into 
_includes folder.
* Do not directly edit the *content* folder. This will need to be generated by 
Jekyll using “jekyll build”.

Lets keep working on the content.

Suresh


On Apr 6, 2016, at 12:35 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:

Hi Nipurn,

This is an excellent thought. I really like the concept and the thought process 
you are walking through the logo design.

If it helps with reviews I checked a coy of the site here - 
http://gw120.iu.xsede.org/airavata-new-site/content/

Suresh

On Apr 6, 2016, at 12:14 PM, Nipurn Doshi 
mailto:nido...@umail.iu.edu>> wrote:

Hi All,

New website is in the works and pure html based content has been added to the 
repo -  https://git-wip-us.apache.org/repos/asf/airavata-site.git .
Git mirror -  https://github.com/apache/airavata-site
There are no special requirements to run the current website redesign 
iterations as they are based out of pure html/css/js. All you have to do is run 
index.html inside the 'co

Re: Website Redesign update

[Pierce, Marlon]

I used jekyll, edited the index.md file and also _include/footer.html. I didn’t 
see any .md file that contained the footer content.

From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 3:16 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

Marlon,

I see that in your last commit you directly edited html files, those will get 
overwritten. We all need to get used to Jekyll. A simple rule of thumb will be 
only .md files are to be edited. All .html are generated files and should be 
considered read-only.

Suresh

On Apr 7, 2016, at 2:39 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:

Also, I didn’t see a Jira so I created one, 
https://issues.apache.org/jira/browse/AIRAVATA-1950. If I missed the 
appropriate Jira, please point me to it.

Thanks,

Marlon


From: marpierc mailto:marpi...@iu.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:36 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

So from scratch use

git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git -b 
"asf-site"

Layout is looking good but the placeholder text needs to be replaced with real 
text.  Nipurn, if you can get this started from the old site, I can iterate it.

Marlon

From: Suresh Marru mailto:smarru.apa...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:17 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update


This is an expected warning since the default (and only) branch is asf-site. 
Git is looking for non existant master branch.

Just do a git checkout asf-site and you should be good.

Suresh


On Apr 7, 2016, at 2:12 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:

I am not able to check out:

 git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git
Cloning into 'airavata-site'...
remote: Counting objects: 158, done.
remote: Compressing objects: 100% (146/146), done.
remote: Total 158 (delta 28), reused 104 (delta 9)
Receiving objects: 100% (158/158), 5.61 MiB | 3.07 MiB/s, done.
Resolving deltas: 100% (28/28), done.
Checking connectivity... done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.


From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 1:16 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

Hi Nipurn,

I converted the website into a Jekyll based site. I still retained all your 
html as it is, but plan to move content into mark down format as much as 
possible so it will be easier to edit and not worry about formatting.

Here is an important change and I documented in the README - 
https://github.com/apache/airavata-site/blob/asf-site/README.md

* Now the sources are in the parent folder of the git repo. You will see that, 
they are the same HTML content with just the header and footer moved into 
_includes folder.
* Do not directly edit the *content* folder. This will need to be generated by 
Jekyll using “jekyll build”.

Lets keep working on the content.

Suresh


On Apr 6, 2016, at 12:35 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:

Hi Nipurn,

This is an excellent thought. I really like the concept and the thought process 
you are walking through the logo design.

If it helps with reviews I checked a coy of the site here - 
http://gw120.iu.xsede.org/airavata-new-site/content/

Suresh

On Apr 6, 2016, at 12:14 PM, Nipurn Doshi 
mailto:nido...@umail.iu.edu>> wrote:

Hi All,

New website is in the works and pure html based content has been added to the 
repo -  https://git-wip-us.apache.org/repos/asf/airavata-site.git .
Git mirror -  https://github.com/apache/airavata-site
There are no special requirements to run the current website redesign 
iterations as they are based out of pure html/css/js. All you have to do is run 
index.html inside the 'content' folder after checking it out.

You will find some links not working and titles having Lorem-ipsum text under 
them which we aim to fill up with relevant content in the coming days. 
Suggestions/comments are most welcome.

Also, feel free to check draft document for the new proposed Apache Airavata 
logo which can also be seen on website iterations, here - 
http://bit.ly/airavata-logo-draft

--
-Sincerely,
Nipurn Doshi

Re: Website Redesign update

[Pierce, Marlon]

Also, I didn’t see a Jira so I created one, 
https://issues.apache.org/jira/browse/AIRAVATA-1950. If I missed the 
appropriate Jira, please point me to it.

Thanks,

Marlon


From: marpierc mailto:marpi...@iu.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:36 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

So from scratch use

git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git -b 
"asf-site"

Layout is looking good but the placeholder text needs to be replaced with real 
text.  Nipurn, if you can get this started from the old site, I can iterate it.

Marlon

From: Suresh Marru mailto:smarru.apa...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:17 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update


This is an expected warning since the default (and only) branch is asf-site. 
Git is looking for non existant master branch.

Just do a git checkout asf-site and you should be good.

Suresh


On Apr 7, 2016, at 2:12 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:

I am not able to check out:


 git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git

Cloning into 'airavata-site'...

remote: Counting objects: 158, done.

remote: Compressing objects: 100% (146/146), done.

remote: Total 158 (delta 28), reused 104 (delta 9)

Receiving objects: 100% (158/158), 5.61 MiB | 3.07 MiB/s, done.

Resolving deltas: 100% (28/28), done.

Checking connectivity... done.

warning: remote HEAD refers to nonexistent ref, unable to checkout.


From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 1:16 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

Hi Nipurn,

I converted the website into a Jekyll based site. I still retained all your 
html as it is, but plan to move content into mark down format as much as 
possible so it will be easier to edit and not worry about formatting.

Here is an important change and I documented in the README - 
https://github.com/apache/airavata-site/blob/asf-site/README.md

* Now the sources are in the parent folder of the git repo. You will see that, 
they are the same HTML content with just the header and footer moved into 
_includes folder.
* Do not directly edit the *content* folder. This will need to be generated by 
Jekyll using “jekyll build”.

Lets keep working on the content.

Suresh


On Apr 6, 2016, at 12:35 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:

Hi Nipurn,

This is an excellent thought. I really like the concept and the thought process 
you are walking through the logo design.

If it helps with reviews I checked a coy of the site here - 
http://gw120.iu.xsede.org/airavata-new-site/content/

Suresh

On Apr 6, 2016, at 12:14 PM, Nipurn Doshi 
mailto:nido...@umail.iu.edu>> wrote:

Hi All,

New website is in the works and pure html based content has been added to the 
repo -  https://git-wip-us.apache.org/repos/asf/airavata-site.git .
Git mirror -  https://github.com/apache/airavata-site
There are no special requirements to run the current website redesign 
iterations as they are based out of pure html/css/js. All you have to do is run 
index.html inside the 'content' folder after checking it out.

You will find some links not working and titles having Lorem-ipsum text under 
them which we aim to fill up with relevant content in the coming days. 
Suggestions/comments are most welcome.

Also, feel free to check draft document for the new proposed Apache Airavata 
logo which can also be seen on website iterations, here - 
http://bit.ly/airavata-logo-draft

--
-Sincerely,
Nipurn Doshi

Re: Website Redesign update

[Pierce, Marlon]

So from scratch use

git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git -b 
"asf-site"

Layout is looking good but the placeholder text needs to be replaced with real 
text.  Nipurn, if you can get this started from the old site, I can iterate it.

Marlon

From: Suresh Marru mailto:smarru.apa...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 2:17 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update


This is an expected warning since the default (and only) branch is asf-site. 
Git is looking for non existant master branch.

Just do a git checkout asf-site and you should be good.

Suresh


On Apr 7, 2016, at 2:12 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:

I am not able to check out:


 git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git

Cloning into 'airavata-site'...

remote: Counting objects: 158, done.

remote: Compressing objects: 100% (146/146), done.

remote: Total 158 (delta 28), reused 104 (delta 9)

Receiving objects: 100% (158/158), 5.61 MiB | 3.07 MiB/s, done.

Resolving deltas: 100% (28/28), done.

Checking connectivity... done.

warning: remote HEAD refers to nonexistent ref, unable to checkout.


From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 1:16 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

Hi Nipurn,

I converted the website into a Jekyll based site. I still retained all your 
html as it is, but plan to move content into mark down format as much as 
possible so it will be easier to edit and not worry about formatting.

Here is an important change and I documented in the README - 
https://github.com/apache/airavata-site/blob/asf-site/README.md

* Now the sources are in the parent folder of the git repo. You will see that, 
they are the same HTML content with just the header and footer moved into 
_includes folder.
* Do not directly edit the *content* folder. This will need to be generated by 
Jekyll using “jekyll build”.

Lets keep working on the content.

Suresh


On Apr 6, 2016, at 12:35 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:

Hi Nipurn,

This is an excellent thought. I really like the concept and the thought process 
you are walking through the logo design.

If it helps with reviews I checked a coy of the site here - 
http://gw120.iu.xsede.org/airavata-new-site/content/

Suresh

On Apr 6, 2016, at 12:14 PM, Nipurn Doshi 
mailto:nido...@umail.iu.edu>> wrote:

Hi All,

New website is in the works and pure html based content has been added to the 
repo -  https://git-wip-us.apache.org/repos/asf/airavata-site.git .
Git mirror -  https://github.com/apache/airavata-site
There are no special requirements to run the current website redesign 
iterations as they are based out of pure html/css/js. All you have to do is run 
index.html inside the 'content' folder after checking it out.

You will find some links not working and titles having Lorem-ipsum text under 
them which we aim to fill up with relevant content in the coming days. 
Suggestions/comments are most welcome.

Also, feel free to check draft document for the new proposed Apache Airavata 
logo which can also be seen on website iterations, here - 
http://bit.ly/airavata-logo-draft

--
-Sincerely,
Nipurn Doshi

Re: Website Redesign update

[Pierce, Marlon]

I am not able to check out:


 git clone  https://git-wip-us.apache.org/repos/asf/airavata-site.git

Cloning into 'airavata-site'...

remote: Counting objects: 158, done.

remote: Compressing objects: 100% (146/146), done.

remote: Total 158 (delta 28), reused 104 (delta 9)

Receiving objects: 100% (158/158), 5.61 MiB | 3.07 MiB/s, done.

Resolving deltas: 100% (28/28), done.

Checking connectivity... done.

warning: remote HEAD refers to nonexistent ref, unable to checkout.


From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Thursday, April 7, 2016 at 1:16 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Website Redesign update

Hi Nipurn,

I converted the website into a Jekyll based site. I still retained all your 
html as it is, but plan to move content into mark down format as much as 
possible so it will be easier to edit and not worry about formatting.

Here is an important change and I documented in the README - 
https://github.com/apache/airavata-site/blob/asf-site/README.md

* Now the sources are in the parent folder of the git repo. You will see that, 
they are the same HTML content with just the header and footer moved into 
_includes folder.
* Do not directly edit the *content* folder. This will need to be generated by 
Jekyll using “jekyll build”.

Lets keep working on the content.

Suresh


On Apr 6, 2016, at 12:35 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:

Hi Nipurn,

This is an excellent thought. I really like the concept and the thought process 
you are walking through the logo design.

If it helps with reviews I checked a coy of the site here - 
http://gw120.iu.xsede.org/airavata-new-site/content/

Suresh

On Apr 6, 2016, at 12:14 PM, Nipurn Doshi 
mailto:nido...@umail.iu.edu>> wrote:

Hi All,

New website is in the works and pure html based content has been added to the 
repo -  https://git-wip-us.apache.org/repos/asf/airavata-site.git .
Git mirror -  https://github.com/apache/airavata-site
There are no special requirements to run the current website redesign 
iterations as they are based out of pure html/css/js. All you have to do is run 
index.html inside the 'content' folder after checking it out.

You will find some links not working and titles having Lorem-ipsum text under 
them which we aim to fill up with relevant content in the coming days. 
Suggestions/comments are most welcome.

Also, feel free to check draft document for the new proposed Apache Airavata 
logo which can also be seen on website iterations, here - 
http://bit.ly/airavata-logo-draft

--
-Sincerely,
Nipurn Doshi

Re: Airavata 0.16 Release Planning

[Pierce, Marlon]

Do we have any outstanding tasks that need to be wrapped up and committed to 
dev?

From: Shameera Rathnayaka 
mailto:shameerai...@gmail.com>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Monday, March 28, 2016 at 11:20 AM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: Airavata 0.16 Release Planning

+1

On Mon, Mar 28, 2016 at 10:50 AM Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi All,

Before we go too far, how about we call a feature freeze and stat working on 
0.16 release? Unless any one is in the middle of a development activity, how 
about we target end of the week to start working on it?

Suresh
--
Shameera Rathnayaka

Re: [GSoC Proposal] In-Situ Simulation Monitoring and Analysis Using Apache Airavata

[Pierce, Marlon]

Agreed, looks good. Please update your submission.

From: Suresh Marru mailto:sma...@apache.org>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Friday, March 25, 2016 at 12:32 PM
To: Airavata Dev mailto:dev@airavata.apache.org>>
Subject: Re: [GSoC Proposal] In-Situ Simulation Monitoring and Analysis Using 
Apache Airavata

You did a great uplift of the proposal, great job.I suggest submit a final copy 
immediately.

Suresh

On Mar 25, 2016, at 12:28 PM, Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>> wrote:

Thank you all for your feedback. I have addressed the many fantastic 
suggestions and believe that the proposal is just about complete in terms of 
content.

If the team could review the updated proposal, I would much appreciate it.

https://docs.google.com/document/d/16BCTV__tA9h2Pnx1annaBIJaU2Jmma1ayTHV6BNMzNA/edit?usp=sharing

Best,
Jeff K.

On Thu, Mar 24, 2016 at 5:47 AM, Sandra Gesing 
mailto:sandra.ges...@nd.edu>> wrote:
Hi Jeff,

I think a graphical sketch of the architecture of the services would be a great 
addition to your proposal.

Thanks,
Sandra


Sandra Gesing
Research Assistant Professor, Department of Computer Science and Engineering
Computational Scientist, Center for Research Computing
University of Notre Dame
http://www3.nd.edu/~sgesing


On Wed, Mar 23, 2016 at 1:50 PM, Miller, Mark 
mailto:mmil...@sdsc.edu>> wrote:
I like this proposal also very much, and agree with what Sandy said as well.

Mark


From: Sandra Gesing [mailto:sandra.ges...@nd.edu]
Sent: Tuesday, March 22, 2016 6:12 PM
To: dev@airavata.apache.org
Subject: Re: [GSoC Proposal] In-Situ Simulation Monitoring and Analysis Using 
Apache Airavata

Hi Jeff,

the proposal looks already excellent. I think it would be beneficial to have 
the feature in Airavata that users can directly search for certain terms in a 
log file and intermediate results additionally to see the whole log file and 
intermediate values.
Thanks,
Sandra


Sandra Gesing
Research Assistant Professor, Department of Computer Science and Engineering
Computational Scientist, Center for Research Computing
University of Notre Dame
http://www3.nd.edu/~sgesing


On Tue, Mar 22, 2016 at 8:03 PM, Jeffery Kinnison 
mailto:jeffery.d.kinniso...@nd.edu>> wrote:
Hello Dev Team,
I have created a draft of my GSoC proposal and made it available as at the link 
below.

https://docs.google.com/document/d/16BCTV__tA9h2Pnx1annaBIJaU2Jmma1ayTHV6BNMzNA/edit?usp=sharing
I would appreciate any comments to improve this proposal.
Best,
Jeff Kinnison

Re: [ GSOC prosposal ]

[Pierce, Marlon]

Jatin and others: please submit now.



On 3/25/16, 8:06 AM, "Jatin Balodhi"  wrote:

>Hi Marlon,
>
>I've made changes to my GSOC proposal draft according to your comments. 
>Can you review it before i submit it to GSOC site .
>
>Thanks
>Jatin

Re: Unable to Start AirAvata Locally

[Pierce, Marlon]

Hi Ang,

XBaya support is deprecated in the current version; 0.14 is the last release to 
support. We are reimplementing the workflow backend services so it would be 
better to look at the API than the clients.

Marlon


From: ANG TAN FONG mailto:an...@um.edu.my>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Friday, March 25, 2016 at 7:03 AM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: Re: Unable to Start AirAvata Locally

Good day Suresh,

I am writing a review paper about the workflow management system.
I am currently looking at Apache Airavata and how it works.
I manage to start airavata after uncomment the validationquery. Thanks.

However, I faced another problem when installing the xbaya-gui.
After executing sh registerSample.sh

 Registering Localhost Computational Resource 

org.apache.thrift.protocol.TProtocolException: Required field 
'securityProtocol' is unset! 
Struct:LOCALSubmission(jobSubmissionInterfaceId:DO_NOT_SET_AT_CLIENTS, 
securityProtocol:null, 
resourceJobManager:ResourceJobManager(resourceJobManagerId:DO_NOT_SET_AT_CLIENTS,
 resourceJobManagerType:FORK))


On Thu, Mar 24, 2016 at 7:21 PM, Suresh Marru 
mailto:sma...@apache.org>> wrote:
Hi Ang,

Thanks for your interest in Airavata. Did you follow the installation 
documentation at - http://airavata.readthedocs.org/en/latest/?

Can you describe your usage scenario a bit? Are you trying to integrate with an 
existing batch scheduler or are you trying to run some simulations locally?

Suresh


On Mar 24, 2016, at 11:54 AM, Supun Nakandala 
mailto:supun.nakand...@gmail.com>> wrote:

Hi Ang,

It seems that you have commented out the validation query in your 
airavata-server.properties (which can be found at /bin). Can you 
share your airavata-server.properties file here.

On Thu, Mar 24, 2016 at 11:40 AM, ANG TAN FONG 
mailto:an...@um.edu.my>> wrote:
Good day,

I get this error when I try to start AirAvata in CentOS 7.
I try to google to solve the problem. However, I can't find any solution.

ERROR org.apache.airavata.registry.core.experiment.catalog.resources.Utils  - 
validationQuery

Attached the log file. Please help. Thanks.

--
Best Regards
Dr. Ang Tan Fong, A.M.P.
Senior Lecturer
Department of Computer System and Technology
Faculty of Computer Science and Information Technology
University of Malaya

" PENAFIAN: E-mel ini dan apa-apa fail yang dikepilkan bersamanya ("Mesej") 
adalah ditujukan hanya untuk kegunaan penerima(-penerima) yang termaklum di 
atas dan mungkin mengandungi maklumat sulit. Anda dengan ini dimaklumkan bahawa 
mengambil apa jua tindakan bersandarkan kepada, membuat penilaian, mengulang 
hantar, menghebah, mengedar, mencetak, atau menyalin Mesej ini atau sebahagian 
daripadanya oleh sesiapa selain daripada penerima(-penerima) yang termaklum di 
atas adalah dilarang. Jika anda telah menerima Mesej ini kerana kesilapan, anda 
mesti menghapuskan Mesej ini dengan segera dan memaklumkan kepada penghantar 
Mesej ini menerusi balasan e-mel. Pendapat-pendapat, rumusan-rumusan, dan 
sebarang maklumat lain di dalam Mesej ini yang tidak berkait dengan urusan 
rasmi Universiti Malaya adalah difahami sebagai bukan dikeluar atau diperakui 
oleh mana-mana pihak yang disebut.


DISCLAIMER: This e-mail and any files transmitted with it ("Message") is 
intended only for the use of the recipient(s) named above and may contain 
confidential information. You are hereby notified that the taking of any action 
in reliance upon, or any review, retransmission, dissemination, distribution, 
printing or copying of this Message or any part thereof by anyone other than 
the intended recipient(s) is strictly prohibited. If you have received this 
Message in error, you should delete this Message immediately and advise the 
sender by return e-mail. Opinions, conclusions and other information in this 
Message that do not relate to the official business of University of Malaya 
shall be understood as neither given nor endorsed by any of the forementioned. "



--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa




--
Best Regards
Dr. Ang Tan Fong, A.M.P.
Senior Lecturer
Department of Computer System and Technology
Faculty of Computer Science and Information Technology
University of Malaya

" PENAFIAN: E-mel ini dan apa-apa fail yang dikepilkan bersamanya ("Mesej") 
adalah ditujukan hanya untuk kegunaan penerima(-penerima) yang termaklum di 
atas dan mungkin mengandungi maklumat sulit. Anda dengan ini dimaklumkan bahawa 
mengambil apa jua tindakan bersandarkan kepada, membuat penilaian, mengulang 
hantar, menghebah, mengedar, mencetak, atau menyalin Mesej ini atau sebahagian 
daripadanya oleh sesiapa selain daripada penerima(-penerima) yang termaklum di 
atas adalah dilarang. Jika anda telah menerima Mesej ini kerana kesilapan, anda 
mesti menghapuskan Mesej ini dengan segera d

Re: [GSoC Proposal] Apache Airavata Monitoring Module

[Pierce, Marlon]

Hi Sidd,

This is a well written proposal. Some comments:

* The monitoring component doesn’t need to directly communicate with GFAC. 
There can be multiple GFACs, so the original GFAC that submitted the request 
may no longer be running.  Also, other components may want to receive updates, 
including external subscribers.  GFAC subscribers can pick up the message and 
act.  There is the issue of managing race conditions between multiple GFACs and 
other distributed computing issues. If a GFAC instance picks up the message and 
wants to act on it but then becomes unresponsive, how do we handle this 
situation? This is my opinion, anyway. It is harder to implement than the 
direct approach.

* You still need your high level architecture diagram and other figures where 
you have placeholders currently.

* You need 2 week deliverables in your timeline. Add discussion of risk and 
risk mitigation, as I mentioned in my general email earlier today.

Marlon


From: Siddharth Jain mailto:ja...@umail.iu.edu>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, March 23, 2016 at 12:52 AM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: Re: [GSoC Proposal] Apache Airavata Monitoring Module

Correction, the cwiki.apache.org link is:
https://cwiki.apache.org/confluence/display/AIRAVATA/[GSoC+Proposal]+Apache+Airavata+Monitoring+Module

On Tue, Mar 22, 2016 at 6:35 PM, Siddharth Jain 
mailto:ja...@umail.iu.edu>> wrote:
Hello all,
I have the first draft of my GSoC proposal on Apache Airavata Monitoring Module 
ready. I will appreciate if you could give any comments or suggestions to 
improve this proposal.

The proposal is available on:
i) 
https://cwiki.apache.org/confluence/display/AIRAVATA/GSoC+Proposal+-+In+Situ+Simulation+Analysis+Using+Airavata

ii) 
https://docs.google.com/document/d/1rm_U-51NzdqfBfTNjOyCzc2eTR2SEg5K1tijhtijcz0/edit?usp=sharing


The cwiki.apache.org link points to what it will 
actually look like, the google docs version is just for convenience of 
commenting.


Best regards,
Siddharth Jain

Re: GSoC proposal - Apache Airavata and Jupyter Integration on the Use Case for Molecular Dynamics

[Pierce, Marlon]

I commented on the document. I would like to see more concrete details of how 
Jupyter and Airavata will interact.  See also general emails about describing 
your risks and risk mitigations, and how you will demonstrate that each 2 week 
milestone is completed.

Marlon

From: Ethan Suh 
mailto:dongyosuh2...@u.northwestern.edu>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Thursday, March 24, 2016 at 4:38 AM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: GSoC proposal - Apache Airavata and Jupyter Integration on the Use 
Case for Molecular Dynamics

Hi,

I have a first draft of my GSoC proposal shared on the GSoC site with the 
Apache Foundation. Here is the link as well: 
https://docs.google.com/document/d/1o4TYCqgMZHbCcGBoGq1RNs2hvR3xdd2-WCerCQPt7K8/edit?usp=sharing.

I am still figuring out the details of my deliverables and milestones with my 
mentor and should include them shortly. Please let me know what comments you 
have about the project.

Thanks,

Ethan Suh
Northwestern University WCAS Class of 2017
Major in Computer Science; Minor in Economics
e...@u.northwestern.edu

GSOC objectives and milestones

[Pierce, Marlon]

For all students: in your proposals, clearly state how you will demonstrate 
each 2 week milestone has been completed.  How can we objectively review each 
milestone and conclude that you have accomplished what you wanted to accomplish?

Think also of the risks for your proposal as a whole and each milestone. What 
could go wrong technically that would prevent you from accomplishing a 
milestone? What is your back up plan?  If you understand the risks and can 
clearly state them, this indicates a deeper understanding of the problems you 
are trying to solve.

Marlon

Re: Note to potential GSOC students

[Pierce, Marlon]

Hi Jatin, I see it. I will be reviewing these over the next few hours.

From: "mywork.ja...@gmail.com<mailto:mywork.ja...@gmail.com>" 
mailto:mywork.ja...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, March 24, 2016 at 9:35 AM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Fwd: Re: Note to potential GSOC students


Hi Marlon,

I have submitted the draft of my proposal to ASF with Apache Airavata in the 
title, in the GSOC portal.

Please let me know if you can see my draft.

Thanks
Jatin


-- Forwarded message --
From: "Pierce, Marlon" mailto:marpi...@iu.edu>>
Date: Thu, Mar 24, 2016 at 6:18 AM -0700
Subject: Re: Note to potential GSOC students
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>


Hi Mangirish, it is there.

From: Mangirish Wagle 
mailto:vaglomangir...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, March 24, 2016 at 12:08 AM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Re: Note to potential GSOC students

Hi Marlon,

I have submitted the first draft of my proposal as per your guidelines to ASF 
with Apache Airavata in the title, in the GSOC portal.
Please let me know if you can see my draft.

Thank you.

Regards,
Mangirish Wagle

On Wed, Mar 23, 2016 at 11:49 AM, Jatin Balodhi 
mailto:mywork.ja...@gmail.com>> wrote:
There's an option of " Apache Software Foundation proposal tag" what should i 
select there?

Thanks
Jatin

 Forwarded Message 
Subject:Re: Note to potential GSOC students
Date:   Wed, 23 Mar 2016 15:36:58 +
From:   Pierce, Marlon <mailto:marpi...@iu.edu>
To: Jatin Balodhi <mailto:mywork.ja...@gmail.com>


Hi Jatin, you have not yet started your application to GSOC program, as far as 
I can tell.  I do not see your application in the project listings.  Do this 
first. Make sure Apache Airavata is in your title.

Marlon


From: Jatin Balodhi mailto:mywork.ja...@gmail.com>>
Reply-To: 
"<mailto:dev@airavata.apache.org>dev@airavata.apache.org<mailto:dev@airavata.apache.org>"
 mailto:dev@airavata.apache.org>>
Date: Wednesday, March 23, 2016 at 11:17 AM
To: 
"<mailto:dev@airavata.apache.org>dev@airavata.apache.org<mailto:dev@airavata.apache.org>"
 mailto:dev@airavata.apache.org>>
Subject: Re: Note to potential GSOC students

Hi Marlon,

I made some changes to my GSOC proposal as you said, can you look at it once 
more?

Thanks
Jatin

On Wednesday 23 March 2016 07:52 PM, Pierce, Marlon wrote:
Please make sure you have started the proposal submission process correctly in 
the GSOC site, <https://summerofcode.withgoogle.com/> 
https://summerofcode.withgoogle.com/.  I see proposal drafts for only about 
half of those students who have expressed interest.

Thanks,

Marlon

GSOC deadline is tomorrow

[Pierce, Marlon]

Proposals are due tomorrow at 19:00 UTC.  See 
https://developers.google.com/open-source/gsoc/timeline.  I recommend that you 
submit 2-3 hours in advance in case there are technical problems with your 
submission.

Marlon

Re: Note to potential GSOC students

[Pierce, Marlon]

Hi Mangirish, it is there.

From: Mangirish Wagle 
mailto:vaglomangir...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, March 24, 2016 at 12:08 AM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: Re: Note to potential GSOC students

Hi Marlon,

I have submitted the first draft of my proposal as per your guidelines to ASF 
with Apache Airavata in the title, in the GSOC portal.
Please let me know if you can see my draft.

Thank you.

Regards,
Mangirish Wagle

On Wed, Mar 23, 2016 at 11:49 AM, Jatin Balodhi 
mailto:mywork.ja...@gmail.com>> wrote:
There's an option of " Apache Software Foundation proposal tag" what should i 
select there?

Thanks
Jatin

 Forwarded Message 
Subject:Re: Note to potential GSOC students
Date:   Wed, 23 Mar 2016 15:36:58 +
From:   Pierce, Marlon <mailto:marpi...@iu.edu>
To: Jatin Balodhi <mailto:mywork.ja...@gmail.com>


Hi Jatin, you have not yet started your application to GSOC program, as far as 
I can tell.  I do not see your application in the project listings.  Do this 
first. Make sure Apache Airavata is in your title.

Marlon


From: Jatin Balodhi mailto:mywork.ja...@gmail.com>>
Reply-To: 
"<mailto:dev@airavata.apache.org>dev@airavata.apache.org<mailto:dev@airavata.apache.org>"
 mailto:dev@airavata.apache.org>>
Date: Wednesday, March 23, 2016 at 11:17 AM
To: 
"<mailto:dev@airavata.apache.org>dev@airavata.apache.org<mailto:dev@airavata.apache.org>"
 mailto:dev@airavata.apache.org>>
Subject: Re: Note to potential GSOC students

Hi Marlon,

I made some changes to my GSOC proposal as you said, can you look at it once 
more?

Thanks
Jatin

On Wednesday 23 March 2016 07:52 PM, Pierce, Marlon wrote:
Please make sure you have started the proposal submission process correctly in 
the GSOC site, <https://summerofcode.withgoogle.com/> 
https://summerofcode.withgoogle.com/.  I see proposal drafts for only about 
half of those students who have expressed interest.

Thanks,

Marlon

Re: [GSOC Proposal] Cloud based clusters for Apache Airavata

[Pierce, Marlon]

The Application Factory component is called “gfac” in the code base.  This is 
the part that handles the interfacing to the remote resource (most often by ssh 
but other providers exist). The Orchestrator routes jobs to GFAC instances.

From: Mangirish Wagle 
mailto:vaglomangir...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Wednesday, March 23, 2016 at 11:56 AM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: [GSOC Proposal] Cloud based clusters for Apache Airavata

Hello Team,

I was drafting the GSOC proposal and I just had a quick question about the 
integration of the project with Apache Airavata.

Which is the component in Airavata that would call the service to provision the 
cloud cluster?

I am looking at the Airavata architecture diagram and my understanding is that 
this would be treated as a new Application and would have a separate 
application interface in 'Application Factory' component. Also the workflow 
orchestrator would be having the intelligence to figure out which jobs to be 
submitted to cloud based clusters.

Please let me know whether my understanding is correct.

Thank you.

Best Regards,
Mangirish Wagle

On Tue, Mar 22, 2016 at 2:28 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Hi Mangirish, please add your proposal to the GSOC 2016 site.

From: Mangirish Wagle 
mailto:vaglomangir...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Thursday, March 17, 2016 at 3:35 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: [GSOC Proposal] Cloud based clusters for Apache Airavata

Hello Dev Team,

I had the opportunity to interact with Suresh and Shameera wherein we discussed 
an open requirement in Airavata to be addressed. The requirement is to expand 
the capabilities of Apache Airavata to submit jobs to cloud based clusters in 
addition to HPC/ HTC clusters.

The idea is to dynamically provision a cloud cluster in an environment like 
Jetstream, based on the configuration figured out by Airavata, which would be 
operated by a distributed system management software like Mesos. An initial 
high level goals would be:-

  1.  Airavata categorizes certain jobs to be run on cloud based clusters and 
figure out the required hardware config for the cluster.
  2.  The proposed service would provision the cluster with the required 
resources.
  3.  An ansible script would configure a Mesos cluster with the resources 
provisioned.
  4.  Airavata submits the job to the Mesos cluster.
  5.  Mesos then figures out the efficient resource allocation within the 
cluster and runs the job and fetches the result.
  6.  The cluster is then deprovisioned automatically when not in use.

The project would mainly focus on point 2 and 6 above.

To start with, I am currently trying to get a working prototype of setting up 
compute nodes on an openstack environment using JClouds (Targetted for 
Jetstream). Also, I am planning to explore the option of using Openstack Heat 
engine to orchestrate the cluster. However, going ahead Airavata would be 
supporting other clouds like Amazon EC2 or Comet cluster, so we need to have a 
generic solution for achieving the goal.

Another approach which might be efficient in terms of performance and time is 
using a container based clouds using Docker, Kubernetes which would have 
substantially less bootstrap time compared to cloud VMs. This would be a future 
prospect as we may not have all the clusters supporting containerization.

This has been considered as a potential GSOC project and I would be working on 
drafting a proposal on this idea.

Any inputs/ comments/ suggestions would be very helpful.

Best Regards,
Mangirish Wagle

Re: [GSoC Proposal] - Integrating Resource Information from Apache Mesos with Apache Airavata’s Job Management Modules

[Pierce, Marlon]

Hi Pankaj,

I still don’t see your application in the GSOC site. Please make sure you have 
done this correctly.

Thanks,

Marlon

From: Pankaj Saha mailto:psa...@binghamton.edu>>
Date: Monday, March 21, 2016 at 5:17 PM
To: marpierc mailto:marpi...@iu.edu>>
Cc: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: [GSoC Proposal] - Integrating Resource Information from Apache 
Mesos with Apache Airavata’s Job Management Modules

Hi Marlon,
Here is the link that I have created.

https://docs.google.com/document/d/1qtFvg4-usT4D_1TDNBsQDFQGZIkH99ideYQ1T3HU9nY/edit?usp=sharing
The draft is created under the GSoC proposal site under Apache foundation with 
the same title.


Thanks
Pankaj



On Mon, Mar 21, 2016 at 4:47 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Hi Pankaj,

I have some comments, but it would be easier if you created a proposal draft in 
the GSOC site. The google doc option for your draft is better than pointing to 
the Airavata wiki. Please make sure you give comment and suggestion permissions.

Marlon


From: Pankaj Saha mailto:psa...@binghamton.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Monday, March 21, 2016 at 11:16 AM
To: dev mailto:dev@airavata.apache.org>>
Subject: [GSoC Proposal] - Integrating Resource Information from Apache Mesos 
with Apache Airavata’s Job Management Modules

Hi Dev Team,

Please review the following GSoC proposal that I plan to submit:
Title: Integrating Resource Information from Apache Mesos with Apache 
Airavata’s Job Management Modules

Abstract:
Apache Airavata provides gateway computing capability across clustered 
environments for scientific users. It abstracts away the complexities of 
submitting jobs to HPC platforms and provides users with an intuitive and 
elegant web-based interface to submit jobs. Apache Mesos is a  distributed 
kernel that manages distributed computing resources as a single computer. As 
Airavata is being extended to use Big Data and Cloud tools to launch jobs in 
cloud environments, it needs to retrieve the resource and job execution 
information from the Big Data framework back to the Apache portal accessible to 
the end user. In this project we will develop code and scripts to be integrated 
with the Airavata that will use the HTTP API of Mesos to continuously fetch the 
complete resource and scheduling information. This information can then be used 
by Airavata to dynamically monitor and improve its job submission strategy in 
cloud environments such as Jetstream.

Introduction:
Apache Mesos provides HTTP API endpoints for scheduler, executor, internal and 
admin related queries. To fetch information regarding a clustered environment 
that is managed by the Mesos master, the API can be accessed via curl requests 
over HTTP. The response to such requests will be received as well formed json 
document. We will parse the json response and present the information in the 
format desired. The retrieved information will include resource usage, resource 
available for further jobs, job status, time elapsed since the job started, 
etc.  Airavata, in turn, will use this information to determine the resource 
usage, performance of the jobs on a job submission, rapid diagnosis on the 
health of the submitted jobs.

We will use the observer pattern to continuously pull information from Cloud 
and big Data Resource Managers, such as Apache Mesos, to Airavata.

Any comment and suggestions would be very helpful.

Thanks
Pankaj

Note to potential GSOC students

[Pierce, Marlon]

Please make sure you have started the proposal submission process correctly in 
the GSOC site, https://summerofcode.withgoogle.com/.  I see proposal drafts for 
only about half of those students who have expressed interest.

Thanks,

Marlon

Re: [GSoC Proposal] - Integrating Resource Information from Apache Mesos with Apache Airavata’s Job Management Modules

[Pierce, Marlon]

Agreed.  You should provide some details on how you will integrate with 
Airavata: which components will you modify/create/extend?

From: Shameera Rathnayaka 
mailto:shameerai...@gmail.com>>
Date: Tuesday, March 22, 2016 at 2:29 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>, marpierc 
mailto:marpi...@iu.edu>>
Subject: Re: [GSoC Proposal] - Integrating Resource Information from Apache 
Mesos with Apache Airavata’s Job Management Modules

Hi Pankaj,

 Airavata architecture haven't change at all since last year, but few internal 
implementations which require minor effort to understand. As you are last year 
GSoC Student, we don't see you need much time (2 weeks, according to the your 
GSoC proposal) to spend to understand Airavata architech. Specially we expect 
task intensive proposal from you. Please revisit your milestones and 
deliverables. We would like to see something integrated with Airavata in you 
mid term evaluation. You can use Community bounding period to explore more 
about backgroud knowledge require such as Apache Mesos. Considering thease 
things you can add more comprehensive task as your milestons.

This proposal is seems to be continuation from your previous year GSoC 
proposal, if you could explain what you have done with previous GSoC and how 
you are going to start  from that will help us to understand the scope of this 
year proposal.

Thanks,
Shameera.

On Mon, Mar 21, 2016 at 5:17 PM Pankaj Saha 
mailto:psa...@binghamton.edu>> wrote:
Hi Marlon,
Here is the link that I have created.

https://docs.google.com/document/d/1qtFvg4-usT4D_1TDNBsQDFQGZIkH99ideYQ1T3HU9nY/edit?usp=sharing
The draft is created under the GSoC proposal site under Apache foundation with 
the same title.


Thanks
Pankaj



On Mon, Mar 21, 2016 at 4:47 PM, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Hi Pankaj,

I have some comments, but it would be easier if you created a proposal draft in 
the GSOC site. The google doc option for your draft is better than pointing to 
the Airavata wiki. Please make sure you give comment and suggestion permissions.

Marlon


From: Pankaj Saha mailto:psa...@binghamton.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Monday, March 21, 2016 at 11:16 AM
To: dev mailto:dev@airavata.apache.org>>
Subject: [GSoC Proposal] - Integrating Resource Information from Apache Mesos 
with Apache Airavata’s Job Management Modules

Hi Dev Team,

Please review the following GSoC proposal that I plan to submit:
Title: Integrating Resource Information from Apache Mesos with Apache 
Airavata’s Job Management Modules

Abstract:
Apache Airavata provides gateway computing capability across clustered 
environments for scientific users. It abstracts away the complexities of 
submitting jobs to HPC platforms and provides users with an intuitive and 
elegant web-based interface to submit jobs. Apache Mesos is a  distributed 
kernel that manages distributed computing resources as a single computer. As 
Airavata is being extended to use Big Data and Cloud tools to launch jobs in 
cloud environments, it needs to retrieve the resource and job execution 
information from the Big Data framework back to the Apache portal accessible to 
the end user. In this project we will develop code and scripts to be integrated 
with the Airavata that will use the HTTP API of Mesos to continuously fetch the 
complete resource and scheduling information. This information can then be used 
by Airavata to dynamically monitor and improve its job submission strategy in 
cloud environments such as Jetstream.

Introduction:
Apache Mesos provides HTTP API endpoints for scheduler, executor, internal and 
admin related queries. To fetch information regarding a clustered environment 
that is managed by the Mesos master, the API can be accessed via curl requests 
over HTTP. The response to such requests will be received as well formed json 
document. We will parse the json response and present the information in the 
format desired. The retrieved information will include resource usage, resource 
available for further jobs, job status, time elapsed since the job started, 
etc.  Airavata, in turn, will use this information to determine the resource 
usage, performance of the jobs on a job submission, rapid diagnosis on the 
health of the submitted jobs.

We will use the observer pattern to continuously pull information from Cloud 
and big Data Resource Managers, such as Apache Mesos, to Airavata.

Any comment and suggestions would be very helpful.

Thanks
Pankaj


--
Shameera Rathnayaka

Re: [GSOC Proposal] Cloud based clusters for Apache Airavata

[Pierce, Marlon]

Hi Mangirish, please add your proposal to the GSOC 2016 site.

From: Mangirish Wagle 
mailto:vaglomangir...@gmail.com>>
Reply-To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Date: Thursday, March 17, 2016 at 3:35 PM
To: "dev@airavata.apache.org" 
mailto:dev@airavata.apache.org>>
Subject: [GSOC Proposal] Cloud based clusters for Apache Airavata

Hello Dev Team,

I had the opportunity to interact with Suresh and Shameera wherein we discussed 
an open requirement in Airavata to be addressed. The requirement is to expand 
the capabilities of Apache Airavata to submit jobs to cloud based clusters in 
addition to HPC/ HTC clusters.

The idea is to dynamically provision a cloud cluster in an environment like 
Jetstream, based on the configuration figured out by Airavata, which would be 
operated by a distributed system management software like Mesos. An initial 
high level goals would be:-

  1.  Airavata categorizes certain jobs to be run on cloud based clusters and 
figure out the required hardware config for the cluster.
  2.  The proposed service would provision the cluster with the required 
resources.
  3.  An ansible script would configure a Mesos cluster with the resources 
provisioned.
  4.  Airavata submits the job to the Mesos cluster.
  5.  Mesos then figures out the efficient resource allocation within the 
cluster and runs the job and fetches the result.
  6.  The cluster is then deprovisioned automatically when not in use.

The project would mainly focus on point 2 and 6 above.

To start with, I am currently trying to get a working prototype of setting up 
compute nodes on an openstack environment using JClouds (Targetted for 
Jetstream). Also, I am planning to explore the option of using Openstack Heat 
engine to orchestrate the cluster. However, going ahead Airavata would be 
supporting other clouds like Amazon EC2 or Comet cluster, so we need to have a 
generic solution for achieving the goal.

Another approach which might be efficient in terms of performance and time is 
using a container based clouds using Docker, Kubernetes which would have 
substantially less bootstrap time compared to cloud VMs. This would be a future 
prospect as we may not have all the clusters supporting containerization.

This has been considered as a potential GSOC project and I would be working on 
drafting a proposal on this idea.

Any inputs/ comments/ suggestions would be very helpful.

Best Regards,
Mangirish Wagle

Re: GSOC proposal

[Pierce, Marlon]

You can also think of the first step as a JavaFX version of the PGA.  You 
should look first at the regular user parts rather than the admin interface in 
the PGA.  You could always use the PGA to manage resources and applications, 
but then connect to the same Airavata server via your workflow desktop composer.

From: marpierc mailto:marpi...@iu.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Tuesday, March 22, 2016 at 2:05 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: GSOC proposal

Start with the UI and show how you will map to the API.  Identity management is 
also an important problem for desktop apps to remote services.

From: Bikramjeet Singh 
mailto:singh.bikramje...@gmail.com>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Date: Tuesday, March 22, 2016 at 2:01 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" 
mailto:dev@airavata.apache.org>>
Subject: Re: GSOC proposal

Hello sir,
Is it the UI part or the API implementation that needs to be detailed?

Thanks

On Tuesday, March 22, 2016, Pierce, Marlon 
mailto:marpi...@iu.edu>> wrote:
Hi Bikramjeet,

Thanks for your interest. Your current proposal lacks a lot of detail, and I’m 
concerned if you will be able to accomplish it. You should take a look, if you 
haven’t at another JavaFX project that builds on the Apache Airavata API:

https://github.com/SciGaP/seagrid-rich-client

This will give you more of an idea about what you will need to do. You can use 
this to strengthen your proposal.  Note this client does not include workflow 
support, so you’ll need to go beyond the SEAGrid client.

Marlon


From: Bikramjeet Singh 
>
Reply-To: 
"dev@airavata.apache.org"
 
>
Date: Monday, March 21, 2016 at 12:19 PM
To: Airavata Dev 
>
Subject: Re: GSOC proposal

Hello,
Yes sir I granted commenting permissions.

Thanks

On Mon, Mar 21, 2016 at 6:49 PM, Pierce, Marlon 
> wrote:
Hi Bikramjeet,

Your proposal is a good start. I would like to make some comments inline in the 
google doc.

Thanks,

Marlon


From: Bikramjeet Singh 
>
Reply-To: 
"dev@airavata.apache.org"
 
>
Date: Thursday, March 17, 2016 at 3:08 PM
To: Airavata Dev 
>
Subject: Re: GSOC proposal

Hi,
I've shared the draft through Gsoc application, kindly review it.

Thanks

On Thu, Mar 17, 2016 at 7:17 PM, Suresh Marru 
> wrote:
Can you start the doc in gsoc application itself and choose google doc for 
draft?

Suresh


> On Mar 17, 2016, at 9:44 AM, Bikramjeet Singh 
> >
>  wrote:
>
> Hi,
> I have a GSOC draft proposal ready, where should I share it for review and 
> will the google doc link work?
>
> Thanks,
> Bikramjeet Singh