Dimuthu and Marcus:
This is great feature and long anticipated. Please let me know when ready and I
can test it from dev.seagrid or dev.amp gateways.
Thanks very much for implementing this.
Sudhakar.
From: DImuthu Upeksha <dimuthu.upeks...@gmail.com>
Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
Date: Sunday, January 2, 2022 at 11:04 PM
To: Airavata Dev <dev@airavata.apache.org>
Subject: [Airavata] Fetching intermediate outputs for an running experiment
Hi All
Marcus and I were working on the feature of grabbing output files from an
already running experiment so that users can determine whether the experiment
is running as expected without waiting for it to complete. This will be
tremendously helpful for long-running and resource-intensive jobs in order to
determine possible early terminations in case of an input or convergence issue.
We added a new API [1] to invoke this feature and this will create a separate
process that is only running a subset of output data transfer workflows of the
parent process. A sample client code might look like below.
List<String> outputNames = new ArrayList<>();
outputNames.add("Gaussian-Application-Output");
apiClient.fetchIntermediateOutputs(token,"exp-id",
outputNames,
"PROCESS_98a8c45c-998c-4998-91e4-18ce915ed13e");
Any failure to this new process will not affect the main experiment but users
can see those errors in the experiment statistics page under the new process
section. The attached screenshot shows how the new process might look like on
the statistics page. Output data will be placed in a separate directory named
'intermediate' on the storage side.
I merged this code to dev environment and it is ready for testing purposes.
This PR [2] will reflect all the file changes from API to Helix level.
Marcus,
We might need to add new buttons / UI components to make this available at the
portal level and we can discuss it here if that makes sense.
Thanks and happy new year
Dimuthu
[1]
https://github.com/apache/airavata/blob/AIRAVATA-3549/thrift-interface-descriptions/airavata-apis/airavata_api.thrift#L989<https://urldefense.com/v3/__https:/github.com/apache/airavata/blob/AIRAVATA-3549/thrift-interface-descriptions/airavata-apis/airavata_api.thrift*L989__;Iw!!DZ3fjg!smVvFfydZKB60LP-hCbEQzuIJH_nh4F8NUL5RK45TYZ-ioit6jh9WtJz_wr4q2TLVOA$>
[2]
https://github.com/apache/airavata/pull/276<https://urldefense.com/v3/__https:/github.com/apache/airavata/pull/276__;!!DZ3fjg!smVvFfydZKB60LP-hCbEQzuIJH_nh4F8NUL5RK45TYZ-ioit6jh9WtJz_wr41W5sOW8$>
