[ 
https://issues.apache.org/jira/browse/AIRAVATA-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17459558#comment-17459558
 ] 

Marcus Christie edited comment on AIRAVATA-3549 at 12/14/21, 11:11 PM:
-----------------------------------------------------------------------

Design of Airavata API server changes:

Airavata API
- void fetchIntermediateOutput(AuthzToken authzToken, String experimentId, 
List[String] outputs)
-- publish message to experiment channel with MessageType.INTERMEDIATE_OUTPUTS
-- ExperimentIntermediateOutputsEvent(experimentId, gatewayId, List[String] 
outputNames)
- OSH.ExperimentHandler.onMessage
-- in handling, check that there isn't already an active OUTPUT_FETCHING task 
for the file(s)
-- we might want a timeout of the OUTPUT_FETCHING task, so we aren't waiting 
forever 
-- create a process, but there are no inputs and only outputs are the specified 
ones
-- also check that experiment isn't already completed
-- create the output data staging tasks, task type OUTPUT_FETCHING
-- publish a PROCESS_LAUNCH event

Helix changes:
-  when getting PROCESS_LAUNCH event, check if it only contains OUTPUT_FETCHING 
tasks. If so, run a new workflow that fetches those output files. 



was (Author: marcuschristie):
Design of Airavata API server changes:

Airavata API

- void fetchIntermediateOutput(AuthzToken authzToken, String experimentId, 
List<String> outputs)
  - publish message to experiment channel with MessageType.INTERMEDIATE_OUTPUTS
  - ExperimentIntermediateOutputsEvent(experimentId, gatewayId, List<String> 
outputNames)
- OSH.ExperimentHandler.onMessage
  - in handling, check that there isn't already an active OUTPUT_FETCHING task 
for the file(s)
  - we might want a timeout of the OUTPUT_FETCHING task, so we aren't waiting 
forever 
  - create a process, but there are no inputs and only outputs are the 
specified ones
  - also check that experiment isn't already completed
  - create the output data staging tasks, task type OUTPUT_FETCHING
  - publish a PROCESS_LAUNCH event

Helix changes:

-  when getting PROCESS_LAUNCH event, check if it only contains OUTPUT_FETCHING 
tasks. If so, run a new workflow that fetches those output files. 

> Fetch intermediate output files
> -------------------------------
>
>                 Key: AIRAVATA-3549
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-3549
>             Project: Airavata
>          Issue Type: New Feature
>          Components: Airavata API, Django Portal, helix implementation
>            Reporter: Marcus Christie
>            Priority: Major
>
> Add fetching of intermediate files for running experiments to the Airavata 
> API. The main use case is for API users to get the output of log files so 
> that they can determine how the experiment is going. An API user may want to 
> cancel a computational job that isn't converging, for example.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Reply via email to