[jira] [Commented] (AIRAVATA-3051) Output files configured with wildcard is not brought back and displayed in the Django summary page.
[ https://issues.apache.org/jira/browse/AIRAVATA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854145#comment-16854145 ] Dimuthu Upeksha commented on AIRAVATA-3051: --- Fixed in [https://github.com/apache/airavata/commit/d4a209d042dd36fe9f86a706e783c3c3ee8a6ec8] > Output files configured with wildcard is not brought back and displayed in > the Django summary page. > --- > > Key: AIRAVATA-3051 > URL: https://issues.apache.org/jira/browse/AIRAVATA-3051 > Project: Airavata > Issue Type: Sub-task > Components: Django Portal >Affects Versions: 0.18 > Environment: https://beta-sciencegateway.brylinski.org >Reporter: Eroma >Assignee: Marcus Christie >Priority: Major > Attachments: Screen Shot 2019-05-30 at 8.40.01 AM.png, Screen Shot > 2019-05-30 at 8.40.08 AM.png, Screen Shot 2019-05-30 at 8.42.46 AM.png > > > # Submitted a job with an application where there are 5 output files. > # The defined outputs are STDERR. STDOUT and two specifically defined (full > output file name given) tar files and another tar file defined with wildcard > (*.tar) > # The job was successfully completed and all the files are available in the > working directory. > # In the PGA storage only the files which the defined with the complete name > was brought back. > # STDOUT, STDERR and the wild card output file is not in the storage. > # Hence they are not displayed and downloadable in the summary page. > # Apart from this in output section of the summary page there are paths > given to all the .tar files as a string. > # Please see all the images attached to this. > # exp ID: > eFindSite_on_May_30,_2019_1:38_AM_85ab2ab9-a1c0-427c-9fb3-ccde9facd8f1 > h5. TODO > * [ ] Don't display an experiment output entry when the type if URI but the > value is not a data product URI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRAVATA-3051) Output files configured with wildcard is not brought back and displayed in the Django summary page.
[
https://issues.apache.org/jira/browse/AIRAVATA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853380#comment-16853380
]
Marcus Christie commented on AIRAVATA-3051:
---
So looks like it is failing to transfer wildcards because it is adding the full
path to the wildcard file to the working directory, see:
https://github.com/apache/airavata/blob/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/staging/OutputDataStagingTask.java#L143
See for example this log line from above:
{noformat}
2019-05-30 07:13:20,988 [TaskStateModelFactory-task_thread-2] WARN
o.a.a.h.i.t.s.DataStagingTask process=PROCESS_c1fc1
629-7bd5-41b9-8cd0-dec05603c81a,
task=TASK_6e4300d5-9692-4b24-b692-5248bbd8a9c5,
experiment=eFindSite_on_May_30,_2019_1
:38_AM_85ab2ab9-a1c0-427c-9fb3-ccde9facd8f1, gateway=csbglsu - Ignoring the
file /scratch/michal/airavata-workdirs/PROC
ESS_c1fc1629-7bd5-41b9-8cd0-dec05603c81a//scratch/michal/airavata-workdirs/PROCESS_c1fc1629-7bd5-41b9-8cd0-dec05603c81a
/eFindSite-screen.tar.gz transfer as it is not available
{noformat}
> Output files configured with wildcard is not brought back and displayed in
> the Django summary page.
> ---
>
> Key: AIRAVATA-3051
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3051
> Project: Airavata
> Issue Type: Sub-task
> Components: Django Portal
>Affects Versions: 0.18
> Environment: https://beta-sciencegateway.brylinski.org
>Reporter: Eroma
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Screen Shot 2019-05-30 at 8.40.01 AM.png, Screen Shot
> 2019-05-30 at 8.40.08 AM.png, Screen Shot 2019-05-30 at 8.42.46 AM.png
>
>
> # Submitted a job with an application where there are 5 output files.
> # The defined outputs are STDERR. STDOUT and two specifically defined (full
> output file name given) tar files and another tar file defined with wildcard
> (*.tar)
> # The job was successfully completed and all the files are available in the
> working directory.
> # In the PGA storage only the files which the defined with the complete name
> was brought back.
> # STDOUT, STDERR and the wild card output file is not in the storage.
> # Hence they are not displayed and downloadable in the summary page.
> # Apart from this in output section of the summary page there are paths
> given to all the .tar files as a string.
> # Please see all the images attached to this.
> # exp ID:
> eFindSite_on_May_30,_2019_1:38_AM_85ab2ab9-a1c0-427c-9fb3-ccde9facd8f1
> h5. TODO
> * [ ] Don't display an experiment output entry when the type if URI but the
> value is not a data product URI
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRAVATA-3051) Output files configured with wildcard is not brought back and displayed in the Django summary page.
[ https://issues.apache.org/jira/browse/AIRAVATA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853372#comment-16853372 ] Marcus Christie commented on AIRAVATA-3051: --- I think Helix OutputDataStagingTask is setting the name to the full path instead of just the filename: https://github.com/apache/airavata/blob/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/staging/OutputDataStagingTask.java#L148 And perhaps the transfer is failing? > Output files configured with wildcard is not brought back and displayed in > the Django summary page. > --- > > Key: AIRAVATA-3051 > URL: https://issues.apache.org/jira/browse/AIRAVATA-3051 > Project: Airavata > Issue Type: Sub-task > Components: Django Portal >Affects Versions: 0.18 > Environment: https://beta-sciencegateway.brylinski.org >Reporter: Eroma >Assignee: Marcus Christie >Priority: Major > Attachments: Screen Shot 2019-05-30 at 8.40.01 AM.png, Screen Shot > 2019-05-30 at 8.40.08 AM.png, Screen Shot 2019-05-30 at 8.42.46 AM.png > > > # Submitted a job with an application where there are 5 output files. > # The defined outputs are STDERR. STDOUT and two specifically defined (full > output file name given) tar files and another tar file defined with wildcard > (*.tar) > # The job was successfully completed and all the files are available in the > working directory. > # In the PGA storage only the files which the defined with the complete name > was brought back. > # STDOUT, STDERR and the wild card output file is not in the storage. > # Hence they are not displayed and downloadable in the summary page. > # Apart from this in output section of the summary page there are paths > given to all the .tar files as a string. > # Please see all the images attached to this. > # exp ID: > eFindSite_on_May_30,_2019_1:38_AM_85ab2ab9-a1c0-427c-9fb3-ccde9facd8f1 > h5. TODO > * [ ] Don't display an experiment output entry when the type if URI but the > value is not a data product URI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
