Hello John,
Thank you very much for the very extended response.
It seems I have enough reading to be calm for a month :-).
Thank you again,
Dmitry
On 5/19/2014 4:42 PM, John Chilton wrote:
Hello Dmitry,
Sorry the delayed response. It occurs to me that data source tools may
be an alternative I should have pointed you to right away -
https://wiki.galaxyproject.org/Admin/Internals/DataSources. I am not
sure they are a perfect fit but regular tools are not a perfect fit
either.
On Tue, May 6, 2014 at 4:26 AM, Dmitry wrote:
Hello John,
If it is at all possible to send the data to the web service instead ...
Unfortunately, I cannot push the data to the web service, because this is
the BES one and the interface is defined in the spec.
As far as I understand reading the mailing list there is no way to pull data
through FTP (only upload).
This is correct.
The API key looks interesting even I do not completely understand it.
Is it possible to manipulate the key(s) programmatically?
It would be great to generate a temporal key, execute the service and remove
the key after that.
When you phrase the problem like this it occurs to me I recently added
this functionality to Galaxy. The LWR (a tool to remotely execute
Galaxy tools) can now be configured to pull files from and send files
to Galaxy with single purpose keys tied to particular jobs. The
details are outlined in the following pull request
(https://bitbucket.org/galaxy/galaxy-central/pull-request/327/implement-job-files-api/diff).
Piggy backing on this is probably the right way to go because you
don't have to worry about whether user's have API keys defined - jobs
have implicit keys not users.
If you wanted to go down this road - you will need to throw Galaxy's
job object or at least the id into the context Galaxy uses to evaluate
tool xml files. This patch
https://gist.github.com/jmchilton/ddec237d2ad63821addc for instances
SHOULD give tools access to a $__job_id__ variable (haven't tested
it).
Then you should be able to construct URLs inside of your Galaxy XML
for your inputs and outputs as follows:
#set $encode_job_id = ${__app__.security.encode_id( $__job_id__ )}
#set $job_key = ${__app__.security.encode_id( $__job_id__, kind="jobs_files" ) }
#set $input_path = str($input)
--url
"http://mygalaxy.com/api/files/$encode_job_id/files?job_key=$job_key&path=$input_path&file_type=input";
#set $encode_job_id = ${__app__.security.encode_id( $__job_id__ )}
#set $job_key = ${__app__.security.encode_id( $__job_id__, kind="jobs_files" ) }
#set $output_path = str($output)
--output_url
"http://mygalaxy.com/api/files/$encode_job_id/files?job_key=$job_key&path=$output_path&file_type=output";
This API endpoint I believe will respond to GETs for the inputs and
POSTs for the output.
There are probably some syntax errors in above tool/cheetah snipplets
(Galaxy doesn't formally support this, I am just giving you some
informal advice on how one might proceed).
In general I would discourage this because it is difficult to do in a
generic way ...
I understand that the problem is when Galaxy is behind a proxy. In my case I
control the URL for the galaxy.
${ __app__.model.User.get( $__user_id__ ).api_keys[0].key }
Again, wouldn't need this if you are using the newer job files API endpoint.
Obviously this e-mail is very informal and hand-wavy because I am
likely too busy to walk through and verify all of these steps. But I
have created a Trello card (https://trello.com/c/JjIF9l4h) to keep
track of this feature request and outline this approach, I will try to
find some time to think about this use case some more in the future.
Sorry I don't have more time to give you a more complete response.
Galaxy is generally very focused on wrapping command-line utilities
for analysis and it doesn't necessarily target web services like this
very well.
-John
I absolutely lost here...
should I know $__user_id__ or this is a variable already known to the tool?
Thank you very much for the help,
Dmitry
On 5/6/2014 4:39 AM, John Chilton wrote:
Hello Dmitry,
This use case isn't really addressed by Galaxy currently.
There is not a generic way of doing grabbing the users credentials
like this or producing URLs from inside the tool. This depends on a
lot of different things - how Galaxy's proxy is configured, etc... so
in many (all?) cases Galaxy would not be able to fetch the users raw
credentials. Your best bet is going to be to use the Galaxy's API -
tools can fetch a user's API key and you can use Galaxy's API to build
a URL that includes this key.
Your users will all need API keys set to use the tool. There have been
some other galaxy-dev discussions about using the API from tools -
http://osdir.com/ml/galaxy-development-source-control/2013-12/msg00105.html.
In general I would discourage this because it is difficult to do in a
generic wa
