[galaxy-dev] Fwd: Storing data in the user session

[Katherine Beaulieu]

-- Forwarded message --
From: Katherine Beaulieu 
Date: Thu, Jul 28, 2016 at 4:32 PM
Subject: Re: [galaxy-dev] Storing data in the user session
To: Björn Grüning 


Hi Bjoern,
Do you think you could contact me to discuss what you've done so far? I
indeed did talk to Eric today but if you have already implemented some of
the back-end stuff, I wouldn't want to be repeating your work and I could
potentially work with you on the user interface for this?
Thanks,
Katherine

On Thu, Jul 28, 2016 at 3:17 PM, Björn Grüning 
wrote:

> Hi Eric,
>
> I think they are. What you described is more the user-facing part.
> My current proof-of-concept does not have a tool-user-interface, it just
> stores the credentials in the framework and makes it accessible through
> the user-object. Which should be enough for dynamic job destinations.
> A dedicated user-interface for tools is a nice idea and we can put this
> ontop I think very easily!
>
> Thanks for sharing!
> Bjoern
>
> Am 28.07.2016 um 21:09 schrieb Eric Rasche:
> > Hey Björn,
> >
> > I talked a bit about this with Katherine today, her group is quite
> > interested in working on this as well.
> >
> > I know we had discussed this a bit at GCC, and I posted a message today,
> > outlining my plans which were formulated when I was at PSU the other
> > month. Were those suggestions compatible with the existing work you'd
> > done on generic KV store stuff?
> >
> > Cheers,
> > Eric
> >
> > On 28. juli 2016 18:59, Björn Grüning wrote:
> >> Hi Steve & Mohammed,
> >>
> >> I'm working on a user-profile extension to get credentials for a user,
> >> but this will take a few more weeks sorry.
> >>
> >> In the meantime can you look at the following idea:
> >>
> >> my_programm.py --credentials $USER_CREDENTIALS --arguments
> >>
> >> The variable $USER_CREDENTIALS is set during job execution with dynamic
> >> destination mappings
> >>
> https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping
> >>
> >> What do you think, would this work for you?
> >> This assumes you as admin can pass along the user credentials if you
> >> know the username.
> >>
> >> If not please get in touch and I will tell you about my plans to extend
> >> the user-preferences maybe you can help me with this to speed things up.
> >>
> >> Ciao,
> >> Bjoern
> >>
> >>
> >>> Hi Bjorn,
> >>>  sure, I don’t want to have the credentials stored in the tool but the
> tool
> >>> needs to act on behalf of the user to retrieve data from the
> repository, so
> >>> it needs access to credentials.  This could for example be a token
> generated
> >>> via an OAuth2 exchange - basically something that is sent along with
> every
> >>> http request to the remote resource to authenticate the user.
> >>>
> >>> My thought is that this could be stored in the user profile if this
> can be made
> >>> available to the tool in the same way that __user_email__ is now.
> >>>
> >>> When you say ‘on the framework level’ do you mean by Galaxy?
> >>>
> >>> Steve
> >>> —
> >>> Department of Computing, Macquarie University
> >>> http://web.science.mq.edu.au/~cassidy
> >>>
>  On 28 Jul 2016, at 5:23 PM, Björn Grüning 
> wrote:
> 
>  Hi Steve,
> 
>  can you explain the larger use-case behind it. Imho you should not
>  populate a tool with credentials, this should be handled on the
>  framework level.
> 
>  Hope you are fine,
>  Bjoern
> 
>  Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
> > Hi again,
> >   I’m looking for the right way to store some user credentials in the
> > galaxy session so that tools can work on behalf of the user with our
> > repository.
> >
> > Currently users have an API key and they need to upload it as a data
> > item to that is then passed to each tool that needs it as input.
> This
> > doesn’t seem like the right solution since the API key becomes part
> of
> > the history and so would be shared if the history were shared.
> >
> > What would be better would be a way of storing the API key in the
> user
> > session and then being able to pass that into a tool.
> >
> > I note that there are a few user session variables available in the
> tool
> > xml file: __user__, __user_email__, __user_name__ and __user_id__.
> > There is also a user preferences page where they can fill out a few
> > details.  However, I can’t see a mechanism to extend this in any way
> and
> > have extra properties in the preferences pane that would then be
> > available to tools via the template file.
> >
> > I can see that the standard practice for the data sources on
> > usegalaxy.org  is to send the user off to a
> > repository website to find data that is then posted back to the
> galaxy
> > server.  This isn’t appropriate for us since the interaction with the
> > repository is not just for downloading a single dataset - tools will
> > query t

[galaxy-dev] Information about SparkGalaxy

[Léo Biscassi]

Hi folks,
I recently saw this tweet [1] about SparkGalaxy. I'm working in one project
which uses spark to process protein-ligand docking data and would like try
use this workflow in my project, can anyone give me more information, e.g.
where I can download it or if it's built in galaxy implementation, about?
I'm pretty excited, this is a great job!

[1] https://twitter.com/galaxyproject/status/758727733643874304

Thanks for your attention!
-- 
Best regards,
*Léo Biscassi*
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Storing data in the user session

[Björn Grüning]

Hi Eric,

I think they are. What you described is more the user-facing part.
My current proof-of-concept does not have a tool-user-interface, it just
stores the credentials in the framework and makes it accessible through
the user-object. Which should be enough for dynamic job destinations.
A dedicated user-interface for tools is a nice idea and we can put this
ontop I think very easily!

Thanks for sharing!
Bjoern

Am 28.07.2016 um 21:09 schrieb Eric Rasche:
> Hey Björn,
> 
> I talked a bit about this with Katherine today, her group is quite
> interested in working on this as well.
> 
> I know we had discussed this a bit at GCC, and I posted a message today,
> outlining my plans which were formulated when I was at PSU the other
> month. Were those suggestions compatible with the existing work you'd
> done on generic KV store stuff?
> 
> Cheers,
> Eric
> 
> On 28. juli 2016 18:59, Björn Grüning wrote:
>> Hi Steve & Mohammed,
>>
>> I'm working on a user-profile extension to get credentials for a user,
>> but this will take a few more weeks sorry.
>>
>> In the meantime can you look at the following idea:
>>
>> my_programm.py --credentials $USER_CREDENTIALS --arguments
>>
>> The variable $USER_CREDENTIALS is set during job execution with dynamic
>> destination mappings
>> https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping
>>
>> What do you think, would this work for you?
>> This assumes you as admin can pass along the user credentials if you
>> know the username.
>>
>> If not please get in touch and I will tell you about my plans to extend
>> the user-preferences maybe you can help me with this to speed things up.
>>
>> Ciao,
>> Bjoern
>>
>>
>>> Hi Bjorn,
>>>  sure, I don’t want to have the credentials stored in the tool but the tool 
>>> needs to act on behalf of the user to retrieve data from the repository, so
>>> it needs access to credentials.  This could for example be a token generated
>>> via an OAuth2 exchange - basically something that is sent along with every 
>>> http request to the remote resource to authenticate the user.  
>>>
>>> My thought is that this could be stored in the user profile if this can be 
>>> made
>>> available to the tool in the same way that __user_email__ is now. 
>>>
>>> When you say ‘on the framework level’ do you mean by Galaxy? 
>>>
>>> Steve
>>> —
>>> Department of Computing, Macquarie University
>>> http://web.science.mq.edu.au/~cassidy
>>>
 On 28 Jul 2016, at 5:23 PM, Björn Grüning  
 wrote:

 Hi Steve,

 can you explain the larger use-case behind it. Imho you should not
 populate a tool with credentials, this should be handled on the
 framework level.

 Hope you are fine,
 Bjoern

 Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
> Hi again,
>   I’m looking for the right way to store some user credentials in the
> galaxy session so that tools can work on behalf of the user with our
> repository. 
>
> Currently users have an API key and they need to upload it as a data
> item to that is then passed to each tool that needs it as input.  This
> doesn’t seem like the right solution since the API key becomes part of
> the history and so would be shared if the history were shared.  
>
> What would be better would be a way of storing the API key in the user
> session and then being able to pass that into a tool.
>
> I note that there are a few user session variables available in the tool
> xml file: __user__, __user_email__, __user_name__ and __user_id__.
> There is also a user preferences page where they can fill out a few
> details.  However, I can’t see a mechanism to extend this in any way and
> have extra properties in the preferences pane that would then be
> available to tools via the template file.  
>
> I can see that the standard practice for the data sources on
> usegalaxy.org  is to send the user off to a
> repository website to find data that is then posted back to the galaxy
> server.  This isn’t appropriate for us since the interaction with the
> repository is not just for downloading a single dataset - tools will
> query the repository, download various kinds of data and possibly upload
> new data, so we need to store user credentials in some way. 
>
> Is there already a good way to achieve this or is this an enhancement to
> Galaxy? 
>
> Thanks,
>
> Steve
>
>
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
> 
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
>

Re: [galaxy-dev] Storing data in the user session

[Eric Rasche]

Hey Björn,

I talked a bit about this with Katherine today, her group is quite
interested in working on this as well.

I know we had discussed this a bit at GCC, and I posted a message today,
outlining my plans which were formulated when I was at PSU the other
month. Were those suggestions compatible with the existing work you'd
done on generic KV store stuff?

Cheers,
Eric

On 28. juli 2016 18:59, Björn Grüning wrote:
> Hi Steve & Mohammed,
>
> I'm working on a user-profile extension to get credentials for a user,
> but this will take a few more weeks sorry.
>
> In the meantime can you look at the following idea:
>
> my_programm.py --credentials $USER_CREDENTIALS --arguments
>
> The variable $USER_CREDENTIALS is set during job execution with dynamic
> destination mappings
> https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping
>
> What do you think, would this work for you?
> This assumes you as admin can pass along the user credentials if you
> know the username.
>
> If not please get in touch and I will tell you about my plans to extend
> the user-preferences maybe you can help me with this to speed things up.
>
> Ciao,
> Bjoern
>
>
>> Hi Bjorn,
>>  sure, I don’t want to have the credentials stored in the tool but the tool 
>> needs to act on behalf of the user to retrieve data from the repository, so
>> it needs access to credentials.  This could for example be a token generated
>> via an OAuth2 exchange - basically something that is sent along with every 
>> http request to the remote resource to authenticate the user.  
>>
>> My thought is that this could be stored in the user profile if this can be 
>> made
>> available to the tool in the same way that __user_email__ is now. 
>>
>> When you say ‘on the framework level’ do you mean by Galaxy? 
>>
>> Steve
>> —
>> Department of Computing, Macquarie University
>> http://web.science.mq.edu.au/~cassidy
>>
>>> On 28 Jul 2016, at 5:23 PM, Björn Grüning  wrote:
>>>
>>> Hi Steve,
>>>
>>> can you explain the larger use-case behind it. Imho you should not
>>> populate a tool with credentials, this should be handled on the
>>> framework level.
>>>
>>> Hope you are fine,
>>> Bjoern
>>>
>>> Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
 Hi again,
   I’m looking for the right way to store some user credentials in the
 galaxy session so that tools can work on behalf of the user with our
 repository. 

 Currently users have an API key and they need to upload it as a data
 item to that is then passed to each tool that needs it as input.  This
 doesn’t seem like the right solution since the API key becomes part of
 the history and so would be shared if the history were shared.  

 What would be better would be a way of storing the API key in the user
 session and then being able to pass that into a tool.

 I note that there are a few user session variables available in the tool
 xml file: __user__, __user_email__, __user_name__ and __user_id__.
 There is also a user preferences page where they can fill out a few
 details.  However, I can’t see a mechanism to extend this in any way and
 have extra properties in the preferences pane that would then be
 available to tools via the template file.  

 I can see that the standard practice for the data sources on
 usegalaxy.org  is to send the user off to a
 repository website to find data that is then posted back to the galaxy
 server.  This isn’t appropriate for us since the interaction with the
 repository is not just for downloading a single dataset - tools will
 query the repository, download various kinds of data and possibly upload
 new data, so we need to store user credentials in some way. 

 Is there already a good way to achieve this or is this an enhancement to
 Galaxy? 

 Thanks,

 Steve


 —
 Department of Computing, Macquarie University
 http://web.science.mq.edu.au/~cassidy
 



 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

-- 

Eric Rasche
Programmer II

Center for Phage Technology
Rm 312A, BioBio
Texas A&M University
Col

Re: [galaxy-dev] Storing data in the user session

[Björn Grüning]

Hi Steve & Mohammed,

I'm working on a user-profile extension to get credentials for a user,
but this will take a few more weeks sorry.

In the meantime can you look at the following idea:

my_programm.py --credentials $USER_CREDENTIALS --arguments

The variable $USER_CREDENTIALS is set during job execution with dynamic
destination mappings
https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping

What do you think, would this work for you?
This assumes you as admin can pass along the user credentials if you
know the username.

If not please get in touch and I will tell you about my plans to extend
the user-preferences maybe you can help me with this to speed things up.

Ciao,
Bjoern


> Hi Bjorn,
>  sure, I don’t want to have the credentials stored in the tool but the tool 
> needs to act on behalf of the user to retrieve data from the repository, so
> it needs access to credentials.  This could for example be a token generated
> via an OAuth2 exchange - basically something that is sent along with every 
> http request to the remote resource to authenticate the user.  
> 
> My thought is that this could be stored in the user profile if this can be 
> made
> available to the tool in the same way that __user_email__ is now. 
> 
> When you say ‘on the framework level’ do you mean by Galaxy? 
> 
> Steve
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
> 
>> On 28 Jul 2016, at 5:23 PM, Björn Grüning  wrote:
>>
>> Hi Steve,
>>
>> can you explain the larger use-case behind it. Imho you should not
>> populate a tool with credentials, this should be handled on the
>> framework level.
>>
>> Hope you are fine,
>> Bjoern
>>
>> Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
>>> Hi again,
>>>   I’m looking for the right way to store some user credentials in the
>>> galaxy session so that tools can work on behalf of the user with our
>>> repository. 
>>>
>>> Currently users have an API key and they need to upload it as a data
>>> item to that is then passed to each tool that needs it as input.  This
>>> doesn’t seem like the right solution since the API key becomes part of
>>> the history and so would be shared if the history were shared.  
>>>
>>> What would be better would be a way of storing the API key in the user
>>> session and then being able to pass that into a tool.
>>>
>>> I note that there are a few user session variables available in the tool
>>> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>>> There is also a user preferences page where they can fill out a few
>>> details.  However, I can’t see a mechanism to extend this in any way and
>>> have extra properties in the preferences pane that would then be
>>> available to tools via the template file.  
>>>
>>> I can see that the standard practice for the data sources on
>>> usegalaxy.org  is to send the user off to a
>>> repository website to find data that is then posted back to the galaxy
>>> server.  This isn’t appropriate for us since the interaction with the
>>> repository is not just for downloading a single dataset - tools will
>>> query the repository, download various kinds of data and possibly upload
>>> new data, so we need to store user credentials in some way. 
>>>
>>> Is there already a good way to achieve this or is this an enhancement to
>>> Galaxy? 
>>>
>>> Thanks,
>>>
>>> Steve
>>>
>>>
>>> —
>>> Department of Computing, Macquarie University
>>> http://web.science.mq.edu.au/~cassidy
>>> 
>>>
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>  https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>  http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] possible bug found - unable to log out of impersonated user when external authentication enabled

[Dannon Baker]

The good news is that I found out why this sounds familiar.  The bad news
is that I committed a fix for this way back in 14.08, here:
https://github.com/galaxyproject/galaxy/commit/5769ea88fa90f3f7c22505cd06aa6f83f940a1e,
which is still not working for you (though it does work for my basic
external auth configuration even back on 15.10).

The only thing that comes to mind is that somehow your remote_user_email
isn't matching up with the admin_users list that's generated here.  Are you
using remote_user_maildomain (this actually does not work correctly for me
in 15.10, so I was unable to test that configuration, though I could see
why it might prevent admin_user from matching)?  Any chance you want to add
a debugging statement or two there and look in the logs to see why that URL
isn't being whitelisted for your user?


On Thu, Jul 28, 2016 at 10:50 AM Ryan G  wrote:

> Hi Dannon - We are using 15.10.
>
> We have an Apache Proxy server sitting in front of Galaxy and handling the
> SSO.  There is no non-standard routing in place.   Yes, when I navigate to
> the /user/logout url, I get the "Access to Galaxy user controls is
> disabled" message.
>
>
> On Thu, Jul 28, 2016 at 12:16 AM, Dannon Baker 
> wrote:
>
>> I did a bit of digging here, and it looks to me like /user/logout should
>> be whitelisted in the middleware, and shouldn't present that "Access to
>> Galaxy user controls is disabled".
>>
>> I might need a little more information.  What revision of Galaxy are you
>> running?  Are you serving Galaxy at a subdomain or is any other nonstandard
>> routing in place?  As an admin user, if you just directly navigate
>> to http:///user/logout, you get the same
>> path-blocked error message?
>>
>> On Wed, Jul 27, 2016 at 1:15 PM Ryan G 
>> wrote:
>>
>>> Thanks.  Clearing out the browser history resolved it but there should
>>> be a better way...
>>>
>>> On Wed, Jul 27, 2016 at 12:48 PM, Dannon Baker 
>>> wrote:
>>>
 Hey, thanks for this.  I've made an issue (
 https://github.com/galaxyproject/galaxy/issues/2671)  and will check
 it out today.

 On Wed, Jul 27, 2016 at 12:44 PM Ryan G 
 wrote:

> Hi all - We are using a SSO system so that user's authenticate
> externally from Galaxy.
>
> when I impersonate a user then try to log out as that user, I'm unable
> to.  I get the page "
> Access to Galaxy user controls is disabled
>
> User controls are disabled when Galaxy is configured for external
> authentication."
>
>
> If I close the browser down, and restart, I get my SSO log in page,
> but when I get to Galaxy, I'm still impersonating the user.
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/


>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] possible bug found - unable to log out of impersonated user when external authentication enabled

[Ryan G]

Hi Dannon - We are using 15.10.

We have an Apache Proxy server sitting in front of Galaxy and handling the
SSO.  There is no non-standard routing in place.   Yes, when I navigate to
the /user/logout url, I get the "Access to Galaxy user controls is
disabled" message.


On Thu, Jul 28, 2016 at 12:16 AM, Dannon Baker 
wrote:

> I did a bit of digging here, and it looks to me like /user/logout should
> be whitelisted in the middleware, and shouldn't present that "Access to
> Galaxy user controls is disabled".
>
> I might need a little more information.  What revision of Galaxy are you
> running?  Are you serving Galaxy at a subdomain or is any other nonstandard
> routing in place?  As an admin user, if you just directly navigate
> to http:///user/logout, you get the same
> path-blocked error message?
>
> On Wed, Jul 27, 2016 at 1:15 PM Ryan G 
> wrote:
>
>> Thanks.  Clearing out the browser history resolved it but there should be
>> a better way...
>>
>> On Wed, Jul 27, 2016 at 12:48 PM, Dannon Baker 
>> wrote:
>>
>>> Hey, thanks for this.  I've made an issue (
>>> https://github.com/galaxyproject/galaxy/issues/2671)  and will check it
>>> out today.
>>>
>>> On Wed, Jul 27, 2016 at 12:44 PM Ryan G 
>>> wrote:
>>>
 Hi all - We are using a SSO system so that user's authenticate
 externally from Galaxy.

 when I impersonate a user then try to log out as that user, I'm unable
 to.  I get the page "
 Access to Galaxy user controls is disabled

 User controls are disabled when Galaxy is configured for external
 authentication."


 If I close the browser down, and restart, I get my SSO log in page, but
 when I get to Galaxy, I'm still impersonating the user.


 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
>>>
>>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Trying to understand python dependencies

[Tony Schreiner]

Thanks
I'm not worried about it replacing the system python. In general, I am just
trying to learn more about how environments are set for jobs.

Tony

On Wed, Jul 27, 2016 at 10:23 PM, Raymond Wan  wrote:

> Hi Anthony,
>
>
> On Wed, Jul 27, 2016 at 11:16 PM, Tony Schreiner
>  wrote:
> > I have galaxy server 15.10 running on Ubuntu 14.04, which has python
> 2.7.5.
> >
> > I was in the process of adding matplotlib from the toolchest. It has a
> > dependency on package_python_2_7.
> >
> > Before proceeding, I'd like to understand a bit how that works. Does that
> > install a whole new python, and how then does it get invoked?
>
>
> I have a limited understanding of Galaxy compared to Gildas and many
> others here, but it seems that Galaxy and its tools are separated from
> the rest of the system.  So, installing a tool won't affect the system
> unless your users have set a PATH to Galaxy's tool directory.
>
> For example, when it installs the bwa tool, a bwa executable will get
> installed deep within Galaxy's directories.  But as long as your users
> set their PATH to /usr/bin or /usr/local/bin or wherever you may have
> bwa installed, they'll be fine.  We have this setup where the server
> that runs Galaxy also serves users via ssh.  So far, so good --
> Galaxy's tool installations don't seem to affect the system's packages
> (whether it is something minor like bwa or major like Python).  I'm
> afraid I don't know how it is invoked behind-the-scenes, though.
>
> The only issue I've encountered so far is the other way.  Software
> installation into the system affects Galaxy.  This was the case with
> Samtools and it happened to me just a couple of months ago.  You can
> take a look at my question and the many helpful replies I got here:
>
> http://dev.list.galaxyproject.org/Unsorted-BAM-file-gt-Galaxy-crash-td4669457.html
> .  It seems to be the case of just samtools for me (so far).
>
> I hoe this helps!
>
> Ray
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Password Parameter Opinions

[Eric Rasche]

Hi Katherine,

Not controversial at all ;) different sub-groups of the galaxy community
just have different needs. I'm in one of those that needs this, but I
likewise agree with the devs that a "password" field is not an optimal
solution.

I've proposed something a while back which might achieve what you need,
but it is a long ways (~1-2 months on my current calendar) from
implementation which I realise isn't very helpful right now. (If there
are people who are motivated for this idea, I would be happy to work
with them on this. I'm more interested in the functionality than I am in
any credit for implementing it ;) )

I'm proposing a framework for "Credentials for Remote Services" as this
is a pretty generic problem. The gist of it is:

  * There are tools which need user provided credentials for talking to
remote services.
  * The tool should be able to register that it would like all
credentials under a certain namespace (e.g. a tool in my apollo
suite would want all credentials under the "apollo" path). E.g. in a
tool:
  o The command block uses something like "my_script.sh
$apollo_password"
  o The inputs contains something like ''
  o And perhaps somewhere else in the tool we have
'
  o Galaxy, upon seeing the credentials block, would expose a user
configuration setting, allowing the user to register as many
services as they want with this form for entering credentials.
  o The tool, upon looking through the user's configured credentials
in Galaxy, would allow the user to select which credentials they
wish to use.

I am not tackling the encryption portion too heavily, credentials would
be encrypted with the galaxy id_secret going into the database (and
probably salted as well), and then exposed as environment variables orso
when the job is constructed. Anything past that is increasingly less
important for me, any sys admin running Galaxy has access to do much
more malicious things anyway.

There are a couple of specifics I still need to work out (mostly syntax
things for how tools should declare these variables, and how they should
consume them) but I think this would be a much more generic option and
allow for more exciting things in the future. This is what I envision
people selecting in the UI.


Regards,
Eric

On 28. juli 2016 10:53, Katherine Beaulieu wrote:
> Hi Everyone,
> I know this is sort of a controversial topic so I will try to tread
> lightly. It seems that for a couple of years the idea of a password
> parameter for Galaxy tools has come up a lot but has also been
> rejected a lot although I believe there is a lot of merit to it. I
> have found at least two other times where people have tried to
> implement this and it was rejected, but there must have been a reason
> why they needed that parameter. My situation currently is that I have
> created two tools in Galaxy which communicate with another software
> called iRODS which needs authentication from the user. In my own
> department, there is another tool that was created which needed
> authentication from the user but the developer had to create some
> other workaround for this problem. 
>
> In terms of the work I have done for this so far, I have been working
> off the latest release version of Galaxy. I have implemented the field
> so that it is obfuscated in the tool form view as well as on the tool
> info page. I will next be looking into storing the parameter in an
> environment variable so it doesn't get passed through the command line
> and encrypting the password before it enters the database. Speaking
> with a co-worker of mine, we discussed the possibility of implementing
> two option, irreversible encryption of the password as it gets entered
> into the database but then making the tool not workflow compatible, or
> using regular encryption and making the tool workflow compatible. I
> look forward to hearing your opinions and ideas.
>
> Katherine
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

-- 
Eric Rasche
Programmer II

Center for Phage Technology
Rm 312A, Biochemistry & Biophysics
Texas A&M University
College Station, TX 77843
e...@tamu.edu 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] TestToolShed down?

[Peter Cock]

For me too, thanks!

Peter

On Thu, Jul 28, 2016 at 11:06 AM, Nicola Soranzo  wrote:
> Works for me now.
>
> Cheers,
> Nicola
>
>
> On 28/07/16 09:44, Peter Cock wrote:
>>
>> Hello all,
>>
>> The main Tool Shed seems fine, but the Test Tool Shed is not:
>> https://testtoolshed.g2.bx.psu.edu/
>>
>>> Galaxy
>>>
>>> The Tool Shed could not be reached
>>>
>>> You are seeing this message because a request to the
>>> Tool Shed timed out or was refused. This may be a temporary
>>> issue which could be resolved by retrying the operation you
>>> were performing. If you receive this message repeatedly or
>>> for an extended amount of time, please report it to the
>>> Galaxy Team at galaxy-b...@bx.psu.edu with details on
>>> what you were trying to do and the URL in the address bar.
>>> More information on the current state of Galaxy services
>>> may be available on the Galaxy status page or the
>>> @galaxyproject Twitter feed.
>>
>> The https://status.galaxyproject.org/ page reports it should
>> be operational.
>>
>> It responds to ping (same server name and IP address as the
>> main Tool Shed):
>>
>>
>> $ ping -c 4 testtoolshed.g2.bx.psu.edu
>> PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
>> ttl=45 time=130 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
>> ttl=45 time=129 ms
>> --- radegast.galaxyproject.org ping statistics ---
>> 4 packets transmitted, 4 received, 0% packet loss, time 3133ms
>> rtt min/avg/max/mdev = 129.744/129.955/130.358/0.348 ms
>>
>>
>> $ ping -c 4 toolshed.g2.bx.psu.edu
>> PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
>> ttl=45 time=130 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
>> ttl=45 time=130 ms
>> --- radegast.galaxyproject.org ping statistics ---
>> 4 packets transmitted, 4 received, 0% packet loss, time 3135ms
>> rtt min/avg/max/mdev = 129.785/129.977/130.196/0.179 ms
>>
>>
>>
>> Peter
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Password Parameter Opinions

[Mohamed Kassam]

Hello Katherine,
You will save a lot the work some people are doing now.

You are not alone to face the same problem. We discussed last time about
that, iRODS, etc...

I think it is a good idea to emcrypt the password  before it enters the
database. That Galaxy don't keep any trace of user name or password on the
system.
I don't know how much it is difficult for that. I tried to modify the
databsae but I had some issues. So I updated this morning the new version
of Galaxy. I will check if I can manage.
Let know us what change you made by putting on the github.

Thanks in advance,

Mohamed

2016-07-28 12:53 GMT+02:00 Katherine Beaulieu <
katherine.beaulieu...@gmail.com>:

> Hi Everyone,
> I know this is sort of a controversial topic so I will try to tread
> lightly. It seems that for a couple of years the idea of a password
> parameter for Galaxy tools has come up a lot but has also been rejected a
> lot although I believe there is a lot of merit to it. I have found at least
> two other times where people have tried to implement this and it was
> rejected, but there must have been a reason why they needed that parameter.
> My situation currently is that I have created two tools in Galaxy which
> communicate with another software called iRODS which needs authentication
> from the user. In my own department, there is another tool that was created
> which needed authentication from the user but the developer had to create
> some other workaround for this problem.
>
> In terms of the work I have done for this so far, I have been working off
> the latest release version of Galaxy. I have implemented the field so that
> it is obfuscated in the tool form view as well as on the tool info page. I
> will next be looking into storing the parameter in an environment variable
> so it doesn't get passed through the command line and encrypting the
> password before it enters the database. Speaking with a co-worker of mine,
> we discussed the possibility of implementing two option, irreversible
> encryption of the password as it gets entered into the database but then
> making the tool not workflow compatible, or using regular encryption and
> making the tool workflow compatible. I look forward to hearing your
> opinions and ideas.
>
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Password Parameter Opinions

[Katherine Beaulieu]

Hi Everyone,
I know this is sort of a controversial topic so I will try to tread
lightly. It seems that for a couple of years the idea of a password
parameter for Galaxy tools has come up a lot but has also been rejected a
lot although I believe there is a lot of merit to it. I have found at least
two other times where people have tried to implement this and it was
rejected, but there must have been a reason why they needed that parameter.
My situation currently is that I have created two tools in Galaxy which
communicate with another software called iRODS which needs authentication
from the user. In my own department, there is another tool that was created
which needed authentication from the user but the developer had to create
some other workaround for this problem.

In terms of the work I have done for this so far, I have been working off
the latest release version of Galaxy. I have implemented the field so that
it is obfuscated in the tool form view as well as on the tool info page. I
will next be looking into storing the parameter in an environment variable
so it doesn't get passed through the command line and encrypting the
password before it enters the database. Speaking with a co-worker of mine,
we discussed the possibility of implementing two option, irreversible
encryption of the password as it gets entered into the database but then
making the tool not workflow compatible, or using regular encryption and
making the tool workflow compatible. I look forward to hearing your
opinions and ideas.

Katherine
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] TestToolShed down?

[Nicola Soranzo]

Works for me now.

Cheers,
Nicola

On 28/07/16 09:44, Peter Cock wrote:

Hello all,

The main Tool Shed seems fine, but the Test Tool Shed is not:
https://testtoolshed.g2.bx.psu.edu/


Galaxy

The Tool Shed could not be reached

You are seeing this message because a request to the
Tool Shed timed out or was refused. This may be a temporary
issue which could be resolved by retrying the operation you
were performing. If you receive this message repeatedly or
for an extended amount of time, please report it to the
Galaxy Team at galaxy-b...@bx.psu.edu with details on
what you were trying to do and the URL in the address bar.
More information on the current state of Galaxy services
may be available on the Galaxy status page or the
@galaxyproject Twitter feed.

The https://status.galaxyproject.org/ page reports it should
be operational.

It responds to ping (same server name and IP address as the
main Tool Shed):


$ ping -c 4 testtoolshed.g2.bx.psu.edu
PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
ttl=45 time=130 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
ttl=45 time=129 ms
--- radegast.galaxyproject.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3133ms
rtt min/avg/max/mdev = 129.744/129.955/130.358/0.348 ms


$ ping -c 4 toolshed.g2.bx.psu.edu
PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
ttl=45 time=130 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
ttl=45 time=130 ms
--- radegast.galaxyproject.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3135ms
rtt min/avg/max/mdev = 129.785/129.977/130.196/0.179 ms



Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] workflow problems

[Cristel Thomas]

Ha good to know. We're at 16.01. I was waiting for 16.07 so I wouldn't have
to upgrade twice in a month.
Thank you.

On Wednesday, July 27, 2016, Dannon Baker  wrote:

> Yep, thought it was familiar and that we'd fixed it.  Cristel, let us know
> if you have trouble with upgrading -- I just confirmed that this isn't an
> issue in the current 16.04 release or on the dev branch.
>
> On Wed, Jul 27, 2016 at 10:27 PM Aysam Guerler  > wrote:
>
>> I remember this issue and that it was fixed. Updating to a newer revision
>> should resolve it.
>>
>> On Wed, Jul 27, 2016 at 10:06 PM, Dannon Baker > > wrote:
>>
>>> This is indeed weird behavior.  What revision is your Galaxy at?
>>>
>>> On Wed, Jul 27, 2016 at 9:07 PM Cristel Thomas >> > wrote:
>>>
 Hi all,
 I ran into a weird behavior on my local instance of galaxy and I was
 wondering if anyone had the same experience (and if so what did you do to
 fix it?).

 When trying to run a workflow by clicking on the workflow name, it
 works. When trying to run the workflow by clicking on the arrow next to the
 name and then 'run', or by clicking on run from the workflow editor, it
 takes me to the interactive tour page.

 Anyone else?

 Thanks,
 Cris
 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] TestToolShed down?

[Peter Cock]

Hello all,

The main Tool Shed seems fine, but the Test Tool Shed is not:
https://testtoolshed.g2.bx.psu.edu/

> Galaxy
>
> The Tool Shed could not be reached
>
> You are seeing this message because a request to the
> Tool Shed timed out or was refused. This may be a temporary
> issue which could be resolved by retrying the operation you
> were performing. If you receive this message repeatedly or
> for an extended amount of time, please report it to the
> Galaxy Team at galaxy-b...@bx.psu.edu with details on
> what you were trying to do and the URL in the address bar.
> More information on the current state of Galaxy services
> may be available on the Galaxy status page or the
> @galaxyproject Twitter feed.

The https://status.galaxyproject.org/ page reports it should
be operational.

It responds to ping (same server name and IP address as the
main Tool Shed):


$ ping -c 4 testtoolshed.g2.bx.psu.edu
PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
ttl=45 time=130 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
ttl=45 time=129 ms
--- radegast.galaxyproject.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3133ms
rtt min/avg/max/mdev = 129.744/129.955/130.358/0.348 ms


$ ping -c 4 toolshed.g2.bx.psu.edu
PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
ttl=45 time=130 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
ttl=45 time=130 ms
--- radegast.galaxyproject.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3135ms
rtt min/avg/max/mdev = 129.785/129.977/130.196/0.179 ms



Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Storing data in the user session

[Mohamed Kassam]

Hi Steve
so what you can do create a web authentification tool with Galaxy and bring
back the information as it is done with the ucsc genopme browser.
Maybe it should work, then you don't need to show the password in the
history .

Mohamed

2016-07-28 10:17 GMT+02:00 Steve Cassidy :

> Hi Mohamed,
>
> The problem with this solution is that the password again becomes part of
> the history since it is one of the inputs to the tool.  If I want to
> publish this workflow then the password can’t be visible.  For this reason
> password storage needs to be outside of workflows and handled somewhere
> else in Galaxy.
>
> Steve
>
>
>
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
> On 28 Jul 2016, at 6:04 PM, Mohamed Kassam  wrote:
>
> Hi Steve,
> there is another solution is that the user is giving his credential via
> the tool. The only thing is missing the tag password on the xml then you
> can encrypt the password.
> I tried to add the new paramter on the build.py but I have some errors,
> don't know if someone arrived to fixed it.
>
> Mohamed
>
> 2016-07-28 10:00 GMT+02:00 Steve Cassidy :
>
>> Hi Mohamed,
>>In my case, the credentials on Galaxy are not generally the same as on
>> the remote system so we need some way to store a second set of credentials
>> in the galaxy session.
>>
>> So, for example, user Fred has an account on Galaxy (f...@here.com) and
>> wants to get data from Alveo as input to a workflow. For the Alveo tools to
>> get data, they need Fred’s access token from Alveo which Fred can retrieve
>> by logging in to the Alveo website.  Fred enters the access key into his
>> preferences in Galaxy.  This key is then passed to one or more tools that
>> access Alveo on his behalf:
>>
>>  - the query tool finds words matching a given pattern from a spoken
>> corpus (needs the token)
>>  - the get-data tool retrieves the audio files for each word (needs the
>> token)
>>  - the automatic alignment tool finds the locations of the vowel sounds
>> in each word
>>  - the annotation storer tool sends these locations back to Alveo as a
>> new annotation set (needs the token)
>>
>> and so on, at various times in the workflow, the tools need to talk to
>> the repository to query, get data or add new data.
>>
>> At the moment, I need the token to be a data item in the history, but it
>> would be better if it were part of the user session in Galaxy.
>>
>> Steve
>>
>>
>> —
>> Department of Computing, Macquarie University
>> http://web.science.mq.edu.au/~cassidy
>>
>> On 28 Jul 2016, at 5:46 PM, Mohamed Kassam  wrote:
>>
>> Hello Cassidy,
>> I have the same problem like you because I need the user credential to
>> access to an external tools.
>> I tried the user session variables available in the tool
>>  xml file: __user__, __user_email__, __user_name__ and __user_id__. I t
>> works perfectly I don't know if you need the password, because this is the
>> only information is missing.
>>
>> Mohamed
>>
>> 2016-07-28 9:42 GMT+02:00 Steve Cassidy :
>>
>>> Hi Bjorn,
>>>  sure, I don’t want to have the credentials stored in the tool but the
>>> tool
>>> needs to act on behalf of the user to retrieve data from the repository,
>>> so
>>> it needs access to credentials.  This could for example be a token
>>> generated
>>> via an OAuth2 exchange - basically something that is sent along with
>>> every
>>> http request to the remote resource to authenticate the user.
>>>
>>> My thought is that this could be stored in the user profile if this can
>>> be made
>>> available to the tool in the same way that __user_email__ is now.
>>>
>>> When you say ‘on the framework level’ do you mean by Galaxy?
>>>
>>> Steve
>>> —
>>> Department of Computing, Macquarie University
>>> http://web.science.mq.edu.au/~cassidy
>>>
>>> > On 28 Jul 2016, at 5:23 PM, Björn Grüning 
>>> wrote:
>>> >
>>> > Hi Steve,
>>> >
>>> > can you explain the larger use-case behind it. Imho you should not
>>> > populate a tool with credentials, this should be handled on the
>>> > framework level.
>>> >
>>> > Hope you are fine,
>>> > Bjoern
>>> >
>>> > Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
>>> >> Hi again,
>>> >>   I’m looking for the right way to store some user credentials in the
>>> >> galaxy session so that tools can work on behalf of the user with our
>>> >> repository.
>>> >>
>>> >> Currently users have an API key and they need to upload it as a data
>>> >> item to that is then passed to each tool that needs it as input.  This
>>> >> doesn’t seem like the right solution since the API key becomes part of
>>> >> the history and so would be shared if the history were shared.
>>> >>
>>> >> What would be better would be a way of storing the API key in the user
>>> >> session and then being able to pass that into a tool.
>>> >>
>>> >> I note that there are a few user session variables available in the
>>> tool
>>> >> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>>> >> There is also a user p

Re: [galaxy-dev] Storing data in the user session

[Steve Cassidy]

Hi Mohamed,

The problem with this solution is that the password again becomes part of the 
history since it is one of the inputs to the tool.  If I want to publish this 
workflow then the password can’t be visible.  For this reason password storage 
needs to be outside of workflows and handled somewhere else in Galaxy.

Steve



—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

On 28 Jul 2016, at 6:04 PM, Mohamed Kassam 
mailto:k.mam...@gmail.com>> wrote:

Hi Steve,
there is another solution is that the user is giving his credential via the 
tool. The only thing is missing the tag password on the xml then you can 
encrypt the password.
I tried to add the new paramter on the build.py but I have some errors, don't 
know if someone arrived to fixed it.

Mohamed

2016-07-28 10:00 GMT+02:00 Steve Cassidy 
mailto:steve.cass...@mq.edu.au>>:
Hi Mohamed,
   In my case, the credentials on Galaxy are not generally the same as on the 
remote system so we need some way to store a second set of credentials in the 
galaxy session.

So, for example, user Fred has an account on Galaxy 
(f...@here.com) and wants to get data from Alveo as input 
to a workflow. For the Alveo tools to get data, they need Fred’s access token 
from Alveo which Fred can retrieve by logging in to the Alveo website.  Fred 
enters the access key into his preferences in Galaxy.  This key is then passed 
to one or more tools that access Alveo on his behalf:

 - the query tool finds words matching a given pattern from a spoken corpus 
(needs the token)
 - the get-data tool retrieves the audio files for each word (needs the token)
 - the automatic alignment tool finds the locations of the vowel sounds in each 
word
 - the annotation storer tool sends these locations back to Alveo as a new 
annotation set (needs the token)

and so on, at various times in the workflow, the tools need to talk to the 
repository to query, get data or add new data.

At the moment, I need the token to be a data item in the history, but it would 
be better if it were part of the user session in Galaxy.

Steve


—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

On 28 Jul 2016, at 5:46 PM, Mohamed Kassam 
mailto:k.mam...@gmail.com>> wrote:

Hello Cassidy,
I have the same problem like you because I need the user credential to access 
to an external tools.
I tried the user session variables available in the tool
 xml file: __user__, __user_email__, __user_name__ and __user_id__. I t works 
perfectly I don't know if you need the password, because this is the only 
information is missing.

Mohamed

2016-07-28 9:42 GMT+02:00 Steve Cassidy 
mailto:steve.cass...@mq.edu.au>>:
Hi Bjorn,
 sure, I don’t want to have the credentials stored in the tool but the tool
needs to act on behalf of the user to retrieve data from the repository, so
it needs access to credentials.  This could for example be a token generated
via an OAuth2 exchange - basically something that is sent along with every
http request to the remote resource to authenticate the user.

My thought is that this could be stored in the user profile if this can be made
available to the tool in the same way that __user_email__ is now.

When you say ‘on the framework level’ do you mean by Galaxy?

Steve
—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

> On 28 Jul 2016, at 5:23 PM, Björn Grüning 
> mailto:bjoern.gruen...@gmail.com>> wrote:
>
> Hi Steve,
>
> can you explain the larger use-case behind it. Imho you should not
> populate a tool with credentials, this should be handled on the
> framework level.
>
> Hope you are fine,
> Bjoern
>
> Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
>> Hi again,
>>   I’m looking for the right way to store some user credentials in the
>> galaxy session so that tools can work on behalf of the user with our
>> repository.
>>
>> Currently users have an API key and they need to upload it as a data
>> item to that is then passed to each tool that needs it as input.  This
>> doesn’t seem like the right solution since the API key becomes part of
>> the history and so would be shared if the history were shared.
>>
>> What would be better would be a way of storing the API key in the user
>> session and then being able to pass that into a tool.
>>
>> I note that there are a few user session variables available in the tool
>> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>> There is also a user preferences page where they can fill out a few
>> details.  However, I can’t see a mechanism to extend this in any way and
>> have extra properties in the preferences pane that would then be
>> available to tools via the template file.
>>
>> I can see that the standard practice for the data sources on
>> usegalaxy.org 
>> > is to send the user off to a
>> repository website to find data that is then 

Re: [galaxy-dev] Storing data in the user session

[Mohamed Kassam]

Hi Steve,
there is another solution is that the user is giving his credential via the
tool. The only thing is missing the tag password on the xml then you can
encrypt the password.
I tried to add the new paramter on the build.py but I have some errors,
don't know if someone arrived to fixed it.

Mohamed

2016-07-28 10:00 GMT+02:00 Steve Cassidy :

> Hi Mohamed,
>In my case, the credentials on Galaxy are not generally the same as on
> the remote system so we need some way to store a second set of credentials
> in the galaxy session.
>
> So, for example, user Fred has an account on Galaxy (f...@here.com) and
> wants to get data from Alveo as input to a workflow. For the Alveo tools to
> get data, they need Fred’s access token from Alveo which Fred can retrieve
> by logging in to the Alveo website.  Fred enters the access key into his
> preferences in Galaxy.  This key is then passed to one or more tools that
> access Alveo on his behalf:
>
>  - the query tool finds words matching a given pattern from a spoken
> corpus (needs the token)
>  - the get-data tool retrieves the audio files for each word (needs the
> token)
>  - the automatic alignment tool finds the locations of the vowel sounds in
> each word
>  - the annotation storer tool sends these locations back to Alveo as a new
> annotation set (needs the token)
>
> and so on, at various times in the workflow, the tools need to talk to the
> repository to query, get data or add new data.
>
> At the moment, I need the token to be a data item in the history, but it
> would be better if it were part of the user session in Galaxy.
>
> Steve
>
>
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
> On 28 Jul 2016, at 5:46 PM, Mohamed Kassam  wrote:
>
> Hello Cassidy,
> I have the same problem like you because I need the user credential to
> access to an external tools.
> I tried the user session variables available in the tool
>  xml file: __user__, __user_email__, __user_name__ and __user_id__. I t
> works perfectly I don't know if you need the password, because this is the
> only information is missing.
>
> Mohamed
>
> 2016-07-28 9:42 GMT+02:00 Steve Cassidy :
>
>> Hi Bjorn,
>>  sure, I don’t want to have the credentials stored in the tool but the
>> tool
>> needs to act on behalf of the user to retrieve data from the repository,
>> so
>> it needs access to credentials.  This could for example be a token
>> generated
>> via an OAuth2 exchange - basically something that is sent along with every
>> http request to the remote resource to authenticate the user.
>>
>> My thought is that this could be stored in the user profile if this can
>> be made
>> available to the tool in the same way that __user_email__ is now.
>>
>> When you say ‘on the framework level’ do you mean by Galaxy?
>>
>> Steve
>> —
>> Department of Computing, Macquarie University
>> http://web.science.mq.edu.au/~cassidy
>>
>> > On 28 Jul 2016, at 5:23 PM, Björn Grüning 
>> wrote:
>> >
>> > Hi Steve,
>> >
>> > can you explain the larger use-case behind it. Imho you should not
>> > populate a tool with credentials, this should be handled on the
>> > framework level.
>> >
>> > Hope you are fine,
>> > Bjoern
>> >
>> > Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
>> >> Hi again,
>> >>   I’m looking for the right way to store some user credentials in the
>> >> galaxy session so that tools can work on behalf of the user with our
>> >> repository.
>> >>
>> >> Currently users have an API key and they need to upload it as a data
>> >> item to that is then passed to each tool that needs it as input.  This
>> >> doesn’t seem like the right solution since the API key becomes part of
>> >> the history and so would be shared if the history were shared.
>> >>
>> >> What would be better would be a way of storing the API key in the user
>> >> session and then being able to pass that into a tool.
>> >>
>> >> I note that there are a few user session variables available in the
>> tool
>> >> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>> >> There is also a user preferences page where they can fill out a few
>> >> details.  However, I can’t see a mechanism to extend this in any way
>> and
>> >> have extra properties in the preferences pane that would then be
>> >> available to tools via the template file.
>> >>
>> >> I can see that the standard practice for the data sources on
>> >> usegalaxy.org  is to send the user off to a
>> >> repository website to find data that is then posted back to the galaxy
>> >> server.  This isn’t appropriate for us since the interaction with the
>> >> repository is not just for downloading a single dataset - tools will
>> >> query the repository, download various kinds of data and possibly
>> upload
>> >> new data, so we need to store user credentials in some way.
>> >>
>> >> Is there already a good way to achieve this or is this an enhancement
>> to
>> >> Galaxy?
>> >>
>> >> Thanks,
>> >>
>> >> St

Re: [galaxy-dev] Storing data in the user session

[Steve Cassidy]

Hi Mohamed,
   In my case, the credentials on Galaxy are not generally the same as on the 
remote system so we need some way to store a second set of credentials in the 
galaxy session.

So, for example, user Fred has an account on Galaxy 
(f...@here.com) and wants to get data from Alveo as input 
to a workflow. For the Alveo tools to get data, they need Fred’s access token 
from Alveo which Fred can retrieve by logging in to the Alveo website.  Fred 
enters the access key into his preferences in Galaxy.  This key is then passed 
to one or more tools that access Alveo on his behalf:

 - the query tool finds words matching a given pattern from a spoken corpus 
(needs the token)
 - the get-data tool retrieves the audio files for each word (needs the token)
 - the automatic alignment tool finds the locations of the vowel sounds in each 
word
 - the annotation storer tool sends these locations back to Alveo as a new 
annotation set (needs the token)

and so on, at various times in the workflow, the tools need to talk to the 
repository to query, get data or add new data.

At the moment, I need the token to be a data item in the history, but it would 
be better if it were part of the user session in Galaxy.

Steve


—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

On 28 Jul 2016, at 5:46 PM, Mohamed Kassam 
mailto:k.mam...@gmail.com>> wrote:

Hello Cassidy,
I have the same problem like you because I need the user credential to access 
to an external tools.
I tried the user session variables available in the tool
 xml file: __user__, __user_email__, __user_name__ and __user_id__. I t works 
perfectly I don't know if you need the password, because this is the only 
information is missing.

Mohamed

2016-07-28 9:42 GMT+02:00 Steve Cassidy 
mailto:steve.cass...@mq.edu.au>>:
Hi Bjorn,
 sure, I don’t want to have the credentials stored in the tool but the tool
needs to act on behalf of the user to retrieve data from the repository, so
it needs access to credentials.  This could for example be a token generated
via an OAuth2 exchange - basically something that is sent along with every
http request to the remote resource to authenticate the user.

My thought is that this could be stored in the user profile if this can be made
available to the tool in the same way that __user_email__ is now.

When you say ‘on the framework level’ do you mean by Galaxy?

Steve
—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

> On 28 Jul 2016, at 5:23 PM, Björn Grüning 
> mailto:bjoern.gruen...@gmail.com>> wrote:
>
> Hi Steve,
>
> can you explain the larger use-case behind it. Imho you should not
> populate a tool with credentials, this should be handled on the
> framework level.
>
> Hope you are fine,
> Bjoern
>
> Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
>> Hi again,
>>   I’m looking for the right way to store some user credentials in the
>> galaxy session so that tools can work on behalf of the user with our
>> repository.
>>
>> Currently users have an API key and they need to upload it as a data
>> item to that is then passed to each tool that needs it as input.  This
>> doesn’t seem like the right solution since the API key becomes part of
>> the history and so would be shared if the history were shared.
>>
>> What would be better would be a way of storing the API key in the user
>> session and then being able to pass that into a tool.
>>
>> I note that there are a few user session variables available in the tool
>> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>> There is also a user preferences page where they can fill out a few
>> details.  However, I can’t see a mechanism to extend this in any way and
>> have extra properties in the preferences pane that would then be
>> available to tools via the template file.
>>
>> I can see that the standard practice for the data sources on
>> usegalaxy.org 
>> > is to send the user off to a
>> repository website to find data that is then posted back to the galaxy
>> server.  This isn’t appropriate for us since the interaction with the
>> repository is not just for downloading a single dataset - tools will
>> query the repository, download various kinds of data and possibly upload
>> new data, so we need to store user credentials in some way.
>>
>> Is there already a good way to achieve this or is this an enhancement to
>> Galaxy?
>>
>> Thanks,
>>
>> Steve
>>
>>
>> —
>> Department of Computing, Macquarie University
>> http://web.science.mq.edu.au/~cassidy
>> 
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>>
>> To search Galaxy ma

Re: [galaxy-dev] Storing data in the user session

[Mohamed Kassam]

Hello Cassidy,
I have the same problem like you because I need the user credential to
access to an external tools.
I tried the user session variables available in the tool
 xml file: __user__, __user_email__, __user_name__ and __user_id__. I t
works perfectly I don't know if you need the password, because this is the
only information is missing.

Mohamed

2016-07-28 9:42 GMT+02:00 Steve Cassidy :

> Hi Bjorn,
>  sure, I don’t want to have the credentials stored in the tool but the tool
> needs to act on behalf of the user to retrieve data from the repository, so
> it needs access to credentials.  This could for example be a token
> generated
> via an OAuth2 exchange - basically something that is sent along with every
> http request to the remote resource to authenticate the user.
>
> My thought is that this could be stored in the user profile if this can be
> made
> available to the tool in the same way that __user_email__ is now.
>
> When you say ‘on the framework level’ do you mean by Galaxy?
>
> Steve
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
> > On 28 Jul 2016, at 5:23 PM, Björn Grüning 
> wrote:
> >
> > Hi Steve,
> >
> > can you explain the larger use-case behind it. Imho you should not
> > populate a tool with credentials, this should be handled on the
> > framework level.
> >
> > Hope you are fine,
> > Bjoern
> >
> > Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
> >> Hi again,
> >>   I’m looking for the right way to store some user credentials in the
> >> galaxy session so that tools can work on behalf of the user with our
> >> repository.
> >>
> >> Currently users have an API key and they need to upload it as a data
> >> item to that is then passed to each tool that needs it as input.  This
> >> doesn’t seem like the right solution since the API key becomes part of
> >> the history and so would be shared if the history were shared.
> >>
> >> What would be better would be a way of storing the API key in the user
> >> session and then being able to pass that into a tool.
> >>
> >> I note that there are a few user session variables available in the tool
> >> xml file: __user__, __user_email__, __user_name__ and __user_id__.
> >> There is also a user preferences page where they can fill out a few
> >> details.  However, I can’t see a mechanism to extend this in any way and
> >> have extra properties in the preferences pane that would then be
> >> available to tools via the template file.
> >>
> >> I can see that the standard practice for the data sources on
> >> usegalaxy.org  is to send the user off to a
> >> repository website to find data that is then posted back to the galaxy
> >> server.  This isn’t appropriate for us since the interaction with the
> >> repository is not just for downloading a single dataset - tools will
> >> query the repository, download various kinds of data and possibly upload
> >> new data, so we need to store user credentials in some way.
> >>
> >> Is there already a good way to achieve this or is this an enhancement to
> >> Galaxy?
> >>
> >> Thanks,
> >>
> >> Steve
> >>
> >>
> >> —
> >> Department of Computing, Macquarie University
> >> http://web.science.mq.edu.au/~cassidy
> >> 
> >>
> >>
> >>
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>  https://lists.galaxyproject.org/
> >>
> >> To search Galaxy mailing lists use the unified search at:
> >>  http://galaxyproject.org/search/mailinglists/
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Storing data in the user session

[Steve Cassidy]

Hi Bjorn,
 sure, I don’t want to have the credentials stored in the tool but the tool 
needs to act on behalf of the user to retrieve data from the repository, so
it needs access to credentials.  This could for example be a token generated
via an OAuth2 exchange - basically something that is sent along with every 
http request to the remote resource to authenticate the user.  

My thought is that this could be stored in the user profile if this can be made
available to the tool in the same way that __user_email__ is now. 

When you say ‘on the framework level’ do you mean by Galaxy? 

Steve
—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

> On 28 Jul 2016, at 5:23 PM, Björn Grüning  wrote:
> 
> Hi Steve,
> 
> can you explain the larger use-case behind it. Imho you should not
> populate a tool with credentials, this should be handled on the
> framework level.
> 
> Hope you are fine,
> Bjoern
> 
> Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
>> Hi again,
>>   I’m looking for the right way to store some user credentials in the
>> galaxy session so that tools can work on behalf of the user with our
>> repository. 
>> 
>> Currently users have an API key and they need to upload it as a data
>> item to that is then passed to each tool that needs it as input.  This
>> doesn’t seem like the right solution since the API key becomes part of
>> the history and so would be shared if the history were shared.  
>> 
>> What would be better would be a way of storing the API key in the user
>> session and then being able to pass that into a tool.
>> 
>> I note that there are a few user session variables available in the tool
>> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>> There is also a user preferences page where they can fill out a few
>> details.  However, I can’t see a mechanism to extend this in any way and
>> have extra properties in the preferences pane that would then be
>> available to tools via the template file.  
>> 
>> I can see that the standard practice for the data sources on
>> usegalaxy.org  is to send the user off to a
>> repository website to find data that is then posted back to the galaxy
>> server.  This isn’t appropriate for us since the interaction with the
>> repository is not just for downloading a single dataset - tools will
>> query the repository, download various kinds of data and possibly upload
>> new data, so we need to store user credentials in some way. 
>> 
>> Is there already a good way to achieve this or is this an enhancement to
>> Galaxy? 
>> 
>> Thanks,
>> 
>> Steve
>> 
>> 
>> —
>> Department of Computing, Macquarie University
>> http://web.science.mq.edu.au/~cassidy
>> 
>> 
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Storing data in the user session

[Björn Grüning]

Hi Steve,

can you explain the larger use-case behind it. Imho you should not
populate a tool with credentials, this should be handled on the
framework level.

Hope you are fine,
Bjoern

Am 28.07.2016 um 03:15 schrieb Steve Cassidy:
> Hi again,
>I’m looking for the right way to store some user credentials in the
> galaxy session so that tools can work on behalf of the user with our
> repository. 
> 
> Currently users have an API key and they need to upload it as a data
> item to that is then passed to each tool that needs it as input.  This
> doesn’t seem like the right solution since the API key becomes part of
> the history and so would be shared if the history were shared.  
> 
> What would be better would be a way of storing the API key in the user
> session and then being able to pass that into a tool.
> 
> I note that there are a few user session variables available in the tool
> xml file: __user__, __user_email__, __user_name__ and __user_id__.
>  There is also a user preferences page where they can fill out a few
> details.  However, I can’t see a mechanism to extend this in any way and
> have extra properties in the preferences pane that would then be
> available to tools via the template file.  
> 
> I can see that the standard practice for the data sources on
> usegalaxy.org  is to send the user off to a
> repository website to find data that is then posted back to the galaxy
> server.  This isn’t appropriate for us since the interaction with the
> repository is not just for downloading a single dataset - tools will
> query the repository, download various kinds of data and possibly upload
> new data, so we need to store user credentials in some way. 
> 
> Is there already a good way to achieve this or is this an enhancement to
> Galaxy? 
> 
> Thanks,
> 
> Steve
> 
> 
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
> 
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/