[galaxy-dev] Sharing galaxy defined workflow with link leads to Internal Server Error

[NGS & Bioinformatics Platform]

Hi All,

We have an issue with our self-hosted galaxy environment. When we want to share 
a workflow  (Workflow -> workflow Name -> Share or Download)  the links under 
'Share' section leads to 'Internal server Error'. Did we miss something ?  See 
attachment for screenshots and log file.

Thank you for helping us.



###
Checked as being free of known viruses.

Scientific Institute of Public Health, Brussels, Belgium
Wetenschappelijk Instituut Volksgezondheid, Brussel, België
Institut scientifique de Santé publique, Bruxelles, Belgique
Visit our website: http://www.wiv-isp.be 
#
DISCLAIMER: Please see https://www.wiv-isp.be/EmailDisclaimer
#

[pid: 4812|app: 0|req: 516/3307] 192.168.3.28 () {50 vars in 920 bytes} [Wed 
Nov 23 14:35:52 2016] GET /u/bioit/w/imported-resistance-characterization => 
generated 777 bytes in 905 msecs (HTTP/1.1 500) 1 headers in 63 bytes (1 
switches on core 2)
10.0.0.26 - - [23/Nov/2016:14:36:45 +0200] "GET / HTTP/1.1" 200 - 
"https://galaxy.wiv-isp.be/workflow/list_published; "Mozilla/5.0 (Windows NT 
6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 
Safari/537.36"
[pid: 4812|app: 0|req: 517/3308] 10.0.0.26 () {50 vars in 903 bytes} [Wed Nov 
23 14:36:45 2016] GET / => generated 386037 bytes in 191 msecs (HTTP/1.1 200) 2 
headers in 73 bytes (1 switches on core 1)
10.0.0.26 - - [23/Nov/2016:14:36:45 +0200] "GET /api/genomes HTTP/1.1" 200 - 
"https://galaxy.wiv-isp.be/; "Mozilla/5.0 (Windows NT 6.1; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 Safari/537.36"
[pid: 4812|app: 0|req: 518/3309] 10.0.0.26 () {52 vars in 889 bytes} [Wed Nov 
23 14:36:45 2016] GET /api/genomes => generated 9634 bytes in 17 msecs 
(HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 0)
10.0.0.26 - - [23/Nov/2016:14:36:45 +0200] "GET 
/api/datatypes?extension_only=False HTTP/1.1" 200 - 
"https://galaxy.wiv-isp.be/; "Mozilla/5.0 (Windows NT 6.1; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 Safari/537.36"
[pid: 4813|app: 0|req: 2792/3310] 10.0.0.26 () {52 vars in 933 bytes} [Wed Nov 
23 14:36:45 2016] GET /api/datatypes?extension_only=False => generated 21730 
bytes in 41 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 3)
10.0.0.26 - - [23/Nov/2016:14:36:46 +0200] "GET /welcome HTTP/1.1" 302 - 
"https://galaxy.wiv-isp.be/; "Mozilla/5.0 (Windows NT 6.1; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 Safari/537.36"
[pid: 4813|app: 0|req: 2794/3311] 10.0.0.26 () {50 vars in 895 bytes} [Wed Nov 
23 14:36:46 2016] GET /welcome => generated 304 bytes in 31 msecs (HTTP/1.1 
302) 3 headers in 108 bytes (1 switches on core 1)
10.0.0.26 - - [23/Nov/2016:14:36:46 +0200] "GET /history/current_history_json 
HTTP/1.1" 200 - "https://galaxy.wiv-isp.be/; "Mozilla/5.0 (Windows NT 6.1; 
Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 
Safari/537.36"
[pid: 4813|app: 0|req: 2794/3312] 10.0.0.26 () {50 vars in 912 bytes} [Wed Nov 
23 14:36:46 2016] GET /history/current_history_json => generated 3360 bytes in 
87 msecs (HTTP/1.1 200) 5 headers in 156 bytes (1 switches on core 2)
10.0.0.26 - - [23/Nov/2016:14:36:46 +0200] "GET 
/api/histories/73bf562cbe439524/contents?v=dev=23477a427001b60d%2Ce48b76eff580fe1e
 HTTP/1.1" 200 - "https://galaxy.wiv-isp.be/; "Mozilla/5.0 (Windows NT 6.1; 
Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 
Safari/537.36"
[pid: 4813|app: 0|req: 2795/3313] 10.0.0.26 () {50 vars in 990 bytes} [Wed Nov 
23 14:36:46 2016] GET 
/api/histories/73bf562cbe439524/contents?v=dev=23477a427001b60d%2Ce48b76eff580fe1e
 => generated 52584 bytes in 197 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 
switches on core 0)
10.0.0.26 - - [23/Nov/2016:14:36:46 +0200] "GET 
/api/histories/73bf562cbe439524?keys=size%2Cnon_ready_jobs HTTP/1.1" 200 - 
"https://galaxy.wiv-isp.be/; "Mozilla/5.0 (Windows NT 6.1; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.3 Safari/537.36"
[pid: 4813|app: 0|req: 2796/3314] 10.0.0.26 () {50 vars in 969 bytes} [Wed Nov 
23 14:36:46 2016] GET 
/api/histories/73bf562cbe439524?keys=size%2Cnon_ready_jobs => generated 42 
bytes in 26 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 3)
192.168.3.28 - - [23/Nov/2016:14:37:01 +0200] "GET 
/u/bioit/w/imported-resistance-characterization HTTP/1.1" 500 - 
"http://10.0.0.219/workflow/sharing?id=581bf952f093ed97; "Mozilla/5.0 (Windows 
NT 6.1; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
Error - : 'NoneType' object has no attribute 
'replace'
URL: http://10.0.0.219/u/bioit/w/imported-resistance-characterization
File 'lib/galaxy/web/framework/middleware/error.py', line 151 in __call__
  app_iter = self.application(environ, 

[galaxy-dev] Mapping LDAP groups to Galaxy groups/roles

[Ignacio EGUINOA]

Hi all, 

I'm currently looking for some help to manage groups/roles in Galaxy. 
The instance I'm working with uses LDAP authentication (all Galaxy registered 
users are in LDAP) and the datasets that are uploaded have permissions 
'patterns' that can be mapped to groups already available in the LDAP server. 
Current approach is to manually add roles and groups and assign the registered 
users to them. Since the amount of users and groups is considerable large and 
can also change, I was hoping to have a more automatic system such that, when a 
new dataset is added, the required group/roles are already available to be 
assigned. 
I guess that, since the LDAP system is only used for authentication purposes, 
there is no way to directly map all the groups and users to galaxy (and 
actually it would be an overkill). But, at least I would like to get to map the 
primary LDAP group in a somehow automatic way. 
The only approach I could come up with is to manage the group/roles creation 
through the API, assuming I have a list of the relevant group names at a 
certain point in time, and then update the link of users to the available 
groups/roles by using information from LDAP. 
When I first deployed the instance I added an extra check during user 
authentication to verify that the user belongs to a specific LDAP group which 
grants access to the Galaxy instance. To do this, a list of groups the user 
belongs to is obtained from the server. I was thinking about extending this 
step to also update the user's Galaxy profile, associating it with the groups 
that are already created in Galaxy. 
Since this will require some digging into Galaxy code, I first want ask if 
there is any alternative solution for this kind of situation or some work was 
already done.. 

Thanks in advance 



Ignacio EGUINOA - Predoctoral fellow 
Applied Bioinformatics And Biostatistics 

VIB Department of Plant Systems Biology 
Ghent University 
Technologiepark 927 - 9052 Ghent - Belgium 
Tel. +32(0)9 331 36 95 
www.psb.ugent.be 



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Error running bedgraph to bigwig on local galaxy

[Evan Clark]

Yes, these alignments were done to ensemble, not UCSC.

---Regards, Evan Clark

Reminder: If this message is requesting you to verify your account or provides 
links for you to login to your account, you are advised to ignore such 
requests. Neither FAU nor legitimate financial institutions will ever ask you 
for this information via email or via a link in an email.


Are you using the chromosome lengths for the correct organism and
source? GL000192.1 is a contig name used by Ensembl and Gencode (but
not UCSC!) for GRCh37 (aka hg19).

Devon
--
Devon Ryan, Ph.D.
Email: dpr...@dpryan.com
Data Manager/Bioinformatician
Max Planck Institute of Immunobiology and Epigenetics
Stübeweg 51
79108 Freiburg
Germany


On Wed, Nov 23, 2016 at 4:12 PM, evan clark  wrote:
> When attempting to generate a bigwig file I receive this error.
>
> hashMustFindVal: 'GL000192.1' not found Fatal error: Matched on Error Error
> running wigToBigWig.
>
> I have added the chrom len files for galaxy and update the location path but
> an unsure what could be causing this issue.
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Moving Galaxy Install From One Server To Another

[Peter Briggs]

Hi Peter

Thanks for your comments - at least it doesn't sound like I'm missing 
the "perfect" way to migrate the installed tools. In the longer term 
moving towards bioconda dependency resolution will probably be more stable.


Also re breaking tool repos: even repos that aren't updated on the 
toolshed can break over time, for example if the URL for an executable 
moves, or if an implicit Python dependency is updated and breaks an 
install (these are both things that I'm seeing in my tests).


Thanks again

Best wishes

Peter

On 23/11/16 11:40, Peter Cock wrote:

On Wed, Nov 23, 2016 at 10:49 AM, Peter Briggs
 wrote:

Hello Evan, Hans-Rudolf

I'm just in the middle of doing a similar migration for our local production
server, and Hans-Rudolf's advice seems sound to me. Definitely moving the
core Galaxy server has been relatively straightforward.

However: for the installed tools, I'm not sure that changing the paths in
the env.sh files is sufficient - in our installation the absolute paths
seemed to be baked into a lot of other files under the 'tool_dependencies'
directory - including things like compiled files (e.g. static and shared
libraries). So for many of the tools I wouldn't feel confident that they
would still work after the move.


Yes, I've seen that with various third party tool installations (out side of
Galaxy), so sadly you cannot in general move the files to a different path :(

I don't know if we can do anything about this directly in Galaxy, or even
in BioConda?


My plan has been to reinstall each of the tools from the toolshed (i.e.
uninstall via the admin interface then reinstall the same tool repository
revision(s) using the API), but I don't feel able to recommend this approach
either as in my testing this has also had problems - ranging from some tool
revisions no longer being available, through to more serious issues (such as
tool dependencies which used to work but since become broken). I figured I'd
just have to knuckle down and work through each problem as I encountered it.


One simple reason for this is stale URLs, where Galaxy's cache can help
but is another step for tool wrapper authors to do when setting up a new
Galaxy dependency: https://github.com/galaxyproject/cargo-port

However, even well intentioned Tool Shed updates could also break things :(


If anyone else has experiences with these kinds of migrations then I'm also
very interested to know what worked (and what didn't)!


Sorry - the closest I've been is setting up a new Galaxy server in
parallel with our old server, and manually installing "missing" tools
via the Tool Shed.

Peter



--
Peter Briggs peter.bri...@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Is there demand for rate limiting in Galaxy?

[Martin Čech]

Max:

you are correct, the dev branch is what would be the best to develop
against. Also feel free to open PR with work in progress at any point. I am
sure you will get useful feedback.

Thank you for using Galaxy,

Martin

On Wed, Nov 23, 2016 at 5:40 AM Maximilian Friedersdorff 
wrote:

>
>
> 
> Un o’r 4 prifysgol uchaf yn y DU a’r orau yng Nghymru am fodlonrwydd
> myfyrwyr.
> (Arolwg Cenedlaethol y Myfyrwyr 2016)
> www.aber.ac.uk
>
> Top 4 UK university and best in Wales for student satisfaction
> (National Student Survey 2016)
> www.aber.ac.uk
>
>
>
> -- Forwarded message --
> From: Maximilian Friedersdorff 
> To:
> Cc: Maximilian Friedersdorff , "
> galaxy-dev@lists.galaxyproject.org" 
> Date: Wed, 23 Nov 2016 10:40:11 +
> Subject: Re: [galaxy-dev] Is there demand for rate limiting in Galaxy?
> > I think there is a demand and partially Galaxy has support for these
> > kind of limits for a user. Please have a look at:
> >
> >
> https://github.com/galaxyproject/galaxy/blob/dev/config/job_conf.xml.sample_advanced#L725
>
> Yes I'm aware of the existing limits.  An ideal (from my point of view)
> implementation would simply add another possible type to these.
>
> > An extension to these would be super cool!
>
> Ok that's all I needed to hear :-).  I guess I should base any changes
> on the [dev] branch?
>
> Thanks
>
> Max
>
> PS. My apologies for the email signature below, over which I have no
> control.
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Is there demand for rate limiting in Galaxy?

[Maximilian Friedersdorff]

Un o’r 4 prifysgol uchaf yn y DU a’r orau yng Nghymru am fodlonrwydd myfyrwyr.
(Arolwg Cenedlaethol y Myfyrwyr 2016)
www.aber.ac.uk

Top 4 UK university and best in Wales for student satisfaction
(National Student Survey 2016)
www.aber.ac.uk
--- Begin Message ---
> I think there is a demand and partially Galaxy has support for these
> kind of limits for a user. Please have a look at:
>
> https://github.com/galaxyproject/galaxy/blob/dev/config/job_conf.xml.sample_advanced#L725

Yes I'm aware of the existing limits.  An ideal (from my point of view)
implementation would simply add another possible type to these.

> An extension to these would be super cool!

Ok that's all I needed to hear :-).  I guess I should base any changes
on the [dev] branch?

Thanks

Max

PS. My apologies for the email signature below, over which I have no control.


signature.asc
Description: PGP signature
--- End Message ---
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Is there demand for rate limiting in Galaxy?

[Björn Grüning]

Hi Maximilian,

I think there is a demand and partially Galaxy has support for these
kind of limits for a user. Please have a look at:

https://github.com/galaxyproject/galaxy/blob/dev/config/job_conf.xml.sample_advanced#L725

An extension to these would be super cool!
Thanks,
Bjoern

Am 23.11.2016 um 10:51 schrieb Maximilian Friedersdorff:
> 
> 
> 
> Un o’r 4 prifysgol uchaf yn y DU a’r orau yng Nghymru am fodlonrwydd myfyrwyr.
> (Arolwg Cenedlaethol y Myfyrwyr 2016)
> www.aber.ac.uk
> 
> Top 4 UK university and best in Wales for student satisfaction
> (National Student Survey 2016)
> www.aber.ac.uk
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Is there demand for rate limiting in Galaxy?

[Maximilian Friedersdorff]

Un o’r 4 prifysgol uchaf yn y DU a’r orau yng Nghymru am fodlonrwydd myfyrwyr.
(Arolwg Cenedlaethol y Myfyrwyr 2016)
www.aber.ac.uk

Top 4 UK university and best in Wales for student satisfaction
(National Student Survey 2016)
www.aber.ac.uk
--- Begin Message ---
Hi All,

I want to make a galaxy instance publicly and freely available.  However
some/all of the tools installed are computationally expensive.
Computing time is expensive and resources are limited, so there should
be some sort of rate limiting to prevent abuse. In my case the most
sensible route would be to limit users to a certain number of CPU hours
per time period in addition to concurrent job limits and maximum job
length.

Is there any demand for something like this to be implemented within
galaxy?  If so I would like to implement this functionality
``properly'', rather than as a quick hack.

Thanks


signature.asc
Description: PGP signature
--- End Message ---
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Moving Galaxy Install From One Server To Another

[Hans-Rudolf Hotz]

Hi Evan

Well, in theory, this is a simple task, since Galaxy mostly works with 
relative paths. Of course, it all depends on how sophisticated your 
installation is and how different the old and the new 'machine' is (eg 
same OS?)


So my advice is: just do it or rather try it.

 - make a copy of your installation on the new hardware (at first it is 
sufficient to copy only the latest sub directories of database/files/)


 - make a copy of your PostgreSQL database

 - connect your copied installation to the PostgreSQL database copy and 
test, and test, and test


 - pay special attention to tools you have installed via the toolshed. 
You might need to change some absolute paths in the 'env.sh' files 
located in the tool_dependencies/ directory.



Once your happy with all the tests, do an rsync on the database/ 
directory and connect to the original PostgreSQL database.



Regards, Hans-Rudolf




On 11/22/2016 07:52 PM, evan clark wrote:

Is there a good method for moving a fully functional galaxy install from
one machine to another in a different directory?
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/