[galaxy-dev] Build the Galaxy under Cluster
Hi everyone, I want to install my local Galaxy under the Cluster with TORQUE.I have a question about how they work together successfully.In addition,after my local Galaxy and TORQUE install successfully,anything else should I do to the Cluster Galaxy?Thank you very much. shenwiyn___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hi,
I was thinking how to tackle the build failures like the atlas library
on OS-X or the ones from xlib and ease the writing of the shell_command
a little bit.
Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
was also sometimes forced to use some bash tricks.
What about to make that more easy and powerful, and as a plus more
consistent with the tool-command tag, and use cheetah for that kind of
work.
Something like that could work easily:
#if ${os.environ}.getVar('X11_LIB_DIR', None):
./configure ... --x-includes=
#else:
./configure
#end if
Also we could define some reserved variables like $__is64__ and cheetah
evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
variable and we do not need to replace it later manually.
What do you think any comments?
If no one has any objections I can try to implement an .
Thanks,
Björn
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hello Björn,
This would be great! I'm not sure about the type attribute - wrote:
> Hi,
>
> I was thinking how to tackle the build failures like the atlas library
> on OS-X or the ones from xlib and ease the writing of the shell_command
> a little bit.
>
> Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
> was also sometimes forced to use some bash tricks.
>
> What about to make that more easy and powerful, and as a plus more
> consistent with the tool-command tag, and use cheetah for that kind of
> work.
>
> Something like that could work easily:
>
> #if ${os.environ}.getVar('X11_LIB_DIR', None):
> ./configure ... --x-includes=
> #else:
> ./configure
> #end if
>
> Also we could define some reserved variables like $__is64__ and cheetah
> evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
> variable and we do not need to replace it later manually.
>
> What do you think any comments?
> If no one has any objections I can try to implement an type="cheetah_command">.
>
> Thanks,
> Björn
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Question about bowtie2_indeces.loc
Dbkey specifies the base assembly; unique build ID species the particular assembly. This is most often used for human assemblies: dbkey is hg19 and build ID might specify some combinations of autosomes, sex chromosomes, and mitichondrial DNA. J. On Jul 6, 2013, at 6:16 PM, Perez, Ricardo wrote: > Dear all, > When adding a genome I understand that the format are added in the following > way: > > > > My question is, what is the difference between the unique_build_id and the > dbkey? > > Thank you for your time, > --Ricardo Perez > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hi Greg,
> Hello Björn,
>
> This would be great! I'm not sure about the type attribute - type="cheetah_command" may be ok, but maybe a more generic type like type="template_command" would be better.
> I'm not quite sure, so go with what you feel is best.
I come up with the following:
#if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR',
False):
./configure --prefix=$env.INSTALL_DIR --x-includes=
$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
#else:
./configure --prefix=$env.INSTALL_DIR
#end if
$env is the global access point to the env variables and its working
fine for the few repositories I tested it.
I had hoped to get it a little bit prettier but fabric and cheetah
doesn't work well together.
If that is still satisfying I would clean up my code and create a pull
request [1].
What do you think?
Cheers,
Bjoern
[1] or patch, I heard rumours that you do not like pull requests ;)?
> Thanks for looking into this!
>
> Greg Von Kuster
>
>
> On Jul 7, 2013, at 5:50 AM, Björn Grüning
> wrote:
>
> > Hi,
> >
> > I was thinking how to tackle the build failures like the atlas library
> > on OS-X or the ones from xlib and ease the writing of the shell_command
> > a little bit.
> >
> > Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
> > was also sometimes forced to use some bash tricks.
> >
> > What about to make that more easy and powerful, and as a plus more
> > consistent with the tool-command tag, and use cheetah for that kind of
> > work.
> >
> > Something like that could work easily:
> >
> > #if ${os.environ}.getVar('X11_LIB_DIR', None):
> > ./configure ... --x-includes=
> > #else:
> > ./configure
> > #end if
> >
> > Also we could define some reserved variables like $__is64__ and cheetah
> > evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
> > variable and we do not need to replace it later manually.
> >
> > What do you think any comments?
> > If no one has any objections I can try to implement an > type="cheetah_command">.
> >
> > Thanks,
> > Björn
> >
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hi Björn,
This is great. A pull request or a patch is fine. Whatever one is most
convenient for you. The code doesn't have to be pretty, just functional. We
can always pretty it up later.
Thanks very much for doing this!
Greg
On Jul 7, 2013, at 1:19 PM, Björn Grüning
wrote:
> Hi Greg,
>
>> Hello Björn,
>>
>> This would be great! I'm not sure about the type attribute - > type="cheetah_command" may be ok, but maybe a more generic type like > type="template_command" would be better.
>> I'm not quite sure, so go with what you feel is best.
> I come up with the following:
>
>
> #if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR', False):
>./configure --prefix=$env.INSTALL_DIR
> --x-includes=$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
>#else:
> ./configure --prefix=$env.INSTALL_DIR
>#end if
>
>
> $env is the global access point to the env variables and its working fine for
> the few repositories I tested it.
> I had hoped to get it a little bit prettier but fabric and cheetah doesn't
> work well together.
>
> If that is still satisfying I would clean up my code and create a pull
> request [1].
> What do you think?
>
> Cheers,
> Bjoern
>
> [1] or patch, I heard rumours that you do not like pull requests ;)?
>
>> Thanks for looking into this!
>>
>> Greg Von Kuster
>>
>>
>> On Jul 7, 2013, at 5:50 AM, Björn Grüning
>> wrote:
>>
>> > Hi,
>> >
>> > I was thinking how to tackle the build failures like the atlas library
>> > on OS-X or the ones from xlib and ease the writing of the shell_command
>> > a little bit.
>> >
>> > Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
>> > was also sometimes forced to use some bash tricks.
>> >
>> > What about to make that more easy and powerful, and as a plus more
>> > consistent with the tool-command tag, and use cheetah for that kind of
>> > work.
>> >
>> > Something like that could work easily:
>> >
>> > #if ${os.environ}.getVar('X11_LIB_DIR', None):
>> >./configure ... --x-includes=
>> > #else:
>> >./configure
>> > #end if
>> >
>> > Also we could define some reserved variables like $__is64__ and cheetah
>> > evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
>> > variable and we do not need to replace it later manually.
>> >
>> > What do you think any comments?
>> > If no one has any objections I can try to implement an > > type="cheetah_command">.
>> >
>> > Thanks,
>> > Björn
>> >
>>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
Hi Greg,
bitbucket has a downtime so I have attached a patch. I will create
tomorrow a pull request if I find time.
It will work as described and we have now a new variable (__is64bit__)
that we can use to determine if python is running under 64bit (assuming
that the rest is also 64bit-ish).
Attached is a ported version of emboss as test case (Dave B wrapper).
Have a nice evening!
Bjoern
> Hi Björn,
>
>
> This is great. A pull request or a patch is fine. Whatever one is
> most convenient for you. The code doesn't have to be pretty, just
> functional. We can always pretty it up later.
>
>
> Thanks very much for doing this!
>
>
> Greg
>
> On Jul 7, 2013, at 1:19 PM, Björn Grüning
> wrote:
>
> > Hi Greg,
> >
> > > Hello Björn,
> > >
> > > This would be great! I'm not sure about the type attribute - > > type="cheetah_command" may be ok, but maybe a more generic type like
> > > > > I'm not quite sure, so go with what you feel is best.
> > I come up with the following:
> >
> >
> > #if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR',
> > False):
> >./configure --prefix=$env.INSTALL_DIR --x-includes=
> > $env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
> >#else:
> > ./configure --prefix=$env.INSTALL_DIR
> >#end if
> >
> >
> > $env is the global access point to the env variables and its working
> > fine for the few repositories I tested it.
> > I had hoped to get it a little bit prettier but fabric and cheetah
> > doesn't work well together.
> >
> > If that is still satisfying I would clean up my code and create a
> > pull request [1].
> > What do you think?
> >
> > Cheers,
> > Bjoern
> >
> > [1] or patch, I heard rumours that you do not like pull
> > requests ;)?
> >
> > > Thanks for looking into this!
> > >
> > > Greg Von Kuster
> > >
> > >
> > > On Jul 7, 2013, at 5:50 AM, Björn Grüning
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I was thinking how to tackle the build failures like the atlas library
> > > > on OS-X or the ones from xlib and ease the writing of the shell_command
> > > > a little bit.
> > > >
> > > > Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
> > > > was also sometimes forced to use some bash tricks.
> > > >
> > > > What about to make that more easy and powerful, and as a plus more
> > > > consistent with the tool-command tag, and use cheetah for that kind of
> > > > work.
> > > >
> > > > Something like that could work easily:
> > > >
> > > > #if ${os.environ}.getVar('X11_LIB_DIR', None):
> > > > ./configure ... --x-includes=
> > > > #else:
> > > > ./configure
> > > > #end if
> > > >
> > > > Also we could define some reserved variables like $__is64__ and cheetah
> > > > evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
> > > > variable and we do not need to replace it later manually.
> > > >
> > > > What do you think any comments?
> > > > If no one has any objections I can try to implement an > > > type="cheetah_command">.
> > > >
> > > > Thanks,
> > > > Björn
> > > >
> > >
> >
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > http://lists.bx.psu.edu/
> >
> > To search Galaxy mailing lists use the unified search at:
> > http://galaxyproject.org/search/mailinglists/
>
>
tool_dependencies.xml
Description: XML document
diff -r 434335f80161 lib/galaxy/util/template.py
--- a/lib/galaxy/util/template.py Fri Jun 21 04:37:07 2013 -0400
+++ b/lib/galaxy/util/template.py Mon Jul 08 00:55:36 2013 +0200
@@ -6,4 +6,4 @@
def fill_template( template_text, context=None, **kwargs ):
if not context:
context = kwargs
-return str( Template( source=template_text, searchList=[context] ) )
\ No newline at end of file
+return str( Template( source=template_text, searchList=[context] ) )
diff -r 434335f80161 lib/tool_shed/galaxy_install/tool_dependencies/common_util.py
--- a/lib/tool_shed/galaxy_install/tool_dependencies/common_util.py Fri Jun 21 04:37:07 2013 -0400
+++ b/lib/tool_shed/galaxy_install/tool_dependencies/common_util.py Mon Jul 08 00:55:36 2013 +0200
@@ -11,6 +11,15 @@
log = logging.getLogger( __name__ )
+def get_env_substitutions( install_dir ):
+# # Added for compatibility with CloudBioLinux.
+# TODO: Add tool_version substitution for compat with CloudBioLinux.
+return { "INSTALL_DIR" : install_dir,
+"system_install" : install_dir,
+#"REPOSITORY_INSTALL_DIR": get_tool_shed_repository_install_dir( app, tool_shed_repository ),
+"__is64bit__": sys.maxsize > 2**32, #checks if the python interpreter is 64bit, we assume that the underlying system is also 64bit
+}
+
def create_env_var_dict( elem, tool_dependency_install_dir=None, t
[galaxy-dev] phastCons scores all NaN
Hi all, I am a new user to Galaxy. Recently I uploaded a bed file to Galaxy and then converted it to hg18 format in order to run a phastCons scoring on it. The fields returned were all NaN, though. I then converted the hg18 format to hg17 in order to see if that would make a difference, but it still returned NaN each time. At this point, in order to test if it was simply my data that had a problem, I found and viewed a specific region in the UCSC genome browser to verify that it did, in fact, include regions with conservation, I entered that sequence into Galaxy in bed format and attempted to get phastCons scores for it, but still received NaN for the results. At this point, I have no idea what is going on. I believe that this must be a bug, since I am fairly certain that my bed files are in the correct format. All of these were run on the main Galaxy server on and off during the past week. If anyone could help me figure out what is wrong, I would appreciate it very much. Thanks, Diana ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] gzipped fastq reader
Dan, Do these readers support gzip files? reader = fastqVerboseErrorReader reader = fastqReader Do I have to define a special type in galaxy for gzipped files or will the fastq type be ok? Ideally, I would like to keep my files zipped and not have galaxy unzip them, since they triple in size when unzipped. I'm happy to do a push request if you don't support this but I want to make sure I'm in line with your roadmap. I have written a simple tool to convert Illumina fastq to mapsplice fastq. Does that already exist already somewhere? -Robert ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy on a server without internet connexion try to contact toolshed.g2.bx.psu.edu
Hi,
I installed Galaxy on a server located on a secure Network without internet
connexion following this instructions:
http://wiki.galaxyproject.org/Admin/Config/Eggs
For an offline usage I changed the following parameters:
Desable eggs check in run.sh
FETCH_EGGS to 0
And the toolshed check option in universe_wsgi.ini
enable_tool_shed_check = False
On the first run Galaxy run, but when I tried to restart Galaxy try to contact
toolshed.g2.bx.psu.edu :
#---
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=bowtie_wrappers&owner=devteam&changeset_revision=0c7e4eadfb3c&from_install_manager=True
raised the exception:
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=bowtie_color_wrappers&owner=devteam&changeset_revision=fd0914e451c5&from_install_manager=True
raised the exception:
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=lastz&owner=devteam&changeset_revision=0801f8207d30&from_install_manager=True
raised the exception:
The URL
http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=lastz_paired_reads&owner=devteam&changeset_revision=96825cee5c25&from_install_manager=True
raised the exception:
Traceback (most recent call last):
File "/usr/local/Galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py",
line 35, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File "/usr/local/Galaxy/galaxy-dist/lib/galaxy/app.py", line 55, in __init__
verify_tools( self, db_url, kwargs.get( 'global_conf', {} ).get(
'__file__', None ), self.config.database_engine_options )
File
"/usr/local/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/migrate/check.py",
line 55, in verify_tools
tool_shed_accessible, missing_tool_configs_dict =
common_util.check_for_missing_tools( app, tool_panel_configs,
latest_tool_migration_script_number )
File "/usr/local/Galaxy/galaxy-dist/lib/tool_shed/util/common_util.py", line
67, in check_for_missing_tools
return tool_shed_accessible, missing_tool_configs_dict
UnboundLocalError: local variable 'missing_tool_configs_dict' referenced before
assignment
#---
I Fixed this problem by adding a enable_tool_shed_check check in
lib/galaxy/app.py
I'm new in the Galaxy World, so perhaps that is not an optimal way to fix this
problem.
diff --git a/lib/galaxy/app.py b/lib/galaxy/app.py
--- a/lib/galaxy/app.py
+++ b/lib/galaxy/app.py
@@ -51,8 +51,9 @@
from galaxy.model.migrate.check import create_or_verify_database
create_or_verify_database( db_url, kwargs.get( 'global_conf', {}
).get( '__file__', None ), self.config.database_engine_options, app=self )
# Alert the Galaxy admin to tools that have been moved from the
distribution to the tool shed.
-from tool_shed.galaxy_install.migrate.check import verify_tools
-verify_tools( self, db_url, kwargs.get( 'global_conf', {} ).get(
'__file__', None ), self.config.database_engine_options )
+if self.config.get_bool( 'enable_tool_shed_check', False ):
+from tool_shed.galaxy_install.migrate.check import verify_tools
+verify_tools( self, db_url, kwargs.get( 'global_conf', {} ).get(
'__file__', None ), self.config.database_engine_options )
# Object store manager
self.object_store = build_object_store_from_config(self.config)
# Setup the database engine and ORM
Regards,
Ghislain
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Regarding galaxy wrappers: "lumi", "DESeq" "edgeR"
Hi, These are the R libraries. Have anyone written a wrapper for these libraries "lumi", "DESeq" "edgeR"? Thanks, Sachit ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Postgresql database not working
Hi, So, I found my mistake. The problem was in a way a copy and paste error in the universe_wsgi.ini. My prefix and cookie_path had different values, the one '/galaxy' the other '/Galaxy'. After setting them to the same, it worked like a charm :) Cheers, Sajoscha On Jul 1, 2013, at 5:00 PM, Sajoscha Sauer wrote: > Hi all, > > I just installed the newest version of Galaxy with: hg clone > https://bitbucket.org/galaxy/galaxy-dist > > I use Apache httpd as proxy and ldap authentication. If I leave the database > as it is set by default everything works fine. > > sqlite:///./database/universe.sqlite?isolation_level=IMMEDIATE > > But if I now change to postgres my histories cannot be fetched. I'm not sure > how to explain it, but every time I refresh the history panel a new history > is created. > The database connection is fine, it also creates the history entries in the > database, but something is wrong. > > Has someone an idea or can help? > > Thanks a lot! Cheers, > > Sajoscha > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Regarding galaxy wrappers: "lumi", "DESeq" "edgeR"
(Sachit, please don't cross-post to the dev and user lists?) There's a beta version of an edgeR/deseq/voom wrapper and a companion htseq based count matrix maker in the test toolshed statistics section owned by me - fubar. Definitely not quite ready for production but installable and I'd appreciate feedback from anyone brave enough to try them out. Best installed on a test instance you plan on destroying because the R 3.0 dependency is about to swap over to Bjorns better one - should be done later this week when I get some time. Be warned it takes a long time to auto-install - 10-15 minutes or so on my laptop. On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari wrote: > Hi, > > These are the R libraries. Have anyone written a wrapper for these > libraries "lumi", "DESeq" "edgeR"? > > Thanks, > > Sachit > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=en&user=UCUuEM4J ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy Server Processes Dying?
Hi, Galaxy Developers, I have what I'm hoping is a fairly simple inquiry for the Galaxy community; basically, our production Galaxy server processes appear to be dying off over time. Our production Galaxy instance implements apache web scaling features so I have a number of server processes, for example my apache Apache configuration has: BalancerMember http://127.0.0.1:8080 BalancerMember http://127.0.0.1:8081 BalancerMember http://127.0.0.1:8082 BalancerMember http://127.0.0.1:8083 BalancerMember http://127.0.0.1:8084 BalancerMember http://127.0.0.1:8085 Nothing unconventional as I understand it. Similarly, my galaxy config has matching [server:ws3], [server:ws2] configuration blocks for each of these processes. When I restart Galaxy, everything is all fine and good. I'll see a server listening on each one of these ports (if I do something like lsof -i TCP -P, for example). What appears to be happening, is that for whatever reason, these server processes seem to die off over time (i.e eventually nothing is listening on ports 8080-8085). This process can take days, and at the time when no servers are available, Apache will begin throwing 503 service unavailable errors. I am fairly confident this process is gradual, for example I just checked now and the Galaxy was still available, however one server had died (the one on TCP port 8082). I do do have a single separate job manager and two job handlers; at this point I believe this problem to be related to the servers only (i.e. the job manager and job handlers do not app! ear to be crashing). Now, I believe that late last week I might have 'caught' the last server process dying, just by coincidence, although I am not 100% certain. Here is the Traceback as it occurred: galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:47:12,011 (6822/39485.sc01) PBS job state changed from Q to R galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,565 (6822/39485.sc01) PBS job state changed from R to C galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,566 (6822/39485.sc01) PBS job has completed successfully galaxy.jobs DEBUG 2013-07-02 08:54:36,685 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-07-02 08:54:36,812 loading metadata from file for: HistoryDatasetAssociation 6046 galaxy.jobs DEBUG 2013-07-02 08:54:38,153 job 6822 ended galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:49,130 (6812/39473.sc01) PBS job state changed from R to E galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job state changed from E to C galaxy.jobs.runners.pbs ERROR 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job failed: Unknown error: -11 galaxy.jobs.runners ERROR 2013-07-02 08:54:52,267 (unknown) Unhandled exception calling fail_job Traceback (most recent call last): File "/group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 58, in run_next method(arg) File "/group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 560, in fail_job if pbs_job_state.stop_job: AttributeError: 'AsynchronousJobState' object has no attribute 'stop_job' Now, I have some questions regarding this issue; 1) It appears to me that although this is a sub-optimal solution, restarting Galaxy solves this problem (i.e. server processes will be listening after restarting Galaxy). Is it possible, or safe, or sane to just restart a single server on a singe port? Ideally I would actually like to fix the problem that is causing my server processes to crash, although I figured it wouldn't hurt to ask this question regardless. 2) Similar to the question above, is it possible to configure Galaxy in a way that server processes re-spawn in a self-service manner (i.e. is this a feature of Galaxy, for example, because server processes dying regularly is either a known issue or expected and tolerable (but undesired) behaivor)? 3) To me, the error messages above aren't very meaningful, other than the Traceback appears to be PBS-related. Would anybody be able comment on the problem above (i.e. have you seen something like this), or comment on Galaxy server processes dying in general? I have done some brief searching of the Galaxy mailing list for server crashes and did not find anything suggesting this is a common problem. 4) I am not 100% confident at this point that the Traceback above is what killed the server process. Does anybody know of a specific string I can search for (a literal) to identify when a server process actually dies? I have done some basic review of log data (our Galaxy server generates lots of logs), and Traceback does not appear to be a valid string to uniquely identify a server crash (they occur too frequently). I currently have logging configured at DEBUG. In case this is relevant, I am using the following change set for Galaxy: > hg parents changeset: 9320:47ddf167c9f1 branch: stable tag: tip user:Nate Co
Re: [galaxy-dev] Regarding galaxy wrappers: "lumi", "DESeq" "edgeR"
Sorry about that Ross. I would try your wrapper of "edgeR" and would happily report you the bug. Can you make me the link of Galaxy Test Toolshed? Thanks, Sachit On Mon, Jul 8, 2013 at 11:51 AM, Ross wrote: > (Sachit, please don't cross-post to the dev and user lists?) > There's a beta version of an edgeR/deseq/voom wrapper and a companion > htseq based count matrix maker in the test toolshed statistics section > owned by me - fubar. Definitely not quite ready for production but > installable and I'd appreciate feedback from anyone brave enough to try > them out. Best installed on a test instance you plan on destroying because > the R 3.0 dependency is about to swap over to Bjorns better one - should be > done later this week when I get some time. > > Be warned it takes a long time to auto-install - 10-15 minutes or so on my > laptop. > > > On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari < > sachit.techner...@gmail.com> wrote: > >> Hi, >> >> These are the R libraries. Have anyone written a wrapper for these >> libraries "lumi", "DESeq" "edgeR"? >> >> Thanks, >> >> Sachit >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > > > > -- > Ross Lazarus MBBS MPH; > Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 > http://scholar.google.com/citations?hl=en&user=UCUuEM4J > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Regarding galaxy wrappers: "lumi", "DESeq" "edgeR"
Hi Sachit, It's packaged up so it installs from the test toolshed directly via the admin interface of your test instance, but you'll need a good reliable network as there is a lot that gets downloaded and installed. Like other toolshed tools, it would be possible but not advisable to download and unpack the archive at http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_test and install manually but your instance will need R 3.0.1 with bioconductor/edgeR/deseq/limma installed - if it's your first try, it will probably take some time and effort to read all the relevant docs to do that or the R/BioC mailing lists can provide assistance if you need it. The edgeR tool requires a count matrix (reads per sample per contig) as input which the htseq tool http://testtoolshed.g2.bx.psu.edu/view/fubar/htseq_bams_to_count_matrix will generate from any number of aligned bam/sam files and a GTF file with the appropriate gene model. Again, automatic installation from the toolshed is highly recommended, otherwise you'll need to manually install the tool and the dependencies. Note that edgeR requires replicate samples in each comparison group so the absolute minimum is 4 bam files as input to the count matrix maker. p values from analyses without replicates are not interpretable in any biologically meaningful way and are meaningless in terms of the usual frequentist interpretation in my opinion. On Mon, Jul 8, 2013 at 4:28 PM, Sachit Adhikari wrote: > Sorry about that Ross. I would try your wrapper of "edgeR" and would > happily report you the bug. Can you make me the link of Galaxy Test > Toolshed? > > Thanks, > > Sachit > > > On Mon, Jul 8, 2013 at 11:51 AM, Ross wrote: > >> (Sachit, please don't cross-post to the dev and user lists?) >> There's a beta version of an edgeR/deseq/voom wrapper and a companion >> htseq based count matrix maker in the test toolshed statistics section >> owned by me - fubar. Definitely not quite ready for production but >> installable and I'd appreciate feedback from anyone brave enough to try >> them out. Best installed on a test instance you plan on destroying because >> the R 3.0 dependency is about to swap over to Bjorns better one - should be >> done later this week when I get some time. >> >> Be warned it takes a long time to auto-install - 10-15 minutes or so on >> my laptop. >> >> >> On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari < >> sachit.techner...@gmail.com> wrote: >> >>> Hi, >>> >>> These are the R libraries. Have anyone written a wrapper for these >>> libraries "lumi", "DESeq" "edgeR"? >>> >>> Thanks, >>> >>> Sachit >>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >> >> >> >> -- >> Ross Lazarus MBBS MPH; >> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 >> http://scholar.google.com/citations?hl=en&user=UCUuEM4J >> > > -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=en&user=UCUuEM4J ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-user] Regarding galaxy wrappers: "lumi", "DESeq" "edgeR"
Hi Sachit, we are working currently on a DESeq2 wrapper. Not yet finised sorry. There is also a edgeR Test repository from Ross Lazarus under: http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_test Work is in progress, help is welcome! Ciao, Bjoenr > Hi, > > > These are the R libraries. Have anyone written a wrapper for these > libraries "lumi", "DESeq" "edgeR"? > > > Thanks, > > > Sachit > ___ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
