Re: [galaxy-dev] sqlite galaxy datatype metadata
Hi JJ,
I think this is a great idea.
Do you want to make the pull request for this? Or I can merge it into my fork
if you like.
Cheers
Ira
On 26 Jul 2014, at 3:22 am, Jim Johnson johns...@umn.edu wrote:
Ira,
Thanks for getting a sqlite datatype into galaxy. I was wanting to subclass
a sqlite datatype for another application: cistrome CEAS
I had made a sqlite datatype that captured a little metadata that I thought
could be useful for the display peek, and perhaps for filtering.
Is this worth considering?
The peek for a CEAS DB would include table names, column names, and row count:
SQLite Database
GeneTable
(chrom,name,strand,txStart,txEnd,cdsStart,cdsEnd,exonCount,exonStarts,exonEnds,name2)
[24892]
GenomeBGP
(chrom,promoter,bipromoter,downstream,gene,rel_loc,rel_loc_cds,roi,chroms) [7]
GenomeBGS
(chrom,promoter,bipromoter,downstream,gene,rel_loc,rel_loc_cds,roi,Ns) [7]
GenomePieP (chrom,promoter,downstream,gene,enhancer,chroms) [7]
GenomePieS (chrom,promoter,downstream,gene,enhancer,total) [7]
$ hg diff /Users/jj/gxt/gxt/lib/galaxy/datatypes/binary.py
diff -r f002131cb905 lib/galaxy/datatypes/binary.py
--- a/lib/galaxy/datatypes/binary.pyFri Jul 25 12:01:34 2014 -0400
+++ b/lib/galaxy/datatypes/binary.pyFri Jul 25 12:13:05 2014 -0500
@@ -20,7 +20,7 @@
from bx.seq.twobit import TWOBIT_MAGIC_NUMBER, TWOBIT_MAGIC_NUMBER_SWAP,
TWOBIT_MAGIC_SIZE
-from galaxy.datatypes.metadata import MetadataElement
+from galaxy.datatypes.metadata import
MetadataElement,ListParameter,DictParameter
from galaxy.datatypes import metadata
from galaxy.datatypes.sniff import *
import dataproviders
@@ -550,8 +550,36 @@
@dataproviders.decorators.has_dataproviders
class SQlite ( Binary ):
+Class describing a Sqlite database
+MetadataElement( name=tables, default=[], param=ListParameter,
desc=Database Tables, readonly=True, visible=True, no_value=[] )
+MetadataElement( name=table_columns, default={}, param=DictParameter,
desc=Database Table Columns, readonly=True, visible=True, no_value={} )
+MetadataElement( name=table_row_count, default={},
param=DictParameter, desc=Database Table Row Count, readonly=True,
visible=True, no_value={} )
file_ext = sqlite
+def init_meta( self, dataset, copy_from=None ):
+Binary.init_meta( self, dataset, copy_from=copy_from )
+
+def set_meta( self, dataset, overwrite = True, **kwd ):
+try:
+tables = []
+columns = dict()
+rowcounts = dict()
+conn = sqlite3.connect(dataset.file_name)
+c = conn.cursor()
+tables_query = SELECT name,sql FROM sqlite_master WHERE
type='table' ORDER BY name
+rslt = c.execute(tables_query).fetchall();
+for table,sql in rslt:
+tables.append(table)
+columns[table] = re.sub('^.*\((.*)\)$','\\1',sql).split(',')
+for table in tables:
+row_query = SELECT count(*) FROM %s % table
+rowcounts[table] = c.execute(row_query).fetchone()[0];
+dataset.metadata.tables = tables
+dataset.metadata.table_columns = columns
+dataset.metadata.table_row_count = rowcounts
+except Exception, exc:
+pass
+
# Connects and runs a query that should work on any real database
# If the file is not sqlite, an exception will be thrown and the sniffer
will return false
def sniff( self, filename ):
@@ -567,7 +595,14 @@
def set_peek( self, dataset, is_multi_byte=False ):
if not dataset.dataset.purged:
-dataset.peek = SQLite Database
+lines = ['SQLite Database']
+if dataset.metadata.tables:
+for table in dataset.metadata.tables:
+try:
+lines.append('%s (%s) [%s]' %
(table,','.join(dataset.metadata.table_columns.get(table,[])),dataset.metadata.table_row_count[table]))
+except:
+continue
+dataset.peek = '\n'.join(lines)
dataset.blurb = data.nice_size( dataset.get_size() )
else:
dataset.peek = 'file does not exist'
Thanks,
JJ
--
James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Working with setup_ruby_environment tag
Hi Bjoern, Ross just gave me some pointers and I realised that I could just set these environment variables myself by doing action type=set_environment environment_variable action=prepend_to name=PATH$INSTALL_DIR/bin/environment_variable environment_variable action=set_to name=GEM_HOME$INSTALL_DIR//environment_variable /action In my protk tool definition. RUBYLIB isn't set ... hopefully this will not be an issue since I guess that's mostly just needed for Gem installation/compilation. This part works now :) Thanks Ira On 30/10/2013, at 3:51 PM, Ira Cooke iraco...@gmail.com wrote: Hi Bjoern, Changing the title to reflect the proper topic I see now that the gem executables are installed under $INSTALLDIR/bin so I added this to the environment for my protk install. That got me to the point where my executable would run ... but now I'm seeing that in my env.sh file I don't really have my ruby environment setup properly (ie RUBYLIB and GEM_HOME are not set). This resulted in my tool running but it couldn't find my gem. So I tried adding ruby explicitly as a tool dependency for my downstream tool (this seems wrong though ... since it is required by protk ... and if I include protk I should get it automatically right?). After doing this though I was getting an error on installing protk about List index out of range. Just to summarize, I can't see how to setup my dependencies properly so that the executables installed by protk get run with the correct environment. The repository I'm trying to install is http://testtoolshed.g2.bx.psu.edu/view/iracooke/omssa I'm sure I've just got muddled somewhere ... but it feels like I'm just messing around and getting it wrong. I'd rather do it in the way you envisioned. If you could give me some pointers on what I've done wrong it would help alot Thanks Ira On 29/10/2013, at 8:14 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Ahhh ... Thanks .. I can't believe I missed that. Thanks for spotting it. No problem! I'm not sure I understand enough about how this works to answer your ruby_home question. Ill play with this more tomorrow and see how things work with my tools and maybe have more feedback then Sure, thanks for working on that! Here is the promised trello card: https://trello.com/c/l9ZlGa8R Ciao, Bjoern Cheers Ira On 29 Oct 2013, at 7:17 pm, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, as far as I can see you missed the actions tag. That results in no log message at all, since no action type is executed. Galaxy should probably give an error here. Or better validate that during tool upload. I will fill a Trello Card for it. The following code snipped worked for me: ?xml version=1.0? tool_dependency package name=protk version=1.2.5 install version=1.0 actions action type=setup_ruby_environment repository changeset_revision=a0494c6e1c63 name=package_ruby_2_0 owner=bgruening toolshed=http://testtoolshed.g2.bx.psu.edu; package name=ruby version=2.0 / /repository packageprotk=1.2.6.pre1/package /action /actions /install readme Installs version 1.2.5 of the protk rubygem from rubygems.org /readme /package /tool_dependency Keep in mind that the repository tag should be filled. If you have a actions and action tag it will be filled automatically by the TS during upload. But better to check :) To populate the RUBY_HOME env var ... do you think its better to set it via the installation automatically? My idea was to just require ruby in the tool? If you want to set it in your protk definition you need to use the $ENV[] construct. Cheers, Bjoern Hi Bjoern, Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. Cheers Ira On 29/10/2013, at 10:06 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, I can reproduce that error here, will try to study it tomorrow more deeply. For the meantime can you try to remove the following lines? action type=set_environment_for_install repository name=package_ruby_2_0 owner=bgruening package name=ruby version=2.0 / /repository /action It's not necessary. Will get back to you tomorrow. Bjoern Hi All, The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the Installing phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool
Re: [galaxy-dev] Working with setup_ruby_environment tag
Hi Bjoern, I'll test, but I wonder if the problem is that GEM_PATH is set instead of GEM_HOME. I think GEM_HOME needs to be set also. It's zoo of environment variables out there :)!! Also ... I think GEM_HOME should be set using set_to not prepend_to ... I am using the latest galaxy-central (default branch). Cheers Ira On 31 Oct 2013, at 12:33 am, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Ira, that is strange, because PATH and GEM_HOME should be set automatically for you. Please have a look at https://github.com/bgruening/galaxytools/blob/master/orphan_tool_dependencies/package_ruby2_bioruby_1_4/tool_dependencies.xml http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_ruby2_bioruby_1_4 That definition results on my test environment in: GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH And the code for that is here: https://bitbucket.org/galaxy/galaxy-central/src/77d58fdd1c2e0dfde374276209279c5e94b1e4c0/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py?at=default#cl-431 Are you using the latest galaxy central? Regarding RUBYLIB, I think we need to set it to specify the right ruby version, because many can be installed in parallel. Some holds for RUBYHOME or? I see your point in having it set automatically and came with the attached patch. Here I need to assume that package_ruby_x_x will have RUBYLIB and RUBYHOME specified, but I think that is a save assumption. Nevertheless, we should document it. With that new patch you will end up with RUBYLIB=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/; export RUBYLIB RUBYHOME=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63; export RUBYHOME GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH PATH=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH Thanks very much for your feedback. I do not have any ruby repository so I really appropriate your testing! Bjoern Hi Bjoern, Ross just gave me some pointers and I realised that I could just set these environment variables myself by doing action type=set_environment environment_variable action=prepend_to name=PATH$INSTALL_DIR/bin/environment_variable environment_variable action=set_to name=GEM_HOME$INSTALL_DIR//environment_variable /action In my protk tool definition. RUBYLIB isn't set ... hopefully this will not be an issue since I guess that's mostly just needed for Gem installation/compilation. This part works now :) Thanks Ira On 30/10/2013, at 3:51 PM, Ira Cooke iraco...@gmail.com wrote: Hi Bjoern, Changing the title to reflect the proper topic I see now that the gem executables are installed under $INSTALLDIR/bin so I added this to the environment for my protk install. That got me to the point where my executable would run ... but now I'm seeing that in my env.sh file I don't really have my ruby environment setup properly (ie RUBYLIB and GEM_HOME are not set). This resulted in my tool running but it couldn't find my gem. So I tried adding ruby explicitly as a tool dependency for my downstream tool (this seems wrong though ... since it is required by protk ... and if I include protk I should get it automatically right?). After doing this though I was getting an error on installing protk about List index out of range. Just to summarize, I can't see how to setup my dependencies properly so that the executables installed by protk get run with the correct environment. The repository I'm trying to install is http://testtoolshed.g2.bx.psu.edu/view/iracooke/omssa I'm sure I've just got muddled somewhere ... but it feels like I'm just messing around and getting it wrong. I'd rather do it in the way you envisioned. If you could give me some pointers on what I've done wrong it would help alot Thanks Ira On 29/10/2013, at 8:14 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Ahhh ... Thanks .. I can't believe I missed that. Thanks for spotting it. No problem! I'm not sure I understand enough about how this works to answer your ruby_home question. Ill play with this more tomorrow and see how things work with my tools and maybe have more feedback then Sure
Re: [galaxy-dev] Working with setup_ruby_environment tag
Hi Bjoern, I've just done some more testing of my tool with a fresh install and a fresh checkout of galaxy. After install of this tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 I have an env.sh that looks like GEM_PATH=/home/vagrant/tool_dependencies/protk/1.2.5/iracooke/package_protk_1_2_5/1100ac98dfd2:$GEM_PATH; export GEM_PATH PATH=/home/vagrant/tool_dependencies/protk/1.2.5/iracooke/package_protk_1_2_5/1100ac98dfd2/bin:$PATH; export PATH Which seems to be enough for me to run the executables in my gem.So I must be wrong about GEM_HOME needing to be set ... sorry for the false alarm there. The thing that I'm now having trouble with is that all the other set_environment tags in my tool definition are ignored. In my tool_dependencies.xml I have action type=set_environment environment_variable action=prepend_to name=PATH$INSTALL_DIR/bin/environment_variable environment_variable action=set_to name=GEM_HOME$INSTALL_DIR//environment_variable environment_variable action=set_to name=PROTK_BLAST_ROOTFROMPATH/environment_variable /action Admittedly some of that stuff is redundant ( I now realise ) ... but other stuff (eg PROTK_BLAST_ROOT ) is important ... but it gets ignored. Is this intended? Seems like I should be able to use the set_environment action as well as the setup_ruby_environment action in the same tool definition? Hope you can clarify. Thanks Ira On 31/10/2013, at 6:06 AM, Ira Cooke mudflatsoftw...@gmail.com wrote: Hi Bjoern, I saw the same SO post ... but it says that GEM_PATH can be multiple paths but one of them should be GEM_HOME. Under your system (I think) GEM_HOME remains unset? Anyway ... I'll test again to double check that this is required for my tool to work. Cheers Ira On 31 Oct 2013, at 6:00 am, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, Hi Bjoern, I'll test, thanks! but I wonder if the problem is that GEM_PATH is set instead of GEM_HOME. I think GEM_HOME needs to be set also. It's zoo of environment variables out there :)!! Mh according to http://stackoverflow.com/questions/11277227/whats-the-difference-between-gem-home-and-gem-path GEM_PATH will contain GEM_HOME and GEM_HOME is used for installation, and I use it for installation of GEMs, See the code. If you think GEM_HOME is necessary to export I will do so. Otherwise I would not export to much. Thanks! Bjoern Also ... I think GEM_HOME should be set using set_to not prepend_to ... I am using the latest galaxy-central (default branch). Cheers Ira On 31 Oct 2013, at 12:33 am, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Ira, that is strange, because PATH and GEM_HOME should be set automatically for you. Please have a look at https://github.com/bgruening/galaxytools/blob/master/orphan_tool_dependencies/package_ruby2_bioruby_1_4/tool_dependencies.xml http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_ruby2_bioruby_1_4 That definition results on my test environment in: GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH And the code for that is here: https://bitbucket.org/galaxy/galaxy-central/src/77d58fdd1c2e0dfde374276209279c5e94b1e4c0/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py?at=default#cl-431 Are you using the latest galaxy central? Regarding RUBYLIB, I think we need to set it to specify the right ruby version, because many can be installed in parallel. Some holds for RUBYHOME or? I see your point in having it set automatically and came with the attached patch. Here I need to assume that package_ruby_x_x will have RUBYLIB and RUBYHOME specified, but I think that is a save assumption. Nevertheless, we should document it. With that new patch you will end up with RUBYLIB=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/; export RUBYLIB RUBYHOME=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63; export RUBYHOME GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH PATH=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH Thanks very much for your feedback. I do not have any ruby repository so I really appropriate
Re: [galaxy-dev] What to do with toolshed tools that get stuck Installing
Ahhh ... Thanks .. I can't believe I missed that. Thanks for spotting it. I'm not sure I understand enough about how this works to answer your ruby_home question. Ill play with this more tomorrow and see how things work with my tools and maybe have more feedback then Cheers Ira On 29 Oct 2013, at 7:17 pm, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, as far as I can see you missed the actions tag. That results in no log message at all, since no action type is executed. Galaxy should probably give an error here. Or better validate that during tool upload. I will fill a Trello Card for it. The following code snipped worked for me: ?xml version=1.0? tool_dependency package name=protk version=1.2.5 install version=1.0 actions action type=setup_ruby_environment repository changeset_revision=a0494c6e1c63 name=package_ruby_2_0 owner=bgruening toolshed=http://testtoolshed.g2.bx.psu.edu; package name=ruby version=2.0 / /repository packageprotk=1.2.6.pre1/package /action /actions /install readme Installs version 1.2.5 of the protk rubygem from rubygems.org /readme /package /tool_dependency Keep in mind that the repository tag should be filled. If you have a actions and action tag it will be filled automatically by the TS during upload. But better to check :) To populate the RUBY_HOME env var ... do you think its better to set it via the installation automatically? My idea was to just require ruby in the tool? If you want to set it in your protk definition you need to use the $ENV[] construct. Cheers, Bjoern Hi Bjoern, Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. Cheers Ira On 29/10/2013, at 10:06 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, I can reproduce that error here, will try to study it tomorrow more deeply. For the meantime can you try to remove the following lines? action type=set_environment_for_install repository name=package_ruby_2_0 owner=bgruening package name=ruby version=2.0 / /repository /action It's not necessary. Will get back to you tomorrow. Bjoern Hi All, The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the Installing phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but since the tool install never completes I can never see this. What I usually do is try to recreate the environment that galaxy sees during tool install in a normal shell .. and then run it manually .. which sometimes tells me the problem ... but not always (see below). The longer version of my problem is below. In my case I am struggling with the following tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 Which uses the new setup_ruby_environment tag ... so the tool should do the following (1) Install the dependency .. Ruby 2.0 ... which it does successfully (2) Install my protk rubygem from rubygems.org .. which it gets stuck doing. The last few lines from my galaxy server log during the failed install are; 10.0.2.2 - - [28/Oct/2013:21:54:26 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh 10.0.2.2 - - [28/Oct/2013:21:54:30 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export
[galaxy-dev] Working with setup_ruby_environment tag
Hi Bjoern, Changing the title to reflect the proper topic I see now that the gem executables are installed under $INSTALLDIR/bin so I added this to the environment for my protk install. That got me to the point where my executable would run ... but now I'm seeing that in my env.sh file I don't really have my ruby environment setup properly (ie RUBYLIB and GEM_HOME are not set). This resulted in my tool running but it couldn't find my gem. So I tried adding ruby explicitly as a tool dependency for my downstream tool (this seems wrong though ... since it is required by protk ... and if I include protk I should get it automatically right?). After doing this though I was getting an error on installing protk about List index out of range. Just to summarize, I can't see how to setup my dependencies properly so that the executables installed by protk get run with the correct environment. The repository I'm trying to install is http://testtoolshed.g2.bx.psu.edu/view/iracooke/omssa I'm sure I've just got muddled somewhere ... but it feels like I'm just messing around and getting it wrong. I'd rather do it in the way you envisioned. If you could give me some pointers on what I've done wrong it would help alot Thanks Ira On 29/10/2013, at 8:14 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Ahhh ... Thanks .. I can't believe I missed that. Thanks for spotting it. No problem! I'm not sure I understand enough about how this works to answer your ruby_home question. Ill play with this more tomorrow and see how things work with my tools and maybe have more feedback then Sure, thanks for working on that! Here is the promised trello card: https://trello.com/c/l9ZlGa8R Ciao, Bjoern Cheers Ira On 29 Oct 2013, at 7:17 pm, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, as far as I can see you missed the actions tag. That results in no log message at all, since no action type is executed. Galaxy should probably give an error here. Or better validate that during tool upload. I will fill a Trello Card for it. The following code snipped worked for me: ?xml version=1.0? tool_dependency package name=protk version=1.2.5 install version=1.0 actions action type=setup_ruby_environment repository changeset_revision=a0494c6e1c63 name=package_ruby_2_0 owner=bgruening toolshed=http://testtoolshed.g2.bx.psu.edu; package name=ruby version=2.0 / /repository packageprotk=1.2.6.pre1/package /action /actions /install readme Installs version 1.2.5 of the protk rubygem from rubygems.org /readme /package /tool_dependency Keep in mind that the repository tag should be filled. If you have a actions and action tag it will be filled automatically by the TS during upload. But better to check :) To populate the RUBY_HOME env var ... do you think its better to set it via the installation automatically? My idea was to just require ruby in the tool? If you want to set it in your protk definition you need to use the $ENV[] construct. Cheers, Bjoern Hi Bjoern, Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. Cheers Ira On 29/10/2013, at 10:06 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, I can reproduce that error here, will try to study it tomorrow more deeply. For the meantime can you try to remove the following lines? action type=set_environment_for_install repository name=package_ruby_2_0 owner=bgruening package name=ruby version=2.0 / /repository /action It's not necessary. Will get back to you tomorrow. Bjoern Hi All, The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the Installing phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but since the tool install never completes I can never see this. What I usually do is try to recreate the environment that galaxy sees during tool install in a normal shell .. and then run it manually .. which sometimes tells me the problem ... but not always (see below). The longer version of my problem is below. In my case I am struggling with the following tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 Which uses the new setup_ruby_environment tag ... so the tool should do the following (1) Install the dependency .. Ruby 2.0 ... which it does successfully (2) Install my protk rubygem from
[galaxy-dev] What to do with toolshed tools that get stuck Installing
Hi All, The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the Installing phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but since the tool install never completes I can never see this. What I usually do is try to recreate the environment that galaxy sees during tool install in a normal shell .. and then run it manually .. which sometimes tells me the problem ... but not always (see below). The longer version of my problem is below. In my case I am struggling with the following tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 Which uses the new setup_ruby_environment tag ... so the tool should do the following (1) Install the dependency .. Ruby 2.0 ... which it does successfully (2) Install my protk rubygem from rubygems.org .. which it gets stuck doing. The last few lines from my galaxy server log during the failed install are; 10.0.2.2 - - [28/Oct/2013:21:54:26 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh 10.0.2.2 - - [28/Oct/2013:21:54:30 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdQaLI7O ruby version 2.0 installed in /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63 [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdJYyhO0 10.0.2.2 - - [28/Oct/2013:21:54:34 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 10.0.2.2 - - [28/Oct/2013:21:47:22 +] POST /admin_toolshed/manage_repositories HTTP/1.1 302 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 10.0.2.2 - - [28/Oct/2013:21:54:37 +] GET /admin_toolshed/monitor_repository_installation?tool_shed_repository_ids=f597429621d6eb2btool_shed_repository_ids=f2db41e1fa331b3e HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 With a normal tool install I would see the last line of this output repeat lots of times as installation is monitored ... but for some reason in this case monitoring just shuts down and I don't see any further updates. If I recreate the situation on the command-line (eg by manually setting the RUBYLIB and GEM_HOME environment variables appropriately I can install my gem without any errors. More details on my setup: I'm using a vagrant VM for testing which is based on Ubuntu
Re: [galaxy-dev] What to do with toolshed tools that get stuck Installing
Hi Bjoern, Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. Cheers Ira On 29/10/2013, at 10:06 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Ira, I can reproduce that error here, will try to study it tomorrow more deeply. For the meantime can you try to remove the following lines? action type=set_environment_for_install repository name=package_ruby_2_0 owner=bgruening package name=ruby version=2.0 / /repository /action It's not necessary. Will get back to you tomorrow. Bjoern Hi All, The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the Installing phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but since the tool install never completes I can never see this. What I usually do is try to recreate the environment that galaxy sees during tool install in a normal shell .. and then run it manually .. which sometimes tells me the problem ... but not always (see below). The longer version of my problem is below. In my case I am struggling with the following tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 Which uses the new setup_ruby_environment tag ... so the tool should do the following (1) Install the dependency .. Ruby 2.0 ... which it does successfully (2) Install my protk rubygem from rubygems.org .. which it gets stuck doing. The last few lines from my galaxy server log during the failed install are; 10.0.2.2 - - [28/Oct/2013:21:54:26 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh 10.0.2.2 - - [28/Oct/2013:21:54:30 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdQaLI7O ruby version 2.0 installed in /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63 [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdJYyhO0 10.0.2.2 - - [28/Oct/2013:21:54:34 +] POST /admin_toolshed/repository_installation_status_updates HTTP/1.1 200 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 10.0.2.2 - - [28/Oct/2013:21:47:22 +] POST /admin_toolshed/manage_repositories HTTP/1.1 302 - http://localhost:8088/admin_toolshed/prepare_for_install; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
[galaxy-dev] Galaxy Proteomics mailing list
Dear All, The galaxy community now has a mailing list dedicated to proteomics. If you're using galaxy to do proteomics, or if you are thinking of doing so then this list if for you. In case you missed it, there is also a proteomics wiki page at http://wiki.galaxyproject.org/Proteomics To subscribe to the galaxy-proteomics list please add your email address to the list here: http://lists.bx.psu.edu/listinfo/galaxy-proteomics See http://wiki.g2.bx.psu.edu/Mailing%20Lists for more information on this and other Galaxy mailing lists, including links to searchable archives of all Galaxy lists. Many thanks to Dave C. and the Galaxy team for setting this up. Thanks Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problems running functional tests
Oops ... I should have thought of that Thanks .. it works now. Ira On 04/06/2013, at 11:20 PM, Dave Bouvier d...@bx.psu.edu wrote: Ira, Yes, I should have mentioned that due to the way functional tests are executed, an environment variable needs to be explicitly exported before the functional test framework will recognize it. The syntax is very nearly the same: export GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies ; sh run_functional_tests.sh -installed --Dave B. On 6/3/13 00:25:53.000, Ira Cooke wrote: Hi Dave, Just following up on this (now quite a bit later sorry). I can't seem to get things to work using your suggested workaround. Using a fresh checkout of galaxy-central 9931:716f29e4c77d I run GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies sh run_functional_tests.sh -installed In my universe_wsgi.ini I have tool_dependency_dir = /home/iracooke/tool_dependencies When running tests I find that the dependencies fail to be resolved (ie ... I get errors like the following). galaxy.jobs.handler INFO 2013-04-28 11:01:17,955 (3) Job dispatched galaxy.tools DEBUG 2013-04-28 11:01:18,594 Building dependency shell command for dependency 'galaxy_protk' galaxy.tools WARNING 2013-04-28 11:01:18,595 Failed to resolve dependency on 'galaxy_protk', ignoring galaxy.tools DEBUG 2013-04-28 11:01:18,596 Building dependency shell command for dependency 'trans_proteomic_pipeline' galaxy.tools WARNING 2013-04-28 11:01:18,596 Failed to resolve dependency on 'trans_proteomic_pipeline', ignoring When I run the tool myself (ie from galaxy ui not from the test script) it resolves the dependencies just fine. Any idea what am I doing wrong? Thanks Ira On 10/05/2013, at 12:10 AM, Dave Bouvier d...@bx.psu.edu wrote: Ira, There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini. This issue is documented in https://trello.com/c/eVT3QWJF --Dave B. On 5/8/13 20:08:08.000, Ira Cooke wrote: Hi All, I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies. The tool I'm considering is xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods='57.021464__at__C' --fix-mods='15.994915__at__M' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///) Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine. When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ). Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods='15.994915__at__M, ' --fix-mods='57.021464__at__C' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database
Re: [galaxy-dev] Problems running functional tests
Hi Dave, Just following up on this (now quite a bit later sorry). I can't seem to get things to work using your suggested workaround. Using a fresh checkout of galaxy-central 9931:716f29e4c77d I run GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies sh run_functional_tests.sh -installed In my universe_wsgi.ini I have tool_dependency_dir = /home/iracooke/tool_dependencies When running tests I find that the dependencies fail to be resolved (ie ... I get errors like the following). galaxy.jobs.handler INFO 2013-04-28 11:01:17,955 (3) Job dispatched galaxy.tools DEBUG 2013-04-28 11:01:18,594 Building dependency shell command for dependency 'galaxy_protk' galaxy.tools WARNING 2013-04-28 11:01:18,595 Failed to resolve dependency on 'galaxy_protk', ignoring galaxy.tools DEBUG 2013-04-28 11:01:18,596 Building dependency shell command for dependency 'trans_proteomic_pipeline' galaxy.tools WARNING 2013-04-28 11:01:18,596 Failed to resolve dependency on 'trans_proteomic_pipeline', ignoring When I run the tool myself (ie from galaxy ui not from the test script) it resolves the dependencies just fine. Any idea what am I doing wrong? Thanks Ira On 10/05/2013, at 12:10 AM, Dave Bouvier d...@bx.psu.edu wrote: Ira, There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini. This issue is documented in https://trello.com/c/eVT3QWJF --Dave B. On 5/8/13 20:08:08.000, Ira Cooke wrote: Hi All, I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies. The tool I'm considering is xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods='57.021464__at__C' --fix-mods=' 15.994915__at__M' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///) Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine. When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ). Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods='15.994915__at__M, ' --fix-mods='57.021464__at__C' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M,' --fix-mods=' 57.021464__at__C' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR Is there something wrong with the way I've defined my tool dependencies for this tool
Re: [galaxy-dev] [galaxy-iuc] Suggestion for improved feedback on failing tests
Hi Peter, Yes that's most likely part of it ... thanks for the link to that thread. I think I am suffering from the same issue. ... but I'm also assuming when my test results come back I'll still need to figure out where my repository dependencies failed. Last test result I saw it looked like a failure compiling ruby ... but it's hard to know what's missing to fix it. Cheers Ira On 14/05/2013, at 6:54 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, May 14, 2013 at 3:32 AM, Ira Cooke iraco...@gmail.com wrote: Hi All, My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem You may be suffering from the same missing test results problem as me, see the long thread Missing test results on (Test) Tool Shed: http://dev.list.galaxyproject.org/Missing-test-results-on-Test-Tool-Shed-td4659531.html Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Suggestion for improved feedback on failing tests
Hi All, My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem One potential improvement that I think could make it easier to debug this situation would be to show test status for repositories even when they contain no tools (the test would simply attempt an install and show the installation log under the test details). This would be particularly useful for repositories that exist purely to install a dependency eg http://testtoolshed.g2.bx.psu.edu/view/iracooke/galaxy_protk Naturally a tool with proper functional tests would fail if its dependency installations fail .. but it would be a huge help in narrowing down the issue to be able to see where those failures occurred. Another point is that this is pretty much the only way of getting feedback on why a tool might fail installation on the build-bot without having a perfect clone of the build-bot locally. Cheers Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-iuc] Script to help maintain toolshed repos across toolsheds
Hi Bjoern, It's good to know I'm not the only one to encounter this. I think it tends to bite when you have alot of dependencies. I use the same method as you (uploading one repo .. then another and so on) ... using the script each time to automate the upload. BTW I made a little addition to the script today so it automatically commits and checks in the changes via mercurial ... that makes the workflow really quite smooth. Your suggestion of possibly allowing the revision tag to be blank makes alot of sense ... it could simply imply (always use the latest available). That would certainly make things much much easier for those who aren't using a script. Cheers Ira On 07/05/2013, at 7:49 PM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi, nice work Ira! That problem also bothers me. I have written a similar script in python, but its not documented ;) I really think a more advanced solution like Peter's are needed midterm. To fix the revision problems: What about leaving the revision tag blank means that the toolshed should insert the latest revision of a tool dependency during upload. For me the vast majority of repo uploads referring to the latest dependencies of other repo's. So I ending up with uploading one repo, running my script to insert the new revision tag, uploading the second, rerun the script and so on ... Ciao, Bjoern Helo Peter and Ira, These are great ideas and contributions, and I'll make sure to incorporate some version of them into the Tool Shed framework as soon as possible. I've created the following Trello card for this. Thanks! https://trello.com/card/toolshed-enable-dependency-definitions-across-tool-sheds/506338ce32ae458f6d15e4b3/838 On May 6, 2013, at 11:05 AM, Peter Cock wrote: On Thu, May 2, 2013 at 10:24 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, May 2, 2013 at 7:16 AM, Ira Cooke iraco...@gmail.com wrote: Hi all, I've written a script to help deal with the problem of maintaining toolshed tools across multiple toolsheds (eg test and release) The problem I encountered was that switching between test and production versions of a suite of tools can be quite painful because every repository definition like this repository toolshed=http://toolshed.g2.bx.psu.edu; name=proteomics_datatypes owner=iracooke changeset_revision=463328a6967f/ needs to be updated to a different toolshed url and (by extension) a different changeset revision. The idea with this script is that you should be able to point it at a directory containing a toolshed repository and it will create a copy of that repository in which the toolshed urls (and changeset revisions) have been updated to correct values for a different toolshed. I'm not sure how others are dealing with this issue (perhaps there is another easier way) .. but I've found this helped me alot so I thought I'd share https://bitbucket.org/iracooke/galaxy_repo_bundler/ Cheers Ira Thanks Ira, I've not made as heavy use of inter-repository dependencies as you, but thus far I have ignored the problem (only a couple of my repositories are affected), in the hope this limitation will be fixed sooner rather than later. Peter Thinking out loud, another way to solve this would be to allow multiple equivalent repository entries as a group where any one would be OK. e.g. For v0.0.5 of my seq_filter_by_id tool, repository toolshed=http://toolshed.g2.bx.psu.edu; owner=peterjc name=seq_filter_by_id changeset_revision=abdd608c869b/ for http://toolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id/abdd608c869b or: repository toolshed=http://testtoolshed.g2.bx.psu.edu; owner=peterjc name=seq_filter_by_id changeset_revision=66d1ca92fb38/ for http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id/66d1ca92fb38 Something like this maybe?: any_one_of repository toolshed=http://toolshed.g2.bx.psu.edu; owner=peterjc name=seq_filter_by_id changeset_revision=abdd608c869b/ repository toolshed=http://testtoolshed.g2.bx.psu.edu; owner=peterjc name=seq_filter_by_id changeset_revision=66d1ca92fb38/ /any_one_of Of course more generally we might also post this on a local public Tool Shed as well. The point is then the same repository_dependencies.xml could be used on either tool shed without modification. Fixing declaring a dependency on an external Tool Shed would be better - but perhaps we'll need both or some sort of mirroring federated system in the long term? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists
[galaxy-dev] Script to help maintain toolshed repos across toolsheds
Hi all, I've written a script to help deal with the problem of maintaining toolshed tools across multiple toolsheds (eg test and release) The problem I encountered was that switching between test and production versions of a suite of tools can be quite painful because every repository definition like this repository toolshed=http://toolshed.g2.bx.psu.edu; name=proteomics_datatypes owner=iracooke changeset_revision=463328a6967f/ needs to be updated to a different toolshed url and (by extension) a different changeset revision. The idea with this script is that you should be able to point it at a directory containing a toolshed repository and it will create a copy of that repository in which the toolshed urls (and changeset revisions) have been updated to correct values for a different toolshed. I'm not sure how others are dealing with this issue (perhaps there is another easier way) .. but I've found this helped me alot so I thought I'd share https://bitbucket.org/iracooke/galaxy_repo_bundler/ Cheers Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-iuc] Proteomics category on test toolshed
Hi Greg, I should probably add to this some background info about my dependencies .. which are a bit weird. Basically I have a bunch of repositories that all depend on a rubygem I wrote. This gem is installed under its own gemset using rvm. Installation of this repository is essential before any other repositories can be installed because it contains all the instructions on how to build stuff etc. As you've seen, I have defined this repo as a dependency of all my repositories .. which works fine .. but when I have a chain of dependencies I have problems. (eg Repo omssa - protk_omssa - galaxy_protk ) .. I define omssa so that it depends on both galaxy_protk and protk_omssa ... and in repo protk_omssa I tell it that it depends on galaxy_protk. When I go to do an install of protk_omssa it works fine ... but when I try to install omssa without first installing the other dependencies it attempts to install other repos before installing galaxy_protk. I hope that makes sense. As another issue I realise that it might be a bit dodgy to install things under the galaxy users home directory. I tried to get around this but it's tricky. RVM for instance doesn't seem to like being installed anywhere other than ~/.rvm . In the end I also decided to keep my ~/.protk directory in its same position because this allows an admin user to more easily manage centrally installed databases (that should be in ~/.protk/Databases ). I have taken some care about encapsulating dependencies for different tool versions though. This is done by installing the protk rubygem into a gemset that is labelled according to the protk version. I also install all other protk related tools into the normal galaxy tool directories ... and protk knows where these are by virtue of setting environment variables (eg PROTK_TPP_DIR) in the relevant env.sh scripts. Anyway I'm reasonably happy with the dependency system .. it would be nice to get the issue with chained dependencies worked out though. Cheers Ira On 06/03/2013, at 1:25 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Ira, Thanks for all of the work you've been doing n your repositories - I've been getting email updates from your repository changes. I'm not quite sure what is causing this behavior. I've taken a glance at your many dependency definitions, and they look correct, so I'm not seeing why certain directory paths are required by some repositories. Required paths like this are handled with complex repository dependencies (which you have correctly defined), so you may have uncovered a corner-case issue that I didn't encounter during development. This may take a bit of time to figure out and resolve, but I'll let you know as soon as we uncover and resolve the problem. Thanks! Greg Von Kuster On Mar 5, 2013, at 1:49 AM, Ira Cooke wrote: Hi, I've just uploaded a set of updates to my proteomics tools on the toolshed. These contain my best attempt to implement automatic dependency installation ... and for the most part it seems to work. I have encountered the following issues though (1) I don't see any way to ensure that one repository_dependency is installed before all others. In my case this is important because I have an installer package (galaxy_protk) which needs to be installed before other package installs will work. For example .. try installing http://toolshed.g2.bx.psu.edu/view/iracooke/omssa (2) Running multiple repo installs at once is not reliable (sorry this is so vague). When I have many dependencies being installed at once I get the following error (for example) Could not cd to ./database/tmp/tmpbIqBW8/galaxy_protk/./database/tmp/tmp3vhKn6/galaxy_protk It looks like a path is not being constructed correctly. If I wait until the currently installed repo is finished .. then try uninstalling and reinstalling it works. Sorry this is vague but if you try installing any of the following repositories you should see what I mean http://toolshed.g2.bx.psu.edu/view/iracooke/xtandem http://toolshed.g2.bx.psu.edu/view/iracooke/msgfplus http://toolshed.g2.bx.psu.edu/view/iracooke/tpp_prophets Anyway .. and just so it doesn't just sound like I'm complaining ... I think the dependency installation stuff is hugely useful. It took me a while to figure out all the details .. but I think it makes sense now. Cheers Ira ___ galaxy-iuc mailing list galaxy-...@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-iuc ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Proteomics category on test toolshed
Hi, I've just uploaded a set of updates to my proteomics tools on the toolshed. These contain my best attempt to implement automatic dependency installation ... and for the most part it seems to work. I have encountered the following issues though (1) I don't see any way to ensure that one repository_dependency is installed before all others. In my case this is important because I have an installer package (galaxy_protk) which needs to be installed before other package installs will work. For example .. try installing http://toolshed.g2.bx.psu.edu/view/iracooke/omssa (2) Running multiple repo installs at once is not reliable (sorry this is so vague). When I have many dependencies being installed at once I get the following error (for example) Could not cd to ./database/tmp/tmpbIqBW8/galaxy_protk/./database/tmp/tmp3vhKn6/galaxy_protk It looks like a path is not being constructed correctly. If I wait until the currently installed repo is finished .. then try uninstalling and reinstalling it works. Sorry this is vague but if you try installing any of the following repositories you should see what I mean http://toolshed.g2.bx.psu.edu/view/iracooke/xtandem http://toolshed.g2.bx.psu.edu/view/iracooke/msgfplus http://toolshed.g2.bx.psu.edu/view/iracooke/tpp_prophets Anyway .. and just so it doesn't just sound like I'm complaining ... I think the dependency installation stuff is hugely useful. It took me a while to figure out all the details .. but I think it makes sense now. Cheers Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-iuc] Installing repositories in a specific order
Hi Peter, The problem is that I have multiple repositories in my repository_dependency file ... but they need to be installed in a specific order. I can't seem to get galaxy to install them in the correct order. Ira PS: Sorry about the totally irrelevant subject line ... I've changed it. On 05/03/2013, at 9:01 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, Mar 5, 2013 at 6:49 AM, Ira Cooke iraco...@gmail.com wrote: Hi, I've just uploaded a set of updates to my proteomics tools on the toolshed. These contain my best attempt to implement automatic dependency installation ... and for the most part it seems to work. I have encountered the following issues though (1) I don't see any way to ensure that one repository_dependency is installed before all others. In my case this is important because I have an installer package (galaxy_protk) which needs to be installed before other package installs will work. For example .. try installing http://toolshed.g2.bx.psu.edu/view/iracooke/omssa I think in this case you just need to use a repository_dependencies.xml file (but note currently this only lets you point at other repositories in the same Tool Shed instance, i.e. http://toolshed.g2.bx.psu.edu/ here): http://wiki.galaxyproject.org/DefiningRepositoryDependencies Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Is the galaxy main toolshed compatible with local installs of galaxy-dist?
I was wondering whether the general policy is to keep the toolshed at toolshed.g2.bx.psu.edu compatible with galaxy-dist or galaxy-central. The reason I ask is that I went to install a tool from a production server I'm trying to setup based on galaxy-dist ... but got an error; The resource could not be found. No action for /admin_toolshed/prepare_for_install Searching for this in the source code, I found that it exists in galaxy-central ... but not in galaxy-dist Am I right in guessing that this error is due to toolshed.g2.bx.psu.edu being powered by a recent toolshed version that is incompatible with galaxy-dist? If so, it would be great to get clarification as to whether this is to be expected in future. My personal feeling is that the main toolshed would be much more useful if kept compatible with galaxy-dist. Cheers Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Best way to allow users to configure a display application url for a toolshed tool
Hi Dan and Greg,
Thanks for your answers on this. So far I have managed to make things work
following Dan's suggestion ... except that I can't seem to get my configuration
data file to install into tool-data. I'm not sure why this might be since all
my XXX.loc.sample files install into tool-data fine. I tried adding a file to
my tool repository under tool-data/protk_site_config.txt.sample ... and had
hoped it would result in a file being installed (when the tools are installed)
into tool-data/protk_site_config.txt .. but unfortunately this doesn't happen.
If I hack things and copy the file in place manually then Dan's instructions
work (after modifying my display applications).
Anyway, I'll investigate this a bit more tomorrow. Hopefully you can shed some
light on why my .loc.sample files are copied fine, but my other file is not.
Does galaxy only install tool-data files with the .loc.sample naming
convention? Or is there some other internal filter/check that could be stopping
my config file being installed.
Thanks
Ira
On 03/05/2012, at 8:35 PM, Greg Von Kuster wrote:
Hi Ira,
You'll need to include an entry for the display application in a
datatype_conf.xml file in the tool shed repository- this will result in the
display application being properly installed with the repository and made
functional in the local Galaxy instance. For details about this, see the
following section of the tool shed wiki.
http://wiki.g2.bx.psu.edu/Tool%20Shed#Tool_Shed.Including_datatype_converters_and_display_applications
Let me know if you have any questions on this.
Thanks!
On May 2, 2012, at 11:19 PM, Daniel Blankenberg wrote:
Hi Ira,
You'll want to set this up as a dynamic display application, where the link
id, name, url, port, etc are defined in an external tab-delimited file. Most
of the displays included in the dist are set up this way, one example would
be the display_applications/ucsc/bam.xml.
For an example where the external display application needs to access
Galaxy's datasets directly on a shared filesystem have a look at
display_applications/genetrack.xml.
Thanks for using Galaxy,
Dan
On May 2, 2012, at 10:58 PM, Ira Cooke wrote:
Dear All,
Our toolshed tools include a custom display application. Although the
display application is open source (
https://bitbucket.org/Andrew_Brock/proteomics-visualise ) we do not yet
have a public server for it. Actually, at present it isn't possible to
have the application work properly if it is running on a completely
separate server because it needs access to galaxy's database/files
directory (I'll leave that for another question). For now I just need to
be able to provide a way for administrators of a local galaxy to configure
the url to this display application.
At the moment we have it hard coded .. like this;
url
target_frame=galaxy_mainhttp://${galaxy_url.qp}:8500/init_local?file=${encoded_filename.qp}amp;type=mzml/url
Using the galaxy_url (because we are on the same server) .. and hard coding
the port.
What would be the best way to allow local admins to configure this?
Thanks
Ira
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Best way to allow users to configure a display application url for a toolshed tool
Hi Greg,
Thanks ... that's great. It's all working now.
Ira
On 04/05/2012, at 3:41 AM, Greg Von Kuster wrote:
Hi Ira,
Sample fiel handling for installed tool shed repositories was restricted to
.loc.sample files as you discovered. I've enhanced this feature to copy all
appropriate .sample files contained in the installed repository to the
tool-data directory of the Galaxy instance to which the repository was
installed. This enhancement is available in change set 7145:ea9f3c25a415 in
our central repository. Thanks for reporting this problem!
Greg Von Kuster
On May 3, 2012, at 6:49 AM, Ira Cooke wrote:
Hi Dan and Greg,
Thanks for your answers on this. So far I have managed to make things work
following Dan's suggestion ... except that I can't seem to get my
configuration data file to install into tool-data. I'm not sure why this
might be since all my XXX.loc.sample files install into tool-data fine. I
tried adding a file to my tool repository under
tool-data/protk_site_config.txt.sample ... and had hoped it would result in
a file being installed (when the tools are installed) into
tool-data/protk_site_config.txt .. but unfortunately this doesn't happen.
If I hack things and copy the file in place manually then Dan's instructions
work (after modifying my display applications).
Anyway, I'll investigate this a bit more tomorrow. Hopefully you can shed
some light on why my .loc.sample files are copied fine, but my other file is
not. Does galaxy only install tool-data files with the .loc.sample naming
convention? Or is there some other internal filter/check that could be
stopping my config file being installed.
Thanks
Ira
On 03/05/2012, at 8:35 PM, Greg Von Kuster wrote:
Hi Ira,
You'll need to include an entry for the display application in a
datatype_conf.xml file in the tool shed repository- this will result in the
display application being properly installed with the repository and made
functional in the local Galaxy instance. For details about this, see the
following section of the tool shed wiki.
http://wiki.g2.bx.psu.edu/Tool%20Shed#Tool_Shed.Including_datatype_converters_and_display_applications
Let me know if you have any questions on this.
Thanks!
On May 2, 2012, at 11:19 PM, Daniel Blankenberg wrote:
Hi Ira,
You'll want to set this up as a dynamic display application, where the
link id, name, url, port, etc are defined in an external tab-delimited
file. Most of the displays included in the dist are set up this way, one
example would be the display_applications/ucsc/bam.xml.
For an example where the external display application needs to access
Galaxy's datasets directly on a shared filesystem have a look at
display_applications/genetrack.xml.
Thanks for using Galaxy,
Dan
On May 2, 2012, at 10:58 PM, Ira Cooke wrote:
Dear All,
Our toolshed tools include a custom display application. Although the
display application is open source (
https://bitbucket.org/Andrew_Brock/proteomics-visualise ) we do not yet
have a public server for it. Actually, at present it isn't possible to
have the application work properly if it is running on a completely
separate server because it needs access to galaxy's database/files
directory (I'll leave that for another question). For now I just need to
be able to provide a way for administrators of a local galaxy to
configure the url to this display application.
At the moment we have it hard coded .. like this;
url
target_frame=galaxy_mainhttp://${galaxy_url.qp}:8500/init_local?file=${encoded_filename.qp}amp;type=mzml/url
Using the galaxy_url (because we are on the same server) .. and hard
coding the port.
What would be the best way to allow local admins to configure this?
Thanks
Ira
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] 503 Error when adding a tool from local toolshed
Dear List, I'm trying to check my toolshed tools by installing first on a local toolshed, but running into an error every time I try to add my tools. In order to rule out possible idiosyncracies of my system I've reproduced the error on an Ubuntu VM ... the system was not completely clean ... a few packages required by galaxy and some dependencies of my tools have been installed, but I performed the following steps after creating a fresh galaxy user. # Create new user galaxy # Create galaxy_env via virtualenv.py # Add entry to .bashrc to ensure python in galaxy_env is the default python for the galaxy user # Fresh checkout of galaxy-central # sh run.sh --reload # Add admin user to galaxy by configuring universe_wsgi.ini # Configure tool_sheds_conf.xml as follows; ?xml version=1.0? tool_sheds tool_shed name=Local url=http://127.0.0.1:9009// tool_shed name=Galaxy main tool shed url=http://toolshed.g2.bx.psu.edu// tool_shed name=Galaxy test tool shed url=http://testtoolshed.g2.bx.psu.edu// /tool_sheds # Restart galaxy # Register myself as a user of galaxy # Add database_connection line to community_wsgi.ini - database_connection=sqlite:///./database/community.sqlite?isolation_level=IMMEDIATE # Add admin user to community_wsgi.ini (same as for galaxy admin user) # run sh run_community.sh # Register myself as a user of the local toolshed # Checked out my tools hg clone https://bitbucket.org/iracooke/protk-toolshed # Made a bz2 file of all my tools cd protk-toolshed ./make_package_data.sh # Created a new category on the local toolshed (Proteomics) # Added a tool called protk to the Proteomics category # Uploaded a bz2 file with my tools to the local toolshed. Looking at the tool after upload everything seems fine # Back on galaxy I attempted to load my tools. After clicking the install button in galaxy I get the following traceback (as text) URL: http://127.0.0.1:8080/admin_toolshed/install_repository?tool_shed_url=http://127.0.0.1:9009/repo_info_dict=0a19b3600379cf51a9d8ca26ff0e7754f5e99731:7b2270726f746b223a205b2250726f74656f6d69637320746f6f6c6b6974222c2022687474703a2f2f697261636f6f6b65403132372e302e302e313a393030392f7265706f732f697261636f6f6b652f70726f746b222c2022646330343464623536626634222c202230225d7dincludes_tools=True File '/home/galaxy/central/galaxy-central/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/galaxy/central/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/galaxy/central/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/galaxy/central/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/galaxy/central/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/galaxy/central/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/galaxy/central/galaxy-central/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/home/galaxy/central/galaxy-central/lib/galaxy/web/controllers/admin_toolshed.py', line 371 in install_repository response = urllib2.urlopen( url ) File '/usr/lib/python2.7/urllib2.py', line 126 in urlopen return _opener.open(url, data, timeout) File '/usr/lib/python2.7/urllib2.py', line 400 in open response = meth(req, response) File '/usr/lib/python2.7/urllib2.py', line 513 in http_response 'http', request, response, code, msg, hdrs) File '/usr/lib/python2.7/urllib2.py', line 438 in error return self._call_chain(*args) File '/usr/lib/python2.7/urllib2.py', line 372 in _call_chain result = func(*args) File '/usr/lib/python2.7/urllib2.py', line 521 in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 503: Service Unavailable I've since tried to figure out where this error actually occurs ... and (according to my crude debugging methods) the actual error originates at line 371 in admin_toolshed.py .. which looks like this .. owner = get_repository_owner( clean_repository_clone_url( repository_clone_url ) ) url = '%s/repository/get_readme?name=%sowner=%schangeset_revision=%swebapp=galaxy' % ( tool_shed_url, name, owner, changeset_revision ) response = urllib2.urlopen( url ) ... so galaxy attempts to fetch a url for the tool readme file ... and that is failing. I've tried adding a readme.txt and that didn't solve the issue. Hopefully this is enough for you to be able to reproduce the error ... I'm mainly just keen to get a workable setup for testing my tools, so a
Re: [galaxy-dev] Proteomics category on main toolshed?
Hi Greg, I'm not too sure of the best way to make these available (perhaps showing by ignorance of the way Galaxy handles such databases in the genomics world). We've got a set of commandline tools which support protk-toolshed https://bitbucket.org/iracooke/protk And included among these is a tool manage_db which makes it relatively easy to setup a database for proteomics .. and then update it regularly. The tools also support using a FASTA file in your history as a database for a proteomics search. We need quite a large variety of custom databases (eg subsets of Swissprot for different taxa) because some search engines (eg X!Tandem) can't create those subsets on the fly .. and because we need decoys embedded in the database files. With regards to cloudman ... we are definitely keen to get an instance up and running. At the moment it takes quite a bit of work to properly configure all the components to make this work well ... so a prebuilt instance would be great. Not sure if that answers your question ... but if there is something we could do to improve the integration of our tools into Galaxy I'd be happy to try it. Regards Ira On 18/04/2012, at 11:13 AM, Greg Edwards wrote: Ira, As part of your Proteomics Toolshed, are you thinking of making some Protein databases available ? Latest UniProt/SwissProt (~ 250 MB) or UniProt/TREMBl (~ 8GB) ? That could be very helpful for future Proteomics tools we are developing. http://en.wikipedia.org/wiki/UniProt Also, could it go in the Galaxy Cloudman /mnt/galaxyIndices volume ? That's a question for the Galaxy team I think. Rgds, Greg E On Tue, Apr 3, 2012 at 11:33 PM, Greg Von Kuster g...@bx.psu.edu wrote: No worries, Ira - have a great Easter vacation! On Apr 3, 2012, at 7:20 AM, Ira Cooke wrote: Hi Greg, I'm afraid to say I might have been a bit ahead of myself in asking you to create that Proteomics category. We're almost ready to upload our tools, but at the last minute we've discovered a bunch of issues we need to sort out. I'm going away for Easter and won't be back until the 16th ... so I guess the earliest our tools would be up is after the 16th. Sorry about that. I hope you're OK with a 0 beside the Proteomics category for now. Ira On 30/03/2012, at 10:03 AM, Ira Cooke wrote: Thanks ... that would definitely be a good idea. Well try to implement proper tests for all the tools. Ira On 30/03/2012, at 10:01 AM, Greg Von Kuster wrote: Thanks Ira, Just so you know, the Galaxy functional test framework now supports testing installed tools if the repository tools include tests and test datasets are included in the repository in a test-data subdirectory. Here is the relevant section of the tool shed wiki describing this: http://wiki.g2.bx.psu.edu/Tool%20Shed#Using_Galaxy.27s_functional_test_framework_to_test_tools_installed_into_your_local_Galaxy_instance Greg On Mar 29, 2012, at 6:53 PM, Ira Cooke wrote: Dear Greg, Thanks very much for that. I'll do a last proof-read of the tools and upload later on today. We're working on some better documentation ... to get it all working takes some extra setup steps on the part of the user. Ira On 30/03/2012, at 9:49 AM, Greg Von Kuster wrote: Hello Ira, I've added the category to the main tool shed. Thanks for the request and your upcoming contributions! Greg Von Kuster On Mar 29, 2012, at 5:33 PM, Ira Cooke wrote: Dear list, I would like to add a suite of proteomics tools to the toolshed but there is currently no suitable category. Would it be possible to add a category called Proteomics? Thanks Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Greg Edwards, Port Jackson Bioinformatics gedwar...@gmail.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Proteomics category on main toolshed?
Hi Greg, I'm afraid to say I might have been a bit ahead of myself in asking you to create that Proteomics category. We're almost ready to upload our tools, but at the last minute we've discovered a bunch of issues we need to sort out. I'm going away for Easter and won't be back until the 16th ... so I guess the earliest our tools would be up is after the 16th. Sorry about that. I hope you're OK with a 0 beside the Proteomics category for now. Ira On 30/03/2012, at 10:03 AM, Ira Cooke wrote: Thanks ... that would definitely be a good idea. Well try to implement proper tests for all the tools. Ira On 30/03/2012, at 10:01 AM, Greg Von Kuster wrote: Thanks Ira, Just so you know, the Galaxy functional test framework now supports testing installed tools if the repository tools include tests and test datasets are included in the repository in a test-data subdirectory. Here is the relevant section of the tool shed wiki describing this: http://wiki.g2.bx.psu.edu/Tool%20Shed#Using_Galaxy.27s_functional_test_framework_to_test_tools_installed_into_your_local_Galaxy_instance Greg On Mar 29, 2012, at 6:53 PM, Ira Cooke wrote: Dear Greg, Thanks very much for that. I'll do a last proof-read of the tools and upload later on today. We're working on some better documentation ... to get it all working takes some extra setup steps on the part of the user. Ira On 30/03/2012, at 9:49 AM, Greg Von Kuster wrote: Hello Ira, I've added the category to the main tool shed. Thanks for the request and your upcoming contributions! Greg Von Kuster On Mar 29, 2012, at 5:33 PM, Ira Cooke wrote: Dear list, I would like to add a suite of proteomics tools to the toolshed but there is currently no suitable category. Would it be possible to add a category called Proteomics? Thanks Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Proteomics category on main toolshed?
Thanks ... that would definitely be a good idea. Well try to implement proper tests for all the tools. Ira On 30/03/2012, at 10:01 AM, Greg Von Kuster wrote: Thanks Ira, Just so you know, the Galaxy functional test framework now supports testing installed tools if the repository tools include tests and test datasets are included in the repository in a test-data subdirectory. Here is the relevant section of the tool shed wiki describing this: http://wiki.g2.bx.psu.edu/Tool%20Shed#Using_Galaxy.27s_functional_test_framework_to_test_tools_installed_into_your_local_Galaxy_instance Greg On Mar 29, 2012, at 6:53 PM, Ira Cooke wrote: Dear Greg, Thanks very much for that. I'll do a last proof-read of the tools and upload later on today. We're working on some better documentation ... to get it all working takes some extra setup steps on the part of the user. Ira On 30/03/2012, at 9:49 AM, Greg Von Kuster wrote: Hello Ira, I've added the category to the main tool shed. Thanks for the request and your upcoming contributions! Greg Von Kuster On Mar 29, 2012, at 5:33 PM, Ira Cooke wrote: Dear list, I would like to add a suite of proteomics tools to the toolshed but there is currently no suitable category. Would it be possible to add a category called Proteomics? Thanks Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problems with custom data types and sniffers in shed tools
Hi Greg, Thanks for that fix ... I'll check it out. I think the issue of sniff order is a pretty important one for shed tools that require proprietary datatypes. The problem is that galaxy's default sniffers include some extremely generic sniffers (eg text,xml) which will catch pretty much anything so it's hard to imagine how a tool writer will ever have their sniffers used. Since the potential for conflicts with galaxy's defaults is an issue how about changing sniffer behaviour so that datatypes which subclass always have priority over their parent class ... then among those subclasses the order in which sniffers are listed in datatypes_conf is used as the second priority. I imagine that this would require that the sniff order code be changed so that all datatypes and sniffers are first loaded .. then reshuffled to get the correct order according to the class hierarchy. I believe this option makes sense since a subclass of a datatype should always override its parent .. and I think it would also avoid the potential for conflicts with galaxy's defaults. What do you think? Is that a system that would work? Cheers ira On 08/02/2012, at 3:25 AM, Greg Von Kuster wrote: Hello Ira, Very sorry for the back-and-forth on this. The behavior you encountered was accurate - the environment in which I was testing was not pristine. I have committed a fix for this issue in revision b4ba8b20d78d, which is now available from our central repo. I started looking at changing things so that proprietary sniffers are loaded before the sniffers defined in the Galaxy distribution, and determined that this will be a bit non-trivial. Problems arise due to potential conflicts between a proprietary datatype that has the same extension as a datatype defined in the Galaxy distribution. The approach I've taken to deal with this is that datatypes in the Galaxy distribution get loaded first, and then any conflicting proprietary datatypes will be ignored. Loading proprietary datatypes first will not allow for this rule. I'll gladly listen to advice from the community on this, but in the meantime, proprietary datatypes and sniffers are loaded after those in the distribution. Thanks for your patience and help on this issue. Greg On Feb 6, 2012, at 11:37 PM, Ira Cooke wrote: Hi Greg, As far as I know there is nothing in my local setup that affects this. As I test I did the following; Checked out a clean copy of galaxy-central Edited my universe_wsgi.ini as follows; - Changed the port to 8300 - Added an admin user - Uncommented this line tool_config_file = tool_conf.xml,shed_tool_conf.xml Edited shed_tool_conf.xml to point to ../shed_tools_galaxy_central (which is an empty directory) Started up galaxy ./run.sh --reload I then went straight to admin - Search and browse tool sheds and selected the main galaxy toolshed. I then selected the gmap tool and told it to install. My paster log is at the bottom of this mail. My system is Mac OSX 10.7.2 and I'm running python 2.7.1 I then went to the relevant bit of code and inserted some print statements. I found that; On line 191 in registry.py there is; module = __import__( datatype_module ) and the value of datatype_module is galaxy.datatypes.gmap this line seems to be causing the exception ... I'm not a python programmer so I don't really know how module importing works ... could this be related to the version of python I'm running? alternatively should the imported_modules variable be used here .. since wouldn't galaxy.datatypes.gmap suggest that the module needs to be located inside the main galaxy lib rather than in the shed_tools. Strange that I'm the only one who gets these errors? Any ideas what it could be? Ira -- paster log -- Cloning http://toolshed.g2.bx.psu.edu/repos/jjohnson/gmap destination directory: gmap requesting all changes adding changesets adding manifests adding file changes added 11 changesets with 32 changes to 17 files updating to branch default resolving manifests getting README getting gmap.xml getting gmap_build.xml getting gsnap.xml getting iit_store.xml getting lib/galaxy/datatypes/gmap.py getting snpindex.xml getting tool-data/datatypes_conf.xml getting tool-data/gmap_indices.loc.sample 9 files updated, 0 files merged, 0 files removed, 0 files unresolved galaxy.util.shed_util DEBUG 2012-02-07 14:25:35,100 Updating cloned repository to revision 93911bac43da resolving manifests 0 files updated, 0 files merged, 0 files removed, 0 files unresolved docutils WARNING 2012-02-07 14:25:35,512 string:10: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent. galaxy.util.shed_util DEBUG 2012-02-07 14:25:35,589 Adding new row (or updating an existing row) for repository 'gmap' in the tool_shed_repository table. docutils WARNING 2012-02-07 14:25
Re: [galaxy-dev] Problems with custom data types and sniffers in shed tools
Hi Greg, Thanks .. that sounds like a good solution. I'll be happy to test it when its ready. Ira On 08/02/2012, at 10:22 AM, Greg Von Kuster wrote: Ira, Thanks for the feedback - I'll come up with something that allows for proprietary sniffers to be loaded first and let you know when it's available. It shouldn't take me too long. I will probably continue to keep the rule of ignoring conflicts in proprietary datatypes, so sniffers for conflicting datatypes will probably be ignored as well. However, hopefully conflicts will not exist, or if one is found, it will get fixed. Greg On Feb 7, 2012, at 6:16 PM, Ira Cooke wrote: Hi Greg, Thanks for that fix ... I'll check it out. I think the issue of sniff order is a pretty important one for shed tools that require proprietary datatypes. The problem is that galaxy's default sniffers include some extremely generic sniffers (eg text,xml) which will catch pretty much anything so it's hard to imagine how a tool writer will ever have their sniffers used. Since the potential for conflicts with galaxy's defaults is an issue how about changing sniffer behaviour so that datatypes which subclass always have priority over their parent class ... then among those subclasses the order in which sniffers are listed in datatypes_conf is used as the second priority. I imagine that this would require that the sniff order code be changed so that all datatypes and sniffers are first loaded .. then reshuffled to get the correct order according to the class hierarchy. I believe this option makes sense since a subclass of a datatype should always override its parent .. and I think it would also avoid the potential for conflicts with galaxy's defaults. What do you think? Is that a system that would work? Cheers ira On 08/02/2012, at 3:25 AM, Greg Von Kuster wrote: Hello Ira, Very sorry for the back-and-forth on this. The behavior you encountered was accurate - the environment in which I was testing was not pristine. I have committed a fix for this issue in revision b4ba8b20d78d, which is now available from our central repo. I started looking at changing things so that proprietary sniffers are loaded before the sniffers defined in the Galaxy distribution, and determined that this will be a bit non-trivial. Problems arise due to potential conflicts between a proprietary datatype that has the same extension as a datatype defined in the Galaxy distribution. The approach I've taken to deal with this is that datatypes in the Galaxy distribution get loaded first, and then any conflicting proprietary datatypes will be ignored. Loading proprietary datatypes first will not allow for this rule. I'll gladly listen to advice from the community on this, but in the meantime, proprietary datatypes and sniffers are loaded after those in the distribution. Thanks for your patience and help on this issue. Greg On Feb 6, 2012, at 11:37 PM, Ira Cooke wrote: Hi Greg, As far as I know there is nothing in my local setup that affects this. As I test I did the following; Checked out a clean copy of galaxy-central Edited my universe_wsgi.ini as follows; - Changed the port to 8300 - Added an admin user - Uncommented this line tool_config_file = tool_conf.xml,shed_tool_conf.xml Edited shed_tool_conf.xml to point to ../shed_tools_galaxy_central (which is an empty directory) Started up galaxy ./run.sh --reload I then went straight to admin - Search and browse tool sheds and selected the main galaxy toolshed. I then selected the gmap tool and told it to install. My paster log is at the bottom of this mail. My system is Mac OSX 10.7.2 and I'm running python 2.7.1 I then went to the relevant bit of code and inserted some print statements. I found that; On line 191 in registry.py there is; module = __import__( datatype_module ) and the value of datatype_module is galaxy.datatypes.gmap this line seems to be causing the exception ... I'm not a python programmer so I don't really know how module importing works ... could this be related to the version of python I'm running? alternatively should the imported_modules variable be used here .. since wouldn't galaxy.datatypes.gmap suggest that the module needs to be located inside the main galaxy lib rather than in the shed_tools. Strange that I'm the only one who gets these errors? Any ideas what it could be? Ira -- paster log -- Cloning http://toolshed.g2.bx.psu.edu/repos/jjohnson/gmap destination directory: gmap requesting all changes adding changesets adding manifests adding file changes added 11 changesets with 32 changes to 17 files updating to branch default resolving manifests getting README getting gmap.xml getting gmap_build.xml getting gsnap.xml getting iit_store.xml getting lib/galaxy/datatypes/gmap.py getting
[galaxy-dev] Error when attempting to get_updates from toolshed
Dear Greg, In the latest version of the galaxy-central code there seems to be a bug preventing me from updating my tools. When I go to update a tool I get the following error URL: http://cookie.latrobe.edu.au:8300/admin_toolshed/browse_repositories?sort=namef-deleted=Falseoperation=Get+updatesid=a3329e0bcb13f2c9 File '/var/www/galaxy/galaxyapp/eggs/Paste-1.6-py2.7.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/var/www/galaxy/galaxyapp/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/var/www/galaxy/galaxyapp/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/var/www/galaxy/galaxyapp/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/var/www/galaxy/galaxyapp/lib/galaxy/web/framework/__init__.py', line 173 in decorator return func( self, trans, *args, **kwargs ) File '/var/www/galaxy/galaxyapp/lib/galaxy/web/controllers/admin_toolshed.py', line 96 in browse_repositories return self.check_for_updates( trans, **kwd ) File '/var/www/galaxy/galaxyapp/lib/galaxy/web/framework/__init__.py', line 173 in decorator return func( self, trans, *args, **kwargs ) File '/var/www/galaxy/galaxyapp/lib/galaxy/web/controllers/admin_toolshed.py', line 654 in check_for_updates ( tool_shed_url, url_for( '/', qualified=True ), repository.name, repository.owner, repository.changeset_revision ) ValueError: unsupported format character '/' (0x2f) at index 1 Looking at the code there seems to be a typo on line 702 of admin_toolshed.py It reads url = '%/srepository/check_for_updates?galaxy_url=%sname=%sowner=%schangeset_revision=%swebapp=galaxy' % \ But I think it should be url = '%s/repository/check_for_updates?galaxy_url=%sname=%sowner=%schangeset_revision=%swebapp=galaxy' % \ When I change this though I still get further errors ... seems like a query to the database is not finding my toolshedrepository The new error is URL: http://127.0.0.1:8300/admin_toolshed/update_to_changeset_revision?tool_shed_url=http://127.0.0.1:9009/name=protkowner=iracookechangeset_revision=21e996f8c650latest_changeset_revision=21e996f8c650 File '/Users/iracooke/Sources/galaxy-proteomics/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/Users/iracooke/Sources/galaxy-proteomics/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/Users/iracooke/Sources/galaxy-proteomics/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/Users/iracooke/Sources/galaxy-proteomics/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/Users/iracooke/Sources/galaxy-proteomics/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/Users/iracooke/Sources/galaxy-proteomics/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/Users/iracooke/Sources/galaxy-proteomics/lib/galaxy/web/framework/__init__.py', line 173 in decorator return func( self, trans, *args, **kwargs ) File '/Users/iracooke/Sources/galaxy-proteomics/lib/galaxy/web/controllers/admin_toolshed.py', line 772 in update_to_changeset_revision id=trans.security.encode_id( repository.id ), AttributeError: 'NoneType' object has no attribute 'id' If I browse my database I can find an entry in the tool_shed_repositories table which seems to have all the right data. I'm getting these errors on the latest version of the galaxy-central code. Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Problem fetching updates to toolshed tool
Hi, We've been running into an issue when attempting to create a new toolshed tool. We have the toolshed and the main galaxy web app setup and running, but updating our tool to the latest version always fails. When clicking Get updates on our installed tool we find that the initial request is sent fine to the community application, but the reply is missing a slash after the galaxy url. This causes the request to fail. Our solution to this problem was to modify lib/galaxy/webapps/community/controllers/repository.py on line 795 to include a slash after the galaxy url. After we did this though we encountered another problem .. this time in lib/galaxy/web/controllers/admin_toolshed.py . In this case there is a named parameter to a method but the name of the parameter does not match the method definition. The problem was on 688 where name = name should read repository_name=name. After making these changes the get updates function seems to work fine. Hope this helps. Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Installation problems on osx
Good to know that I'm not the only one. Also, I didn't put the rescramble commands in my original email ( In case there are other mac users googling this thread). cd galaxy-dist/scripts/ rm -r ../eggs/numpy-1.3.0-py2.6-macosx-10.6-universal-ucs2.egg rm -r ../eggs/bx_python-0.7.0_494c2d1d68b3-py2.6-macosx-10.6-universal-ucs2.egg python scramble.py numpy python scramble.py bx_python Of course, you also need developer tools installed. Ira On 23/06/2011, at 2:16 AM, Michael Steder wrote: Hi Ira, I had the same problem and scrambling the eggs myself seemed to be the workaround. I'm not sure why we're having problems. It does seem to be specific to mac though, as I can run this same checkout on linux without any workarounds. ~Mike On Jun 21, 2011, at 11:47 PM, Ira Cooke wrote: Hi, I recently downloaded galaxy and had some problems installing it on my machine (running Mac OSX 10.6.7 ). I've found a workaround to my problem, but I thought I should post here in case it is useful. I obtained the standard galaxy distribution using hg clone https://bitbucket.org/galaxy/galaxy-dist/ I then tried running sh run.sh but got; Traceback (most recent call last): File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object I googled this error and found a post suggesting that I try virtualenv ... I tried that but it made no difference. Python --version returns 2.6.1 After some digging I tried rescrambling the numpy and then the bx_python egg. After rescrambling the latter sh run.sh worked. Anyway I'm not sure if other mac users have had similar problems. In my case I'm pretty sure I had installed things with easy_install in the past ...but I assume using virtualenv would have solved that problem. Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Installation problems on osx
Hi, I recently downloaded galaxy and had some problems installing it on my machine (running Mac OSX 10.6.7 ). I've found a workaround to my problem, but I thought I should post here in case it is useful. I obtained the standard galaxy distribution using hg clone https://bitbucket.org/galaxy/galaxy-dist/ I then tried running sh run.sh but got; Traceback (most recent call last): File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/iracooke/Sources/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object I googled this error and found a post suggesting that I try virtualenv ... I tried that but it made no difference. Python --version returns 2.6.1 After some digging I tried rescrambling the numpy and then the bx_python egg. After rescrambling the latter sh run.sh worked. Anyway I'm not sure if other mac users have had similar problems. In my case I'm pretty sure I had installed things with easy_install in the past ...but I assume using virtualenv would have solved that problem. Ira ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
