Re: [galaxy-dev] How to upload local files in Galaxy
From: Fields, Christopher J [mailto:cjfie...@illinois.edu] [SNIP] if you follow the guidelines for FTP import, any method used (not just FTP, but scp, sftp, grid-ftp, etc.) to get data into the 'FTP' import folder works as long as permissions on the data are set so the galaxy user on the cluster end can read the data. From our investigations, the galaxy user needs to *own* the file, not just be able to read it. The reason is that galaxy does a shutil.move, which calls copy2 to give the new file the same permissions as the old, which calls copystat, which calls os.chmod, which requires galaxy to be the owner. This seems to be true at least on our old version of Galaxy, though perhaps it's been fixed in the last few months. We had our local cluster admins set up a link to the user's galaxy import folder in their home directory, so users can basically do this: scp mydata.fastq.gz usern...@biocluster.igb.illinois.edu:galaxy-upload We wanted to do just that, but we kept getting permission errors. In the end, we needed to add a cron job that runs every minute as root and does a chown galaxy on any files appearing in the upload directories. Please let me know if I'm missing something. I'd be happy to turn off the cron. Thanks, -Amir Karger ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] adding tools using toolshed is not working
Hi, I just updated my local Galaxy installation to the latest version. I am trying out the feature to add tools directly from the toolshed, but this is not working. It results in this error: File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxyTools/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/galaxyTools/galaxy-central/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/home/galaxyTools/galaxy-central/lib/galaxy/web/controllers/admin_toolshed.py', line 481 in install_repository install_tool_dependencies=install_tool_dependencies ) File '/home/galaxyTools/galaxy-central/lib/galaxy/util/shed_util.py', line 1228 in load_repository_contents dist_to_shed=False ) File '/home/galaxyTools/galaxy-central/lib/galaxy/util/shed_util.py', line 342 in create_or_update_tool_shed_repository installed_changeset_revision=changeset_revision, NameError: global name 'changeset_revision' is not defined And my shed_tools_conf.xml looks like this: ?xml version=1.0? toolbox tool_path=../shed_tools /toolbox Should I put more configuration in this file? Any help is appreciated. Thanks! Mattias ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] adding tools using toolshed is not working
Hello Mattias, Did you pull from the Galaxy central repo? The latest change set is 7262:4fbd05095ca7, which is what you should be running in order to use the tool shed. Let me know if you in fact have the latest from central - your line numbers in your log below look to be different from mine. Thanks, Greg Von Kuster On Jun 13, 2012, at 11:38 AM, Hollander, Mattias de wrote: Hi, I just updated my local Galaxy installation to the latest version. I am trying out the feature to add tools directly from the toolshed, but this is not working. It results in this error: File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxyTools/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/galaxyTools/galaxy-central/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/home/galaxyTools/galaxy-central/lib/galaxy/web/controllers/admin_toolshed.py', line 481 in install_repository install_tool_dependencies=install_tool_dependencies ) File '/home/galaxyTools/galaxy-central/lib/galaxy/util/shed_util.py', line 1228 in load_repository_contents dist_to_shed=False ) File '/home/galaxyTools/galaxy-central/lib/galaxy/util/shed_util.py', line 342 in create_or_update_tool_shed_repository installed_changeset_revision=changeset_revision, NameError: global name 'changeset_revision' is not defined And my shed_tools_conf.xml looks like this: ?xml version=1.0? toolbox tool_path=../shed_tools /toolbox Should I put more configuration in this file? Any help is appreciated. Thanks! Mattias ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] File Type Validators
Hi, I am data curator on the ENCODE project. I was looking at the file formats that you support: http://galaxy.genenetwork.org:8080/static/formatHelp.html http://galaxy.genenetwork.org:8080/static/formatHelp.html%29 and was wondering if you had any file validation software you were using? Thanks, Venkat S. Malladi Data Wrangler ENCODE Project ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Error in db upgrade when updating to May11 distribution
Hello, I get an error when trying to upgrade my mysql db with sh manage_db.sh upgrade from version 94-95 (see below). Will cause problems with my installation? sh manage_db.sh upgrade 93 - 94... Migration script to create handler column in job table. done 94 - 95... Migration script to create table for tracking history_dataset_association subsets. (OperationalError) (1059, Identifier name 'ix_history_dataset_association_subset_history_dataset_association_id' is too long) u'CREATE INDEX ix_history_dataset_association_subset_history_dataset_association_id ON history_dataset_association_subset (history_dataset_association_id)' () done 95 - 96... Migration script to add column to openid table for provider. Remove any OpenID entries with nonunique GenomeSpace Identifier done 96 - 97... Migration script to add the ctx_rev column to the tool_shed_repository table. done Thanks in advance, Liisa___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error in db upgrade when updating to May11 distribution
Hi Liisa, This won't cause any problems with your instance; if you use tools in Trackster, they'll run a tad slower due to this problem. We'll fix this soon, and then you can downgrade/upgrade to create the requisite index if you want. Best, J. On Jun 13, 2012, at 2:22 PM, Liisa Koski wrote: Hello, I get an error when trying to upgrade my mysql db with sh manage_db.sh upgrade from version 94-95 (see below). Will cause problems with my installation? sh manage_db.sh upgrade 93 - 94... Migration script to create handler column in job table. done 94 - 95... Migration script to create table for tracking history_dataset_association subsets. (OperationalError) (1059, Identifier name 'ix_history_dataset_association_subset_history_dataset_association_id' is too long) u'CREATE INDEX ix_history_dataset_association_subset_history_dataset_association_id ON history_dataset_association_subset (history_dataset_association_id)' () done 95 - 96... Migration script to add column to openid table for provider. Remove any OpenID entries with nonunique GenomeSpace Identifier done 96 - 97... Migration script to add the ctx_rev column to the tool_shed_repository table. done Thanks in advance, Liisa___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] tool_data_table_config.xml.sample
Hi Greg Thanks for the tip. The .file items appear when I tar the directory on my Mac, so I work around that issue and tar the directory on a UNIX server. Thus said I happily did the same with another directory that contains tools that are all dependent on the tool_data_table_conf.xml.sample and the tool-data/cg_crr_files.loc.sample file. This directory has the following files: cgatools/README.txt cgatools/tool_data_table_conf.xml.sample cgatools/datatypes_conf.xml cgatools/lib/galaxy/datatypes/completegenomics.py cgatools/tool-data/cg_crr_files.loc.sample cgatools/tools/cg_cgatools/listvariants.xml cgatools/tools/cg_cgatools/junctiondiff.xml cgatools/tools/cg_cgatools/snpdiff.xml cgatools/tools/cg_cgatools/calldiff.xml cgatools/tools/cg_cgatools/testvariants.xml cgatools/tools/cg_cgatools/varfilter.xml cgatools/tools/cg_cgatools/listtestvariants.xml cgatools/tools/cg_cgatools/join.xml cgatools/tools/cg_cgatools/varfilter_wrapper.pl When I upload this tar ball to the repository, one tool, calldiff, which is alphabetically the first one in the list, loads properly and for the others I get the following error message: Metadata was defined for some items in revision 'e183f52194a5'. Correct the following problems if necessary and reset metadata. join.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. junctiondiff.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. listtestvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. listvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. snpdiff.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. testvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. varfilter.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. For the invalid tools is show this explanation: [Errno 2] No such file or directory: '/var/opt/galaxy/g2cmnty/tmp/work_tmp/cg_crr_files.loc' But I know the file is there …. Any suggestions? - I attached the tar ball for a closer look. Thanks Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.commailto:bcr...@completegenomics.com From: Greg Von Kuster g...@bx.psu.edumailto:g...@bx.psu.edu Date: Wednesday, June 13, 2012 6:09 AM To: Birgit Crain bcr...@completegenomics.commailto:bcr...@completegenomics.com Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Hello Birgit, Your archive included several ._ scripts for some reason. None of these were valid xml files, but since they have a .xml extension, the tool shed attempts to parse them which results in the error message you got: not well-formed (invalid token): line 1, column 0 Here is the contents of your achive: -rw-r--r-- 1 gvk wheel 6148 Jun 6 16:09 .DS_Store -rw-r--r-- 1 gvk wheel171 May 25 13:44 ._Calculate_TestVariants_Variant_Frequencies.xml -rwxr-xr-x 1 gvk wheel171 May 3 19:19 ._Calculate_TestVariants_Variant_Frequencies_0_1_0.pl -rw-r--r-- 1 gvk wheel171 May 31 13:39 ._List_Unique_Variants.xml -rwxr-xr-x 1 gvk wheel171 May 3 12:50 ._List_Unique_Variants_2_1_0.pl -rw-r--r-- 1 gvk wheel 1950 May 25 13:44 Calculate_TestVariants_Variant_Frequencies.xml -rwxr-xr-x 1 gvk wheel 9242 May 3 19:19 Calculate_TestVariants_Variant_Frequencies_0_1_0.pl -rw-r--r-- 1 gvk wheel 13625 May 31 13:39 List_Unique_Variants.xml -rwxr-xr-x 1 gvk wheel 22321 May 3 12:50 List_Unique_Variants_2_1_0.pl I eliminated all of the ._ files and created a new archive which uploaded successfully. I've attached the archive. Greg Von Kuster On Jun 12, 2012, at 4:38 PM, Birgit Crain wrote: Thanks, Greg I tried uploading the same tar ball without the tool_data_table_config.xml.sample file and the tools load fine (to troubleshoot I added only tools that actually do not depend on the file). As soon as I add the .sample file to the tar ball I get the error message. Thanks, Birgit From: Greg Von Kuster g...@bx.psu.edumailto:g...@bx.psu.edu Date: Tuesday, June 12, 2012 1:18 PM To: Birgit Crain bcr...@completegenomics.commailto:bcr...@completegenomics.com Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu
[galaxy-dev] scripts in welcome.html?
To stop users forgetting whether they're on our development or production servers I'd like to put a simple script in welcome.html to echo the hostname (or some other stuff) I've tried both javascript and php but no luck. Is this supported or should I dig further into our Apache config? Or is there a better way to do it? Thanx, Russell Smithies === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] scripts in welcome.html?
It does of course work, just my scripting skills have faded with lack of use :( --Russell From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Smithies, Russell Sent: Thursday, 14 June 2012 10:37 a.m. To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] scripts in welcome.html? To stop users forgetting whether they're on our development or production servers I'd like to put a simple script in welcome.html to echo the hostname (or some other stuff) I've tried both javascript and php but no luck. Is this supported or should I dig further into our Apache config? Or is there a better way to do it? Thanx, Russell Smithies Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] CloudMan: Command not found
You can probably modfiy /home/galaxy/.sge_request to make it work for now, but for portability of tools I'd still recommend the requirements tag. Changes to .sge_request will not be persisted after shutdown, so you'll have to redo it with each new cluster. On Jun 13, 2012, at 9:25 PM, Jose Navas wrote: Hi Dannon, thank you for your help. I've tried the requirements tag option with the 'env.sh' script. I have added ~100 XML files... there isn't any option to modify the environment variables directly? If not I will modify all my XML files thanks! Jose Subject: Re: [galaxy-dev] CloudMan: Command not found From: dannonba...@me.com Date: Wed, 13 Jun 2012 21:05:02 -0400 CC: galaxy-dev@lists.bx.psu.edu To: josenavasmol...@hotmail.com Instead of modifying the environment variables directly, I've found it easier to use Galaxy's built in tool dependency framework. All you'd have to do is add a requirements type=package tag to your custom tools to specify the package directory that needs to be checked at runtime. See http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies, specifically the Managed Tool Dependencies section. Many of the other tools on the cloud volume use the 'env.sh' method discussed there. For example, blast has the 'blast' package requirement, and in the /mnt/galaxyTools/blast/default directory you'll find an 'env.sh' file that configures the environment at runtime. -Dannon On Jun 13, 2012, at 8:49 PM, Jose Navas wrote: Hi Galaxy Team, I'm modifying my Galaxy CloudMan instance by adding custom tools. I've installed my tools under the /mnt/galaxyTools/tools folder and I've modified the .bashrc files from the sgeadmin and galaxy users to add the needed paths to the PATH and PYTHONPATH variables. When I'm in Galaxy and I try to launch one of my tools, it fails and shows the 'Command not found' error. Where I should add the paths to make Galaxy now where are my executables? Whan I log into the isntance through ssh and I use the galaxy user, it knows where are my executables. What I'm missing? Thanks, Jose ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
