[galaxy-dev] large drill down selection UI component?
Hi all, I was wondering if there were any existing implementations of, or any plans for, a drill down selection component that supports thousands of items? For example a taxonomy based drill down. Thanks, Andrew ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] SLURM and hidden success
Hi John, Thanks so much for the reply. After investigating this more today it turns out, as you might have suspected, SLURM was a red herring. Despite our attempts to faithfully rsync everything between the two servers it looks like there was a problem with our workflows in the new database. Strangely every single workflow that was previously created had a "hide action" set for their outputs despite the past and present configuration of the tool wrappers. Any newly created workflow does not display this behavior. It happens that the steps with errors were being displayed despite the workflow weirdness thanks to an error check in lib/galaxy/jobs/actions/post.py Thanks again, Andrew On Mon, Nov 11, 2013 at 1:39 PM, John Chilton wrote: > This is really odd. I see no code in the job runner stuff at all that > could cause this behavior outside the context of the dataset being > marked hidden as part of a workflow - let alone something DRM specific > that could cause this. Are you rerunning an existing job that has been > marked this way in a workflow. Does this happen if you click new tools > outside the context of workflows or past jobs. > > Can you find the corresponding dataset via the history API or in the > database and determine if they indeed are having visible set to False > - this I guess is what should cause a dataset to become "hidden"? > > -John > > On Fri, Nov 8, 2013 at 11:40 AM, Andrew Warren > wrote: > > Hello all, > > > > We are in the process of switching from SGE to SLURM for our galaxy > setup. > > We are currently experiencing a problem where jobs that are completely > > successful (no text in their stderr file and the proper exit code) are > being > > hidden after the job completes. Any job that fails or has some text in > the > > stderr file is not hidden (note: hidden not deleted; they can be viewed > by > > selecting 'Unhide Hidden Datasets'). > > > > Our drmaa.py is at changeset 10961:432999eabbaa > > Our drmaa egg is at drmaa = 0.6 > > And our SLURM version is 2.3.5 > > > > And we are currently passing no parameters for > default_cluster_job_runner = > > drmaa:/// > > > > We have the same code base on both clusters but only observe this > behavior > > when using SLURM. > > Any pointers or advice would be greatly appreciated. > > > > Thanks, > > Andrew > > > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] SLURM and hidden success
Hello all, We are in the process of switching from SGE to SLURM for our galaxy setup. We are currently experiencing a problem where jobs that are completely successful (no text in their stderr file and the proper exit code) are being hidden after the job completes. Any job that fails or has some text in the stderr file is not hidden (note: hidden not deleted; they can be viewed by selecting 'Unhide Hidden Datasets'). Our drmaa.py is at changeset 10961:432999eabbaa Our drmaa egg is at drmaa = 0.6 And our SLURM version is 2.3.5 And we are currently passing no parameters for default_cluster_job_runner = drmaa:/// We have the same code base on both clusters but only observe this behavior when using SLURM. Any pointers or advice would be greatly appreciated. Thanks, Andrew ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] ENA galaxy import link broken?
Hello, I am trying to use the ENA galaxy import feature. This involves going to the analyze menu in galaxy and clicking the link 'EBI SRA' which is encoded as http://www.ebi.ac.uk/ena/data/search?GALAXY_URL=https%3A//main.g2.bx.psu.edu/tool_runner%3Ftool_id%3Debi_sra_main This does not seem to be working at present. I manage a Galaxy site that uses a similar setup encoded as (also doesn't work) http://www.ebi.ac.uk/ena/data/search?GALAXY_URL=http%3A//rnaseq.pathogenportal.org/tool_runner%3Ftool_id%3Debi_sra_main I sent an email off to ENA but haven't got a response. Any ideas? Thanks for any help, Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Unable to read builds file: publicbuilds.txt
Hello all, With recent additions to lib/galaxy/util/__init__.py I have started getting the following in the std out of bowtie and cufflinks (and probably other tools as well): ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' Indeed I do not have files for publicbuilds.txt or builds.txt created. Is the best way to deal with this just to create these files (even though I am not using them)? Are these files being read every time a web transaction goes down due to the property in lib/galaxy/web/framework/__init__.py ? @property def ucsc_builds( self ): Thanks, Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] potential bug in DefaultToolAction collecting chrom info
Hello, In the file lib/galaxy/tools/actions/__init__.py for the execute function of the DefaultToolAction class line 198 has the following code: if trans.user and ( 'dbkeys' in trans.user.preferences ) and ( input_dbkey in trans.user.preferences[ 'dbkeys' ] ): In this case trans.user.preferences[ 'dbkeys' ] is just returning a json string version of the dictionary. The result is that if input_dbkey is a substring of any part of this then it will be returned as True. Something like this would fix (though with the next lines of code it would be converting from json twice): if trans.user and ( 'dbkeys' in trans.user.preferences ) and ( input_dbkey in from_json_string(trans.user.preferences[ 'dbkeys' ] )): Cheers, Andrew ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] advanced parameters: all or nothing?
Thanks for the pointer to the updated wrapper and the optional tag. So is the behavior of the optional="true" tag that if the field form is left empty for that parameter then it will not be passed to the tool? I am wondering if a more sophisticated behavior on the UI side of things for the "optional" tag would still allow for recording that the parameter is using the default value for the version of that tool but not actually pass that parameter on the command line in case it influences the behavior of the tool? Thinking of parameters that should have a default value but whose actual presence on the command line changes what happens. Though, I guess this could be done by simply deleting the default value in the optional parameter has one? Maybe if the optional tag enabled the parameter itself to be toggled on/off (grayed out/regular text)? This way a block of advanced parameters could be enabled without requiring that they all be passed to the tool (or deleting all their values if they are marked optional). Just a small note, the update to the tophat wrapper was to the maximum number of alignments instead of mismatches. Thanks again, Andrew On Fri, Feb 10, 2012 at 11:28 AM, Peter Cock wrote: > On Fri, Feb 10, 2012 at 4:18 PM, Jeremy Goecks > wrote: > >> On Thu, Feb 9, 2012 at 11:27 PM, Andrew Warren > wrote: > >>> Hello, > >>> > >>> I was recently adjusting advanced parameters when running Tophat in > Galaxy > >>> and noticed that when advanced parameters are used, every field is > converted > >>> and submitted as command line parameter to the tool at runtime. Without > >>> changing any of the default values I get a different tophat result > than if > >>> advanced parameters are left off. > >> > >> That sounds like a bug in the wrapper. > > > > Recent Tophat versions have changed parameters and default values > > again. I made the following changes in 3bd8bed55631 to make the > > wrapper compatible with Tophat 1.4.0: > > > > *removed junctions_filter parameter > > *changed the default value for max-mismatches from 40 to 20 > > > > This should ensure that enabling advanced parameters but leaving > > them untouched yields the same results as using default parameters. > > > > J. > > Wouldn't it be better not to have the default value 40 (or 20) in the > wrapper > at all? i.e. Leave it out, so that by default that argument isn't used, so > that > tophat uses the default it wants to use. > > Peter > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] advanced parameters: all or nothing?
Hello, I was recently adjusting advanced parameters when running Tophat in Galaxy and noticed that when advanced parameters are used, every field is converted and submitted as command line parameter to the tool at runtime. Without changing any of the default values I get a different tophat result than if advanced parameters are left off. I'm curious if the Galaxy team has considered a mechanism for disabling an individual parameter? Or perhaps a way of individually enabling parameters from within the Advanced Parameter block? Just trying to think of a way to use one advanced parameter without using all of them. Cheers, Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] software installs: PATH vs env.sh
Thanks Ross, the -v option is definitely a good way to go that I had not considered. For anyone else following this thread, the tool dependency environment system was just commented on and documented by Nate. See the following links: http://galaxy-development-list-archive.2308389.n4.nabble.com/Documentation-for-Galaxy-s-tool-dependency-environment-system-td4321481.html http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies Thanks! -Andrew On Tue, Jan 24, 2012 at 8:27 PM, Ross wrote: > > Without knowing exactly what 'a cluster with a similar setup to > CloudMan' really means, if you're using SGE, passing the galaxy > process's path to the job is usually accomplished by setting the -V > flag in sge_request > eg on an ubuntu 11.10 machine with 'default' as the queue > (vgalaxy)galaxy@iaas1:$ cat /var/lib/gridengine/default/common/sge_request > # system wide defaults > -V > -cwd > > works for me. > > > > On Wed, Jan 25, 2012 at 12:12 PM, Andrew Warren wrote: > > Thanks for the replies. One extra element I failed to properly describe is > > that we are running on a cluster with a similar setup to CloudMan. This > > means we have a galaxyTools/tools directory with folders for each tool, and > > subfolders for each version with a "default" symlink to the version > > currently in use. Each subfolder has a "env.sh" script which is added by the > > dependency manager to the qsub script to be processed at runtime . I was > > thinking that one of the benefits of using the "requirements" tag is that it > > would/could allow for tool and version specific dependency chains. For > > instance if a version of tophat only runs with a certain version of bowtie > > but you want to make the newest beta version of bowtie also available to run > > independently. > > > > Right now the (sort of problem) I am having is that in order for the > > necessary PATH information to be transmitted to the compute nodes I have to > > have bowtie and samtools listed as requirements in the tool wrapper. The > > PATH variable of the galaxy user doesn't transmit to the compute nodes > > (despite having it set before launching the main instance of galaxy). With > > our current setup I have seen this with several different tools that call > > other tools and so I am wondering if my cluster/cloud setup has gone wrong > > somewhere. Is there a "right" way or place to specify the runtime PATH so > > that it will be transmitted to the compute nodes? Just trying to figure this > > out in terms of "best practices" since my current setup seems to require > > modification of the default wrappers to transmit PATH information to the > > compute nodes. > > > > Thanks again, > > Andrew > > > > On Sun, Jan 22, 2012 at 8:00 AM, Hans-Rudolf Hotz wrote: > >> > >> > >> > >> On 01/22/2012 01:41 AM, Anthonius deBoer wrote: > >>> > >>> All tools need to be in the path of the user running galaxy. > >>> > >> > >> but can be simply added to the path by adding them to the 'run.sh' script > >> > >> Regards, Hans > >> > >> > >> > >>> Regards, > >>> > >>> Thon > >>> > >>> Thon de Boer, Ph.D > >>> Bioinformatics Guru > >>> > >>> T: +1.650.799.6839 | E-mail: thondeb...@me.com > >>> > >>> http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b > >>> > >>> "IMPORTANT NOTICE: This email message is legally privileged, confidential > >>> and is for the use of the individual or entity to whom it is addressed. If > >>> you have received this email message by error, please immediately notify > >>> us > >>> by email and delete the message. Thank you." > >>> > >>> On Jan 21, 2012, at 3:50 PM, Andrew Warren wrote: > >>> > >>>> Hello, > >>>> > >>>> We recently transitioned from a CloudMan instance of galaxy to our own > >>>> cluster and started having problems with calls to tools from within > >>>> other tools. For example when Tophat calls bowtie-inspect its not > >>>> finding the executable. To fix this I listed bowtie in the > >>>> requirements section of the tophat wrapper like so: > >>>> > >>>> > >>>> Find splice junctions using RNA-seq data > >>>> tophat --version > >>>> > >>>> t
Re: [galaxy-dev] software installs: PATH vs env.sh
Thanks for the replies. One extra element I failed to properly describe is that we are running on a cluster with a similar setup to CloudMan. This means we have a galaxyTools/tools directory with folders for each tool, and subfolders for each version with a "default" symlink to the version currently in use. Each subfolder has a "env.sh" script which is added by the dependency manager to the qsub script to be processed at runtime . I was thinking that one of the benefits of using the "requirements" tag is that it would/could allow for tool and version specific dependency chains. For instance if a version of tophat only runs with a certain version of bowtie but you want to make the newest beta version of bowtie also available to run independently. Right now the (sort of problem) I am having is that in order for the necessary PATH information to be transmitted to the compute nodes I have to have bowtie and samtools listed as requirements in the tool wrapper. The PATH variable of the galaxy user doesn't transmit to the compute nodes (despite having it set before launching the main instance of galaxy). With our current setup I have seen this with several different tools that call other tools and so I am wondering if my cluster/cloud setup has gone wrong somewhere. Is there a "right" way or place to specify the runtime PATH so that it will be transmitted to the compute nodes? Just trying to figure this out in terms of "best practices" since my current setup seems to require modification of the default wrappers to transmit PATH information to the compute nodes. Thanks again, Andrew On Sun, Jan 22, 2012 at 8:00 AM, Hans-Rudolf Hotz wrote: > > > On 01/22/2012 01:41 AM, Anthonius deBoer wrote: > >> All tools need to be in the path of the user running galaxy. >> >> > but can be simply added to the path by adding them to the 'run.sh' script > > Regards, Hans > > > > Regards, >> >> Thon >> >> Thon de Boer, Ph.D >> Bioinformatics Guru >> >> T: +1.650.799.6839 | E-mail: thondeb...@me.com >> >> http://www.linkedin.com/pub/**thon-de-boer/1/1ba/a5b<http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b> >> >> "IMPORTANT NOTICE: This email message is legally privileged, confidential >> and is for the use of the individual or entity to whom it is addressed. If >> you have received this email message by error, please immediately notify us >> by email and delete the message. Thank you." >> >> On Jan 21, 2012, at 3:50 PM, Andrew Warren wrote: >> >> Hello, >>> >>> We recently transitioned from a CloudMan instance of galaxy to our own >>> cluster and started having problems with calls to tools from within >>> other tools. For example when Tophat calls bowtie-inspect its not >>> finding the executable. To fix this I listed bowtie in the >>> requirements section of the tophat wrapper like so: >>> >>> >>> Find splice junctions using RNA-seq data >>> tophat --version >>> >>> tophat >>> bowtie >>> samtools >>> >>> >>> Now I am wondering, is it generally expected that all tools used by >>> galaxy will have their executables on the user galaxy's PATH? Is the >>> above a good solution? Or is there something else likely amiss with >>> our galaxy setup? I think we recently pulled updates for some major >>> tool_shed release but I haven't been able to determine if any of the >>> tools listed above were affected by that. >>> >>> Wish I were in Český Krumlov asking this question. Missed the >>> registration deadline...doh. >>> >>> Thanks, >>> Andrew Warren >>> >>> __**_ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >>> >> >> __**_ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] software installs: PATH vs env.sh
Hello, We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so: Find splice junctions using RNA-seq data tophat --version tophat bowtie samtools Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that. Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh. Thanks, Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] bug? workflow controller item slug for shared workflows
Potential bug? I haven't seen any effects of this, and I don't really know whats going on with slugs, but I just wanted to mention it in case its an issue. Starting on line 149 of lib/galaxy/web/controllers/workflow.py there is a section of code for creating slugs for shared workflows. There is a loop there for slug creation and the following check OUTSIDE the loop: if slug_set: trans.sa_session.flush() Should that be in the loop? Otherwise its just flushing the session if the last workflow gets a slug created. -Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] History reports misconfiguration of the Galaxy job running system
Hi Nate, I am running in daemon mode so this out of paster.log. The job gets submitted to the PBS queue normally but the error shows up in history panel right away (also the working directory shows up during the cufflinks run): galaxy.jobs DEBUG 2011-10-24 17:51:25,567 dispatching job 595 to pbs runner galaxy.jobs INFO 2011-10-24 17:51:25,772 job 595 dispatched galaxy.jobs.runners.pbs DEBUG 2011-10-24 17:51:26,153 (595) submitting file /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/pbs/595.sh galaxy.jobs.runners.pbs DEBUG 2011-10-24 17:51:26,155 (595) command is: python /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/tools/ngs_rna/cufflinks_wrapper.py --input=/opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/files/001/dataset_1866.dat --assembled-isoforms-output=/opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/files/002/dataset_2161.dat --num-threads="6" -I 30 -F 0.05 -j 0.05-g /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/files/001/dataset_1699.dat -b --ref_file=/opt/rnaseq_data/indices/bowtie/Salmonella/14028S/Salmonella_enterica_subsp_enterica_serovar_Typhimurium_str_14028S.fna --dbkey=14028S --index_dir=/opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/tool-data galaxy.jobs.runners.pbs DEBUG 2011-10-24 17:51:26,157 (595) queued in batch queue as 303.localhost galaxy.jobs DEBUG 2011-10-24 17:51:26,599 dispatching job 596 to pbs runner galaxy.jobs INFO 2011-10-24 17:51:26,738 job 596 dispatchedgalaxy.jobs.runners.pbs DEBUG 2011-10-24 17:51:26,767 (595/303.localhost) PBS job state changed from N to R Then later with no errors in paster.log when the cufflinks job is finishing (notice that it doesn't try to copy the contents of the working directory for this job like it normally does because galaxy thinks the job wasn't submitted to the queue even though it was): galaxy.jobs.runners.pbs DEBUG 2011-10-24 19:05:57,402 (595/303.localhost) PBS job has left queue galaxy.jobs.runners.pbs DEBUG 2011-10-24 19:05:57,402 (596/304.localhost) PBS job state changed from Q to R galaxy.jobs.runners.pbs DEBUG 2011-10-24 21:29:01,995 (596/304.localhost) PBS job has left queue galaxy.jobs.runners.pbs DEBUG 2011-10-24 21:29:01,995 (597/305.localhost) PBS job state changed from Q to R galaxy.jobs DEBUG 2011-10-24 21:29:02,999 finish(): Moved /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/job_working_directory/596/isoforms.fpkm_tracking to /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/files/002/dataset_2163.dat as directed by from_work_dir galaxy.jobs DEBUG 2011-10-24 21:29:03,555 finish(): Moved /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/job_working_directory/596/genes.fpkm_tracking to /opt/hts_software/galaxy-parent/galaxy_server/galaxy-dist/database/files/002/dataset_2162.dat as directed by from_work_dir Thanks, Andrew On Mon, Nov 7, 2011 at 4:00 PM, Nate Coraor wrote: > On Oct 25, 2011, at 2:43 AM, Andrew Warren wrote: > > > Hello, > > I recently encountered a problem when trying to run Cufflinks on eight > BAM files on our galaxy instance (via the multi-input toggle) and received > the error: "Unable to run job due to a misconfiguration of the Galaxy job > running system" for some, but not all, of the cufflinks jobs that appear in > the history. These particular BAM files were copied over from the history > of a larger workflow where they were successfully run through cufflinks. In > the case of the problem workflow run, the cufflinks jobs are all > successfully submitted to Torque PBS and continue to run and finish but > many have this error displayed in the History. The jobs with the error > displayed fail to copy files from the working directory to the database > directory despite running to completion. We recently updated to > galaxy-central changeset 6131:be6c89c33639. I have seen this error multiple > times for this workflow, even after restarting galaxy. Does anyone have any > ideas of what might be going wrong? > > Hi Andrew, > > The error you're receiving indicates that there should also be a traceback > logged to the Galaxy server log file or output when this occurs. Could you > check the log/output for such a traceback? > > Thanks, > --nate > > > > > Thanks for any help, > > Andrew Warren > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > >
[galaxy-dev] an attempt on multiple multiple inputs
Hello galaxy dev, I have done some work on enabling multiple input files for multiple tool parameters in a workflow. This could come in handy when running multiple paired end data files for RNA Seq through a pipeline (I know this has been discussed on a few different threads but I couldn't find any when writing this up). The changes cause the workflow controller to step through the input data in synchronized fashion going down the list (of selected inputs). When running a workflow with multiple, multiple inputs the default ordering from the history can be problematic so I also did a little work on a new display/interaction method that I called "on_deck." If you set the tool xml config to have parameter tag display="on_deck" then it will enable this interaction method. e.g. from the bowtie wrapper: The changes to run.mako seem a little hacked in to me (I am still getting a handle on the mako format). I wasn't sure where to stick the javascript that is in there. It only lets you go forward if the number of inputs, for all multiple toggled inputs, is equal. The code is at https://bitbucket.org/aswarren/galaxy-cid 8cf8d25557b2 Hope this helps somebody. Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] updating configuration files after code update
When first running Galaxy there are a number of files that are created including universe_wsgi.ini. After doing an 'hg pull -u' the sample version of this file will be updated but not the specific server instance of the file that was initially created. I was wondering if there are any other files like this that need to be updated/merged/replaced as development continues? Also, does anyone have any good practices or methods for updating configuration files? (right now I'm just using a visual diff and merge to get the new options for universe_wsgi.ini) Sorry if this has been covered somewhere or asked before, I couldn't find any material on it. Thanks, Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] History reports misconfiguration of the Galaxy job running system
Hello, I recently encountered a problem when trying to run Cufflinks on eight BAM files on our galaxy instance (via the multi-input toggle) and received the error: "Unable to run job due to a misconfiguration of the Galaxy job running system" for some, but not all, of the cufflinks jobs that appear in the history. These particular BAM files were copied over from the history of a larger workflow where they were successfully run through cufflinks. In the case of the problem workflow run, the cufflinks jobs are all successfully submitted to Torque PBS and continue to run and finish but many have this error displayed in the History. The jobs with the error displayed fail to copy files from the working directory to the database directory despite running to completion. We recently updated to galaxy-central changeset 6131:be6c89c33639. I have seen this error multiple times for this workflow, even after restarting galaxy. Does anyone have any ideas of what might be going wrong? Thanks for any help, Andrew Warren ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy-dev Digest, Vol 63, Issue 8
I would also like to voice my support for this feature. I wrote a wrapper for bowtie that converts the SAM output to BAM after bowtie is finished just to avoid the hassle of letting galaxy "know" that the SAM file existed (didn't want to run Tophat). After thinking about how I would go about deleting an existing output it occurred to me that a "deleting tool" would require some extra logic since you would probably want to prevent the output port on a workflow node/tool from being connected to the input of another node if the output is going to be deleted. I was wondering if it might make sense to modify the "flagged output" feature (the asterisk) of the galaxy tools nodes to delete the non-flagged outputs instead of just hiding them? Or perhaps just mark them as deleted so they will be taken care of by the cleanup scripts? In this same line of thinking, it might make sense to have a flag for the input ports that specify that the input will be consumed/deleted after the tool has successfully run. This would address the case where you wanted to use the output of a tool before it is removed. Cheers, Andrew >I haven't had a chance to do anything on this yet, but I'll see if I can work something out in the near future.> > >-Dannon > >On Sep 7, 2011, at 9:34 PM, Glen Beane wrote: > > > On Sep 7, 2011, at 8:10 PM, Edward Kirton wrote: > >> i'm resurrecting this thread to see if there's any more support for the idea of deleting intermediate files in a workflow. i think this is an important feature to have. oftentimes a workflow creates many intermediate files no one will ever look at. and leaving it up to the user to cleanup their data files is asking too much. there's another ticket regarding allow users to still be able to preview the metadata of deleted workflow history items and together these would go together nicely. >> > > I am _very_ interested in this feature > > -- > Glen L. Beane > Senior Software Engineer > The Jackson Laboratory > (207) 288-6153 > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] suggestion for multithreading
So would the current correct method for setting up multi-threaded jobs on a cluster be to specify custom runners in the [galaxy:tool_runners] section of the universe config file for EVERY tool that uses a multiple threads (assuming the default is set to one)? For example, for the bowtie program and a queue named "galaxy": *bowtie = pbs:///galaxy/-l ppn=4,mem=16gb/* * * Is this currently the only way for galaxy to inform the queuing system how many threads a program will use? And does this mean that without custom runners in the config file any muti-threaded program that has multiple instances in an asychronous workflow has the opportunity to overload a cluster node since the queuing system doesn't "know" how many threads the program will be using? Just want to make sure I'm not missing out on the latest and greatest method for process management. :) Thanks, Andrew * * Louise-Amélie Schmitt wrote: > > > > default_cluster_job_runner will remain for backwards compatibility, but > > we'll ship a sample job_conf.xml that runs everything locally by > > default. > > > > --nate > > Haha, and I did that before realizing I could do just what I needed by > writing tool-specific *pbs*:// URLs at the end of the config file... I'm such > an idiot. Haha, okay, I don't think i even noticed since I was distracted by your implementation being a step in the way we want to go with it. > But I really like what you did of it and I have a couple of questions. > > Concerning the single-threaded tools, what would happen if the number of > threads set in the xml file was >1 ? It'd consume extra slots, but the tool itself would just run as usual. > Could it be possible to forbid a tool to run on a given node? Hrm. In *PBS* you could do it using node properties/neednodes or resource requirements. I'd have to think a bit about how to do this in a more general way in the XML. --nate > > Thanks, > L-A > > > > > >> > >> Peter > >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] API workflow_execute.py using ldda
Hello,
I have been working on running a workflow programmatically on my local
galaxy instance using library datasets.
It took me a few hours to figure out the problem I was having so I thought I
would document it here in case it helps anyone. Also, I have a question
regarding 'ldda' IDs (see below).
After looking at this post:
http://gmod.827538.n3.nabble.com/Running-a-workflow-programatically-td2572556.htmland
the comments in workflow_execute.py I was trying to run a workflow in
the following way:
1) get the dataset ID's from the library
./display.py
http://localhost:8080/api/libraries/ebfb8f50c6abde6d/contents
Collection Members
--
#1:
/api/libraries/ebfb8f50c6abde6d/contents/c6ca0ddb55be603a016bf252082cbf1f
name: /
type: folder
id: c6ca0ddb55be603a016bf252082cbf1f
#2: /api/libraries/ebfb8f50c6abde6d/contents/a0e1686ccf7a028a
name: /http://smane.vbi.vt.edu/SRR014335.fastq
type: file
id: a0e1686ccf7a028a
#3: /api/libraries/ebfb8f50c6abde6d/contents/98a64dc1a6526c8e
name: /http://smane.vbi.vt.edu/SRR014339.fastq
type: file
id: 98a64dc1a6526c8e
#4: /api/libraries/ebfb8f50c6abde6d/contents/2764e29c8c7b7454
name: /http://smane.vbi.vt.edu/Yeast.gtf
type: file
id: 2764e29c8c7b7454
2) get input IDs for the workflow
./display.py http://localhost:8080/api/workflows/f2db41e1fa331b3e
Member Information
--
url: /api/workflows/f2db41e1fa331b3e
inputs: {'102': {'value': '', 'label': 'Illumina2'}, '104': {'value': '',
'label': 'ReferenceAnnotation'}, '96': {'value': '', 'label': 'Illumina1'}}
id: f2db41e1fa331b3e
Then I ran:
python /opt/galaxy_server/galaxy-dist/scripts/api/workflow_execute.py
http://localhost:8080/api/workflows f2db41e1fa331b3e 'Test API'
'102=ldda=a0e1686ccf7a028a' '104=ldda=2764e29c8c7b7454'
'96=ldda=98a64dc1a6526c8e'
This gives the following error at the end of the stack trace for HTTP Error
500: *ValueError: invalid literal for int() with base 10: 'file.37'*
*
*
After looking through the code for a while I saw that when the "src id type"
is given as "ld" then the decoded name is split and the number is used to
retrieve the ldda or "library_dataset_dataset_association"
So instead I ran the above command with: '102=ld=a0e1686ccf7a028a'
'104=ld=2764e29c8c7b7454' '96=ld=98a64dc1a6526c8e'
And everything worked, woo!
This brings me to my question: Where can I find ldda ID's ? I imagine I
could find them in my database and there looks like some functions in
lib/galaxy/web/api/workflows.py for handling them but I am having trouble
locating them via an API script or on the library management interface.
Note: I am assuming that the ID's returned by display.py are in fact "ld"
ID's since this is the ID type I used to avoid the error listed above.
Thanks,
Andrew
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
