[galaxy-dev] Fwd: Running galaxy from eclipse running PyDev

[Eric Kuyt]

Hi All,

I am Running my local galaxy instance in Eclipse - pydev to make debugging
somewhat easier, but now I run into loads of Code errors. It seems that
pydev doesn't understand the Bunch() Class frequently used in Galaxy.

But also code errors that maybe could be fixed. such as:

lib/galaxy/jobs/runner/lwr.py:232

#worker = threading.Thread( ( name=LwrJobRunner.thread-%d % i ), #
target=self.run_next )
--
#worker = threading.Thread( name=LwrJobRunner.thread-%d % i ,
#target=self.run_next )

Are these real errors or am I doing something wrong.

Is there anyone with more experience in running Galaxy from within Eclipse?

Thanks,

Eric
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] psiblast

[Peter Cock]

On Mon, Feb 4, 2013 at 7:32 PM, Peter Cock p.j.a.c...@googlemail.com wrote:


 On Monday, February 4, 2013, Luobin Yang wrote:

 Hi, Peter,

 Thanks for pointing them out!

 The phi-blast and psi-blast actually share the same binary, both use the
 psiblast program in the NCBI BLAST+ tools. When the option -phi_pattern is
 used, then it does phi-blast searching.

 Best,
 Luobin



 Could you give a few command line examples of how you typically use
 psiblast,  focussing in the main options only?

 I would find that helpful when putting together a new wrapper - since
 including all the options right away is overwhelming.

 Thanks,

 Peter

I suspect we would also need to define another datatype for the PSSM
pattern files. That makes things more complicated.

Peter
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Fwd: Running galaxy from eclipse running PyDev

[Eric Kuyt]

Hi Dannon,

Thanks for the fast reply

On 5 February 2013 14:49, Dannon Baker dannonba...@me.com wrote:

 Hi Eric,

The threading.Thread issue indicated below has been fixed (the changeset is
 already available in galaxy-dist) and isn't eclipse-centric.


Ok somehow I didn't get this update from
https://bitbucket.org/galaxy/galaxy-dist even today.


 From eclipse, how are you launching galaxy?


I installed a Mercurial plugin in Eclipse, checked out from '
https://bitbucket.org/galaxy/galaxy-dist; ran ./run.sh one time to download
the Eggs necessary, installed the eggs in my eclipse PYTHONPATH and put
paster.py serve universe_wsgi.ini in a run configuration.

The running of galaxy works nicely, my logs are handled by Eclipse and
breakpointing works. So I was quite happy with everything, until I asked
PyDev for a Code analysis.

It turns out that PyDev did not understand the Bunch() classes like
model.Job.states.QUEUED, but also functions in classes that don't have
self as a variable or udefined variables that are called. Probably in real
life this is not a big problem, because galaxy is running quite flawlessly,
so maybe I just misconfigured PyDev.

Because of this I was curious if more people tried the Eclipse way or
maybe another IDE with debugging enabled. Because I think an IDE can be a
great advantage over text-editor developing.


 Personally for debugging I've found it easier to use the debugging
 middleware for anything that happens inside a web transaction (sometimes
 intentionally raising an exception just to kick out to it), and a simple
 addition of 'import pdb; pdb.set_trace()' for creating breakpoints that I
 can work through via pdb in the terminal.


Yeah maybe it's better just to debug this way but it feels a bit low tech
with all the nice PyDev features in Eclipse.

Thanks again,

Eric




 -Dannon


 On Feb 5, 2013, at 5:29 AM, Eric Kuyt eric.ku...@wur.nl wrote:

  Hi All,
 
  I am Running my local galaxy instance in Eclipse - pydev to make
 debugging somewhat easier, but now I run into loads of Code errors. It
 seems that pydev doesn't understand the Bunch() Class frequently used in
 Galaxy.
 
  But also code errors that maybe could be fixed. such as:
 
  lib/galaxy/jobs/runner/lwr.py:232
 
  #worker = threading.Thread( ( name=LwrJobRunner.thread-%d % i ), #
 target=self.run_next )
  --
  #worker = threading.Thread( name=LwrJobRunner.thread-%d % i ,
 #target=self.run_next )
 
  Are these real errors or am I doing something wrong.
 
  Is there anyone with more experience in running Galaxy from within
 Eclipse?
 
  Thanks,
 
  Eric
 
 
 
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
 
   http://lists.bx.psu.edu/




___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] psiblast

[Luobin Yang]

Hi, Peter,

You are right, I thought there was already a datatype defined for PSSM, but
it turned out there isn't one yet. I attached a draft tool configuration
file for psiblast, it was based on the blastp configuration file, please
revise as you like.

The options that are specific to psiblast are the following:
1. num_iterations
2. pseudocount
3. inclusion_ethresh

Other options should be the same as the blastp program. So a typical
command line would be the same as blastp except adding above options if
non-default values are used. I can see there are two optional output files,
one is the PSSM and the other is ASCII PSSM. Besides using a typical query
input, psiblast can take a PSSM file or a multiple sequence alignment file
as the input.

Luobin

On Tue, Feb 5, 2013 at 4:53 AM, Peter Cock p.j.a.c...@googlemail.comwrote:

 On Mon, Feb 4, 2013 at 7:32 PM, Peter Cock p.j.a.c...@googlemail.com
 wrote:
 
 
  On Monday, February 4, 2013, Luobin Yang wrote:
 
  Hi, Peter,
 
  Thanks for pointing them out!
 
  The phi-blast and psi-blast actually share the same binary, both use the
  psiblast program in the NCBI BLAST+ tools. When the option -phi_pattern
 is
  used, then it does phi-blast searching.
 
  Best,
  Luobin
 
 
 
  Could you give a few command line examples of how you typically use
  psiblast,  focussing in the main options only?
 
  I would find that helpful when putting together a new wrapper - since
  including all the options right away is overwhelming.
 
  Thanks,
 
  Peter

 I suspect we would also need to define another datatype for the PSSM
 pattern files. That makes things more complicated.

 Peter

tool id=ncbi_psiblast_wrapper name=NCBI BLAST+ psiblast version=0.0.15
descriptionSearch protein database with protein query sequence(s)/description
!-- If job splitting is enabled, break up the query file into parts --
parallelism method=multi split_inputs=query split_mode=to_size split_size=1000 shared_inputs=subject,histdb merge_outputs=output1/parallelism
version_commandpsiblast -version/version_command
command
## The command is a Cheetah template which allows some Python based syntax.
## Lines starting hash hash are comments. Galaxy will turn newlines into spaces
psiblast
#if str($input_opts.input_opts_selector) == query
-query $query
#else if str($input_opts.input_opts_selector) == pssm
-in_pssm $in_pssm
#else if str($input_opts.input_opts_selector) == msa
-in_msa $in_msa
#end if

#if $db_opts.db_opts_selector == db:
  -db ${db_opts.database.fields.path}
#elif $db_opts.db_opts_selector == histdb:
  -db ${os.path.join($db_opts.histdb.extra_files_path,'blastdb')}
#else:
  -subject $db_opts.subject
#end if

-evalue $evalue_cutoff
-out $output1

##Set the extended list here so if/when we add things, saved workflows are not affected
#if str($out_format)==ext:
-outfmt 6 std sallseqid score nident positive gaps ppos qframe sframe qseq sseq qlen slen
#else:
-outfmt $out_format
#end if

-num_threads 8

#if $adv_opts.adv_opts_selector==advanced:
$adv_opts.filter_query
-matrix $adv_opts.matrix
## Need int(str(...)) because $adv_opts.max_hits is an InputValueWrapper object not a string
## Note -max_target_seqs overrides -num_descriptions and -num_alignments
#if (str($adv_opts.max_hits) and int(str($adv_opts.max_hits))  0):
-max_target_seqs $adv_opts.max_hits
#end if
#if (str($adv_opts.word_size) and int(str($adv_opts.word_size))  0):
-word_size $adv_opts.word_size
#end if
##Ungapped disabled for now - see comments below
##$adv_opts.ungapped
$adv_opts.parse_deflines
## End of advanced options:
#end if

#if $psiblast_opts.psiblast_opts_selector==advanced
-num_iterations $psiblast_opts.num_iterations
-pseudocount $psiblast_opts.pseudocount
-inclusion_ethresh $psiblast_opts.inclusion_ethresh
#end if

#if $pssm_output_condition.pssm_output_selector == yes 
-out_pssm $pssm_output
#end if

#if $pssm_ascii_output_condition.pssm_ascii_output_selector == yes 
-out_ascii_pssm $pssm_ascii_output
#end if

/command
stdio
!-- Anything other than zero is an error --
exit_code range=1: /
exit_code range=:-1 /
!-- In case the return code has not been set propery check stderr too --
regex match=Error: /
regex match=EXception: /
/stdio
inputs
conditional name=input_opts
param name=input_opts_selector type=select label=Select the input file type
option value=query selected=trueQuery/option
option value=msaMultiple Sequence Alignment file/option
option value=pssmPSI-BLAST checkpoint file/option
/param
when value=query
param name=query type=data format=fasta label=Protein query sequence(s)/ 
/when
when value=msa
param name=in_msa type=data format=msa,phylip label=Multiple Sequence Alignment File /
/when
when value=pssm
param name=in_pssm type=data 

Re: [galaxy-dev] permissions on library contents

[John Chilton]

Only setting library permissions has been implemented. A new methods
would need to be added to this controller if you would like to read
library permission.

If you want to change the library permissions using the API, you can
post against that URL and include the following options in your
payload.

LIBRARY_ACCESS_in = list of role ids
LIBRARY_MODIFY_in = list of role ids
LIBRARY_ADD_in = list of role ids
LIBRARY_MANAGE_in = list of role ids

-John


On Tue, Feb 5, 2013 at 10:58 AM, Olivia Doppelt
olivia.dopp...@pasteur.fr wrote:
 Hello,

 I'm looking for a way to change library permissions using galaxy API.

 When I look at the API documentation at the url


 https://galaxy-dist.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/permissions.html#PermissionsController

 There is supposed to be a way to do that but when I try to access my local
 galaxy url library permissions, the route isn't valid
 (http://localhost:8080/api/libraries/f2db41e1fa331b3e/permissions)

 Could you please help me with that ?

 Best regards,

 --
 Olivia Doppelt-Azeroual, PhD
 CIB - Institut Pasteur, PARIS

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Error! Sam-to-bam samtools command not found

[Hakeem Almabrazi]

Hi,

I installed a local galaxy on Centos 6 and I followed the instructions on how 
to install samtools/bwa/bowtie found here 
http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup.

I setup the correct env. for these programs correctly.  Since I have no issues 
with the bwa and bowtie so I assumed I got everything correctly but that was 
not true since I keep getting the following error when I try Sam-To-Bam...
.
galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external 
set_meta for job 62 finished
galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or 
stdio handling; checking stderr for success
galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata 
externally failed for HistoryDatasetAssociation 76: Error Setting BAM Metadata: 
/bin/sh: samtools: command not found
galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish 
method failed
Traceback (most recent call last):
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 
156, in run_job
job_wrapper.finish( stdout, stderr, exit_code )
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, line 412, 
in finish
dataset.datatype.set_meta( dataset, overwrite = False ) #call 
datatype.set_meta directly for the initial set_meta call during dataset creation
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 
213, in set_meta
raise Exception, Error Setting BAM Metadata: %s % stderr
Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found
galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external 
metadata files



* Samtools is in the path:  echo $PATH
/usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/


* I setup the env.sh as recommended. more 
//usr/local/galaxy/software/samtools/0.1.16/env.sh
#Configure PATH to Samtools binaries
PATH=/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH
export PATH


* And here what I have in the bin: ls -lh 
/usr/local/galaxy/software/samtools/0.1.16/bin/
total 1.5M
-rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl
-rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl
-rwxr-xr-x 1 galaxy galaxy  19K Jan 29 12:53 export2sam.pl
-rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl
-rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl
-rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl
-rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl
-rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools
-rwxr-xr-x 1 galaxy galaxy  14K Jan 29 12:53 samtools.pl
-rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 soap2sam.pl
-rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 wgsim_eval.pl
-rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 zoom2sam.pl


* I could ran the samtools from anywhere in the system including from 
galaxy account.


* I looked at this thread which the guy has similar problem but he was 
trying to set it up in a cluster and installing it on Ubunto.  I read the whole 
thing and tried everything in there but no luck

http://gmod.827538.n3.nabble.com/Exception-Error-Setting-BAM-Metadata-bin-sh-samtools-not-found-td3915349.html


* I have installed R and Python/rpy and set their env. as required (I 
THINK) assuming this might be a cause but I do not see anything in the error 
log that will indicate that...

Can anyone help in debugging this issue please?

Thanks

hakeem



___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Why

[Dannon Baker]

Have you restarted your galaxy instance after changing the setting?  And, in 
that drop-down, can you confirm that 'filesystem paths' is not an option?

Also, please keep conversations on-list so that others might be able to help 
and/or benefit from the exchange.

-Dannon


On Feb 5, 2013, at 12:15 PM, hudai...@icgeb.org wrote:

 I set  allow_library_path_paste = True in universe_wsgi.ini  and still don't 
 have the option of copying locally on manage data libraries page.
 
 http://i.imgur.com/PEeCAnc.png?1
 
 Should I set some other parameters to True?
 
 quote author='Dannon Baker'
 The absolute fastest method would be to enable filepath upload for data
 libraries (in universe_wsgi.ini -- allow_library_path_paste = True).  Once
 that's set, you can paste the absolute file path into the upload box, and
 check the box for not copying the data.  This will leave the data where it
 is on the external drive, though it will make it available in the Data
 Library.  Obviously if you use this method, you'll need to have that
 external drive attached (and always at the same mount point) to use the
 data.
 
 If you actually *do* want to copy this data off the external drive, simply
 don't check that 'no copy' checkbox and the data will be copied directly
 through the filesystem and not uploaded.
 
 -Dannon
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Error! Sam-to-bam samtools command not found

[Dannon Baker]

Hakeem,

What's going on here is that the BAM metadata setting function doesn't use the 
normal dependency methods and thus doesn't read 'env.sh' (which does look 
correct), requiring 'samtools' to be available on the path of the galaxy user.  
How are you executing galaxy itself?  Are you positive that at launch, samtools 
is in the PATH?  One simple way to inject this basic dependency would be to add 
it to the PATH in run.sh itself, as galaxy loads.

-Dannon




On Feb 5, 2013, at 12:17 PM, Hakeem Almabrazi halmabr...@idtdna.com wrote:

 Hi,
  
 I installed a local galaxy on Centos 6 and I followed the instructions on how 
 to install samtools/bwa/bowtie found 
 herehttp://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup. 
  
 I setup the correct env. for these programs correctly.  Since I have no 
 issues with the bwa and bowtie so I assumed I got everything correctly but 
 that was not true since I keep getting the following error when I try 
 Sam-To-Bam…
 .
 galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external 
 set_meta for job 62 finished
 galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or 
 stdio handling; checking stderr for success
 galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata 
 externally failed for HistoryDatasetAssociation 76: Error Setting BAM 
 Metadata: /bin/sh: samtools: command not found
 galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish 
 method failed
 Traceback (most recent call last):
   File /usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 
 156, in run_job
 job_wrapper.finish( stdout, stderr, exit_code )
   File /usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, line 412, 
 in finish
 dataset.datatype.set_meta( dataset, overwrite = False ) #call 
 datatype.set_meta directly for the initial set_meta call during dataset 
 creation
   File /usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 
 213, in set_meta
 raise Exception, Error Setting BAM Metadata: %s % stderr
 Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found
 galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external 
 metadata files
 
  
 · Samtools is in the path:  echo $PATH
 /usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/
  
 · I setup the env.sh as recommended. more 
 //usr/local/galaxy/software/samtools/0.1.16/env.sh
 #Configure PATH to Samtools binaries
 PATH=/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH
 export PATH
  
 · And here what I have in the bin: ls -lh 
 /usr/local/galaxy/software/samtools/0.1.16/bin/
 total 1.5M
 -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl
 -rwxr-xr-x 1 galaxy galaxy  19K Jan 29 12:53 export2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl
 -rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl
 -rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools
 -rwxr-xr-x 1 galaxy galaxy  14K Jan 29 12:53 samtools.pl
 -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 soap2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 wgsim_eval.pl
 -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 zoom2sam.pl
  
 · I could ran the samtools from anywhere in the system including from 
 “galaxy” account.
  
 · I looked at this thread which the guy has similar problem but he 
 was trying to set it up in a cluster and installing it on Ubunto.  I read the 
 whole thing and tried everything in there but no luck….
 http://gmod.827538.n3.nabble.com/Exception-Error-Setting-BAM-Metadata-bin-sh-samtools-not-found-td3915349.html
  
 · I have installed R and Python/rpy and set their env. as required (I 
 THINK) assuming this might be a cause but I do not see anything in the error 
 log that will indicate that…
  
 Can anyone help in debugging this issue please?
  
 Thanks
  
 hakeem
  
  
  
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  

Re: [galaxy-dev] Error! Sam-to-bam samtools command not found

[Hakeem Almabrazi]

Thank you Dannon for pointing me to the right direction. I have been using a 
script suggested in one of this forum to start it service galaxy start/stop.  
I never thought that would be a source of issue since it is calling run.sh 
script anyway.  However, I just tried sh run.sh and I did not see this issue 
at all.  

Here is part of the /etc/init.d/galaxy script  I am using to start and stop the 
server

.
#!/bin/bash

SERVICE_NAME=galaxy
RUN_AS=galaxy
RUN_IN=/usr/local/galaxy/galaxy-dist

#--- main actions

start() {
echo Starting $SERVICE_NAME... 
cmd=cd $RUN_IN  sh run.sh --daemon
case $(id -un) in
$RUN_AS)
eval $cmd
;;
root)
su - $RUN_AS -c $cmd
;;
*)
echo *** ERROR *** must be $RUN_AS or root in order to 
control this service 2
exit 1
esac
echo ...done.
}

...

So what does that mean?  Is there something wrong with this script?  

Thank you...
 

-Original Message-
From: Dannon Baker [mailto:dannonba...@me.com] 
Sent: Tuesday, February 05, 2013 11:37 AM
To: Hakeem Almabrazi
Cc: Galaxy Dev
Subject: Re: [galaxy-dev] Error! Sam-to-bam samtools command not found

Hakeem,

What's going on here is that the BAM metadata setting function doesn't use the 
normal dependency methods and thus doesn't read 'env.sh' (which does look 
correct), requiring 'samtools' to be available on the path of the galaxy user.  
How are you executing galaxy itself?  Are you positive that at launch, samtools 
is in the PATH?  One simple way to inject this basic dependency would be to add 
it to the PATH in run.sh itself, as galaxy loads.

-Dannon




On Feb 5, 2013, at 12:17 PM, Hakeem Almabrazi halmabr...@idtdna.com wrote:

 Hi,
  
 I installed a local galaxy on Centos 6 and I followed the instructions on how 
 to install samtools/bwa/bowtie found 
 herehttp://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup. 
  
 I setup the correct env. for these programs correctly.  Since I have no 
 issues with the bwa and bowtie so I assumed I got everything correctly but 
 that was not true since I keep getting the following error when I try 
 Sam-To-Bam...
 .
 galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external 
 set_meta for job 62 finished
 galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or 
 stdio handling; checking stderr for success
 galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata 
 externally failed for HistoryDatasetAssociation 76: Error Setting BAM 
 Metadata: /bin/sh: samtools: command not found
 galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish 
 method failed
 Traceback (most recent call last):
   File /usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 
 156, in run_job
 job_wrapper.finish( stdout, stderr, exit_code )
   File /usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, line 412, 
 in finish
 dataset.datatype.set_meta( dataset, overwrite = False ) #call 
 datatype.set_meta directly for the initial set_meta call during dataset 
 creation
   File /usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 
 213, in set_meta
 raise Exception, Error Setting BAM Metadata: %s % stderr
 Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found
 galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external 
 metadata files
 
  
 * Samtools is in the path:  echo $PATH
 /usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/
  
 * I setup the env.sh as recommended. more 
 //usr/local/galaxy/software/samtools/0.1.16/env.sh
 #Configure PATH to Samtools binaries
 PATH=/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH
 export PATH
  
 * And here what I have in the bin: ls -lh 
 /usr/local/galaxy/software/samtools/0.1.16/bin/
 total 1.5M
 -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl
 -rwxr-xr-x 1 galaxy galaxy  19K Jan 29 12:53 export2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl
 -rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl
 -rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl
 -rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools
 -rwxr-xr-x 1 galaxy galaxy  14K Jan 29 12:53 samtools.pl
 -rwxr-xr-x 1 

Re: [galaxy-dev] Problem referencing EMBOSS tools in universe_wsgi.ini

[Nate Coraor]

On Jan 29, 2013, at 3:59 PM, Guest, Simon wrote:

 We have set up our local Galaxy installation to submit jobs by default 
 through HTCondor, using the drmaa tool runner.  We override this on a 
 tool-by-tool basis in universe_wsgi.ini, to specify certain tools we always 
 want to run locally.
  
 The problem is that EMBOSS tools have a tool id like ‘emboss: infoseq46’, and 
 it’s not possible to reference a tool with an id containing a colon in the 
 universe_wsgi.ini file (because colon is the character which separates a 
 configuration item name from its value).
  
 It seems to me there are two possible solutions:
 1. Implement some form of escape character processing in the config file 
 parser (on top of Python’s ConfigParser) so we can write tool ids in that 
 file like ‘emboss%3a infoseq46’
 2. Change all the EMBOSS tools to have ids like ‘emboss-infoseq46’ rather 
 than using a colon and space.
  
 For now, we are modifying our XML files locally (with a script) to do 2.
  
 Are there any comments from Galaxy developers on which of these solutions 
 they prefer, or any reason why (2) shouldn’t be incorporated in the Galaxy 
 mainline?

Hi Simon,

Significant changes to the way that job run parameters are defined will be 
included in a future release.  In specific, there'll be no more job runner 
URLs, and parameters will be specified with a more robust XML language.  These 
changes aren't stable enough to go out in the next release (scheduled for the 
end of this week), but they should be in the one after that.

--nate

  
 cheers,
 Simon
  
 
 ===
 Attention: The information contained in this message and/or attachments
 from AgResearch Limited is intended only for the persons or entities
 to which it is addressed and may contain confidential and/or privileged
 material. Any review, retransmission, dissemination or other use of, or
 taking of any action in reliance upon, this information by persons or
 entities other than the intended recipients is prohibited by AgResearch
 Limited. If you have received this message in error, please notify the
 sender immediately.
 ===
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Question regarding frequency of pbs_mom authentications

[Nate Coraor]

On Jan 30, 2013, at 11:34 AM, Daniel Patrick Sullivan wrote:

 Hi, Galaxy Developers,
 
 I have a couple of hopefully quick questions regarding PBS integration with 
 Galaxy;
 
 I noticed (based on taking a look at the PBS log data) that Galaxy 
 authenticates rather frequently to the pbs_mom server (please see the log 
 data at the end of this email).
 
 Is the frequency of these connections common (i.e. sometimes several times 
 per second).  Is there any way to throttle this?  Has anybody seen a problem 
 resulting from the large number of authentication request from the Galaxy 
 server to the pbs_mom server?  It is suspected that the frequency of these 
 connections is causing a problem on our scheduler node.  Thank-you so much 
 for taking the time to answer my questions.

Hi Dan,

Galaxy attempts to check the status of running jobs frequently to make the job 
life cycle as quick as possible.  We run three handlers all pointed at the same 
Torque server and have not seen adverse effects from it.

--nate

 
 [a.cri.dsullivan@sc01 server_logs]$ tail -f 20130130
 01/30/2013 10:30:50;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:50;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=12
 01/30/2013 10:30:51;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=10
 01/30/2013 10:30:51;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:52;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=10
 01/30/2013 10:30:52;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:53;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=10
 01/30/2013 10:30:53;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:54;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=10
 01/30/2013 10:30:54;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:55;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=10
 01/30/2013 10:30:55;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:56;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:56;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=12
 01/30/2013 10:30:57;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:57;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=10
 01/30/2013 10:30:58;0100;PBS_Server;Req;;Type AuthenticateUser request 
 received from s.cri.gal...@crigalaxy.uchicago.edu, sock=9
 01/30/2013 10:30:58;0100;PBS_Server;Req;;Type StatusJob request received from 
 s.cri.gal...@crigalaxy.uchicago.edu, sock=12
 
 Thank-you,
 
 Dan Sullivan
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Velvet program files

[Thyssen, Gregory - ARS]

So I am setting up my local Galaxy on a workstation for my own use.
I have installed several programs from the Assembly Main Tool Shed.
None works.
Are these just the wrappers?  Do I need to install, say, Velvet somewhere 
outside of Galaxy?
How do I do this?

My errors:
6: velveth on data 2
error
An error occurred running this job:Unable to run velveth: No such file or 
directory

7: Abyss on data 5
error
An error occurred running this job: ERROR: Expected exactly 3 arguments; got: 
67 /media/My Passport/IM-Data2Bio-10262012/IM-mut1/Im-mutant-bulk_1.fastq 
/home/galaxy/galaxy-dist/database/files/000/dataset_9.dat

8: MIRA contigs (FASTA)
error
An error occurred running this job: Traceback (most recent call last): File 
/home/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/peterjc/mira_assembler/3e7eca1f5d04/mira_assembler/tools/sr_assembly/mira.py,
 line 33, in mira_ver = get_version() File /home/galaxy/shed_tool


15: Singleton reads reads for velvet
error
An error occurred running this job: Error running preppereads.sh /bin/sh: 
/users/galaxy/galaxyscripts/prepare_pe_reads_for_velvet.sh: No such file or 
directory


Thanks,
Greg





This electronic message contains information generated by the USDA solely for 
the intended recipients. Any unauthorized interception of this message or the 
use or disclosure of the information it contains may violate the law and 
subject the violator to civil or criminal penalties. If you believe you have 
received this message in error, please notify the sender and delete the email 
immediately.
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Fwd: Desire to contribute

[Matthew Paul]

Dear Galaxy Project community,

I am working with a group of students at College of Charleston
of South Carolina. Being interested in bioinformatics and software
engineering, we chose to work on Galaxy for our open source class project.
We are subscribed to the appropriate mailing list, have been accessing
Trello and are becoming familiar with the Galaxy architecture. Our first
assignment is to identify and fix a bug, but unfortunately the bugs
reported seem to be going right over our heads.Where would be a good place
to start, so that we may be able to contribute to your system
(documentation, etc)? We are looking forward to your response.

Thank you,
Matt Paul

-- Forwarded message --
From: Matthew Paul mrp...@g.cofc.edu
Date: Tue, Feb 5, 2013 at 8:10 PM
Subject: Desire to contribute
To: galaxy-dev@lists.bx.psu.edu


Dear Galaxy Project community,
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] One-to-many relationship assocation

[Ted Goldstein]

I am trying to define a colleagues association table that can be used to notify 
people when there colleagues make changes. Think of it as a Facebook 
followers type of thing.
The relationship is not necessary inverse.  I may follow James Watson, he might 
not (yet) follow me. ;-)


Anyhow, here is where I am at, but SQL Alchemy wants a fully bidirectional 
relationship and complains


Colleague_Association.table = Table( colleague_association, metadata,
Column( id, Integer, primary_key=True ),
Column( user_id, Integer, ForeignKey( galaxy_user.id ), index=True ),
Column( colleague_user_id, Integer, ForeignKey( galaxy_user.id ), 
index=True ) ,
Column( deleted, Boolean, default=False )
)

assign_mapper( context, Colleague_Association, Colleague_Association.table, 
properties=dict( user=relation( User, backref=colleagues ), 
 colleague=relation( User ) ))

There are examples on the web for doing this in more recent versions of 
SQLAlchemy, but I can't find one that uses the (now deprecated) assign_mapper 
method.

Any ideas?

Thanks,
Ted


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/