Re: [galaxy-dev] Scatterplot: error fetching data
OK, will let you know. Update planned for next week. Cheers, Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 01/14/2013 06:40 PM, Jeremy Goecks wrote: I think this bug has been fixed in the most recent galaxy distribution; please update and let us know if it's still an issue. Thanks, J. On Jan 14, 2013, at 9:49 AM, Joachim Jacob wrote: Hi all, Trying to construct a small scatterplot of a tabular file with two columns, only integers between 1 and 9. The error that the scatter plot shows after clicking 'Draw' in the 'Data controls' is: ERROR fetching data: error Internal Server Error Any hint of what can be wrong? Cheers Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Enhancing tool id="subtract_query1"
Hi all, One of our users had a problem, that the subtract datasets tool did not behave. She had just uploaded two datasets, one column each (actually in txt format), one with ~30 lines, another with 106, and used the subtract tool. It did not work because of trailing white spaces (spaces, tabs and EOL) in the second file. Looking at the code, I had made a small adjustment then: adding a line: line = line.rstrip() (Line 32 in ~/galaxy-dist/tools/new_operations/subtract_query.py). Anyway, how to proceed that this fix is included in Galaxy main? I understand that these tools eventually will be migrated to the toolshed? Is it useful to do a pull request for this small enhancement? Or can somebody of the devteam add this patch? Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Anonymous use of workflows and visualization?
On Mon, Jan 14, 2013 at 4:13 PM, Sveinung Gundersen wrote: > Hello all, > > We are wondering if it is possible to activate anonymous use of the workflows > and the visualization (trackster) features of Galaxy, somehow? > > This is a strict requirement for the acceptance of our new paper in a > specific journal. > > Hack solutions are also welcome! > > Thanks, > Sveinung I'd also been been worried about a similar requirement to ensure reviewers could have anonymous access to software run as a service. In that case even providing dummy accounts to the reviewers might not have been allowed (since the administrator could easily 'spy' on what the reviewers did or didn't try). Given this potential barrier to publishing a paper using or extending Galaxy, I would hope extending Galaxy so that users can choose to share a workflow (or visualization) with anyone - even an anonymous user - would be a priority. Is there a trello issue on this? Searching there isn't very intuitive to me... http://galaxyproject.org/trello Thanks, Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Anonymous use of workflows and visualization?
> I'd also been been worried about a similar requirement to ensure reviewers > could have anonymous access to software run as a service. In that case > even providing dummy accounts to the reviewers might not have been > allowed (since the administrator could easily 'spy' on what the reviewers > did or didn't try). > > Given this potential barrier to publishing a paper using or extending > Galaxy, I would hope extending Galaxy so that users can choose to > share a workflow (or visualization) with anyone - even an anonymous > user - would be a priority. > > Is there a trello issue on this? Searching there isn't very intuitive to me... > http://galaxyproject.org/trello I've created a card for anonymous use of workflows and visualizations: https://trello.com/c/gja5TtwF Thanks, J. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] FTP and data library uploads
On Jan 14, 2013, at 11:45 PM, Shantanu Pavgi (Campus) wrote: > > On Jan 9, 2013, at 12:41 PM, Nate Coraor wrote: > >> Hi Shantanu, >> >> The code in question is actually in the upload tool, >> tools/data_source/upload.py. In general, you should be able to minimize the >> number of copy and delete steps if you put new_file_path, file_path, and >> ftp_upload_dir in the same filesystem. >> >> --nate >> > > Thanks for the reply Nate. I understand that both data libraries and FTP > upload methods call upload tool, but I am not following how it is being > called by them. For example, FTP upload seems to be doing newline conversion > as mentioned earlier, whereas, data library seems to be skipping it (or doing > it in a different manner, i.e., not in-place??). Which files should I be > looking at to follow FTP and data library upload calls to understand such > differences? Data library uploads, if instructed not to copy data in to Galaxy, will not modify the file on disk. The code from the library side begins at lib/galaxy/webapps/galaxy/controllers/library_common.py, and then moves through lib/galaxy/tools/actions/upload_common.py, where the library upload becomes a job. From there it goes to the job running system, where tools/data_source/upload.py is executed. --nate > > -- > Shantanu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] DRMAA runner weirdness
In our case someone had installed and started a second development instance of galaxy but used the same database as the first development instance. So the ids were mixed up and and causing some jobs to crash. Yuck! Thanks, Liisa From: Nate Coraor To: Liisa Koski Cc: kellr...@soe.ucsc.edu, "galaxy-dev@lists.bx.psu.edu" , galaxy-dev-boun...@lists.bx.psu.edu Date: 14/01/2013 10:48 AM Subject:Re: [galaxy-dev] DRMAA runner weirdness On Jan 11, 2013, at 9:32 AM, Liisa Koski wrote: > Hello, > Can you please post the link to this patch? I do not see it in the mail thread and I too have noticed some issues with the DRMAA job running since updating to the Oct. 23rd distribution. I don't know if it is related yet but I'd like to try the patch to see. I have two local instances of Galaxy (prod and dev). On my dev instance (which is fully up to date), when I run the same job multiple times, sometimes it finishes and sometimes it dies, this is independent of which node it runs on. My prod instance is still at the Oct. 03 distribution and does not experience this problem. So I am afraid to update our production instance. > > Thanks in advance, > Liisa Hi Liisa, Here's the one that Kyle is referring to: https://bitbucket.org/galaxy/galaxy-central/commits/c015b82b3944f967e2c859d5552c00e3e38a2da0 However, this patch should only fix the problem of the server segfaulting when deleting certain jobs (ones that have not yet been dispatched to the cluster). --nate > > > > > From:Kyle Ellrott > To:Nate Coraor > Cc:"galaxy-dev@lists.bx.psu.edu" > Date:10/01/2013 07:44 PM > Subject:Re: [galaxy-dev] DRMAA runner weirdness > Sent by:galaxy-dev-boun...@lists.bx.psu.edu > > > > I did a merge of galaxy-central that included the patch you posted today. The scheduling problem seems to have gone away. Although I'm still getting back 'Job output not returned from cluster' for errors. This seems odd, as the system previously would output stderr correctly. > > Kyle > > > On Thu, Jan 10, 2013 at 8:30 AM, Nate Coraor wrote: > On Jan 9, 2013, at 12:18 AM, Kyle Ellrott wrote: > > > I'm running a test Galaxy system on a cluster (merged galaxy-dist on Janurary 4th). And I've noticed some odd behavior from the DRMAA job runner. > > I'm running a multithread system, one web server, one job_manager, and three job_handlers. DRMAA is the default job runner (the command for tophat2 is drmaa://-V -l mem_total=7G -pe smp 2/), with SGE 6.2u5 being the engine underneath. > > > > My test involves trying to run three different Tophat2 jobs. The first two seem to start up (and get put on the SGE queue), but the third stays grey, with the job manager listing it in state 'new' with command line 'None'. It doesn't seem to leave this state. Both of the jobs that actually got onto the queue die (reasons unknown, but much to early, probably some tophat/bowtie problem), but one job is listed in error state with stderr as 'Job output not returned from cluster', while the other job (which is no longer in the SGE queue) is still listed as running. > > Hi Kyle, > > It sounds like there are bunch of issues here. Do you have any limits set as to the number of concurrent jobs allowed? If not, you may need to add a bit of debugging information to the manager or handler code to figure out why the 'new' job is not being dispatched for execution. > > For the 'error' job, more information about output collection should be available from the Galaxy server log. If you have general SGE problems this may not be Galaxy's fault. You do need to make sure that the stdout/stderr files are able to be properly copied back to the Galaxy server upon job completion. > > For the 'running' job, make sure you've got 'set_metadata_externally = True' in your Galaxy config. > > --nate > > > > > Any ideas? > > > > > > Kyle > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Enhancing tool id="subtract_query1"
Even for a small change a pull request is preferred since they allow the origin of the change to be tracked. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Can't see history settings, reload etc. after recent update
Do you see the same problem if you access main.g2.bx.psu.edu? What browser are you using? You may need to clear your browser cache. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Mon, Jan 14, 2013 at 11:03 PM, Anthonius deBoer wrote: > Hi, > > I just updated to the latest galaxy-dist, but now my history pane is not > showing the options and no reload etc. > Attached a picture, not sure it will come through... > > Any ideas? Now my galaxy is totally broken...I was hoping not to have to > worry about it as much by switching to the galaxy-dist,from > galaxy-central. > > Hope I can revert the changes > > Thon > > [image: Capture.JPG] > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > <>___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Synchronize Data Library with Directory
I installed a local Galaxy server with Apache proxy, kerberos authentication and submitting to a cluster with DRMAA. Everything works great, so far. Sequence data is NFS mounted on the Galaxy server and all cluster nodes at /shared/ngs/. I would like to configure one Data Library for each lab, add the data from the lab's ngs directory to their library, and synchronize the library and directory. I used "allow_library_path_paste = True" and "Upload files from filesystem paths" to add data to a library from an ngs lab directory. This works great the first time, but if you do it a second time the data is duplicated in the library. 1) Is there a way to synchronize a data library and directory? 2) If so, is there a way to automate the synchronization, perhaps scripted with cron? Thanks for your help! -Erik ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy with Univa Grid Engine (UGE) instead of SGE?
Hello all,
Our local Galaxy server had been running happily under SGE, using
one of the last free releases (not sure exactly which - I could ask).
Due to concerns about long term maintenance, the SysAdmin has
moved us to an SGE compatible setup - Univa Grid Engine (UGE).
However, in at least one respect this is not a drop in replacement,
while other cluster usage appears to be working fine our Galaxy
installation is not, e.g.
galaxy.jobs.runners.drmaa DEBUG 2013-01-15 17:14:33,660 (331:842)
submitting file
/mnt/galaxy/galaxy-central/database/pbs/galaxy_331:842.sh
galaxy.jobs.runners.drmaa DEBUG 2013-01-15 17:14:33,661 (331:842)
command is: /mnt/galaxy/galaxy-central/extract_dataset_parts.sh
/mnt/galaxy/galaxy-central/database/job_working_directory/000/331/task_0;
blastp -query
"/mnt/galaxy/galaxy-central/database/job_working_directory/000/331/task_0/dataset_344.dat"
-db "/var/local/blast/ncbi/nr" -task blastp -evalue 0.001 -out
/mnt/galaxy/galaxy-central/database/job_working_directory/000/331/task_0/dataset_373.dat
-outfmt 5 -num_threads 8
galaxy.jobs.runners.drmaa ERROR 2013-01-15 17:14:33,666 Uncaught
exception queueing job
Traceback (most recent call last):
File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py",
line 146, in run_next
self.queue_job( obj )
File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py",
line 234, in queue_job
job_id = self.ds.runJob(jt)
File
"/mnt/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.6.egg/drmaa/__init__.py",
line 331, in runJob
_h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate)
File "/mnt/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.6.egg/drmaa/helpers.py",
line 213, in c
return f(*(args + (error_buffer, sizeof(error_buffer
File "/mnt/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.6.egg/drmaa/errors.py",
line 90, in error_check
raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value))
DeniedByDrmException: code 17: error: no suitable queues
Debugging this by attempting a manual submission,
$ qsub /mnt/galaxy/galaxy-central/database/pbs/galaxy_331:842.sh
Unable to run job: Colon (':') not allowed in objectname.
Exiting.
Renaming the file to replace the colon with (say) an underscore allows
a manual qsub to work fine with UGE. I've edited Galaxy to avoid the
colons (patch below) but the submission still fails.
Additionally removing the SGE specific settings in universe_wsgi.ini did
allow the job to be submitted I am still having problems. Perhaps I need
to fix all the other filenames too (e.g. stdout, stderr, error code), or do that
in one go by removing the colon in the job name?
Has anyone else tried Galaxy under UGE, and do you have any advice?
Thanks,
Peter
--
Quick filename hack to avoid colons in job script filenames - might
be better to avoid this in the job name itself?
$ hg diff
diff -r 1bfe2768026a lib/galaxy/jobs/runners/drmaa.py
--- a/lib/galaxy/jobs/runners/drmaa.py Mon Jan 14 17:21:25 2013 +
+++ b/lib/galaxy/jobs/runners/drmaa.py Tue Jan 15 18:44:31 2013 +
@@ -191,7 +191,7 @@
job_name = ''.join( map( lambda x: x if x in ( string.letters
+ string.digits + '_' ) else '_', job_name ) )
jt = self.ds.createJobTemplate()
-jt.remoteCommand = "%s/galaxy_%s.sh" %
(self.app.config.cluster_files_directory, job_wrapper.get_id_tag())
+jt.remoteCommand = ("%s/galaxy_%s.sh" %
(self.app.config.cluster_files_directory,
job_wrapper.get_id_tag())).replace(":", "_")
jt.jobName = job_name
jt.outputPath = ":%s" % ofile
jt.errorPath = ":%s" % efile
@@ -229,6 +229,7 @@
log.debug("(%s) submitting file %s" % ( galaxy_id_tag,
jt.remoteCommand ) )
log.debug("(%s) command is: %s" % ( galaxy_id_tag, command_line ) )
+log.debug("(%s) spec: %s" % ( galaxy_id_tag, native_spec))
# runJob will raise if there's a submit problem
if self.external_runJob_script is None:
job_id = self.ds.runJob(jt)
@@ -423,7 +424,7 @@
drm_job_state.ofile = "%s.drmout" % os.path.join(os.getcwd(),
job_wrapper.working_directory, job_wrapper.get_id_tag())
drm_job_state.efile = "%s.drmerr" % os.path.join(os.getcwd(),
job_wrapper.working_directory, job_wrapper.get_id_tag())
drm_job_state.ecfile = "%s.drmec" % os.path.join(os.getcwd(),
job_wrapper.working_directory, job_wrapper.get_id_tag())
-drm_job_state.job_file = "%s/galaxy_%s.sh" %
(self.app.config.cluster_files_directory, job.get_id())
+drm_job_state.job_file = ("%s/galaxy_%s.sh" %
(self.app.config.cluster_files_directory, job.get_id())).replace(":",
"_")
drm_job_state.job_id = str( job_id )
drm_job_state.runner_url = job_wrapper.get_job_runner_url()
job_wrapper.command_line = job.get_command_line()
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Ga
[galaxy-dev] local galaxy tool shed install
Having followed the directions in "HostingALocalToolShed" I ran run_community.sh with the default configurations in community_wsgi.ini. It executed without error but failed to start the tool shed as http://127.0.0.1:9009 returns "Unable to Connect" error Thanks for your help Mark This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] local galaxy tool shed install
Hi Mark, What version of Galaxy are you running, and what does your tool shed paster log show? Greg Von Kuster On Jan 15, 2013, at 5:49 PM, wrote: > Having followed the directions in “HostingALocalToolShed” I ran > run_community.sh with the default configurations in community_wsgi.ini. It > executed without error but failed to start the tool shed as > http://127.0.0.1:9009 returns “Unable to Connect” error > > Thanks for your help > > Mark > > > > This message may contain confidential information. If you are not the > designated recipient, please notify the sender immediately, and delete the > original and any copies. Any use of the message by you is > prohibited.___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Workflows
Hi Sorry if these questions are obvious but I just don't know how to find the answers. I'm trying to get one of the API examples to work in http://wiki.galaxyproject.org/Learn/API/Examples . I've got my API key but how do I get/find the workflow id (f2db41e1fa331b3e in the examples). I've created workflow but don't know how to access this key. Also in the first example it states /api/workflows However I don't have the "api/workflows" directory structure in my galaxy-dist, do I need to create this? Thanks Neil ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] envirtual and import numpy problem
Hi, First it started with the error message: self.check_version_conflict(). So I use the virtualenv as instructed in wiki. But I got a new error: ImportError: No module named rpy/numpy... I will not have this importerror if I dont use the virtualenv. How to solve this problem? Thanks, Huayan ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
