Re: [galaxy-dev] Upload via FTP - Assertion Error on Change ownership of working dictory
Thanks for the tip Eric. I was able to figure it out. The key to this,
was you suggested was permission issue.
I had to modify sudoers so tty was not required. This will allow
scripts to change ownership to the correct user so SGE can submit
them.
Thanks!
-Adam
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
System Administrator, HPC Cluster
Office of Information Technology
http://hpc.oit.uci.edu/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Mon, Feb 3, 2014 at 9:57 AM, Eric Rasche wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Looks like a permission issue. Is your FTP server set to use the same
> user/group as galaxy?
>
> For reference: my proftpd is configured to use LDAP user information,
> but then it forces the user/group id of uploaded files to be that of galaxy
>
> # Force those numbers even if LDAP finds a valid UID/GID
> LDAPDefaultUID {galaxy's UID}
> LDAPDefaultGID {galaxy group GID}
>
>
> Hope that helps
>
>
>
> On 02/03/2014 11:23 AM, Adam Brenner wrote:
>> Anyone have any ideas on this?
>>
>> Thanks,
>> -Adam
>>
>>
>> --
>> Adam Brenner
>> Computer Science, Undergraduate Student
>> Donald Bren School of Information and Computer Sciences
>>
>> System Administrator, HPC Cluster
>> Office of Information Technology
>> http://hpc.oit.uci.edu/
>>
>> University of California, Irvine
>> www.ics.uci.edu/~aebrenne/
>> aebre...@uci.edu
>>
>>
>> On Thu, Jan 30, 2014 at 2:33 PM, Adam Brenner wrote:
>>> Howdy,
>>>
>>> I am having trouble uploading data on our local instance of Galaxy
>>> here. The method is via FTP and here is the error I am seeing in the
>>> logs:
>>>
>>>
>>> galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of
>>> external set_meta for job 1367 finished
>>> galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of
>>> working directory with: /usr/bin/sudo -E
>>> scripts/external_chown_script.py
>>> /som/galaxy/database/job_working_directory/001/1367 root 0
>>> galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change
>>> ownership of /som/galaxy/database/job_working_directory/001/1367,
>>> failing
>>> Traceback (most recent call last):
>>> File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 870, in
>>> finish
>>> self.reclaim_ownership()
>>> File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line
>>> 1316, in reclaim_ownership
>>> self._change_ownership( self.galaxy_system_pwent[0], str(
>>> self.galaxy_system_pwent[3] ) )
>>> File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line
>>> 1302, in _change_ownership
>>> assert p.returncode == 0
>>> AssertionError
>>>
>>>
>>> Any idea on how to fix the issue?
>>> Thanks,
>>> -Adam
>>>
>>> --
>>> Adam Brenner
>>> Computer Science, Undergraduate Student
>>> Donald Bren School of Information and Computer Sciences
>>>
>>> System Administrator, HPC Cluster
>>> Office of Information Technology
>>> http://hpc.oit.uci.edu/
>>>
>>> University of California, Irvine
>>> www.ics.uci.edu/~aebrenne/
>>> aebre...@uci.edu
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client. To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>> http://galaxyproject.org/search/mailinglists/
>>
>
> - --
> Eric Rasche
> Programmer II
> Center for Phage Technology
> Texas A&M University
> College Station, TX 77843
> 404-692-2048
> e...@tamu.edu
> rasche.e...@yandex.ru
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.17 (GNU/Linux)
>
> iQIcBAEBAgAGBQJS79hyAAoJEMqDXdrsMcpV2tIP/3k218ARn3VIvYlhTT0SRGsP
> 8Qgt1w0AgPUSv897MsoS3IQwVqNG4ZJtTPJWgyd+asi0DDPb1lERWSeCXHRr1QSn
> gQmT3Q2sW2tM7sx1OINwMqQ34Hly0bOJiN3/ifOxNaqp3LYjqfWmEvHszQUb3OS1
> mS4JGrrAuZC4z38KohLC93FRLRwA+tOtw6Cw35ViFJZ+MnAWXcalew+WdhnCJmUg
> jqXqWHLk8+d3K/FKkn0xkSL5UK9MMn4d6g/TBV3UeG0vOIhFzY3r3sxQKEjtT5x6
> /p4qeCAgYWZ8woFQn+mSyrLmYCC5U+jKvtlsuu4la/GhNxFTj9ohppi6SDRn5bsf
> I8WDXQS7Hu
Re: [galaxy-dev] Upload via FTP - Assertion Error on Change ownership of working dictory
Anyone have any ideas on this? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences System Administrator, HPC Cluster Office of Information Technology http://hpc.oit.uci.edu/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Jan 30, 2014 at 2:33 PM, Adam Brenner wrote: > Howdy, > > I am having trouble uploading data on our local instance of Galaxy > here. The method is via FTP and here is the error I am seeing in the > logs: > > > galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of > external set_meta for job 1367 finished > galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of > working directory with: /usr/bin/sudo -E > scripts/external_chown_script.py > /som/galaxy/database/job_working_directory/001/1367 root 0 > galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change > ownership of /som/galaxy/database/job_working_directory/001/1367, > failing > Traceback (most recent call last): > File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 870, in > finish > self.reclaim_ownership() > File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line > 1316, in reclaim_ownership > self._change_ownership( self.galaxy_system_pwent[0], str( > self.galaxy_system_pwent[3] ) ) > File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line > 1302, in _change_ownership > assert p.returncode == 0 > AssertionError > > > Any idea on how to fix the issue? > Thanks, > -Adam > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > System Administrator, HPC Cluster > Office of Information Technology > http://hpc.oit.uci.edu/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Upload via FTP - Assertion Error on Change ownership of working dictory
Howdy, I am having trouble uploading data on our local instance of Galaxy here. The method is via FTP and here is the error I am seeing in the logs: galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of external set_meta for job 1367 finished galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /som/galaxy/database/job_working_directory/001/1367 root 0 galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change ownership of /som/galaxy/database/job_working_directory/001/1367, failing Traceback (most recent call last): File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 870, in finish self.reclaim_ownership() File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 1316, in reclaim_ownership self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) ) File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 1302, in _change_ownership assert p.returncode == 0 AssertionError Any idea on how to fix the issue? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences System Administrator, HPC Cluster Office of Information Technology http://hpc.oit.uci.edu/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tardis job splitter
Alan, At first glance this look promising. I am a little leery of tools that claim to do parallel processing. However I would like to test it out on our HPC cluster here at UCI. Few questions: Could you explain how your tool actually does the parallel processing on something that is sequential? For example in your PDF you mention the fastq example, but I do not see the explanation as to how it "splits" up the work across multiple cores/nodes. Does it simply split the sequence string N times and then merges the results? > * our current implementation is quite naive in the distributed compute API > it uses - it supports launching condor job files (and also native > sub-processes) - our plan is to replace that with using the drmaa API We are strictly a SGE (Son of Grid Engine) cluster with a lot of work done by Joseph Farran (check pointing, freeq system, etc). Using DRMAA APIs would be great. If this tool can parallel fastq jobs along with BAM as described, it would be a great improvement for a number of people here. ~Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Oct 28, 2013 at 7:39 PM, McCulloch, Alan wrote: > dear all, > > > > There have been a few posts lately about doing distributed computing via > Galaxy – i.e. > > job splitters etc – below a contribution of some ideas we have developed > > and applied in our work, where we have arranged for some Galaxy tools to > execute in parallel > > on our cluster. > > > > We have developed a job-splitter script "tardis.py" (available from > > https://bitbucket.org/agr-bifo/tardis), which takes marked-up > > standard unix commands that run an application or tool. The mark-up is > > prefixed to the input and output command-line options. Tardis strips off the > > mark-up, and re-writes the commands to refer to split inputs and outputs, > which are then > > executed in parallel e.g. on a distributed compute resource. Tardis knows > > the output files to expect and how to join them back together. > > > > (This was referred to in our GCC2013 talk > > http://wiki.galaxyproject.org/Events/GCC2013/Abstracts#Events.2FGCC2013.2FAbstracts.2FTalks.A_layered_genotyping-by-sequencing_pipeline_using_Galaxy > ) > > > > Any reasonable unix based data processing or analysis command may be marked > up and run > > using tardis, though of course tardis needs to know how to split and join > the data. Our approach > > also assumes a “symmetrical” HPC cluster configuration, in the sense that > each node sees the same > > view of the file system (and has the required underlying application > installed). We use tardis > > to support both Galaxy and command-line based compute. > > > > Background / design pattern / motivating analogy: Galaxy provides a high > level > > "end to end" view of a workflow; the HPC cluster resource that one uses then > involves > > spraying chunks of data out into parallel processes, usually in the form of > some kind of > > distributed compute cluster - but an end-user looking at a Galaxy history, > should ideally not be able > > to tell whether the workflow was run as a single process on the server, or > > via many parallel processes on the cluster (apart from the fact that when > run > > in parallel on the cluster, its alot faster!). We noticed that the TCP / IP > layered networking > > protocol stack provides a useful metaphor and design pattern - with the > "end to end" topology > > of a Galaxy workflow corresponding to the transport layer of TCP/ IP; and > the distribution > > of computation across a cluster corresponding to the next TCP/IP layer down > - the packet-routing > > layer. > > > > This picture suggested a strongly layered approach to provisioning > > Galaxy with parallelised compute on split data, and hence to an approach in > which the > > footprint in the Galaxy code-base, of parallel / distributed compute > support, should ideally > > (from the layered-design point of view) be minimal and superficial. Thus in > our approach so far, > > the only footprint is in the tool config files, where we arrange the > templating to > > (optionally) prefix the required tardis mark-up to the input and output > command options, and > > the tardis script name to the command as a whole. tardis then takes care of > rewriting and > > launching all of the jobs, and finally joining the results back together and > putting them wh
Re: [galaxy-dev] Problem With Galaxy
Is this a local galaxy setup that you host ... or the public one? It sounds like a cache issue on your end (clear your browsers cache). If this is a local setup that you host, perhaps you are not loading CSS/javascript? Are those being blocked? Are you running a reverse proxy? Have the redirects been setup to handle static content? To many unknowns ~Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Oct 28, 2013 at 3:59 PM, Caitlin Grube wrote: > I am having an issue trying to do anything. I click on a tool, the screen > pops up in the middle but then when I move my mouse over to that screen, the > drop down menus all pull down in a large black screen so that I cannot get > to anything. The screen also does not go away if I try to click away from > it, but only if I choose one of the options from the list. I would click > analyze data which would take back to where I started. This happens for > every tool that I try. I did not have this problem this morning or > afternoon, but I've been trying to get it to work for almost an hour. As > soon as my mouse hits that center area (top, bottom, middle), the screen > comes down, but only after I select a tool. Please help. > > Caitlin Grube > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy Install - Debug Report FROM address as Admin and not User?
Martin, Thanks for the response. Yes, I do notice the reply-to address does include the user, but wouldn't it be better if the FROM address was from the actual user reporting the error? When checking my email, for example, I do not want to see emails that are FROM me, to MYSELF when its really from our users. Do you know which of the python files contain the code that relates to this? Or point me in the right direction so I can change this? Thanks! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Oct 1, 2013 at 10:52 AM, Martin Čech wrote: > Hi Adam, > > you are right about the behaviour, this is what Galaxy does. However it > sends the email to the user too, so "Reply all" will reach the user's email > address. This is how the feature was meant to be used. > > Martin > > > > On Tue, Oct 1, 2013 at 12:59 PM, Adam Brenner wrote: >> >> Howdy, >> >> On our local Galaxy install, everytime a user submits a bug report, it >> does email all the admins -- as defined in universe_wsgi.ini -- but >> the FROM address is not of the user, but rather the first email >> address from the admin list. >> >> Is this normal? May we have it changed so the FROM address is that of the >> user? >> >> Thanks, >> -Adam >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of Information Technology >> http://www.oit.uci.edu/rcs/ >> >> University of California, Irvine >> www.ics.uci.edu/~aebrenne/ >> aebre...@uci.edu >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Local Galaxy Install - Debug Report FROM address as Admin and not User?
Howdy, On our local Galaxy install, everytime a user submits a bug report, it does email all the admins -- as defined in universe_wsgi.ini -- but the FROM address is not of the user, but rather the first email address from the admin list. Is this normal? May we have it changed so the FROM address is that of the user? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Need your helps about Galaxy
You have to determine if the old version of galaxy ( galaxylocal.genenetwork.org:8080/) and the new one, if any database changes have occurred between them -- most likely. If not, and you can check, by looking at the hg history changes via the bitbucket website: https://bitbucket.org/galaxy/galaxy-central/src Assuming database has changed, you can upgrade your old galaxy: http://wiki.galaxyproject.org/Admin/Get%20Galaxy#Keep_your_code_up_to_datestart up the instance, upgrade the DB and then dump the database. Perhaps you can get away without upgrading your old galaxy instance by only importing a few tables (such as users) into the new galaxy DB..but I am not sure. Perhaps someone more familiar with the Galaxy DB can comment on this. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Sep 23, 2013 at 12:50 PM, Lei Yan wrote: > Dear James, > > Thanks so much for your info. > Dumping database is a good idea. But the problem is that the two Galaxy > are running on different servers, and the Galaxy versions are very > different. > Thanks again. > > > Lei Yan > Center for Integrative and Translational Genomics > UTHSC > > > On Mon, Sep 23, 2013 at 2:37 PM, James Taylor wrote: > >> There is not a simple way to do this. You will need to dump the >> accounts from your old database and load into the new database. Are >> they running the same database server? >> >> Please send questions like this to the galaxy-dev mailing list. >> >> -- >> James Taylor, Associate Professor, Biology/CS, Emory University >> >> >> On Mon, Sep 23, 2013 at 3:31 PM, Lei Yan wrote: >> > Dear James, >> > >> > I built a new cloud Galaxy instance today. >> > Would you please tell me how to transfer some given accounts from an old >> > Galaxy to this new one? >> > This is old Galaxy: http://galaxylocal.genenetwork.org:8080/ >> > This is new Galaxy: http://galaxyclass.genenetwork.org/ >> > Thanks a lot. >> > >> > >> > Lei Yan >> > Center for Integrative and Translational Genomics >> > UTHSC >> > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Local Galaxy Install - Multiple Input Files for Workflow Missing?
Howdy, On our local galaxy cluster, I have gotten a report that the multiple input files selection on workflows is missing. How do I enable it? Looking at the past mailing list[1] this is the item that is missing from our setup -- the tooltip icon. We are running a version of galaxy that is roughly ~one month old: c42567f43aa7. [1]: http://dev.list.galaxyproject.org/Looking-for-recommendations-How-to-run-galaxy-workflows-in-batch-td4362836.html#a4362874 Thanks! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Limit number of Jobs to Trinity
Carlos, Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya wrote: > Hi guys, > > I have a running local galaxy installation on one of our servers. I've > just installed Trinity for de novo assembly of transcripts and it will use > up to 32 processors. However i would like to limit the number of concurrent > jobs only for this tool. Do you know if it would be possible to limit the > number of concurrent jobs per tool basis? > > Bests, > > Carlos > > -- > Carlos Canchaya > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
> Setting the environment for each tool just prior to running the tool is > clearly better. > > What I'm doing currently is loading an environment module just before running > a tool in Galaxy. On our HPC cluster, I have configured Galaxy to source the following from our modules system: 1) Global file as defined in universe_wsgi.ini environment_setup_file=setup-modules-for-galaxy.sh ^^ This setups the module environment so commands like "module load " work. It does something similar to: source /data/apps/modules/Modules/default/init/bash 2>&1 &> /dev/null 2) Configured tool_dependency_dir as described here: http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies tool_dependency_dir=tool_dependency For each requirement as specified the xml, galaxy will automatically look in tool_dependency//default/env.sh Each dependancy's env.sh then use our favorite command: module load blah/version Cumbersome? Yup! But I believe its a nice elegant way of getting thing to work with no mucking around in the xml file. > This will enable my native EMBOSS wrapper to run the defined version of the > (system-installed) EMBOSS suite. > However, it would only be worth me writing this extension if there's a > likelihood of getting it into Galaxy. Is there? I truly like the idea and think it would. However, stepping into the Galaxy devs shoes, a solution like this might only be best for those using modules (small subset of users). Hopefully others can give their thoughts. The con I see in this is that Galaxy already has the env.sh solution which in my setup described above, does support modules > In the meantime, there's work going on in Galaxy land to package every > application wanted in Galaxy, > by writing install scripts, encapsulating environment variables in tool XML > files, etc. (There is difficulty > here in writing install scripts which work on every platform, which I have > commented on previously.) I saw that post last week and while I like the idea, folks like me who really do care about the version of GCC we use (and compile our own versions of gcc) will not use it. However, that is going off topic now :-) Will wait and see what others say... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Sep 12, 2013 at 4:03 PM, Guest, Simon wrote: >> > Another vote for the excellent modules system from us at AgResearch >> >> Galaxy's dependency injection works in a similar way, without requiring >> all the external dependencies. We don't need unload since we compose a >> unique environment for every tool execution. This is much cleaner that >> loading everything into the environment before running Galaxy, we want >> tools to be executed with as clean an environment as possible. > > Setting the environment for each tool just prior to running the tool is > clearly better. > > What I'm doing currently is loading an environment module just before running > a tool in Galaxy. So far, I have done this in the tool XML file by prefixing > the command with a module load command, e.g. > > . /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence > [snipped] > > The ugly ". /etc/profile" is needed for technical reasons. I don't like > this, as it's a bit hacky, but what it achieves is useful and flexible. > > I envisage an extension to the tool XML syntax which would let me write it > like this, say: > >version="5.0.0">EMBOSS > infoseq -sequence [snipped] > > This will enable my native EMBOSS wrapper to run the defined version of the > (system-installed) EMBOSS suite. However, it would only be worth me writing > this extension if there's a likelihood of getting it into Galaxy. Is there? > > Here's where I'm coming from. I already have most of the applications I want > installed on my system. I am working on updating the RPMs to support > side-by-side installs of multiple versions, with version selection by the > environment modules facility. This is a small change (and improvement) on > what I have today. It's a work in progress, but it's a manageable amount of > work. It's looking likely that this multi-version packaging approach may be > adopted by an official CentOS Scientific Repo, and this could change the > landscape in terms of packaging of scientific applications for major Linux > distributions. Who knows, that's all in the future. > > In the meantime, there's work going on in Galaxy
[galaxy-dev] Creating Galaxy Admin / Local Install Mailing List - Updates?
Howdy, I know this has been discussed before, but I can not find the email threads on this...but what was the status on creating a galaxy admin / local install mailing list? The Devel list is very chatty and while my filters do a good job, I think it will be easier for others to find help on their own if it was on a separate mailing list It would be easier to search and look back at previous threads. However, the added benefit of having the devel list is that we get feedback from Galaxy Developers As the saying godes: Just food for thought, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Stable Galaxy for local install
> What I want to do in a hurry is to fix the problem of "Select" in "Filter > and Sort" .It is a step very important for me to Sort SAM file for > Cufflinks.Thank you very much for all of your help. Per my last email:. $ hg clone https://bitbucket.org/galaxy/galaxy-dist/ $ cd galaxy-dist $ hg update stable If you are running into issues, can you be more specific...provide error message? Steps on how to reproduce your issue? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 6:58 PM, cngbgal...@gmail.com wrote: > Hi all, > What I want to do in a hurry is to fix the problem of "Select" in "Filter > and Sort" .It is a step very important for me to Sort SAM file for > Cufflinks.Thank you very much for all of your help. > > ____ > cngbgalaxy > > From: Adam Brenner > Date: 2013-09-12 00:41 > To: cngbgalaxy > CC: joachim.jacob; galaxy-dev@lists.bx.psu.edu; galaxy-...@bx.psu.edu > Subject: Re: [galaxy-dev] Stable Galaxy for local install > Galaxy development occurs in two repositories, galaxy-central and > galaxy-dist. Changes are stabilized in galaxy-central and then pushed > to galaxy-dist roughly once a month.[1] > > So > > $ hg clone https://bitbucket.org/galaxy/galaxy-dist/ > $ cd galaxy-dist > $ hg update stable > > [1]: http://wiki.galaxyproject.org/Admin/Get%20Galaxy > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Wed, Sep 11, 2013 at 8:42 AM, cngbgalaxy wrote: >> Hi Joachim, >> Thank you very much for your help. >> Actually I find some problems about such versions of Galaxy in >> https://bitbucket.org/galaxy/galaxy-dist/downloads . >> We can not view the data in history by clicking the eyes icon on the >> c42567f43aa7 version.The "Select" in "Filter and Sort" fail to work on the >> 1ae95b3aa98d (release_2013.08.12 ) version and the 524f246ca853 >> (release_2013.06.03 ),though I don't know whether it is a bug or not. >> As a result,I feel confused about which version of Galaxy to install and >> avoid the above problems. >> >> >> cngbgalaxy >> >> From: Joachim Jacob | VIB | >> Date: 2013-09-11 22:38 >> To: test galaxy >> CC: galaxy-dev; galaxy-dev >> Subject: Re: [galaxy-dev] Stable Galaxy for local install >> The latest code from the 'stable' branch of Galaxy is preferred. See the >> news brief accompanying the update (release_2013.08.12) on >> http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12 >> >> You need mercurial (http://mercurial.selenic.com/) for installing >> Galaxy. Report bugs at https://trello.com/b/75c1kASa/galaxy-development >> >> Cheers >> Joachim >> >> Joachim Jacob >> Contact details: http://www.bits.vib.be/index.php/about/80-team >> >> >> On 09/11/2013 03:55 AM, test galaxy wrote: >>> Hi everyone, >>> I want to install my local Galaxy with less bugs,and I don't know >>> which version is more stable and work well.Could anyone give same >>> advise?Thank you very much. >>> >>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>>http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>>http://galaxyproject.org/search/mailinglists/ >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Stable Galaxy for local install
Galaxy development occurs in two repositories, galaxy-central and galaxy-dist. Changes are stabilized in galaxy-central and then pushed to galaxy-dist roughly once a month.[1] So $ hg clone https://bitbucket.org/galaxy/galaxy-dist/ $ cd galaxy-dist $ hg update stable [1]: http://wiki.galaxyproject.org/Admin/Get%20Galaxy -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 8:42 AM, cngbgalaxy wrote: > Hi Joachim, > Thank you very much for your help. > Actually I find some problems about such versions of Galaxy in > https://bitbucket.org/galaxy/galaxy-dist/downloads . > We can not view the data in history by clicking the eyes icon on the > c42567f43aa7 version.The "Select" in "Filter and Sort" fail to work on the > 1ae95b3aa98d (release_2013.08.12 ) version and the 524f246ca853 > (release_2013.06.03 ),though I don't know whether it is a bug or not. > As a result,I feel confused about which version of Galaxy to install and > avoid the above problems. > > > cngbgalaxy > > From: Joachim Jacob | VIB | > Date: 2013-09-11 22:38 > To: test galaxy > CC: galaxy-dev; galaxy-dev > Subject: Re: [galaxy-dev] Stable Galaxy for local install > The latest code from the 'stable' branch of Galaxy is preferred. See the > news brief accompanying the update (release_2013.08.12) on > http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12 > > You need mercurial (http://mercurial.selenic.com/) for installing > Galaxy. Report bugs at https://trello.com/b/75c1kASa/galaxy-development > > Cheers > Joachim > > Joachim Jacob > Contact details: http://www.bits.vib.be/index.php/about/80-team > > > On 09/11/2013 03:55 AM, test galaxy wrote: >> Hi everyone, >> I want to install my local Galaxy with less bugs,and I don't know >> which version is more stable and work well.Could anyone give same >> advise?Thank you very much. >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >>http://galaxyproject.org/search/mailinglists/ > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] PATH and local jobs
Ryan, On our HPC setup, we use the modules system but its the same issue you are running into. A number of methods exists for setting up the correct path, for you, I believe the easiest would be one of the two: 1) In your init script: source (or setup) PATHS as you have them in your .bash_profile. /etc/init.d/galaxy OR 2) Within Galaxy, edit universe_wsgi.ini and look for: environment_setup_file and create a file that will setup the correct $PATH for you. You can also invoke the Tool Dependency options for each specific tool you load as shown in[1]. [1]: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 10:18 AM, Ryan Davis wrote: > Hi, > > ** ** > > We have a local server running Galaxy and a majority of the tools work > correctly (bowtie, BWA, samtools and etc…). I am having issues with some > of the FASTX-Toolkit tools. I am starting Galaxy with an init script on > Centos 6 and the tools are in my $PATH and work at the command line, just > not in Galaxy. The wiki suggest to “set $PATH in the startup file or > use.” > > It leaves me hanging and don’t know what the other option is. Can someone > share with me their workaround? > > ** ** > > Here is the wiki page: > http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies > > ** ** > > Here is the section I am hoping will help: > > ** ** > Local Jobs > > Changes to $PATH can be persisted by setting them in your shell's startup > file(s). This typically means ~/.bash_profile for bash, but please see > the bash documentation on startup > files<http://www.gnu.org/software/bash/manual/bashref.html#Bash-Startup-Files> > or the *INVOCATION* section of the bash(1) man page to understand the > intricacies of how that file is read. Of particular importance, if you are > starting Galaxy in a method other than manual invocation from a shell > prompt (with sh run.sh) such as with an init script, it is likely that > your startup file will not be read. In this instance, you should set > $PATH in the startup file or use. > > ** ** > > ** ** > > Thanks! > > Ryan > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ZFS storage recommendations
> optimize performance? The purpose is to share the database over NFS to the > Galaxy VM. Are you moving the PostgreSQL/MySQL database over to ZFS or the actual storage of your datasets over to ZFS? I am assuming the latter. Remember, ZFS is just a file system, you still need a protocol, like NFS, to export the data to each of your machines. This is going to be your bottleneck, luckily the NFS clients supports write caching as described in our test here[1]. On our HPC cluster, we run xfs filesystem on top of Gluster and we have another filesystem using xfs on top of FraunhoferFS. I did test out ZFS on Linux roughly a year ago and in terms of Read/Write on a single machine it was slower than native ext4 and XFS. This of course was to be expected. However the added benefit of ZFS may be more favorable in your case, Snapshots, disk management, ZIL/SSD caching, etc. If you use a distributed filesystem like Gluster, FraunhoferFS, or even Lustre (2.x branch supports ZFS!) you will most certainly get some very good read/write speeds. However, it sounds like you are using a single machine, so your read/write is going to be slower than native ext4 and XFS --- trust but verify, run your own read/write tests. When I was using ZFS, this ZFSBuild[2] website was most helpful. Let me know if you have any other questions, -Adam [1]: http://www.gluster.org/pipermail/gluster-users/2012-September/034295.html [2]: http://www.zfsbuild.com/ -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Sep 10, 2013 at 3:29 AM, Joachim Jacob | VIB | wrote: > Hi all, > > I am performing some tests to move my galaxy database to ZFS. Does anybody > have experience with ZFS on linux, and some recommendations/experiences to > optimize performance? The purpose is to share the database over NFS to the > Galaxy VM. > > > Thanks, > Joachim. > > -- > Joachim Jacob > Contact details: http://www.bits.vib.be/index.php/about/80-team > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] questions on Galaxy installation under Apache
Jim, You need to enable Apache's proxy service. See my pervious post here: http://dev.list.galaxyproject.org/Running-Galaxy-through-Apache-td4659452.html Let us know if you run into further issues, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Sep 9, 2013 at 12:04 PM, Jingchao Zhang wrote: > Hello everyone, > > I installed both Galaxy and Apache in a Red Hat linux machine and they work > fine individually. If I open http://localhost then a welcome page comes out > showing the httpd is working. In galaxy I changed host=0.0.0.0 and "sh > run.sh", then the http://localhost:8080 will direct me to the Galaxy page. > > Now I want to put Galaxy under Apache and here is what I did: > 1. First I comment the "host=0.0.0.0" line to stop the build in web server > in Galaxy and then "sh run.sh" to make Galaxy running. > 2. Then I put these lines > RewriteEngine On > RewriteRule ^/galaxy$ /galaxy/ [R] > RewriteRule ^/static/style/(.*) > /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] > RewriteRule ^/static/scripts/(.*) > /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] > RewriteRule ^/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] > RewriteRule ^/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico > [L] > RewriteRule ^/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] > RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] > into my "httpd.conf" file. Then I restart the httpd service. > > Also, I made changes to the "universe_wsgi.ini" file as instructed and > restart Galaxy: > > [filter:proxy-prefix] > use = egg:PasteDeploy#prefix > prefix = /galaxy > > [app:main] > filter-with = proxy-prefix > cookie_path = /galaxy > > After I do these, either http://localhost:8080 nor http://localhost/galaxy > works. I find two threads talked about this problem before: > http://user.list.galaxyproject.org/Installing-galaxy-with-Apache-td4654499.html > http://dev.list.galaxyproject.org/Running-Galaxy-behind-apache2-td4624545.html > but my Red Hat httpd does not has this > "/etc/apache2/sites-available/default" directory and only has > "/etc/httpd/conf/httpd.conf". > > Could someone please tell me where should I put those lines in the > "httpd.conf" file? I have tries various places and none of them works. Any > help would be deeply appreciated. > > Thanks, > Jim > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Typo in Wiki Page - Purge History and Datasets
http://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets > There are 5 scripts included in the Galaxy distribution that > can be used to clean up unwanted histories, libraries and > datasets. There are located in the > <<>>/scripts/cleanup_datasets > directory and are named: Should be changed to 6 scripts Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] URL problem
Alper, Nginx or Apache? Did you modify universe_wsgi.ini and add the proxy settings? Does the information on http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Using_a_proxy_serverhelp you? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, Aug 30, 2013 at 12:48 PM, Jennifer Jackson wrote: > Hi Alper, > There may be permissions problems if just for these two links, but I am > not an expert on this. Moving post over to the galaxy-dev list, which will > give it better exposure to the development team and community. You'll want > to post here directly with any future local install questions. > http://wiki.galaxyproject.org/**Support#Mailing_Lists<http://wiki.galaxyproject.org/Support#Mailing_Lists> > Best, > Jen > Galaxy team > > > On 8/26/13 10:49 AM, Kucukural, Alper wrote: > >> Hi, >> I have a problem in galaxy to get host/domain name in two different pages. >> First one is in the tool installation from toolshed, I got the error >> below, >> >> # >> Not Found >> >> The requested URL /admin_toolshed/prepare_for_**install was not found on >> this server. >> >> # >> >> The second one is in the saved histories. When I click the buttons of the >> saved histories. I got the similar error like below. >> # >> Not Found >> >> The requested URL /history/list was not found on this server. >> >> # >> >> I haven't seen these any other pages yet. >> >> My installation is working on LDAP authentication with Proxy. So, I could >> not find a place to set the domain or host name in these two places that >> they can actually find the requested URLs. >> >> In the paster.log file. I don't get any error when I install a tool or >> go to another history. It doesn't report any error. >> >> Thanks for your help, >> >> Alper Kucukural >> __**_ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at usegalaxy.org. Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> >> http://lists.bx.psu.edu/**listinfo/galaxy-dev<http://lists.bx.psu.edu/listinfo/galaxy-dev> >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> >> >> http://galaxyproject.org/**search/mailinglists/<http://galaxyproject.org/search/mailinglists/> >> > > -- > Jennifer Hillman-Jackson > http://galaxyproject.org > > __**_ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/**search/mailinglists/<http://galaxyproject.org/search/mailinglists/> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Instance and current website are not the same
Hamid, Read the information here: http://snpeff.sourceforge.net/SnpEff_manual.html#galaxy -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Aug 28, 2013 at 2:16 PM, Tevfik Hamdi Kitapci wrote: > Hi, > I installed a local instance of galaxy browser on my local machine using the > steps in: > > http://wiki.galaxyproject.org/Admin/Get%20Galaxy > > I tried both the default (hg update default) and stable versions (hg update > stable) > > but the current website has different features on the left panel then the > local instance that I downloaded. Specifically I want to use the SnpEff > feature under the NGS toolbox Beta on the left panel of the website. But I > can't find this feature on the local instance is there anyway to install a > local instance of the current version of the website? > > Thanks > Best Regards > Hamdi Kitapci > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Login hook
I am not aware of Galaxy having any sort of login hook as you described --- without writing custom code. This brings up the added challenge of keeping your custom modifications current with the latest release of galaxy. Another approach, and I am not sure how feasible this is on your setup, is to create all these added services during the user creation[1] process. On our HPC cluster this is the method we implemented. All our users can use SFTP, rsync, BBCP, etc to upload to the cluster and the user moves their data to galaxy by the "FTP Trick[2]" to their "dropbox" folder that galaxy has access to. Once data is moved to the dropbox folder, users login via the galaxy web interface and select the data via the Get Data -> Upload File interface. This might be a simpler approach, create these services for each new user during their account creation. You will be forced to use the command line interface to create new users, but I believe its far simpler then using postgres notify or the other methods you described. [1]: http://dev.list.galaxyproject.org/user-creation-using-API-td4657726.html [2]: http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Aug 25, 2013 at 9:39 AM, Maddhi, Srinivas wrote: > Hello, > > Does Galaxy provide a post-login hook that could be used to trigger an > operating system level action upon successful login ? > > Here is the scenario and use-case where that would come in handy: > * Local instance of Galaxy (Nov 2012 release) on a Linux server with a > PostgreSQL DB > * Upload via FTP feature is enabled > * Users are authenticated via LDAP and nginx > * Users should be able to securely upload files (to their sub-directory > within the configured FTP upload directory) > > The current method being employed to fulfill this use-case is through a > modification to Š/galaxy-dist/lib/galaxy/web/framework/__init__.py file > that calls a a shell script which creates an OS-level user and associated > FTP upload directory. Files are uploaded by users via sFTP (SSHD > configured to restrict Galaxy users to a chroot'ed directory). > > Options considered: > * Shell/Python script that periodically queries the "galaxy_user" table > and creates an OS-level user and FTP upload directory (not yet tried; > introduces delay between user creation and FTP capability unless frequency > of polling is high) > * ProFTPD with sFTP and LDAP modules (increased administrative overhead > compared to using built-in SSHD; wasn't able to get auto directory > creation to work due to TBD cause) > * PostgreSQL listen-notify to call a sFTP user creation script when an > insert occurs in the "galaxy_user" table (not yet tried) > > > Thank you. > > -- > Senior Systems Administrator > Iowa Institute for Human Genetics > ITS Research Services, The University of Iowa > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Help with installation
The issue here is the last line:
OperationalError: (OperationalError) disk I/O error u'PRAGMA
table_info("dataset")' ()
Do you have any security rules setup on the server, quotes, disk
issues, etc? You can always run a strace on run.sh for a better
understanding of whats going on.
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Thu, Aug 8, 2013 at 4:12 AM, Jurgens de Bruin wrote:
> To all,
>
> I am getting the following error when I run sh run.sh :
>
> galaxy.model.migrate.check DEBUG 2013-08-08 07:52:39,479 pysqlite>=2 egg
> successfully loaded for sqlite dialect
> Traceback (most recent call last):
> File
> "/scratch/sysusers/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py",
> line 35, in app_factory
> app = UniverseApplication( global_conf = global_conf, **kwargs )
> File "/scratch/sysusers/galaxy/galaxy-dist/lib/galaxy/app.py", line 52, in
> __init__
> create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get(
> '__file__', None ), self.config.database_engine_options, app=self )
> File
> "/scratch/sysusers/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py",
> line 51, in create_or_verify_database
> dataset_table = Table( "dataset", meta, autoload=True )
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py",
> line 318, in __new__
> table._init(name, metadata, *args, **kw)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py",
> line 381, in _init
> self._autoload(metadata, autoload_with, include_columns)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py",
> line 408, in _autoload
> self, include_columns, exclude_columns
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 2425, in run_callable
> return conn.run_callable(callable_, *args, **kwargs)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1968, in run_callable
> return callable_(self, *args, **kwargs)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py",
> line 260, in reflecttable
> return insp.reflecttable(table, include_columns, exclude_columns)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py",
> line 370, in reflecttable
> for col_d in self.get_columns(table_name, schema, **tblkw):
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py",
> line 223, in get_columns
> **kw)
> File "", line 1, in
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py",
> line 47, in cache
> ret = fn(self, con, *args, **kw)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py",
> line 713, in get_columns
> (pragma, qtable)))
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1449, in execute
> params)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1628, in _execute_text
> statement, parameters
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1698, in _execute_context
> context)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
> line 1691, in _execute_context
> context)
> File
> "/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py",
> line 331, in do_execute
> cursor.execute(statement, parameters)
> OperationalError: (OperationalError) disk I/O error u'PRAGMA
> table_info("dataset")'
Re: [galaxy-dev] Dynamically decide where a Galaxy tool runs
> This is something our research group is interested in, but it's not possible > to specify which cluster the particular tool should run on ahead of time. Incorrect (at least to my knowledge). Using the new way jobs are handled within Galaxy via the XML[1] files you can send specific jobs to specific destinations which could be different "clusters." For our cluster, we handle the file uploading "tool" to a different job runner then the cluster (SGE). This was accomplished by where id is the galaxy tool id and destination is where the tool should be ran, cluster, local, etc. [1] http://wiki.galaxyproject.org/Admin/Config/Jobs -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, Jul 12, 2013 at 11:42 AM, Saliya Ekanayake wrote: > Hi, > > I see in documentation that Galaxy tools may be configured to run on a > cluster (http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster). > This is something our research group is interested in, but it's not possible > to specify which cluster the particular tool should run on ahead of time. > > Therefore, is it possible to specify the cluster to run when the tool is > actually being used when composing the workflow from Galaxy? > > Thank you, > Saliya > > -- > Saliya Ekanayake esal...@gmail.com > Cell 812-391-4914 Home 812-961-6383 > http://saliya.org > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Build the Galaxy under Cluster
May you make your question more specific? Have you read the online docs: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Jul 7, 2013 at 1:35 AM, shenwiyn wrote: > Hi everyone, > I want to install my local Galaxy under the Cluster with TORQUE.I have a > question about how they work together successfully.In addition,after my > local Galaxy and TORQUE install successfully,anything else should I do to > the Cluster Galaxy?Thank you very much. > > > shenwiyn > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Startup problem after upgrade
When you updated, did you download all the files? Was their any conflicts during the update? It appears you are missing a file. Cant locate template for uri 'root/index.mako' Run 'hg update'... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Jul 2, 2013 at 8:10 AM, Bertrand PITOLLAT wrote: > Hi, > > > > I have just upgraded my galaxy instance. > > > > When trying to access my galaxy instance home page, I get the following > error : > > > > 195.221.174.40 - - [02/Jul/2013:15:34:23 +0200] "GET / HTTP/1.1" 500 - "-" > "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) > Chrome/27.0.1453.116 Safari/537.36" > > Error - : Cant locate > template for uri 'root/index.mako' > > URL: http://marmadais.cirad.fr:7090/ > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/middleware/error.py', > line 149 in __call__ > > app_iter = self.application(environ, sr_checker) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', > line 84 in __call__ > > return self.application(environ, start_response) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', > line 633 in __call__ > > return self.application(environ, start_response) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/base.py', > line 132 in __call__ > > return self.handle_request( environ, start_response ) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/base.py', > line 190 in handle_request > > body = method( trans, **kwargs ) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/webapps/galaxy/controllers/root.py', > line 39 in index > > params=kwd ) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/__init__.py', > line 957 in fill_template > > return self.fill_template_mako( filename, **kwargs ) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/__init__.py', > line 964 in fill_template_mako > > template = self.webapp.mako_template_lookup.get_template( filename ) > > File > '/SATA/bioinfo/galaxy_dev/galaxy_dist/eggs/Mako-0.4.1-py2.6.egg/mako/lookup.py', > line 221 in get_template > > "Cant locate template for uri %r" % uri) > > TopLevelLookupException: Cant locate template for uri 'root/index.mako' > > > > Any help welcome. > > > > Bertrand. > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Dependency Check for eXpress - requirements syntax incorrect?
James, Thanks for the response. I had some sort of typo in my express_wrapper.xml file and all is well now. It is now sourcing the correct file. I went ahead and submitted a pull request to have eXpress updated to package, instead of binary: https://bitbucket.org/galaxy/galaxy-central/pull-request/192/express-tool-requirements-tag-updated/diff -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Jun 27, 2013 at 11:03 AM, James Taylor wrote: > Hi Adam, yes, it should be package, the "binary" type has never been > supported but keeps getting propagated for some reason. At some point > it might be nice to have a configurable mapping from the other > dependency types to packages. > > However, changing it to package *should* work. Can you send the > relevant part of the paster log, it should log information about > dependency resolution and the command line. > > -- > James Taylor, Assistant Professor, Biology/CS, Emory University > > > On Thu, Jun 27, 2013 at 1:38 PM, Adam Brenner wrote: >> Howdy, >> >> One of our users is running the eXpress package under NGS: RNA >> Analysis -> eXpress -- galaxy/dist/tools/ngs_rna/express_wrapper.* >> >> I noticed the syntax for express_wrapper.xml has the following: >> eXpress >> any reason why it was set to that, as opposed to: >> eXpress >> >> In our setup, we need to source the env.sh file for express to setup >> the correct environmental variables. Changing from binary to package >> has not solved this and re-running the job never sources our env.sh >> file. >> >> What am I doing wrong? >> >> My tool dependency directory looks like so: >>galaxy/dist/tool_dependency/eXpress/ >>|---default --> 1.3.0 >>|1.3.0/env.sh >> >> Thanks, >> -Adam >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of Information Technology >> http://www.oit.uci.edu/rcs/ >> >> University of California, Irvine >> www.ics.uci.edu/~aebrenne/ >> aebre...@uci.edu >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Tool Dependency Check for eXpress - requirements syntax incorrect?
Howdy, One of our users is running the eXpress package under NGS: RNA Analysis -> eXpress -- galaxy/dist/tools/ngs_rna/express_wrapper.* I noticed the syntax for express_wrapper.xml has the following: eXpress any reason why it was set to that, as opposed to: eXpress In our setup, we need to source the env.sh file for express to setup the correct environmental variables. Changing from binary to package has not solved this and re-running the job never sources our env.sh file. What am I doing wrong? My tool dependency directory looks like so: galaxy/dist/tool_dependency/eXpress/ |---default --> 1.3.0 |1.3.0/env.sh Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] DRMAA Runner URL Specify TORQUE host
Bart, I believe drmaa://-q somehost@queue-name will work. However I could be very wrong. It has been a while since I messed with the actual drmaa runners. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Jun 26, 2013 at 1:28 PM, Bart Gottschalk wrote: > Is it possible to specify the torque host as part of a DRMAA runner URL? I > haven't been able to find a native_options parameter to allow for this. I'm > using the old style cluster configuration. > > drmaa://[native_options]/ > > Also, I haven't been able to find a list of native_options anywhere. Does > such a list exist? If so, where? > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Bowtie2 Specifying -k value (Alignment Search)
One of our users has requested to see if we could add specifying an alignment search option to Bowtie2 (-k and -a option http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#default-mode-search-for-multiple-alignments-report-the-best-one ) Is this something anyone else has added to their galaxy installs? If so, how? I was looking at galaxy/dist/tools/sr_mapping/bowtie2_wrapper.* and it does not seem to challenging to add another field where a user could specify some integer value that we simply pass to the the tag in the xml file. Would this be the correct way to go about this? Thanks in advanced! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bypassing the proxy server when using the API, without breaking the automatic update of the history in the web interface
Why not use Allow All / Deny All rules? ... your Auth Type stuff here ... Deny from all Allow from 127.0.0.1 # Assuming API calls are made locally Satisfy any -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Jun 5, 2013 at 5:15 PM, Anthonius deBoer wrote: > Hi, > > I am trying desperately to setup my GALAXY system so that if you enter the > website, you need to authenticate yourself using the proxy setup... > I also would like to be able to BYPASS the proxy server when an API call is > made... > > In the example proxy setting I have setup in the following way: > > ReWriteRule ^(/api/histories/.*) http://localhost:8080$1 [P,L] > ReWriteRule ^(/api/.*) http://localhost:8081$1 [P,L] > RewriteRule ^(.*) http://localhost:8080$1 [P] > > I then setup a /etc/httpd/conf.d/galaxy.d to require authentication when a user is > re-directed to the localhost:8080 and allow the calls to localhost:8081 to > pass unhindered... > > This works brilliantly EXCEPT for when you want to make an API call that > involves some histories > The call fails, since /api/histories calls are routed through the proxy > server and require authentication, which I want to avoid... > > It seems that histories are "special" so they have their own line in the > configuration file above... > > I tried to change > ReWriteRule ^(/api/histories/.*) http://localhost:8080$1 [P,L] > > to > > ReWriteRule ^(/api/histories/.*) http://localhost:8081$1 [L] > > Which DOES work, but it also breaks the automatic update of the history in > the webbrowser... > I have to reload the page manually to update the history... > > Is there anyway I can get the automatic update to work again, but don't > require autentication? > > Thanks > > Thon > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Config Questions
> 1. We want to use the NGS:Mapping tool 'Map with Bowtie for Illumina' > which requires 'bowtie (0.12.7) with bowtie indexes'. Do we simply > download the bowtie software, place it into > /Users/administrator/galaxy-dist/tools, restart Galaxy, and the tool > is available under NGS:Mapping ? Correct, download the bowtie software, install and verify that it works. Once it does, you can place the binaries (or a symbolic link) to it in the tools dependecy directory (not tools). You need to specify the tools dependency directory in your universe_wsgi.ini file. You can also, see my response to #3. More information: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies > 2. Continuing from question #1, does a type of 'package manager' exist > for automating the installation of tools and dependencies? Can > mercurial accomplish this or is this the purpose of the Tool Shed? Yes and no. Tool Shed is like a package manager, but not. More information here: http://wiki.galaxyproject.org/Tool%20Shed You will still need to install the tools, like bowtie, on your own and not via Galaxy. > 3. What is the purpose of the Galaxy-specific environment file and how > is it used? Going off your question one, you could install bowtie anywhere. I think on OSX its /Applications/Bowtie2.app/ or some other locations /opt/bowti2/ etc. Then simply have a reference to using env.sh to setup the correct environmental paths. In your tools dependency folder you would have the following setup: tools_dependecy/bowtie2/1.03/env.sh tools_dependecy/bowtie2/default --> 1.03/ Note: default is a symbolic link to 1.03 > 4. We have two hard drives: drive1 and drive2. Galaxy is installed on > drive1 and our data files are on drive2. Will Galaxy know how to find > the location of our data files on drive2? Also, will Galaxy know how > to write the result of a job to drive2? Yes, see universe_wsgi.ini for this. Just update the paths. On OSX it could be /Volumes/someNameOfDrive/folder/on/drive > 5. Does Galaxy come configured with an admin account and, if so, how > is it accessed? Yes, see universe_wsgi.ini (admin_users). You need to add an email address of the admin account. > 6. Does Galaxy use as many hardware resources as possible to execute > each job (for example, all 12 cores)? If not, can it be configured to > do so? You will need to configure your job runners and job workers for this. http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster Enjoy! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Jun 5, 2013 at 11:22 AM, Nicholas Kline wrote: > Hi, > > Our lab is in the process of installing a local version of Galaxy on a > computer with the following specs: > > - Mac Pro mid-2012 > - OS X 10.8.3 > - 2 2.4 GHz Intel Xeon Processors > - 6 cores per processor, 12 cores total > - 64 GB RAM > > This installation is meant to be used by a single user. We followed > the instructions at http://wiki.galaxyproject.org/Admin/Get%20Galaxy > and now have a working version of Galaxy available at localhost:8080, > but we have a few questions before using it: > > 1. We want to use the NGS:Mapping tool 'Map with Bowtie for Illumina' > which requires 'bowtie (0.12.7) with bowtie indexes'. Do we simply > download the bowtie software, place it into > /Users/administrator/galaxy-dist/tools, restart Galaxy, and the tool > is available under NGS:Mapping ? > > 2. Continuing from question #1, does a type of 'package manager' exist > for automating the installation of tools and dependencies? Can > mercurial accomplish this or is this the purpose of the Tool Shed? > > 3. What is the purpose of the Galaxy-specific environment file and how > is it used? > > 4. We have two hard drives: drive1 and drive2. Galaxy is installed on > drive1 and our data files are on drive2. Will Galaxy know how to find > the location of our data files on drive2? Also, will Galaxy know how > to write the result of a job to drive2? > > 5. Does Galaxy come configured with an admin account and, if so, how > is it accessed? > > 6. Does Galaxy use as many hardware resources as possible to execute > each job (for example, all 12 cores)? If not, can it be configured to > do so? > > Thank you for your help! > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu
Re: [galaxy-dev] uploading large datasets into a new instance of Galaxy on a Mac Pro
How are you trying to upload the data to Galaxy? Are you using the web interface? Or...? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 31, 2013 at 1:59 PM, Elwood Linney wrote: > > Hello, > I installed Galaxy on a new Mac Pro with 64gb of ram and a solid state first > drive and 3 , 2tb normal drives. > > I can upload from my desktop a smaller file like a reference list or a > cuffdiff file but when I try to load a 16gb fastq file it just does not > load. > > Given that I have my files on the same computer that has Galaxy, and given > that I am not well versed in unix, would anyone have a suggestion as to ways > I could easily upload these large files. > > I would like to think I am just naive and that this has a simple solution. > Hopefully this is the correct list to pose this question. > > Thank you for any help, > > el linney > > For those in the no, I did see this appear on the terminal after I tried to > upload the file through the browser mechanism for upload: > > "Overflow Error: signed integer is greater than maximum" > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Storage question for Galaxy-Admins
Raj, I agree with the points Alex made. It would be much easier to start small and then work your way up. I recently setup galaxy for our HPC cluster for the entire school of medicine. Its only a few ~month into our setup but we are currently using ~300GB of storage. This is part of our larger cluster storage pool of 600TB. We use the gluster filesystem across 6 storage nodes. We are trying fhgfs and other file systems as well (lustre, ceph, etc). If you have any other questions, feel free to ask. -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, May 29, 2013 at 10:34 PM, Raj Ayyampalayam wrote: > Hello, > > I am trying to come up with a number for our local galaxy storage > requirements when we go live (This is an installation for research at our > university). > > I checked out the survey that was done a while back for the GalaxyAdmins. It > seems that the bigger installations have allocated about 200TB for their > installation. > > It would be of great help to me if I can get some feedback on storage used > by local galaxy installations. > If possible please use the following format: > > Total capacity: > Used capacity: > Storage Technology: > Comments: > > Thank you all very much. > -Raj > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Invalid Galaxy URL: None - Installing Tools Shed
Greg, Thanks for the tip on Safari. I have switched to another browser and working now! Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, May 8, 2013 at 6:50 AM, Greg Von Kuster wrote: > In addition, it's helpful to know which browser you are using. Safari has > fairly strict policies on 3rd-party cookies if you have them blocked. Chrome > and firefox are less strict, so unblocking 3rd-party cookies if using Safari > or simply switching to chrome or firefox will probably solve this issue. > > Greg Von Kuster > > On May 8, 2013, at 9:45 AM, Dave Bouvier wrote: > >> Adam, >> >> Normally, the Galaxy URL should be automatically determined and set in a >> cookie during the repository installation process. >> >> To help track down this issue, could you provide the revision of Galaxy >> you're running, and the end of the paster.log file when this error occurs? >> >> >> --Dave B. >> >> On 5/7/13 20:13:38.000, Adam Brenner wrote: >>> Howdy, >>> >>> I am trying to install a tools shed item, Emboss, but when I do this >>> via the admin interface, I get the following text: >>> >>> Repository installation is not possible due to an invalid Galaxy URL: >>> None. You may need to enable cookies in your browser. >>> >>> I have searched the universe_wsgi.ini file and could not find anything >>> that looks like Galaxy URL. Any ideas on how to set the Galaxy URL? >>> >>> Thanks, >>> -Adam >>> >>> -- >>> Adam Brenner >>> Computer Science, Undergraduate Student >>> Donald Bren School of Information and Computer Sciences >>> >>> Research Computing Support >>> Office of Information Technology >>> http://www.oit.uci.edu/rcs/ >>> >>> University of California, Irvine >>> www.ics.uci.edu/~aebrenne/ >>> aebre...@uci.edu >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Invalid Galaxy URL: None - Installing Tools Shed
Howdy, I am trying to install a tools shed item, Emboss, but when I do this via the admin interface, I get the following text: Repository installation is not possible due to an invalid Galaxy URL: None. You may need to enable cookies in your browser. I have searched the universe_wsgi.ini file and could not find anything that looks like Galaxy URL. Any ideas on how to set the Galaxy URL? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Taxonomy fetching not working
Deeps, It does not appear that you have installed taxBuilder on your system. What commands did you use to compile it on your system? If you type 'which taxBuilder' it should return no results (assuming installed incorrectly). You will need to 'make clean' then 'make all' then manually move the binary files it produces into some environmental $PATH. Hope that helps, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 3, 2013 at 3:58 PM, Deepthi Thomas Kannanayakal < dttho...@ucalgary.ca> wrote: > Hi Galaxy team, > > I am having problems with taxonomy information fetching.(Metagenomic > Analysses->Fetch taxonomic representation). > > According to the following link > > > http://dev.list.galaxyproject.org/quot-Fetch-taxonomic-representation-quot-DOESN-T-WORK-td4565489.html > > I have installed libraries from https://bitbucket.org/natefoo/taxonomy > I have installed them in my local directory(not in /usr/local). But I am > still having problem > > /bin/sh: taxBuilder: command not found > > What am I missing here? > > Thanks, > > Deeps > > *Deepthi Theresa Thomas Kannanayakal > > Bioinformatics Database Specialist > Sun Visual Genomic Centre, > Suite 1150, Health Science Centre > 3330 Hospital Drive NW > Department of Biochemistry and Molecular Biology > Faculty of Medicine, University of Calgary > T2N 4N1 > Calgary, Alberta, Canada** > * > *Ph: (403) - 210 6661 Fax: (403) - 210 9538* > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running galaxy using Apache as proxy
Mike, Looking at the configuration, you are not proxy'ing anything. Take a look at my pervious post here and see if that helps: http://dev.list.galaxyproject.org/Running-Galaxy-through-Apache-td4659452.html Let us know how it goes! (I am planning on submitting a re-write for the Apache Proxy page as its missing information..feedback is appreciated). -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 3, 2013 at 11:25 AM, Michael Place wrote: > Hello, > > I am trying to run galaxy using Apache as the proxy server. > > This is intended to be internal to our lab. I have galaxy running and > working. > The problem is that the pages load slowly and then seem to time out after > 10 min or so. > If I login and click get data then go away for a few minutes , then I > come back > and click a tool I get: > > The connection to the server was reset while the page was loading. > > If I retry the page loads. This makes the page very clunky. > > httpd.conf changes: > > > ProxyRequests On > > Order deny,allow > Deny from all > Allow from * > > > > ServerName 192.168.0.240 > RewriteEngine on > RewriteRule ^/galaxy$ /galaxy/ [R] > RewriteRule ^/galaxy/static/style/(.*) > /opt/galaxy-dist/static/june_2007_style/blue/$1 [L] > RewriteRule ^/galaxy/static/scripts/(.*) > /opt/galaxy-dist/static/scripts/packed/$1 [L] > RewriteRule ^/galaxy/static/(.*) > /opt/galaxy-dist/galaxy-dist/static/$1 [L] > RewriteRule ^/galaxy/favicon.ico > /opt/galaxy-dist/static/favicon.ico [L] > RewriteRule ^/galaxy/robots.txt /opt/galaxy-dist/static/robots.txt > [L] > RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] > > > > I have attached the universe_wsgi.ini and my httpd.conf > I am running CentOS. > > Thank you, > > Mike > > > > > > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Galaxy through Apache
Jeff,
Did you ever get this to work?
-Adam
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Wed, May 1, 2013 at 1:52 PM, Jeffrey Long wrote:
> In the Apache access.log, a single access link results in a great many
> output lines; the first two look like this:
>
> 128.233.109.35 - - [01/May/2013:14:31:29 -0600] "GET /galaxy HTTP/1.1" 200
> 5058 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)
> AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"
> 128.233.109.35 - - [01/May/2013:14:31:29 -0600] "GET
> /galaxy/static/style/base.css?v=1367440234 HTTP/1.1" 404 495 "
> http://trove.usask.ca/galaxy"; "Mozilla/5.0 (Macintosh; Intel Mac OS X
> 10_7_5) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4
> Safari/536.29.13"
>
> If I'm reading this right, the second one involves a 404 error. Indeed, I
> don't see any static/style directory in my galaxy location...should there
> be one?
>
> I don't get anything new in apache's error.log as a result of a single
> access attempt.
>
> -Jeff
>
>
>
> On Wed, May 1, 2013 at 6:16 AM, Nate Coraor wrote:
>
>> On Apr 30, 2013, at 5:15 PM, Jeffrey Long wrote:
>>
>> > 1) Any idea which proxy settings in the universe_wsgi file need to be
>> set other than the ones on the website? Those are the only ones I changed.
>> I scrolled down to the 'Advanced Proxy' settings but didn't see anything
>> that looked relevant to me.
>> >
>> > Probably I should have sent this earlier, but this is the traceback I
>> get; it looks like there's a url somewhere that's not getting passed along
>> or re-assembled properly.
>> >
>> > Exception happened during processing of request from ('127.0.0.1',
>> 47424)
>> > Traceback (most recent call last):
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py",
>> line 1053, in process_request_in_thread
>> > self.finish_request(request, client_address)
>> > File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
>> > self.RequestHandlerClass(request, client_address, self)
>> > File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__
>> > self.handle()
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py",
>> line 432, in handle
>> > BaseHTTPRequestHandler.handle(self)
>> > File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
>> > self.handle_one_request()
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py",
>> line 427, in handle_one_request
>> > self.wsgi_execute()
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py",
>> line 287, in wsgi_execute
>> > self.wsgi_start_response)
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/config.py",
>> line 285, in __call__
>> > return self.app(environ, start_response)
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py",
>> line 193, in __call__
>> > path_info = self.normalize_url(path_info, False)[1]
>> > File
>> "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py",
>> line 117, in normalize_url
>> > or self.domain_url_re.search(url)), "URL fragments must start with
>> / or http:// (you gave %r)" % url
>> > AssertionError: URL fragments must start with / or http:// (you gave
>> '.')
>>
>> Hi Jeff,
>>
>> Does the Apache log give any indication of what path is being accessed?
>> You may also want to enable mod_rewrite's debug logging to make sure it's
>> behaving correctly.
>>
>> > 2) Are the static subdirectories strictly necessary, or can I worry
>> about one problem at a time? I've tried both commenting out the proxy
>> lines (in apache2.conf) and leaving them in, and I get the same behaviour.
>>
>> You can exclude the static page rewrites for the purposes of getting it
>> working. Galaxy's internal HTTP s
Re: [galaxy-dev] (no subject)
What version of Python are you using? I believe Python 2.7 is the recommend version. I suggest you grab a fresh copy of galaxy from the stable branch and try again. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 3, 2013 at 12:24 AM, sridhar srinivasan < sridhar2bioi...@gmail.com> wrote: > Dear All, > > i am installing galaxy first time. > > i face problem while installing the python eggs. > while running the Run.sh command it gives as > > /home/sbsuser/.python-eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg-tmp/csamtools.so:6: > RuntimeWarning: __builtin__.file size changed, may indicate binary > incompatibility > � b �n 64 0� c � � � �� ? �� � ��U �#k � ! \ j Z o� [ & � � � [ < > � � �9 - �� ra # ��� �� 8n � �C �U�; B � S l K (p� �� U � & � P w 5 ≠ > ' H š è π ø * ─ ␋ Ð Ä @ ê > > > i couldn't read the message and also i couldn't open the galaxy page > > http://localhost:8080 > > > Could you please suggest tth epossibilities to install. > > Thanks in advance > > Regards > Sridhar > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Load balancing and job configuration
Oliver, The new job running file is also providing me a lot of headaches. The documentation on site is correct, but some of the items are not yet implemented, for example, DRMAA external scripts still need to be in the universe_wsgi.ini, yet the site say to put them in job_conf.xml file! Lot of hair pulling and waiting response from IRC to figure this out...wasn't fun. As for your issue, you want to specify your handlers, not your webserver for your handler types. Let me know if this helps, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, May 2, 2013 at 9:00 AM, Olivier Inizan wrote: > Dear galaxy-dev-list, > > I am trying to use new-style job configuration whith a load-balancing on 2 > web servers and 2 job hanlders. > I have configured load balancing as follows (universe_wsgi.ini): > > # Configuration of the internal HTTP server. > > [server:web0] > use = egg:Paste#http > port = 8083 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 7 > > [server:web1] > use = egg:Paste#http > port = 8082 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 7 > > [server:manager] > use = egg:Paste#http > port = 8079 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler0] > use = egg:Paste#http > port = 8090 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler1] > use = egg:Paste#http > port = 8091 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > > This configuration works fine (manager.log): > galaxy.jobs.manager DEBUG 2013-04-23 12:20:22,901 Starting job handler > galaxy.jobs.runners DEBUG 2013-04-23 12:20:22,902 Starting 5 LocalRunner > workers > galaxy.jobs DEBUG 2013-04-23 12:20:22,904 Loaded job runner > 'galaxy.jobs.runners.local:LocalJobRunner' as 'local' > galaxy.jobs.runners DEBUG 2013-04-23 12:20:22,909 Starting 3 LWRRunner > workers > galaxy.jobs DEBUG 2013-04-23 12:20:22,911 Loaded job runner > 'galaxy.jobs.runners.lwr:LwrJobRunner' as 'lwr' > galaxy.jobs DEBUG 2013-04-23 12:20:22,911 Legacy destination with id > 'local:///', url 'local:///' converted, got params: > galaxy.jobs.handler DEBUG 2013-04-23 12:20:22,911 Loaded job runners > plugins: lwr:local > galaxy.jobs.handler INFO 2013-04-23 12:20:22,912 job handler stop queue > started > galaxy.jobs.handler INFO 2013-04-23 12:20:22,919 job handler queue started > > To use new-style job configuration I have create the following job_conf.xml: > > > > > > > load="galaxy.jobs.runners.local:LocalJobRunner"/> > > > > > > > > > > > > > > When I restart the instance the manager.log outputs: > > galaxy.jobs DEBUG 2013-05-02 17:25:28,402 Loading job configuration from > ./job_conf.xml > galaxy.jobs DEBUG 2013-05-02 17:25:28,402 Read definition for handler > 'server:web0' > galaxy.jobs DEBUG 2013-05-02 17:25:28,403 Read definition for handler > 'server:web1' > galaxy.jobs DEBUG 2013-05-02 17:25:28,403 default set to child > with id or tag 'handlers' > galaxy.jobs DEBUG 2013-05-02 17:25:28,403 default set to > child with id or tag 'local' > galaxy.jobs DEBUG 2013-05-02 17:25:28,403 Done loading job configuration > > So everything seems fine but when I try to run a tool, the corresponding job > is in permanent waiting status (gray color in history). > > I think I have missed something in the configuration process, > Thanks for your help, > > Olivier > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tophat for Illumina is not running
Lilach, Is this on your own install of Galaxy or the public ones? If its on the public ones offered by the Galaxy community, I think you need to post to the users list. If its own your own custom install of Galaxy, you need to check that the job runners have been configured correctly. -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 30, 2013 at 12:29 AM, lilach noy wrote: > Hello, > > I'm new to Galaxy so I think i'm at the right place asking this, though > please let me know if I got it all wrong. > I sent a Tophat query on Thursday about 5 days ago and it is still waiting > to ran (gray background). > > Is to O.k? A bug? Should i keep waiting or stop everything and restart? > > If this is not the address for reporting a suspicious bug, whom does this > concern? > > Thank you very much, > Lilach > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Galaxy through Apache
Jeff, How / Where did you add the apache changes? Are you using VirutalHosts or ... ? Quick glance, everything appears correct. Its been a while since I used Apache (switched everything to nginx) however I believe if you want to setup a Reverse Proxy, you need to add / use the proxy pass lines which does not appear in the Galaxy Wiki. Something like this should work: (untested...) ProxyRequests Off AddDefaultCharset off Order Deny,Allow Deny from all Allow from *.ualberta.ca # Change / Remove as needed ProxyPass /galaxy http://localhost:8080/ ProxyPassReverse /galaxy http://localhost:8080/ # Line below is optional Redirect permanent /galaxy http://mymachine.com/galaxy ProxyVia On Let us know, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, Apr 26, 2013 at 2:56 PM, Jeffrey Long wrote: > Hi folks, > I am wanting to run a galaxy instance and serve it at a location other than > the webserver root, i.e. at http://mymachine.com/galaxy rather than just at > http://mymachine. > > To do this, I've followed the instructions about proxying Galaxy through > Apache at http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy. I'm > pretty sure I've followed the instructions exactly. I enabled the necessary > apache proxy modules and then I added these lines to my apache.conf file: > > > RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule > ^/galaxy/static/style/(.*) > /myplace/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule > ^/galaxy/static/scripts/(.*) /myplace/galaxy-dist/static/scripts/packed/$1 > [L] RewriteRule ^/galaxy/static/(.*) /myplace/galaxy-dist/static/$1 [L] > RewriteRule ^/galaxy/favicon.ico /myplace/galaxy-dist/static/favicon.ico [L] > RewriteRule ^/galaxy/robots.txt /myplace/galaxy-dist/static/robots.txt [L] > RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] > > ...and re-started the apache server without a problem (note: I also tried it > commenting out everything but the first, second and last line in case there > was an issue with the static stuff). > > I then made sure the following lines were present and uncommented in my > universe_wsgi.ini file > > [filter:proxy-prefix] > use = egg:PasteDeploy#prefix > prefix = /galaxy > > [app:main] > > filter-with = proxy-prefix > cookie_path = /galaxy > > > I start up Galaxy. And still, I can only reach it on http://mymachine:8080, > not at http://mymachine/galaxy as I intend. > > Any help would be appreciated! Thanks! > > -Jeff > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Command Line Tool for Creating New Users
Howdy All, I was wondering if anyone has created a command line tool that will add users to the Galaxy user database? I did look into switching to external user authentication via nginx, however, this will result in us using the campus wide kerberos system and allowing ~30,000 people access to Galaxy... and thus offering us less fine grain control to Galaxy. Anything anyone has to share would be much appreciated, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
Okay! I found out what the issue was. I am not sure if this is related to the new Galaxy setup (requiring job_conf.xml) but, I added the following lines to my universe_wsgi.ini, saved, restarted and it worked. drmaa_external_runjob_script = scripts/drmaa_external_runner.py drmaa_external_killjob_script = scripts/drmaa_external_killer.py external_chown_script = scripts/external_chown_script.py So the bigger question is: Is the current documentation wrong and adding drmaa to the job_conf.xml incorrect? If not, then what I am doing wrong in my file? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 23, 2013 at 10:47 AM, Adam Brenner wrote: > Anyone have any ideas as to why its not running? Surely must be a few > users who run Galaxy via a Scheduler using DRMAA... > > My updated job_conf.xml is like so: http://pastebin.com/raw.php?i=QJbay57i > > > Jobs do get submitted to DRMAA, but the scripts are never called to run as > real user. I have DEBUG set to true in universe_wsgi.ini and nothing > regarding errors or debug message are created in paster.log about this. > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Tue, Apr 16, 2013 at 4:55 PM, Adam Brenner wrote: > >> I have the following job_conf.xml file setup for our cluster: >> http://pastebin.com/raw.php?i=0uNhcVJk >> >> When running jobs from my admin account via Galaxy it runs under the same >> user as galaxy is, for testing, this as root. I need for it to run under >> the real user. >> >> The email address / galaxy account I am using to test with does match >> with a user on the cluster -- u...@example.com where user is a valid >> user account with correct permissions to run jobs on our cluster and queue. >> >> I followed the information here: >> http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User >> >> Any ideas as to why galaxy_external* is not being called? The paster.log >> file does not show anything useful regarding calling external scripts. >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of Information Technology >> http://www.oit.uci.edu/rcs/ >> >> University of California, Irvine >> www.ics.uci.edu/~aebrenne/ >> aebre...@uci.edu >> > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
Anyone have any ideas as to why its not running? Surely must be a few users who run Galaxy via a Scheduler using DRMAA... My updated job_conf.xml is like so: http://pastebin.com/raw.php?i=QJbay57i Jobs do get submitted to DRMAA, but the scripts are never called to run as real user. I have DEBUG set to true in universe_wsgi.ini and nothing regarding errors or debug message are created in paster.log about this. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 16, 2013 at 4:55 PM, Adam Brenner wrote: > I have the following job_conf.xml file setup for our cluster: > http://pastebin.com/raw.php?i=0uNhcVJk > > When running jobs from my admin account via Galaxy it runs under the same > user as galaxy is, for testing, this as root. I need for it to run under > the real user. > > The email address / galaxy account I am using to test with does match with > a user on the cluster -- u...@example.com where user is a valid user > account with correct permissions to run jobs on our cluster and queue. > > I followed the information here: > http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User > > Any ideas as to why galaxy_external* is not being called? The paster.log > file does not show anything useful regarding calling external scripts. > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem using FTP upload
This is what I have configured for our cluster: ftp_upload_dir = /som/galaxy/users/ ftp_upload_dir_identifier = username (requires Jon's modification http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html ) ftp_upload_site = None ^^ can not be left blank. Any value should work. The folder setup/structure for /som/galaxy/users/ looks like so. /som/galaxy/users/aebrenne /som/galaxy/users/panteater . Both aebrenne and panteater are users within Galaxy, they just have the @domain.tld part left out. If the folders contain no files, then nothing will be displayed in Galaxy. It does support nested folders, etc... Hopefully that helps, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 23, 2013 at 1:11 AM, Jillian Rowe wrote: > Well, I just tried as the root user and it still doesn't see them. I'm sure root has access everywhere. > ____ > From: Adam Brenner [aebre...@uci.edu] > Sent: Tuesday, April 23, 2013 10:13 AM > To: Jillian Rowe > Cc: galaxy-dev@lists.bx.psu.edu > Subject: Re: [galaxy-dev] Problem using FTP upload > > Um... Lets see Jillian. Can the user who is running the Galaxy process > access the folders in question? Whats the chown/chmod on the folders? > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Mon, Apr 22, 2013 at 11:56 PM, Jillian Rowe > wrote: >> Hi Adam, >> >> Nope, still getting the 'no files found'. Is there any troubleshooting tips you could give me for file permissions? >> >> Best, >> Jillian >> >> From: Adam Brenner [aebre...@uci.edu] >> Sent: Sunday, April 21, 2013 9:42 PM >> To: Jillian Rowe >> Cc: galaxy-dev@lists.bx.psu.edu >> Subject: Re: [galaxy-dev] Problem using FTP upload >> >> Jillian, >> >> Make sure import/users is the full path. IE: /storage/import/users. >> >> If you are using the add-on provided by Jon in: >> http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html >> make sure you add: >> >> ftp_upload_dir_identifier = username >> >> to universe_wsgi.ini. Save and restart. >> >> >> We have the same setup on our HPC cluster. Users copy their data to >> the large storage bricks via scp, rsync, etc... >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of Information Technology >> http://www.oit.uci.edu/rcs/ >> >> University of California, Irvine >> www.ics.uci.edu/~aebrenne/ >> aebre...@uci.edu >> >> >> On Sun, Apr 21, 2013 at 12:00 AM, Jillian Rowe >> wrote: >>> Hello, >>> >>> I would like to use the FTP upload option without actually having an FTP service setup. Instead users will scp to the directory. >>> >>> ftp_upload_dir = import/users >>> ftp_upload_site = notnothing >>> >>> if a create a import/users/username with username as my login I still get the message: >>> >>> Your FTP upload directory contains no files. >>> This Galaxy server allows you to upload files via FTP. To upload some files, log in to the FTP server at localhost using your Galaxy credentials (email address and password). >>> >>> According to this email tot he mailing list http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td3934194.htmlI thought I could do a setup such as I am trying now, but now I am not so sure. >>> >>> Any ideas? >>> >>> Best, >>> Jillian >>> >>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem using FTP upload
Um... Lets see Jillian. Can the user who is running the Galaxy process access the folders in question? Whats the chown/chmod on the folders? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Apr 22, 2013 at 11:56 PM, Jillian Rowe wrote: > Hi Adam, > > Nope, still getting the 'no files found'. Is there any troubleshooting tips > you could give me for file permissions? > > Best, > Jillian > ________ > From: Adam Brenner [aebre...@uci.edu] > Sent: Sunday, April 21, 2013 9:42 PM > To: Jillian Rowe > Cc: galaxy-dev@lists.bx.psu.edu > Subject: Re: [galaxy-dev] Problem using FTP upload > > Jillian, > > Make sure import/users is the full path. IE: /storage/import/users. > > If you are using the add-on provided by Jon in: > http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html > make sure you add: > > ftp_upload_dir_identifier = username > > to universe_wsgi.ini. Save and restart. > > > We have the same setup on our HPC cluster. Users copy their data to > the large storage bricks via scp, rsync, etc... > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Sun, Apr 21, 2013 at 12:00 AM, Jillian Rowe > wrote: >> Hello, >> >> I would like to use the FTP upload option without actually having an FTP >> service setup. Instead users will scp to the directory. >> >> ftp_upload_dir = import/users >> ftp_upload_site = notnothing >> >> if a create a import/users/username with username as my login I still get >> the message: >> >> Your FTP upload directory contains no files. >> This Galaxy server allows you to upload files via FTP. To upload some files, >> log in to the FTP server at localhost using your Galaxy credentials (email >> address and password). >> >> According to this email tot he mailing list >> http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td3934194.html >> I thought I could do a setup such as I am trying now, but now I am not so >> sure. >> >> Any ideas? >> >> Best, >> Jillian >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem using FTP upload
Jillian, Make sure import/users is the full path. IE: /storage/import/users. If you are using the add-on provided by Jon in: http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html make sure you add: ftp_upload_dir_identifier = username to universe_wsgi.ini. Save and restart. We have the same setup on our HPC cluster. Users copy their data to the large storage bricks via scp, rsync, etc... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 21, 2013 at 12:00 AM, Jillian Rowe wrote: > Hello, > > I would like to use the FTP upload option without actually having an FTP > service setup. Instead users will scp to the directory. > > ftp_upload_dir = import/users > ftp_upload_site = notnothing > > if a create a import/users/username with username as my login I still get the > message: > > Your FTP upload directory contains no files. > This Galaxy server allows you to upload files via FTP. To upload some files, > log in to the FTP server at localhost using your Galaxy credentials (email > address and password). > > According to this email tot he mailing list > http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td3934194.html > I thought I could do a setup such as I am trying now, but now I am not so > sure. > > Any ideas? > > Best, > Jillian > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user names with authentication
John, that worked! I was able to upload some sample files using usernames, rather then full email address. No errors, no bugs. Thanks! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 18, 2013 at 2:18 PM, John Chilton wrote: > Crazy coincidence, I had the same problem and hadn't even seen the > original e-mail from Jillian. My pull request implementing this was > merged by Nate earlier this week. > > > https://bitbucket.org/galaxy/galaxy-central/pull-request/157/increased-flexibility-for-ftp-directory/diff > > You can just grab the three changesets out of there and apply them to > your local galaxy repository. Then you just need to set the new > `ftp_upload_dir_identifier` option in universe_wsgi.ini to `username`. > > Hope this helps. > > -John > > On Thu, Apr 18, 2013 at 4:07 PM, Adam Brenner wrote: > > We would also be looking forward to seeing a solution (or hack) like > this. > > We currently have a a gluster filesystem setup where users drop big data > to > > /gl/tmp/user/. Keeping this format would be best. > > > > However, I believe a simple symbolic link would also work with Galaxy. > > > > for i in $(ls /gl/tmp/user); do ln -s /gl/tmp/user > > /gl/tmp/u...@uci.edu;done; > > > > > > -- > > Adam Brenner > > Computer Science, Undergraduate Student > > Donald Bren School of Information and Computer Sciences > > > > Research Computing Support > > Office of Information Technology > > http://www.oit.uci.edu/rcs/ > > > > University of California, Irvine > > www.ics.uci.edu/~aebrenne/ > > aebre...@uci.edu > > > > > > On Sun, Apr 14, 2013 at 1:12 AM, Jillian Rowe > > wrote: > >> > >> Hello, > >> > >> I have a galaxy instance authentication using LDAP. The authentication > >> works fine, and I would like to enable the user upload option. > >> > >> I would like to have: > >> > >> user_library_import_dir = /a/path > >> > >> and for galaxy to look for users with just the username. So if I log in > >> with au...@example.com > >> > >> I want for galaxy to look in /a/path/auser instead of > >> /a/path/au...@example.com > >> > >> Is there anywhere I can set this? > >> > >> Best, > >> Jillian > >> ___ > >> Please keep all replies on the list by using "reply all" > >> in your mail client. To manage your subscriptions to this > >> and other Galaxy lists, please use the interface at: > >> http://lists.bx.psu.edu/ > >> > >> To search Galaxy mailing lists use the unified search at: > >> http://galaxyproject.org/search/mailinglists/ > > > > > > > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user names with authentication
We would also be looking forward to seeing a solution (or hack) like this. We currently have a a gluster filesystem setup where users drop big data to /gl/tmp/user/. Keeping this format would be best. However, I believe a simple symbolic link would also work with Galaxy. for i in $(ls /gl/tmp/user); do ln -s /gl/tmp/user /gl/tmp/u...@uci.edu ;done; -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 14, 2013 at 1:12 AM, Jillian Rowe wrote: > Hello, > > I have a galaxy instance authentication using LDAP. The authentication > works fine, and I would like to enable the user upload option. > > I would like to have: > > user_library_import_dir = /a/path > > and for galaxy to look for users with just the username. So if I log in > with au...@example.com > > I want for galaxy to look in /a/path/auser instead of /a/path/ > au...@example.com > > Is there anywhere I can set this? > > Best, > Jillian > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
I have the following job_conf.xml file setup for our cluster: http://pastebin.com/raw.php?i=0uNhcVJk When running jobs from my admin account via Galaxy it runs under the same user as galaxy is, for testing, this as root. I need for it to run under the real user. The email address / galaxy account I am using to test with does match with a user on the cluster -- u...@example.com where user is a valid user account with correct permissions to run jobs on our cluster and queue. I followed the information here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User Any ideas as to why galaxy_external* is not being called? The paster.log file does not show anything useful regarding calling external scripts. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
So I finally figured it out (thanks to the help from the IRC folks, bag and natefoo). I incorrectly had my job_conf.xml setup where no job runners were set. All is working now. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 14, 2013 at 1:54 PM, Adam Brenner wrote: > Anyone have any ideas? Still have not sorted this out > > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Thu, Apr 11, 2013 at 2:24 PM, Adam Brenner wrote: > >> FYI: Here are the entries in paster.log. If you notice no job_handlers >> are being sent. I performed tests on both nginx reverse proxy and direct >> localhost:8080 (limited to our private network on campus) >> >> 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET >> /tool_runner?tool_id=upload1 HTTP/1.0" 200 - " >> http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; >> Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) >> Version/6.0.3 Safari/536.28.10" >> galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool >> upload1 created job id 8 >> 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index >> HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1"; >> "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 >> (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" >> 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" >> 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 >> (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like >> Gecko) Version/6.0.3 Safari/536.28.10" >> 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET >> /api/histories/56959efe847bb360 HTTP/1.0" 200 - " >> http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac >> OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 >> Safari/536.28.10" >> >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of Information Technology >> http://www.oit.uci.edu/rcs/ >> >> University of California, Irvine >> www.ics.uci.edu/~aebrenne/ >> aebre...@uci.edu >> >> >> On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner wrote: >> >>> Howdy Galaxy Admins, >>> >>> I am installing Galaxy on our HPC cluster and running into a strange >>> issue that I can no longer debug further. I am running the latest galaxy >>> build as of Tuesday April 9th. >>> hg clone https://bitbucket.org/galaxy/galaxy-dist/ >>> hg update stable >>> >>> When a user tries to upload a file via the web interface, nginx reverse >>> proxy setup as described here >>> http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, >>> the upload stays in a state of "Job is waiting to run." >>> >>> The job stays like this for both FTP uploads, and uploads via browser. >>> The dataset is a 319 bytes (both FTP and upload via browser) >>> >>> My universe_wsgi.ini variables are like so: >>> .. under [app:main] >>> new_file_path = /som/galaxy/tmp >>> file_path = /som/galaxy/datasets/files >>> track_jobs_in_database = True >>> job_manager = manager >>> job_handlers = handler0,handler1 >>> ... before [app:main] in [server:main] >>> >>> [server:main] >>> use = egg:Paste#http >>> port = 8080 >>> host = 127.0.0.1 >>> use_threadpool = true >>> threadpool_workers = 5 >>> >>> [server:manager] >>> use = egg:Paste#http >>> port = 8079 >>> host = 127.0.0.1 >>> use_threadpool = true >>> threadpool_workers = 5 >>> >>> [server:handler0] >>> use = egg:Paste#http >>>
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
Anyone have any ideas? Still have not sorted this out -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 2:24 PM, Adam Brenner wrote: > FYI: Here are the entries in paster.log. If you notice no job_handlers are > being sent. I performed tests on both nginx reverse proxy and direct > localhost:8080 (limited to our private network on campus) > > 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET > /tool_runner?tool_id=upload1 HTTP/1.0" 200 - " > http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; > Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) > Version/6.0.3 Safari/536.28.10" > galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool > upload1 created job id 8 > 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index > HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1"; > "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 > (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" > 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 > (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like > Gecko) Version/6.0.3 Safari/536.28.10" > 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET > /api/histories/56959efe847bb360 HTTP/1.0" 200 - " > http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac > OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 > Safari/536.28.10" > > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner wrote: > >> Howdy Galaxy Admins, >> >> I am installing Galaxy on our HPC cluster and running into a strange >> issue that I can no longer debug further. I am running the latest galaxy >> build as of Tuesday April 9th. >> hg clone https://bitbucket.org/galaxy/galaxy-dist/ >> hg update stable >> >> When a user tries to upload a file via the web interface, nginx reverse >> proxy setup as described here >> http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, >> the upload stays in a state of "Job is waiting to run." >> >> The job stays like this for both FTP uploads, and uploads via browser. >> The dataset is a 319 bytes (both FTP and upload via browser) >> >> My universe_wsgi.ini variables are like so: >> .. under [app:main] >> new_file_path = /som/galaxy/tmp >> file_path = /som/galaxy/datasets/files >> track_jobs_in_database = True >> job_manager = manager >> job_handlers = handler0,handler1 >> ... before [app:main] in [server:main] >> >> [server:main] >> use = egg:Paste#http >> port = 8080 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:manager] >> use = egg:Paste#http >> port = 8079 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:handler0] >> use = egg:Paste#http >> port = 8090 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:handler1] >> use = egg:Paste#http >> port = 8091 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> >> Galaxy, for now, is running under the root user and is installed in >> /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS >> system under /som/galaxy/. >> >> I have tried removing the extra job managers and handlers so I just have >> server:main and this still continues to happen. The paster.log files (with >> DEBUG = True) does not show any errors from python or issues writing to >> file. All POST and GET responses are recorded and nothing else. >> >> What I find strange is that if I look in new_file_pat
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
FYI: Here are the entries in paster.log. If you notice no job_handlers are being sent. I performed tests on both nginx reverse proxy and direct localhost:8080 (limited to our private network on campus) 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.0" 200 - " http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool upload1 created job id 8 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /api/histories/56959efe847bb360 HTTP/1.0" 200 - " http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner wrote: > Howdy Galaxy Admins, > > I am installing Galaxy on our HPC cluster and running into a strange issue > that I can no longer debug further. I am running the latest galaxy build as > of Tuesday April 9th. > hg clone https://bitbucket.org/galaxy/galaxy-dist/ > hg update stable > > When a user tries to upload a file via the web interface, nginx reverse > proxy setup as described here > http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the > upload stays in a state of "Job is waiting to run." > > The job stays like this for both FTP uploads, and uploads via browser. The > dataset is a 319 bytes (both FTP and upload via browser) > > My universe_wsgi.ini variables are like so: > .. under [app:main] > new_file_path = /som/galaxy/tmp > file_path = /som/galaxy/datasets/files > track_jobs_in_database = True > job_manager = manager > job_handlers = handler0,handler1 > ... before [app:main] in [server:main] > > [server:main] > use = egg:Paste#http > port = 8080 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:manager] > use = egg:Paste#http > port = 8079 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler0] > use = egg:Paste#http > port = 8090 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler1] > use = egg:Paste#http > port = 8091 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > > Galaxy, for now, is running under the root user and is installed in > /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS > system under /som/galaxy/. > > I have tried removing the extra job managers and handlers so I just have > server:main and this still continues to happen. The paster.log files (with > DEBUG = True) does not show any errors from python or issues writing to > file. All POST and GET responses are recorded and nothing else. > > What I find strange is that if I look in new_file_path I do see my dataset > as > upload_file_data_Zx_1rK > and cat'ing the file shows the correct contents. The items listed in > my file_path (the dataset_*.dat) files are empty/blank..but are created. > > Any ideas as to why its failing? Is it nginx? I only have it redirect > 8080, and none of the other job managers / handlers...is that an issue? Do > you think its permissions on the my file_path or new_file_path directories? > > A very stumped Adam. > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of "Job is waiting to run." The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have it redirect 8080, and none of the other job managers / handlers...is that an issue? Do you think its permissions on the my file_path or new_file_path directories? A very stumped Adam. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
