Re: [galaxy-dev] Starting uWSGI
Hi Nate. Thank you for the reply. So I installed uWSGI inside my Python virtual environment since that was one of the options listed in the documentation, like so: cd ~/galaxy_env/ source bin/activate pip install uwsgi It looks like uWSGI was successfully installed. I then exited/deactivated the virtual environment via 'deactivate'. Next, I tried to start uWSGI: ~/galaxy-dist$ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste config/galaxy.ini ... but it returned this: realpath() of config/galaxy.ini failed: No such file or directory [core/utils.c line 3607] So, I pointed it to universe_wsgi.ini instead, like so: ~/galaxy-dist$ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste universe_wsgi.ini ...and this was returned: [uWSGI] getting INI configuration from universe_wsgi.ini At that point, Terminal just kinda sits there and does not display any output, error messages, etc. Questions: 1. Was I correct in specifying universe_wsgi.ini ? 2. Nothing is returned after [uWSGI] getting INI configuration from universe_wsgi.ini. Is that normal? 3. Should I activate the Python virtual environment first (via source bin/activate) before executing ~/galaxy-dist$ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste universe_wsgi.ini ? 4. Is the next step to start Galaxy, like so? cd galaxy-dist GALAXY_RUN_ALL=1 sh run.sh --daemon I appreciate your help. On Mon, Jan 26, 2015 at 5:22 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Nicholas, Sorry for the somewhat misleading text in the documentation. The part about no extra modules or recompiling refers to nginx itself - uWSGI protocol support is built in to the core functionality of nginx. This does not mean that uWSGI itself is installed. Installing uWSGI is addressed a bit further up: You will also need to have uWSGI installed. There are a variety of ways to do this. It can be installed system-wide by installing from your system's package manager (on Debian and Ubuntu systems, the uwsgi and uwsgi-plugin-python provide the necessary components), or with theeasy_install or pip commands (which will install it to the system's Python site-packages directory). Alternatively, if you are already running Galaxy from a Python virtualenv, you can use pip install uwsgi with that virtualenv's copy of pip to install to that virtualenv as your unprivileged Galaxy user. In the case of Ubuntu sudo apt-get install uwsgi uwsgi-plugin-python should provide the necessary components. --nate On Mon, Jan 26, 2015 at 2:46 PM, Nicholas Kline nx...@case.edu wrote: Hello, I am trying to setup a local production Galaxy server for our department. Our Galaxy server environment is as follows: - Ubuntu 14.04 64-bit - 94 GB Memory - Intel Xeon - Python virtual environment - Nginx - PostgreSQL - ProFTPd Today I tried starting the Galaxy server for the first time. I think the first step in that process is starting uWSGI, correct? To do so, I executed these commands: cd ~/galaxy-dist/ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste config/galaxy.ini However, it appears something failed. This information is returned: The program 'uwsgi' can be found in the following packages: * uwsgi-core * uwsgi-plugin-alarm-curl * uwsgi-plugin-alarm-xmpp * uwsgi-plugin-curl-cron * uwsgi-plugin-emperor-pg * uwsgi-plugin-erlang * uwsgi-plugin-geoip * uwsgi-plugin-graylog2 * uwsgi-plugin-jvm-openjdk-6 * uwsgi-plugin-jvm-openjdk-7 * uwsgi-plugin-ldap * uwsgi-plugin-lua5.1 * uwsgi-plugin-lua5.2 * uwsgi-plugin-php * uwsgi-plugin-psgi * uwsgi-plugin-python * uwsgi-plugin-python3 * uwsgi-plugin-rack-ruby1.9.1 * uwsgi-plugin-router-access * uwsgi-plugin-sqlite3 * uwsgi-plugin-v8 * uwsgi-plugin-xslt Ask your administrator to install one of them According to the galaxy documentation at https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling , uWSGI support is built in to nginx, so no extra modules or recompiling should be required. Also, should '--ini-paste config/galaxy.ini' really be configured like this instead: --ini-paste universe_wsgi.ini ? Thank you ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] LD_LIBRARY_PATH not exporting to galaxy
Well we are using the environment_setup_file option in our galaxy.ini file. Supposedly we can point to a shell script that will be sourced before running each job. In this script we load all of the environment variables using modules. The paths are all correct after this runs. However, the python script that is run (from the XML for the cufflinks tool) does not inherit these environment variables correctly. Below is the content of our env.sh script: #!/bin/bash . /etc/profile.d/modules_sh.sh module load meta/galaxy This definitely loads the environment properly. And here is the cufflinks line from the XML: command interpreter=python cufflinks_wrapper.py --input=$input . etc. This cufflinks wrapper fails because it cannot actually run cufflinks because it cannot find a library that it needs, because the LD_LIBRARY_PATH is empty within its environment. I actually edited the XML to echo the load path to a file before the python script runs and the path was fine, but still the python script doesn't inherit it properly. - Nik. On Wed, Jan 28, 2015 at 6:16 AM, Nate Coraor n...@bx.psu.edu wrote: Hi Nik, Is there any chance that $LD_LIBRARY_PATH is being set in the parent shell but not exported? --nate On Wed, Jan 28, 2015 at 7:04 AM, Nikhil Joshi najo...@ucdavis.edu wrote: To answer your questions from above, in case it can help with this new problem, we are sourcing the modules from an env.sh script whose full path is the value of the environment_setup_file option. This fixed our first problem, but not this second one. We created the modules ourselves for each of the programs. - Nik. On Tue, Jan 27, 2015 at 6:45 PM, Nikhil Joshi najo...@ucdavis.edu wrote: Okay, we fixed that problem, but another tool is now having the same problem. Now I am trying to run cufflinks and I am getting that same error, but this time there is yet another layer of abstraction because the XML is calling a python script. It seems like the LD_LIBRARY_PATH is correct right up until the python wrapper script gets called and then for some reason the environment is not being passed to the script. This seems like a problem with Popen somewhere in galaxy where it is not passing the environment to the subprocess? - Nik. On Tue, Jan 27, 2015 at 4:33 PM, Nikhil Joshi najo...@ucdavis.edu wrote: Hi John, Thanks for your reply. Our sys admin figured out the problem we had to use the environment_setup_file option to load the modules and we had to fix some shell issues where galaxy was using the dash shell instead of bash. Anyways, it's fixed for now! - Nik. On Mon, Jan 26, 2015 at 6:43 AM, John Chilton jmchil...@gmail.com wrote: Had some idea but they were all dead ends. How are you sourcing the modules - in Galaxy's environment, using an environment file, using Galaxy dependencies (env.sh files) or using Galaxy's modules support? Do you have an example of such a module that sets LD_LIBRARY_PATH? One thing to check is the Galaxy's .bashrc file - make sure it isn't resetting LD_LIBRARY_PATH somehow. -John On Fri, Jan 23, 2015 at 9:34 PM, Nikhil Joshi najo...@ucdavis.edu wrote: Hi all, So we have forked the stable branch of galaxy and are making our own modifications. We are running galaxy on Ubuntu 14.04.1 LTS and we're using modules to load the underlying software, i.e. set environment variables. This works just fine for PATH, JAVA_JAR_PATH, PYTHONPATH, etc., however, LD_LIBRARY_PATH is not being passed to the tools. When a tool (such as cuffdiff) tries to access LD_LIBRARY_PATH, it is empty. We've checked to make sure that loading the cufflinks module correctly updates the LD_LIBRARY_PATH, however, when it is used in galaxy, all the other environment variables are set properly, but not LD_LIBRARY_PATH. Does anyone have any ideas on why this would happen and how to fix it? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis
[galaxy-dev] GalaxyAdmins Online Meetup on 19 February
Hello all, The first GalaxyAdmins https://wiki.galaxyproject.org/Community/GalaxyAdmins meetup of 2015 https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_02_19 will happen online on Thursday, 19 February http://bit.ly/1yZJSkN. GalaxyAdmins https://wiki.galaxyproject.org/Community/GalaxyAdmins is a special interest group for Galaxy community members who are responsible for Galaxy installations. A summary of both the user and admin/developer 2014 Galaxy Community Questionnaires https://wiki.galaxyproject.org/News/2014Questionnaire will be presented by Dave Clements https://wiki.galaxyproject.org/DaveClements, a *Galaxy Project Update* will be offered by a Galaxy Team member https://wiki.galaxyproject.org/GalaxyTeam, and Hans-Rudolf Hotz https://wiki.galaxyproject.org/HansrudolfHotz will lead a short discussion on *GalaxyAdmins direction*. Thanks to everyone for letting us know what dates and times worked best for you. This time and day of the week worked for a remarkable 91% of respondents. We are looking forward to starting this group up again, Hans-Rudolf Hotz and Dave Clements -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Starting uWSGI
On Wed, Jan 28, 2015 at 11:55 AM, Nicholas Kline nx...@case.edu wrote: Hi Nate. Thank you for the reply. So I installed uWSGI inside my Python virtual environment since that was one of the options listed in the documentation, like so: cd ~/galaxy_env/ source bin/activate pip install uwsgi It looks like uWSGI was successfully installed. I then exited/deactivated the virtual environment via 'deactivate'. Next, I tried to start uWSGI: ~/galaxy-dist$ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste config/galaxy.ini Calling uwsgi after deactivating the virtualenv means it'd be running from wherever it's found on $PATH, which is probably not the one you just installed into the virtualenv. But if it works, it may not matter. ... but it returned this: realpath() of config/galaxy.ini failed: No such file or directory [core/utils.c line 3607] So, I pointed it to universe_wsgi.ini instead, like so: ~/galaxy-dist$ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste universe_wsgi.ini universe_wsgi.ini was renamed to config/galaxy.ini late last year. However, if you still have it, it's perfectly fine to leave it there and continue using it. ...and this was returned: [uWSGI] getting INI configuration from universe_wsgi.ini At that point, Terminal just kinda sits there and does not display any output, error messages, etc. If uwsgi is running, this suggests it's working. If you have set up the uwsgi proxy in nginx you should be able to try to access Galaxy via the web at this point. More detail may be available in the uwsgi log file - did you set `logto` in the `[uwsgi]` section of universe_wsgi.ini? Questions: 1. Was I correct in specifying universe_wsgi.ini ? The location of the config file is up to you. universe_wsgi.ini in the root of the Galaxy directory was the old default. 2. Nothing is returned after [uWSGI] getting INI configuration from universe_wsgi.ini. Is that normal? Yes. You probably want to set `logto` as described above and start uwsgi in the background, e.g. with the `-d` command line option. 3. Should I activate the Python virtual environment first (via source bin/activate) before executing ~/galaxy-dist$ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste universe_wsgi.ini ? Yes, I would do this. 4. Is the next step to start Galaxy, like so? cd galaxy-dist GALAXY_RUN_ALL=1 sh run.sh --daemon uwsgi is running your Galaxy web processes, so I would check to ensure they are working first. Once you've verified that, you can use the command above to start any job handlers you have configured. --nate I appreciate your help. On Mon, Jan 26, 2015 at 5:22 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Nicholas, Sorry for the somewhat misleading text in the documentation. The part about no extra modules or recompiling refers to nginx itself - uWSGI protocol support is built in to the core functionality of nginx. This does not mean that uWSGI itself is installed. Installing uWSGI is addressed a bit further up: You will also need to have uWSGI installed. There are a variety of ways to do this. It can be installed system-wide by installing from your system's package manager (on Debian and Ubuntu systems, the uwsgi and uwsgi-plugin-python provide the necessary components), or with theeasy_install or pip commands (which will install it to the system's Python site-packages directory). Alternatively, if you are already running Galaxy from a Python virtualenv, you can use pip install uwsgi with that virtualenv's copy of pip to install to that virtualenv as your unprivileged Galaxy user. In the case of Ubuntu sudo apt-get install uwsgi uwsgi-plugin-python should provide the necessary components. --nate On Mon, Jan 26, 2015 at 2:46 PM, Nicholas Kline nx...@case.edu wrote: Hello, I am trying to setup a local production Galaxy server for our department. Our Galaxy server environment is as follows: - Ubuntu 14.04 64-bit - 94 GB Memory - Intel Xeon - Python virtual environment - Nginx - PostgreSQL - ProFTPd Today I tried starting the Galaxy server for the first time. I think the first step in that process is starting uWSGI, correct? To do so, I executed these commands: cd ~/galaxy-dist/ PYTHONPATH=eggs/PasteDeploy-1.5.0-py2.7.egg uwsgi --ini-paste config/galaxy.ini However, it appears something failed. This information is returned: The program 'uwsgi' can be found in the following packages: * uwsgi-core * uwsgi-plugin-alarm-curl * uwsgi-plugin-alarm-xmpp * uwsgi-plugin-curl-cron * uwsgi-plugin-emperor-pg * uwsgi-plugin-erlang * uwsgi-plugin-geoip * uwsgi-plugin-graylog2 * uwsgi-plugin-jvm-openjdk-6 * uwsgi-plugin-jvm-openjdk-7 * uwsgi-plugin-ldap * uwsgi-plugin-lua5.1 * uwsgi-plugin-lua5.2 * uwsgi-plugin-php * uwsgi-plugin-psgi *
