Re: [galaxy-dev] Upstart script to manage a multi instance load balanced installation
On Aug 9, 2013, at 11:53 AM, Seth Sims wrote:
Dear Nate,
Adding su - galaxy as the first line of the pre-start script seems to
work reasonably well. Also it looks like the line that sets the egg cache is
not working properly. My egg cache ends up being /tmp/${SERVER_NAME}_egg/
but things still seem to be working so I've changed that part to use one
directory in /tmp/ for all instances.
Hi Seth,
Is your Galaxy user's home directory /srv/galaxy-dist? Otherwise, `su -
galaxy` would change the working directory to that user's home directory and
the rest of the script would fail.
I was thinking it might work to just use `su -c` for individual commands, e.g.:
pre-start script
echo checking python version
su - galaxy -c cd /srv/galaxy-dist ; python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
su - galaxy -c cd /srv/galaxy-dist ; python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
su - galaxy -c cd /srv/galaxy-dist ; python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
Sincerely,
Seth Sims
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.2
description galaxy master process. Fetches eggs and spawns all of the
servers it finds configured
start on started network-services
# put galaxy root directory here
chdir /srv/galaxy-dist/
pre-start script
su - galaxy
date
echo checking python version
python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
post-stop script
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
date
echo stopping galaxy servers.
for SERVER in ${SERVERS} ; do
echo stopping ${SERVER}
stop galaxy-worker SERVER_NAME=${SERVER}
done
end script
---
*galaxy-worker*
author Seth Sims seth.s...@gmail.com
version 0.0.2
description Starts a galaxy server instance. This is run from the
galaxy.conf file
instance $SERVER_NAME
#make sure we are running as the galaxy user
setuid galaxy
setgid galaxy
#put the galaxy root directory here
chdir /srv/galaxy-dist/
#system users don't have /home/ directories; so point the egg unpack to a
directory in /tmp/
env PYTHON_EGG_CACHE=/tmp/galaxy_eggs/
respawn
script
exec python ./scripts/paster.py serve universe_wsgi.ini
--server-name=${SERVER_NAME}
end script
On Thu, Aug 8, 2013 at 1:52 PM, Nate Coraor n...@bx.psu.edu wrote:
On Jul 15, 2013, at 6:29 PM, Seth Sims wrote:
After some work i've created an Upstart script which can manage a load
balanced galaxy configuration as described in the wiki. I thought that I
would put it on this list for other people to use. The script parses
universe_wsgi.ini just like run.sh and spawns all of the servers it finds.
It comes in two pieces galaxy.conf and galaxy-worker.conf. Once you place
them both in /etc/init and make the proper edits for the environment a
server can be started with sudo start galaxy. The configuration starts
the server at boot time and puts all of the instances under the management
of upstart which deals with pids, logging to syslog and respawning if an
instance crashes.
I have just gotten this working reasonably well but have done basically no
testing so there are bugs to be found. Any comments are welcome if anyone
knows a better way to do something here.
- Seth
Hi Seth,
Thanks for submitting these. I was about to commit them to the contrib/
directory along with the rest of the start scripts, but I was wondering if
you could avoid running the check/fetch scripts as root by just using `su -c`?
--nate
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.1 test
description galaxy master process. Fetches eggs and spawns all of the
servers it finds
Re: [galaxy-dev] Upstart script to manage a multi instance load balanced installation
Dear Nate,
Actually... no, galaxy's home was set to a non-existent directory so
the working directory was being changed to the root of the file system.
However the script still seemed to work. I changed the script to use su -
galaxy -c like you show anyway. There seem to be no significant change to
the log files with the change. But i'm going to use su -c it as you show
anyway.
Sincerely,
Seth Sims
On Wed, Aug 14, 2013 at 1:03 PM, Nate Coraor n...@bx.psu.edu wrote:
On Aug 9, 2013, at 11:53 AM, Seth Sims wrote:
Dear Nate,
Adding su - galaxy as the first line of the pre-start script seems
to work reasonably well. Also it looks like the line that sets the egg
cache is not working properly. My egg cache ends up being
/tmp/${SERVER_NAME}_egg/ but things still seem to be working so I've
changed that part to use one directory in /tmp/ for all instances.
Hi Seth,
Is your Galaxy user's home directory /srv/galaxy-dist? Otherwise, `su -
galaxy` would change the working directory to that user's home directory
and the rest of the script would fail.
I was thinking it might work to just use `su -c` for individual commands,
e.g.:
pre-start script
echo checking python version
su - galaxy -c cd /srv/galaxy-dist ; python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
su - galaxy -c cd /srv/galaxy-dist ; python ./scripts/check_eggs.py
-q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
su - galaxy -c cd /srv/galaxy-dist ; python
./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini |
xargs echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
Sincerely,
Seth Sims
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.2
description galaxy master process. Fetches eggs and spawns all of the
servers it finds configured
start on started network-services
# put galaxy root directory here
chdir /srv/galaxy-dist/
pre-start script
su - galaxy
date
echo checking python version
python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini |
xargs echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
post-stop script
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini |
xargs echo`
date
echo stopping galaxy servers.
for SERVER in ${SERVERS} ; do
echo stopping ${SERVER}
stop galaxy-worker SERVER_NAME=${SERVER}
done
end script
---
*galaxy-worker*
author Seth Sims seth.s...@gmail.com
version 0.0.2
description Starts a galaxy server instance. This is run from the
galaxy.conf file
instance $SERVER_NAME
#make sure we are running as the galaxy user
setuid galaxy
setgid galaxy
#put the galaxy root directory here
chdir /srv/galaxy-dist/
#system users don't have /home/ directories; so point the egg unpack to
a directory in /tmp/
env PYTHON_EGG_CACHE=/tmp/galaxy_eggs/
respawn
script
exec python ./scripts/paster.py serve universe_wsgi.ini
--server-name=${SERVER_NAME}
end script
On Thu, Aug 8, 2013 at 1:52 PM, Nate Coraor n...@bx.psu.edu wrote:
On Jul 15, 2013, at 6:29 PM, Seth Sims wrote:
After some work i've created an Upstart script which can manage a load
balanced galaxy configuration as described in the wiki. I thought that I
would put it on this list for other people to use. The script parses
universe_wsgi.ini just like run.sh and spawns all of the servers it finds.
It comes in two pieces galaxy.conf and galaxy-worker.conf. Once you place
them both in /etc/init and make the proper edits for the environment a
server can be started with sudo start galaxy. The configuration starts
the server at boot time and puts all of the instances under the management
of upstart which deals with pids, logging to syslog and respawning if an
instance crashes.
I have just gotten this working reasonably well but have done
basically no testing so there are bugs to be found. Any comments are
Re: [galaxy-dev] Upstart script to manage a multi instance load balanced installation
Dear Nate,
Adding su - galaxy as the first line of the pre-start script seems to
work reasonably well. Also it looks like the line that sets the egg cache
is not working properly. My egg cache ends up being
/tmp/${SERVER_NAME}_egg/ but things still seem to be working so I've
changed that part to use one directory in /tmp/ for all instances.
Sincerely,
Seth Sims
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.2
description galaxy master process. Fetches eggs and spawns all of the
servers it finds configured
start on started network-services
# put galaxy root directory here
chdir /srv/galaxy-dist/
pre-start script
su - galaxy
date
echo checking python version
python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
post-stop script
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
date
echo stopping galaxy servers.
for SERVER in ${SERVERS} ; do
echo stopping ${SERVER}
stop galaxy-worker SERVER_NAME=${SERVER}
done
end script
---
*galaxy-worker*
author Seth Sims seth.s...@gmail.com
version 0.0.2
description Starts a galaxy server instance. This is run from the
galaxy.conf file
instance $SERVER_NAME
#make sure we are running as the galaxy user
setuid galaxy
setgid galaxy
#put the galaxy root directory here
chdir /srv/galaxy-dist/
#system users don't have /home/ directories; so point the egg unpack to a
directory in /tmp/
env PYTHON_EGG_CACHE=/tmp/galaxy_eggs/
respawn
script
exec python ./scripts/paster.py serve universe_wsgi.ini
--server-name=${SERVER_NAME}
end script
On Thu, Aug 8, 2013 at 1:52 PM, Nate Coraor n...@bx.psu.edu wrote:
On Jul 15, 2013, at 6:29 PM, Seth Sims wrote:
After some work i've created an Upstart script which can manage a load
balanced galaxy configuration as described in the wiki. I thought that I
would put it on this list for other people to use. The script parses
universe_wsgi.ini just like run.sh and spawns all of the servers it finds.
It comes in two pieces galaxy.conf and galaxy-worker.conf. Once you place
them both in /etc/init and make the proper edits for the environment a
server can be started with sudo start galaxy. The configuration starts
the server at boot time and puts all of the instances under the management
of upstart which deals with pids, logging to syslog and respawning if an
instance crashes.
I have just gotten this working reasonably well but have done basically
no testing so there are bugs to be found. Any comments are welcome if
anyone knows a better way to do something here.
- Seth
Hi Seth,
Thanks for submitting these. I was about to commit them to the contrib/
directory along with the rest of the start scripts, but I was wondering if
you could avoid running the check/fetch scripts as root by just using `su
-c`?
--nate
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.1 test
description galaxy master process. Fetches eggs and spawns all of the
servers it finds configured
start on started network-services
# make sure that any eggs we download are at least owned by the galaxy
group.
# we cannot use setuid in this script because only root can issue the
start galaxy-worker
# command. But this way the galaxy instances should still be able to use
their eggs.
setgid galaxy
# put galaxy root directory here
chdir /srv/galaxy-dist/
pre-start script
date
echo checking python version
python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini |
xargs echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
post-stop script
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini |
xargs echo`
date
echo
Re: [galaxy-dev] Upstart script to manage a multi instance load balanced installation
On Jul 15, 2013, at 6:29 PM, Seth Sims wrote:
After some work i've created an Upstart script which can manage a load
balanced galaxy configuration as described in the wiki. I thought that I
would put it on this list for other people to use. The script parses
universe_wsgi.ini just like run.sh and spawns all of the servers it finds. It
comes in two pieces galaxy.conf and galaxy-worker.conf. Once you place them
both in /etc/init and make the proper edits for the environment a server can
be started with sudo start galaxy. The configuration starts the server at
boot time and puts all of the instances under the management of upstart which
deals with pids, logging to syslog and respawning if an instance crashes.
I have just gotten this working reasonably well but have done basically no
testing so there are bugs to be found. Any comments are welcome if anyone
knows a better way to do something here.
- Seth
Hi Seth,
Thanks for submitting these. I was about to commit them to the contrib/
directory along with the rest of the start scripts, but I was wondering if you
could avoid running the check/fetch scripts as root by just using `su -c`?
--nate
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.1 test
description galaxy master process. Fetches eggs and spawns all of the
servers it finds configured
start on started network-services
# make sure that any eggs we download are at least owned by the galaxy group.
# we cannot use setuid in this script because only root can issue the start
galaxy-worker
# command. But this way the galaxy instances should still be able to use
their eggs.
setgid galaxy
# put galaxy root directory here
chdir /srv/galaxy-dist/
pre-start script
date
echo checking python version
python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
post-stop script
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
date
echo stopping galaxy servers.
for SERVER in ${SERVERS} ; do
echo stopping ${SERVER}
stop galaxy-worker SERVER_NAME=${SERVER}
done
end script
---
*galaxy-worker*
author Seth Sims seth.s...@gmail.com
version 0.0.1 test
description Starts a galaxy server instance. This is run from the
galaxy.conf file
instance $SERVER_NAME
#make sure we are running as the galaxy user
setuid galaxy
setgid galaxy
#put the galaxy root directory here
chdir /srv/galaxy-dist/
#having multiple instances of galaxy using the same egg directory was causing
a race
#condition that was stopping the instances from starting correctly. So give
each instance
#its own directory under /tmp
env PYTHON_EGG_CACHE=/tmp/${SERVER_NAME}_egg/
respawn
script
exec python ./scripts/paster.py serve universe_wsgi.ini
--server-name=${SERVER_NAME}
end script
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Upstart script to manage a multi instance load balanced installation
After some work i've created an Upstart script which can manage a load
balanced galaxy configuration as described in the wiki. I thought that I
would put it on this list for other people to use. The script parses
universe_wsgi.ini just like run.sh and spawns all of the servers it finds.
It comes in two pieces galaxy.conf and galaxy-worker.conf. Once you place
them both in /etc/init and make the proper edits for the environment a
server can be started with sudo start galaxy. The configuration starts
the server at boot time and puts all of the instances under the management
of upstart which deals with pids, logging to syslog and respawning if an
instance crashes.
I have just gotten this working reasonably well but have done basically no
testing so there are bugs to be found. Any comments are welcome if anyone
knows a better way to do something here.
- Seth
*galaxy.conf*
author Seth Sims seth.s...@gmail.com
version 0.0.1 test
description galaxy master process. Fetches eggs and spawns all of the
servers it finds configured
start on started network-services
# make sure that any eggs we download are at least owned by the galaxy
group.
# we cannot use setuid in this script because only root can issue the
start galaxy-worker
# command. But this way the galaxy instances should still be able to use
their eggs.
setgid galaxy
# put galaxy root directory here
chdir /srv/galaxy-dist/
pre-start script
date
echo checking python version
python ./scripts/check_python.py
[ $? -ne 0 ] exit 1
echo pre-fetching tossing out expired eggs
python ./scripts/check_eggs.py -q
if [ $? -eq 0 ]; then
echo Some eggs are out of date, attempting to fetch...
python ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo Fetch Successful.
else
echo Fetch failed.
fi
fi
echo starting servers
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
for SERVER in ${SERVERS} ; do
echo starting server ${SERVER}
start galaxy-worker SERVER_NAME=${SERVER}
done
end script
post-stop script
SERVERS=`sed -n 's/^\[server:\(.*\)\]/\1/ p' universe_wsgi.ini | xargs
echo`
date
echo stopping galaxy servers.
for SERVER in ${SERVERS} ; do
echo stopping ${SERVER}
stop galaxy-worker SERVER_NAME=${SERVER}
done
end script
---
*galaxy-worker*
author Seth Sims seth.s...@gmail.com
version 0.0.1 test
description Starts a galaxy server instance. This is run from the
galaxy.conf file
instance $SERVER_NAME
#make sure we are running as the galaxy user
setuid galaxy
setgid galaxy
#put the galaxy root directory here
chdir /srv/galaxy-dist/
#having multiple instances of galaxy using the same egg directory was
causing a race
#condition that was stopping the instances from starting correctly. So give
each instance
#its own directory under /tmp
env PYTHON_EGG_CACHE=/tmp/${SERVER_NAME}_egg/
respawn
script
exec python ./scripts/paster.py serve universe_wsgi.ini
--server-name=${SERVER_NAME}
end script
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
