Hi Bjoern, thanks for the pointer ... We are running the default
dependency_resolvers_conf.xml now and that, along with the symlinks hint
has helped with a lot of the tool dep. issues. But there is still a
pathing issue we are stuck on.
Take for example the trim_galore tool.
The error is
/home/.galaxy/database/jobs_directory/007/7480/tool_script.sh: line 25:
trim_galore: command not found
And the script is:
#!/bin/bash
# The following block can be used by the job system
# to ensure this script is runnable before actually attempting
# to run it.
if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then
exit 42
fi
[ "$CONDA_DEFAULT_ENV" =
"/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3" ] ||
MAX_TRIES=3
COUNT=0
while [ $COUNT -lt $MAX_TRIES ]; do
. /home/.galaxy/tooldeps/_conda/bin/activate
'/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3' >
conda_activate.log 2>&1
if [ $? -eq 0 ];then
break
else
let COUNT=COUNT+1
if [ $COUNT -eq $MAX_TRIES ];then
echo "Failed to activate conda environment! Error was:"
cat conda_activate.log
exit 1
fi
sleep 10s
fi
done ; trim_galore --version >
/scratch2/galaxy/tmp/GALAXY_VERSION_STRING_7477 2>&1; ln -s
'/home/.galaxy/database/files/013/dataset_13139.dat' input_1.fastq &&
trim_galore --phred33 --output_dir ./ input_1.fastq
--dont_gzip && if [ -f input_1_trimmed.fq.gz ] ; then mv
input_1_trimmed.fq.gz input_1_trimmed.fq ; fi && if [ -f
input_1_val_1.fq.gz ] ; then mv input_1_val_1.fq.gz input_1_val_1.fq ;
fi && if [ -f input_2_val_2.fq.gz ] ; then mv input_2_val_2.fq.gz
input_2_val_2.fq ; fi && if [ -f input_1_unpaired_1.fq.gz ] ; then mv
input_1_unpaired_1.fq.gz input_1_unpaired_1.fq ; fi && if [ -f
input_2_unpaired_2.fq.gz ] ; then mv input_2_unpaired_2.fq.gz
input_2_unpaired_2.fq ; fi
In fact most of the tools do this post-upgrade - where they cannot find
a file.
Here is the trace from hisat2 - it fails in a similar way so we conclude
that there is a common underlying fault here that is affecting the
pathing of the scripts.
Any ideas how we should go about finding this error?
Thanks, Gerhard
PS Here is the hisat2 output.
#!/bin/bash
# The following block can be used by the job system
# to ensure this script is runnable before actually attempting
# to run it.
if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then
exit 42
fi
[ "$CONDA_DEFAULT_ENV" =
"/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426"
] ||
MAX_TRIES=3
COUNT=0
while [ $COUNT -lt $MAX_TRIES ]; do
. /home/.galaxy/tooldeps/_conda/bin/activate
'/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426'
> conda_activate.log 2>&1
if [ $? -eq 0 ];then
break
else
let COUNT=COUNT+1
if [ $COUNT -eq $MAX_TRIES ];then
echo "Failed to activate conda environment! Error was:"
cat conda_activate.log
exit 1
fi
sleep 10s
fi
done ; [ "$CONDA_DEFAULT_ENV" =
"/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426"
] ||
MAX_TRIES=3
COUNT=0
while [ $COUNT -lt $MAX_TRIES ]; do
. /home/.galaxy/tooldeps/_conda/bin/activate
'/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426'
> conda_activate.log 2>&1
if [ $? -eq 0 ];then
break
else
let COUNT=COUNT+1
if [ $COUNT -eq $MAX_TRIES ];then
echo "Failed to activate conda environment! Error was:"
cat conda_activate.log
exit 1
fi
sleep 10s
fi
done ; hisat2 --version >
/scratch2/galaxy/tmp/GALAXY_VERSION_STRING_7484 2>&1; set -o | grep -q
pipefail && set -o pipefail; ln -s
'/home/.galaxy/database/files/013/dataset_13248.dat' genome.fa &&
_hisat2-build_ -p ${GALAXY_SLOTS:-1} genome.fa genome && ln -f -s
'/home/.galaxy/database/files/013/dataset_13238.dat' input_f.fastq &&
ln -f -s '/home/.galaxy/database/files/013/dataset_13243.dat'
input_r.fastq && hisat2 -p ${GALAXY_SLOTS:-1} -x 'genome' -1
'input_f.fastq' -2 'input_r.fastq' |
samtools sort - -@ ${GALAXY_SLOTS:-1} -l 6 -o
'/home/.galaxy/database/jobs_directory/007/7484/galaxy_dataset_19227.dat'
The job fails when it cannot find “hisat2-build”.
Fatal error: Exit code 127 ()
/home/.galaxy/database/jobs_directory/007/7484/tool_script.sh: line 41:
hisat2-build: command not found
The PATH variable should be exported with installation
(/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43/env.sh):
PATH=/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43/bin:$PATH;
export PATH
HISAT2_ROOT_DIR=/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43;
export HISAT2_ROOT_DIR
HISAT_ROOT_DIR=/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43;
export HISAT_ROOT_DIR
The conda environment with dependencies seems to be there:
/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426/bin
Björn Grüning <bjoern.gruen...@gmail.com>;
galaxy-dev@lists.galaxyproject.org
------------------------------------------------------------------------
*From:* Björn Grüning <bjoern.gruen...@gmail.com>
*Sent:* 05 July 2018 22:00:58
*To:* Van Wageningen, GERHARD [gerha...@sun.ac.za];
galaxy-dev@lists.galaxyproject.org
*Subject:* Re: [galaxy-dev] Conda problems: Failed to activate conda
environment
Hi Gerhard,
it seems your tool is picking up the old tool_dependency version and not
the conda version of python.
Have a look at
https://github.com/galaxyproject/galaxy/blob/dev/config/dependency_resolvers_conf.xml.sample
<https://github.com/galaxyproject/galaxy/blob/dev/config/dependency_resolvers_conf.xml.sample>
galaxyproject/galaxy
<https://github.com/galaxyproject/galaxy/blob/dev/config/dependency_resolvers_conf.xml.sample>
github.com
galaxy - Data intensive science for everyone.
to configure the order of dependency resolution in Galaxy.
Cheers,
Bjoern
Hi All
I have recently started learning the Galaxy setup starting with an upgrade.
Made some headway but I was wondering if someone could comment on a
problem that has emerged.
The system is a Galaxy 18.05 installation (upgraded from 17.01) on an
Altair PBS scheduler with drmaa setup (working)
Conda problem: Failed to activate conda environment.
Conda works during tool installation or reinstallation.
Galaxy tools have been updated/reinstalled so they all look "green" in
manage tools.
In Manage Dependencies there are a number of "Dependency resolved but
version not found" warnings, otherwise the deps look clean.
When running tools in Galaxy errors occur, after the jobs have correctly
deployed on the cluster.
The galaxy application raised errors like the two below.
"Failed to activate conda environment! Error was:
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File
"/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py",
line 123
raise C"
"
Failed to activate conda environment! Error was: CondaEnvironmentError:
Environment error: Cannot activate environment bash. User does not have
write access for conda symlinks.
"
Galaxy started and stopped using supervisord with config below:
Thanks, hopefully someone has seen this before ...
regards, Gerhard
[program:web]
command = uwsgi --plugin python --virtualenv /home/.galaxy/.venv
--ini-paste /home/.galaxy/config/galaxy.ini
directory = /home/.galaxy
umask = 022
autostart = true
autorestart = true
startsecs = 10
user = galaxy
numprocs = 1
stopsignal = INT
[program:handler]
command = python ./scripts/galaxy-main -c
/home/.galaxy/config/galaxy.ini --server-name=handler%(process_num)s
directory = /home/.galaxy
process_name = handler%(process_num)s
numprocs = 1
umask = 022
autostart = true
autorestart = true
startsecs = 10
user = galaxy
environment =
VIRTUAL_ENV="/home/.galaxy/.venv",PATH="/home/.galaxy/.venv/bin:%(ENV_PATH)s",http_proxy="http://storage2.eth.hpc:3128/",https_proxy="http://storage2.eth.hpc:3128/";
stdout_logfile = /home/.galaxy/handler%(process_num)s.log
redirect_stderr = true
[group:galaxy]
programs = handler, web
<http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
The integrity and confidentiality of this email is governed by these
terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer >
Die integriteit en vertroulikheid van hierdie e-pos word deur die
volgende bepalings gereël. Vrywaringsklousule
<http://www.sun.ac.za/emaildisclaimer >
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
<http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
The integrity and confidentiality of this email is governed by these
terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer >
Die integriteit en vertroulikheid van hierdie e-pos word deur die
volgende bepalings gereël. Vrywaringsklousule
<http://www.sun.ac.za/emaildisclaimer >
