Hi Peter,
On Feb 19, 2014, at 6:09 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote: > On Wed, Feb 19, 2014 at 9:33 PM, Greg Von Kuster <g...@bx.psu.edu> wrote: >> Hi Peter, >> >> Dave B and I just discovered that issue that causes your tests to fail. >> >> The problem lies with our current implementation supporting the >> <move_directory_files> action tag, which is used in your recipe. >> This tag ultimately uses Python's os.listdir via shutil.move. >> >> It turns out that certain Python versions produce different results >> from os.listdir - the order is different. This is a problem with MIRA >> because during the instlllation it is moving a directory of files where >> the directory includes symlinks to other files in the directory. When >> the symlinks are moved after the files to which they link, problems >> occur. Again, this behavior is intermittent depending on the Pyhton >> version ( and perhaps the os ). >> >> In any case, we have a fix for out move_directory_files which >> Dave is now committing. So your tests should pass with tonight's >> run - Let's hope! >> >> Thanks Peter! > > Well done Greg & Dave, > > That explanation makes perfect sense to me - but I can > see it took some digging to solve it! > > There is actually one MIRA binary (mira) and several aliases > (e.g. mirabait) which are really symlinks back to mira. Magic. > > My next query would be: why wasn't the failing action > <move_directory_files> being caught as a failed install? Your installation recipe for mira attempts to download a binary and if that fails, it echoes an error, but still performs <set_environment> actions.. The tool dependency installation process will fall back to attempting the installation using the action tags to handle install from source and compile if downloading a pre-compiled binary fails. However, your recipe does not have include installing from source, so the installation process simply assumed your <set_environment> tags were all that was necessary. If the recipe for installing from source is not too complex, maybe you could add it to your recipe. It's probabluy not critical though since we now have the fix to the framework. Here is the entire log of the installtion process that helped us uncover the problem - notice that upon failure of the initial binary installation, the process "proceeds with install and compile recipe for tool dependency MIRA". tool_shed.galaxy_install.tool_dependencies.fabric_util DEBUG 2014-02-19 04:50:42,285 Successfully downloaded from url: https://downloads.sourceforge.net/project/mira-assembler/MIRA/stable/mira_4.0_linux-gnu_x86_64_static.tar.bz2 tool_shed.galaxy_install.tool_dependencies.install_util ERROR 2014-02-19 04:50:42,318 Error installing tool dependency MIRA version 4.0. Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/install_util.py", line 289, in install_and_build_package_via_fabric tool_dependency = fabric_util.install_and_build_package( app, tool_dependency, actions_dict ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py", line 581, in install_and_build_package destination_dir=os.path.join( action_dict[ 'destination_directory' ] ) ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/td_common_util.py", line 309, in move_directory_files shutil.move( source_file, destination_file ) File "/usr/lib/python2.7/shutil.py", line 301, in move copy2(src, real_dst) File "/usr/lib/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib/python2.7/shutil.py", line 82, in copyfile with open(src, 'rb') as fsrc: IOError: [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpdr4lDp/tmp-toolshed-mtdV8W9PM/mira_4.0_linux-gnu_x86_64_static/bin/mirabait' tool_shed.galaxy_install.tool_dependencies.install_util DEBUG 2014-02-19 04:50:42,541 Error downloading binary for tool dependency MIRA version 4.0: File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/install_util.py", line 289, in install_and_build_package_via_fabric tool_dependency = fabric_util.install_and_build_package( app, tool_dependency, actions_dict ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py", line 581, in install_and_build_package destination_dir=os.path.join( action_dict[ 'destination_directory' ] ) ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/td_common_util.py", line 309, in move_directory_files shutil.move( source_file, destination_file ) File "/usr/lib/python2.7/shutil.py", line 301, in move copy2(src, real_dst) File "/usr/lib/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib/python2.7/shutil.py", line 82, in copyfile with open(src, 'rb') as fsrc: [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpdr4lDp/tmp-toolshed-mtdV8W9PM/mira_4.0_linux-gnu_x86_64_static/bin/mirabait' tool_shed.galaxy_install.tool_dependencies.install_util DEBUG 2014-02-19 04:50:42,541 Proceeding with install and compile recipe for tool dependency MIRA. tool_shed.util.tool_dependency_util DEBUG 2014-02-19 04:50:42,549 Removed tool dependency installation directory: /ToolDepsTest/MIRA/4.0/peterjc/mira4_assembler/133b863a8a40 /var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py:173: DeprecationWarning: With-statements now directly support multiple context managers with settings( warn_only=True ): /var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py:646: DeprecationWarning: With-statements now directly support multiple context managers with settings( warn_only=True ): /var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py:173: DeprecationWarning: With-statements now directly support multiple context managers with settings( warn_only=True ): Warning: local() encountered an error (return code 2) while executing 'echo Your machine details (the output from 'uname' and 'arch'):' tool_shed.galaxy_install.tool_dependencies.install_util DEBUG 2014-02-19 04:50:43,683 Proceeding with install and compile recipe for tool dependency MIRA. tool_shed.util.tool_dependency_util DEBUG 2014-02-19 04:50:43,690 Removed tool dependency installation directory: /ToolDepsTest/MIRA/4.0/peterjc/mira4_assembler/133b863a8a40 tool_shed.util.tool_dependency_util DEBUG 2014-02-19 04:50:43,698 Changing status for tool dependency MIRA from Uninstalled to Installed. > > Thanks, > > Peter > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
