Hi Greg, bitbucket has a downtime so I have attached a patch. I will create tomorrow a pull request if I find time.
It will work as described and we have now a new variable (__is64bit__)
that we can use to determine if python is running under 64bit (assuming
that the rest is also 64bit-ish).
Attached is a ported version of emboss as test case (Dave B wrapper).
Have a nice evening!
Bjoern
> Hi Björn,
>
>
> This is great. A pull request or a patch is fine. Whatever one is
> most convenient for you. The code doesn't have to be pretty, just
> functional. We can always pretty it up later.
>
>
> Thanks very much for doing this!
>
>
> Greg
>
> On Jul 7, 2013, at 1:19 PM, Björn Grüning
> <bjoern.gruen...@pharmazie.uni-freiburg.de> wrote:
>
> > Hi Greg,
> >
> > > Hello Björn,
> > >
> > > This would be great! I'm not sure about the type attribute - <action
> > > type="cheetah_command" may be ok, but maybe a more generic type like
> > > <action type="template_command" would be better.
> > > I'm not quite sure, so go with what you feel is best.
> > I come up with the following:
> >
> > <action type="template_command">
> > #if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR',
> > False):
> > ./configure --prefix=$env.INSTALL_DIR --x-includes=
> > $env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
> > #else:
> > ./configure --prefix=$env.INSTALL_DIR
> > #end if
> > </action>
> >
> > $env is the global access point to the env variables and its working
> > fine for the few repositories I tested it.
> > I had hoped to get it a little bit prettier but fabric and cheetah
> > doesn't work well together.
> >
> > If that is still satisfying I would clean up my code and create a
> > pull request [1].
> > What do you think?
> >
> > Cheers,
> > Bjoern
> >
> > [1] or patch, I heard rumours that you do not like pull
> > requests ;)?
> >
> > > Thanks for looking into this!
> > >
> > > Greg Von Kuster
> > >
> > >
> > > On Jul 7, 2013, at 5:50 AM, Björn Grüning
> > > <bjoern.gruen...@pharmazie.uni-freiburg.de> wrote:
> > >
> > > > Hi,
> > > >
> > > > I was thinking how to tackle the build failures like the atlas library
> > > > on OS-X or the ones from xlib and ease the writing of the shell_command
> > > > a little bit.
> > > >
> > > > Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
> > > > was also sometimes forced to use some bash tricks.
> > > >
> > > > What about to make that more easy and powerful, and as a plus more
> > > > consistent with the tool-command tag, and use cheetah for that kind of
> > > > work.
> > > >
> > > > Something like that could work easily:
> > > >
> > > > #if ${os.environ}.getVar('X11_LIB_DIR', None):
> > > > ./configure ... --x-includes=....
> > > > #else:
> > > > ./configure
> > > > #end if
> > > >
> > > > Also we could define some reserved variables like $__is64__ and cheetah
> > > > evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
> > > > variable and we do not need to replace it later manually.
> > > >
> > > > What do you think any comments?
> > > > If no one has any objections I can try to implement an <action
> > > > type="cheetah_command">.
> > > >
> > > > Thanks,
> > > > Björn
> > > >
> > >
> >
> >
> >
> > ___________________________________________________________
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > http://lists.bx.psu.edu/
> >
> > To search Galaxy mailing lists use the unified search at:
> > http://galaxyproject.org/search/mailinglists/
>
>
tool_dependencies.xml
Description: XML document
diff -r 434335f80161 lib/galaxy/util/template.py
--- a/lib/galaxy/util/template.py Fri Jun 21 04:37:07 2013 -0400
+++ b/lib/galaxy/util/template.py Mon Jul 08 00:55:36 2013 +0200
@@ -6,4 +6,4 @@
def fill_template( template_text, context=None, **kwargs ):
if not context:
context = kwargs
- return str( Template( source=template_text, searchList=[context] ) )
\ No newline at end of file
+ return str( Template( source=template_text, searchList=[context] ) )
diff -r 434335f80161 lib/tool_shed/galaxy_install/tool_dependencies/common_util.py
--- a/lib/tool_shed/galaxy_install/tool_dependencies/common_util.py Fri Jun 21 04:37:07 2013 -0400
+++ b/lib/tool_shed/galaxy_install/tool_dependencies/common_util.py Mon Jul 08 00:55:36 2013 +0200
@@ -11,6 +11,15 @@
log = logging.getLogger( __name__ )
+def get_env_substitutions( install_dir ):
+ # # Added for compatibility with CloudBioLinux.
+ # TODO: Add tool_version substitution for compat with CloudBioLinux.
+ return { "INSTALL_DIR" : install_dir,
+ "system_install" : install_dir,
+ #"REPOSITORY_INSTALL_DIR": get_tool_shed_repository_install_dir( app, tool_shed_repository ),
+ "__is64bit__": sys.maxsize > 2**32, #checks if the python interpreter is 64bit, we assume that the underlying system is also 64bit
+ }
+
def create_env_var_dict( elem, tool_dependency_install_dir=None, tool_shed_repository_install_dir=None ):
env_var_name = elem.get( 'name', 'PATH' )
env_var_action = elem.get( 'action', 'prepend_to' )
diff -r 434335f80161 lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py
--- a/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py Fri Jun 21 04:37:07 2013 -0400
+++ b/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py Mon Jul 08 00:55:36 2013 +0200
@@ -10,6 +10,8 @@
from contextlib import contextmanager
from galaxy import eggs
+from galaxy.util.template import fill_template
+
import pkg_resources
pkg_resources.require('ssh' )
@@ -158,6 +160,7 @@
def install_and_build_package( app, tool_dependency, actions_dict ):
"""Install a Galaxy tool dependency package either via a url or a mercurial or git clone command."""
+
sa_session = app.model.context.current
install_dir = actions_dict[ 'install_dir' ]
package_name = actions_dict[ 'package_name' ]
@@ -193,7 +196,7 @@
dir = common_util.zip_extraction_directory( work_dir, downloaded_filename )
else:
dir = os.path.curdir
- elif action_type == 'shell_command':
+ elif action_type in ['shell_command', 'template_command']:
# <action type="shell_command">git clone --recursive git://github.com/ekg/freebayes.git</action>
# Eliminate the shell_command clone action so remaining actions can be processed correctly.
filtered_actions = actions[ 1: ]
@@ -305,16 +308,31 @@
return_code = handle_command( app, tool_dependency, install_dir, modify_env_command )
if return_code:
return
- elif action_type == 'shell_command':
+ elif action_type == 'shell_command':
with settings( warn_only=True ):
cmd = ''
for env_shell_file_path in env_shell_file_paths:
- for i, env_setting in enumerate( open( env_shell_file_path ) ):
+ for env_setting in open( env_shell_file_path ):
cmd += '%s\n' % env_setting
cmd += action_dict[ 'command' ]
return_code = handle_command( app, tool_dependency, install_dir, cmd )
if return_code:
return
+ elif action_type == 'template_command':
+ env_vars = dict()
+ for env_shell_file_path in env_shell_file_paths:
+ for env_setting in open( env_shell_file_path ):
+ env_string = env_setting.split(';')[0]
+ env_name, env_path = env_string.split('=')
+ env_vars[env_name] = env_path
+ env_vars.update( common_util.get_env_substitutions( install_dir ) )
+
+ with settings( warn_only=True, **env_vars):
+ # we need to import fabric.api.env so that we can acceess all collected env_vars
+ cmd = fill_template( '#from fabric.api import env\n%s' % action_dict[ 'command' ], context=env_vars )
+ return_code = handle_command( app, tool_dependency, install_dir, cmd )
+ if return_code:
+ return
elif action_type == 'download_file':
# Download a single file to the current directory.
url = action_dict[ 'url' ]
diff -r 434335f80161 lib/tool_shed/galaxy_install/tool_dependencies/install_util.py
--- a/lib/tool_shed/galaxy_install/tool_dependencies/install_util.py Fri Jun 21 04:37:07 2013 -0400
+++ b/lib/tool_shed/galaxy_install/tool_dependencies/install_util.py Mon Jul 08 00:55:36 2013 +0200
@@ -361,11 +361,7 @@
def evaluate_template( text ):
""" Substitute variables defined in XML blocks obtained loaded from dependencies file. """
- # # Added for compatibility with CloudBioLinux.
- # TODO: Add tool_version substitution for compat with CloudBioLinux.
- substitutions = { "INSTALL_DIR" : install_dir,
- "system_install" : install_dir }
- return Template( text ).safe_substitute( substitutions )
+ return Template( text ).safe_substitute( common_util.get_env_substitutions( install_dir ) )
if not os.path.exists( install_dir ):
os.makedirs( install_dir )
@@ -386,6 +382,19 @@
action_dict[ 'command' ] = action_elem_text
else:
continue
+ elif action_type == 'template_command':
+ # <action type="template_command">
+ # #if env.PATH:
+ # make
+ # #end if
+ # </action>
+ # cheetah template syntax
+ action_elem_text = action_elem.text.strip()
+ if action_elem_text:
+ action_dict[ 'command' ] = action_elem_text
+ else:
+ continue
+
elif action_type == 'download_by_url':
# <action type="download_by_url">http://sourceforge.net/projects/samtools/files/samtools/0.1.18/samtools-0.1.18.tar.bz2</action>
if action_elem.text:
@@ -679,4 +688,4 @@
parts = []
for arg in args:
parts.append( arg.strip( '/' ) )
- return '/'.join( parts )
\ No newline at end of file
+ return '/'.join( parts )
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
