Re: [galaxy-dev] Putting it all together - toolshed tool + conda dependency

[Steve Cassidy]

Looks like I’m hitting this issue:

https://github.com/galaxyproject/galaxy/issues/2797

Steve

—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

> On 29 Aug 2016, at 11:09 AM, Steve Cassidy  wrote:
> 
> Thanks Marius,
>   I tried a build based on the image you mentioned and get the following 
> error:
> 
> /export/galaxy-central/database/job_working_directory/000/1/tool_script.sh: 
> line 9: /tool_deps/_conda/bin/activate: No such file or directory
> /export/galaxy-central/database/job_working_directory/000/1/galaxy_1.sh: line 
> 66: /tool_deps/_conda/bin/activate: No such file or directory
> 
> I tried again with the 16.07 release image but got the same result.  
> 
> Checking the /tool_deps/_conda/bin/ directory, indeed there is no activate 
> script, in fact it contains:
> 
> root@db343fbf7372:/galaxy-central# ls /tool_deps/_conda/bin/
> 2to3  conda  c_rehash  easy_install-3.5  idle3.5  pippydoc3   
>  python   python3.5
> python3.5m python3-config  pyvenv-3.5  tclsh8.5  wheelxz
> 2to3-3.5  conda-env  easy_install  idle3 openssl  pydoc  pydoc3.5 
>  
> python3  python3.5-config  python3.5m-config  pyvenv  sqlite3 
> unxz  wish8.5
> 
> 
> Any pointers appreciated.
> 
> Thanks,
> 
> Steve
> 
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
> 
>> On 26 Aug 2016, at 6:16 PM, Marius van den Beek  
>> wrote:
>> 
>> Hi Steve,
>> 
>> Looks like you're running an older version of galaxy in the docker 
>> container, since newer galaxy will build the metadata environment in a 
>> separate environment called 'conda-metadata-env',
>> 
>> and we have also changed the way that the handlers receive their Python 
>> environment (that's why sqlalachemy couldn't be found).
>> 
>> Can you try updating the container or building from the dev branch? I think 
>> the necessary changes for conda should be in the dev branch, which you can 
>> pull with docker pull quay.io/bgruening/galaxy:dev.
>> 
>> Best,
>> 
>> Marius
>> 
>> 
>> On Aug 26, 2016 6:49 AM, "Steve Cassidy"  wrote:
>> Ok, probing further trying to understand this. It looks like the job runner 
>> is trying to call set_meta() from galaxy_ext/metadata/set_metadata.py.  In 
>> there there’s a line that tries to set up the PATH:
>> 
>> # insert *this* galaxy before all others on sys.path
>> sys.path.insert( 1, os.path.abspath( os.path.join( os.path.dirname( __file__ 
>> ), os.pardir, os.pardir ) ) )
>> 
>> In my case the result is:
>> 
>> ['/export/galaxy-central/database/job_working_directory/000/3’,
>> '/galaxy-central/lib’,
>> '/galaxy-central/lib’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python27.zip’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/plat-linux2’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/lib-tk’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/lib-old’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/lib-dynload’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/site-packages’,
>> '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/site-packages/setuptools-25.1.6-py2.7.egg’
>> ]
>> 
>> Looking for sqlalchemy, I find it installed at:
>> 
>> /galaxy-central/.venv/lib/python2.7/site-packages/sqlalchemy
>> 
>> so clearly that line of code is not doing what it should. I’m guessing that 
>> the Conda resolver is running the script within a conda env and the BETA 
>> magic hasn’t quite made it to the right place yet…
>> 
>> BTW I was trying to understand the flags you mentioned but I can’t find 
>> GALAXY_CONFIG_ENABLE_BETA_TOOL_COMMAND_ISOLATION anywhere other than in 
>> sample Dockerfiles - is this some kind of magic incantation that creates a 
>> rift in space-time…enquiring minds want to know!!!
>> 
>> Steve
>> —
>> Department of Computing, Macquarie University
>> http://web.science.mq.edu.au/~cassidy
>> 
>>> On 25 Aug 2016, at 9:54 PM, Steve Cassidy  wrote:
>>> 
>>> Nah, definitely baby steps…
>>> 
>>> so, in the repo you point to there seems to be an error in the Dockerfile, 
>>> the ENV line should use the var=value syntax to have more than one setting 
>>> on one line (maybe that’s changed recently in docker).
>>> 
>>> with this I built a new docker image, when I run my first tool it takes an 
>>> age while it’s installing the deps, then it crashes with:
>>> 
>>> Traceback (most recent call last):
>>> File 
>>> "/export/galaxy-central/database/job_working_directory/000/1/set_metadata_QUJQLD.py",
>>>  line 1, in 
>>>   from 

Re: [galaxy-dev] Loading a GUI on galaxy

[Peter van Heusden]

Hi

Galaxy supports visualisation plugins and then it also supports interactive
environments. I don't know how WEKA VisualizaPanel works - is it a web
application? Does it require a server?

On Fri, 26 Aug 2016 at 20:52 Lijo John  wrote:

> Hello All,
> I want to know, whether it is possible to embed graphical user interface
> like that of WEKA VisualizePanel tool in galaxy ?
> I am able to load  the GUI on a local instance of galaxy. But when i am
> trying to access the galaxy server through a remote connection, this GUI is
> not able to load on remote desktop.
>
> Any comments or suggestions would be of great interest !
> --
> Regards,
> Lijo John
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Introducing Conda as a new standard for Galaxy tool dependencies

[Léo Biscassi]

Nice job! I was really anxious for some documentation about this theme!
Thanks all involved!

On Fri, Aug 26, 2016 at 3:38 PM Björn Grüning 
wrote:

> Hi all,
>
> Galaxy 16.07 was just released:
> https://docs.galaxyproject.org/en/master/releases/16.07_announce.html
> and the IUC has some news to share with you.
>
> For a more readable version please see:
> https://docs.galaxyproject.org/en/master/admin/conda_faq.html
>
> 
>
> Galaxy tools (also called wrappers) traditionally use Tool Shed package
> recipes to install their dependencies. At the tool’s installation time
> the recipe is downloaded and executed in order to provide the underlying
> software executables. Introduction of these Galaxy-specific recipes was
> a necessary step at the time, however nowadays there are other more
> mature and stable options to install software in a similar manner. The
> Galaxy community has taken steps to improve the tool dependency system
> in order to enable new features and expand its reach. This document aims
> to describe these and answer the FAQ.
>
> It is a pleasure to announce the adoption of a new standard for tool
> dependencies in Galaxy which has been integrated over the last six
> months: Conda packages!
> Not only do Conda packages make tool dependencies more reliable and
> stable, they are also easier to test and faster to develop than the
> traditional Tool Shed package recipes .
>
> Conda is a package manager like apt-get, yum, pip, brew or guix. We
> don’t want to argue about the relative merits of various package
> managers here, in fact Galaxy supports multiple package managers and we
> welcome community contributions (such as implementing a Guix package
> manager or enhancing the existing brew support to bring it on par with
> Conda).
>
> As a community, we have decided that Conda is the one that best fulfills
> our needs. The following are some of the crucial Conda features that led
> to this decision:
>
> * Installation of packages does not require root privileges
> (Installation at any location  the Galaxy user has write access to)
> * Multiple versions of software can be installed in parallel
> * HPC-ready
> * Faster and more robust package installations through pre-compiled
> packages (no build environment complications)
> * Independent of programming language (R, Perl, Python, Julia, Java,
> pre-compiled binaries, and more)
> * Easy to write recipes (1 YAML description file + 1 Bash install script)
> * An active, large and growing community (with more and more software
> authors managing their own recipes)
> * Extensive documentation: Conda documentation
> (http://conda.pydata.org/docs/building/build.html) and Conda quick-start
> (http://conda.pydata.org/docs/get-started.html)
>
> Below we answer some common questions (collected by Lance Parsons):
>
> 1. How do I enable Conda dependency resolution for existing Galaxy
> installations?
> ---
>
> Most Galaxy administrators have not set up a dependency resolvers
> configuration file (dependency_resolvers_conf.xml) which means they are
> using Galaxy’s default (dependency_resolvers_conf.xml.sample). Galaxy
> has enabled Conda dependency resolution by default since release 16.04
> (if Conda was installed already), so many existing installations can use
> Conda dependencies. Having Conda enabled in
> dependency_resolvers_conf.xml means that Galaxy will look for
> dependencies using the Conda system when it attempts to run tools. If
> conda_auto_install is True, and a dependency is not found, Galaxy will
> attempt to install it using the configured Conda channels. A graphical
> user interface that allows administrators install Conda packages
> directly from Galaxy when tools are installed from the Tool Shed is
> available in the 16.07 release of Galaxy.
>
>
> 2. How do Conda dependencies work? Where do things get installed?
> -
> In contrast to the old dependency system, which was used exclusively by
> Galaxy, Conda is a pre-existing, independent project. With Conda it is
> possible for an admin to install packages without touching Galaxy at all
> – managing your dependencies independently from Galaxy. Galaxy can
> handle these dependencies for you, but admins are not required to use
> Galaxy for dependency management.
>
> There are a few new config options in the galaxy.ini file, but by
> default Galaxy will install Conda (the package manager) and the required
> packages in the /_conda/ directory. In this
> directory, Galaxy will create an envs folder with all of the
> environments managed by Galaxy. Every environment contains a lib, bin,
> share, and include subdirectory, depending on the tool, and is
> sufficient to get a Galaxy tool up and running. Galaxy simply sources
> this folder via Conda and makes everything available before the tool is
> executed on your cluster.
>
> To summarize,