Re: [galaxy-dev] RFC: new

[Björn Grüning]

Hi Rémy,

> Hi,
> 
> Ok, it seems to be great. We had many problems with RPM, R, or Perl
> packages not maintained/"downloadable" any more. The only SPOF I see now is
> Github itself and the Cargo-Port source code ;) Perhaps you should try to
> add it to python/pip ?

Carogo-port is already pip installable. The problem is more to host and
update the big proxy table. For this you always want to have the latest
table so pypi is not practical here. For the code, the checks and the
gsl installer, sure this can be made available in pypi.

Cheers,
Bjoern

> Best,
> Remy
> 
> 2016-01-04 17:50 GMT+01:00 Eric Rasche :
> 
>> Hi Peter,
>>
>>
>> On 01/02/2016 11:54 PM, Peter Cock wrote:
>>
>>
>>
>> On Friday, 1 January 2016, Björn Grüning < 
>> bjoern.gruen...@gmail.com> wrote:
>>
>>> Hi Galaxy developers,
>>>
>>> this is a RFC to get the implementation details right for a new action
>>> type in `tool_dependencies.xml`.
>>>
>>> Since years we try to save a very crucial sustainability problem:
>>>   **Non-sustainable links**!
>>>
>>>
>>> A little bit of history
>>> 
>>>
>>> At first we tried to [mirror
>>> tarballs](https://github.com/bgruening/download_store) with sceptical
>>> sustainability,
>>> like BioC or random FTP servers.
>>> But over time we encountered many more places which we can not trust.
>>> Google-Code, SourceForge etc ...
>>> We tried to mirror the entire BioC history by tracking the SVN history
>>> down and creating tarball for every revision ...  a Herculean task ...
>>> but still limited in scope because there are so many other things that
>>> needs to be archived to make Galaxy and all tools sustainable.
>>>
>>> In the end we ended up with the simplest solution, provide a community
>>> archive where everyone can drop tarballs that they want to be
>>> sustainable. The Galaxy Project was so generous and is funding the
>>> storage but we have plans to mirror and distribute the workload to
>>> universities and other institutes that want to help.
>>>
>>> The biggest problem we needed to solve was the access to the archive.
>>> Who can drop tarballs? How do we control access to prevent abuse of this
>>> system?
>>>
>>> We went ahead and the created the Cargo-Port:
>>> https://github.com/galaxyproject/cargo-port
>>> Access will be controlled by a community and via PR. Add your package
>>> and we will check the content (hopefully) automatically and the tarball
>>> will be mirrored to a storage server.
>>>
>>>
>>> RFC
>>> ---
>>>
>>> So far so good. This RFC is about the usage of Cargo-Port inside of
>>> Galaxy. I would like to propose a new action type that uses the
>>> Cargo-Port directly. It should replace `>> sha256sum="6387238383883...">` and `` and
>>> offer a more transparent and user-friendly solution.
>>> The current state of the art is quite cumbersome since we need to
>>> generate manually the checksum, offer the correct link
>>> and get the same information into Cargo-Port. I would like to streamline
>>> this a little bit and use this as a good opportunity
>>> to fix and work on https://github.com/galaxyproject/galaxy/issues/896.
>>>
>>>
>>> Proposal ``:
>>>  * attribute for Id, Version, Platform, Architecture
>>>  * no URL, no checksum
>>>  * attribute for the URL to cargo-port/urls.tsv
>>>* default to the current github repo
>>>* configurable via galaxy.ini
>>>  * this action will more or less trigger this curl command: `$ curl
>>> https://raw.githubusercontent.com/galaxyproject/cargo-port/master/gsl.py
>>> | python - --package_id augustus_3_1`
>>>* which give us the freedom to change API, columns ... in Cargo-Port
>>> without updating Galaxy core
>>>* the only API that need to keep stable is `gsl`
>>>  * `gsl` will try to download from the original URL, specified in
>>> Cargo-Port. If this does not work we will download our archived one.
>>>  * Changing the current working dir? Is this what we want, e.g.
>>> automatically uncompress and change cwd like `download_by_url`.
>>>* We will need an attribute to not uncompress. A few tools need the
>>> tarballs uncompressed.
>>>
>>>
>>> Single Point of Failure - a small remark
>>> 
>>>
>>> Previously, Galaxy packages relied entirely on the kindness of upstream
>>> to maintain existing packages indefinitely. Obviously not a sustainable
>>> practice. Every time a tarball was moved, we had to hope one of us
>>> retained a copy so that we could ensure reproducibility. With the advent
>>> of the Cargo Port, we now maintain a complete, redundant copy of every
>>> upstream tarball used in IUC and devteam repositories, additionally
>>> adding sha256sums for every file to ensure download integrity. The
>>> community is welcome to request that files they use in their packages be
>>> added as well. We believe this will help combat the single point of
>>> failure by providing at least one level of duplication. The Cargo Port
>>> is considering plans to provide mirrors of its

Re: [galaxy-dev] RFC: new

[Rémy Dernat]

Hi,

Ok, it seems to be great. We had many problems with RPM, R, or Perl
packages not maintained/"downloadable" any more. The only SPOF I see now is
Github itself and the Cargo-Port source code ;) Perhaps you should try to
add it to python/pip ?

Best,
Remy

2016-01-04 17:50 GMT+01:00 Eric Rasche :

> Hi Peter,
>
>
> On 01/02/2016 11:54 PM, Peter Cock wrote:
>
>
>
> On Friday, 1 January 2016, Björn Grüning < 
> bjoern.gruen...@gmail.com> wrote:
>
>> Hi Galaxy developers,
>>
>> this is a RFC to get the implementation details right for a new action
>> type in `tool_dependencies.xml`.
>>
>> Since years we try to save a very crucial sustainability problem:
>>   **Non-sustainable links**!
>>
>>
>> A little bit of history
>> 
>>
>> At first we tried to [mirror
>> tarballs](https://github.com/bgruening/download_store) with sceptical
>> sustainability,
>> like BioC or random FTP servers.
>> But over time we encountered many more places which we can not trust.
>> Google-Code, SourceForge etc ...
>> We tried to mirror the entire BioC history by tracking the SVN history
>> down and creating tarball for every revision ...  a Herculean task ...
>> but still limited in scope because there are so many other things that
>> needs to be archived to make Galaxy and all tools sustainable.
>>
>> In the end we ended up with the simplest solution, provide a community
>> archive where everyone can drop tarballs that they want to be
>> sustainable. The Galaxy Project was so generous and is funding the
>> storage but we have plans to mirror and distribute the workload to
>> universities and other institutes that want to help.
>>
>> The biggest problem we needed to solve was the access to the archive.
>> Who can drop tarballs? How do we control access to prevent abuse of this
>> system?
>>
>> We went ahead and the created the Cargo-Port:
>> https://github.com/galaxyproject/cargo-port
>> Access will be controlled by a community and via PR. Add your package
>> and we will check the content (hopefully) automatically and the tarball
>> will be mirrored to a storage server.
>>
>>
>> RFC
>> ---
>>
>> So far so good. This RFC is about the usage of Cargo-Port inside of
>> Galaxy. I would like to propose a new action type that uses the
>> Cargo-Port directly. It should replace `> sha256sum="6387238383883...">` and `` and
>> offer a more transparent and user-friendly solution.
>> The current state of the art is quite cumbersome since we need to
>> generate manually the checksum, offer the correct link
>> and get the same information into Cargo-Port. I would like to streamline
>> this a little bit and use this as a good opportunity
>> to fix and work on https://github.com/galaxyproject/galaxy/issues/896.
>>
>>
>> Proposal ``:
>>  * attribute for Id, Version, Platform, Architecture
>>  * no URL, no checksum
>>  * attribute for the URL to cargo-port/urls.tsv
>>* default to the current github repo
>>* configurable via galaxy.ini
>>  * this action will more or less trigger this curl command: `$ curl
>> https://raw.githubusercontent.com/galaxyproject/cargo-port/master/gsl.py
>> | python - --package_id augustus_3_1`
>>* which give us the freedom to change API, columns ... in Cargo-Port
>> without updating Galaxy core
>>* the only API that need to keep stable is `gsl`
>>  * `gsl` will try to download from the original URL, specified in
>> Cargo-Port. If this does not work we will download our archived one.
>>  * Changing the current working dir? Is this what we want, e.g.
>> automatically uncompress and change cwd like `download_by_url`.
>>* We will need an attribute to not uncompress. A few tools need the
>> tarballs uncompressed.
>>
>>
>> Single Point of Failure - a small remark
>> 
>>
>> Previously, Galaxy packages relied entirely on the kindness of upstream
>> to maintain existing packages indefinitely. Obviously not a sustainable
>> practice. Every time a tarball was moved, we had to hope one of us
>> retained a copy so that we could ensure reproducibility. With the advent
>> of the Cargo Port, we now maintain a complete, redundant copy of every
>> upstream tarball used in IUC and devteam repositories, additionally
>> adding sha256sums for every file to ensure download integrity. The
>> community is welcome to request that files they use in their packages be
>> added as well. We believe this will help combat the single point of
>> failure by providing at least one level of duplication. The Cargo Port
>> is considering plans to provide mirrors of itself to various
>> universities and another layer of redundancy.
>>
>>
>> Thanks for reading and we appreciate any comments.
>>
>> Eric, Nitesh & Bjoern
>>
>> -- https://gist.github.com/bgruening/48297c27cd72cbadea7a
>>
>>
> Maybe a question for Nitesh,
>
> Would this replace or coexist with related but narrower in scope
> Bioarchive project?
>
> Different scope, coexist.
>
> Bioarchive
>
>- Hosts only bioconduc

Re: [galaxy-dev] RFC: new

[Eric Rasche]

Hi Peter,

On 01/02/2016 11:54 PM, Peter Cock wrote:
>
>
> On Friday, 1 January 2016, Björn Grüning  > wrote:
>
> Hi Galaxy developers,
>
> this is a RFC to get the implementation details right for a new action
> type in `tool_dependencies.xml`.
>
> Since years we try to save a very crucial sustainability problem:
>   **Non-sustainable links**!
>
>
> A little bit of history
> 
>
> At first we tried to [mirror
> tarballs](https://github.com/bgruening/download_store) with sceptical
> sustainability,
> like BioC or random FTP servers.
> But over time we encountered many more places which we can not trust.
> Google-Code, SourceForge etc ...
> We tried to mirror the entire BioC history by tracking the SVN history
> down and creating tarball for every revision ...  a Herculean task ...
> but still limited in scope because there are so many other things that
> needs to be archived to make Galaxy and all tools sustainable.
>
> In the end we ended up with the simplest solution, provide a community
> archive where everyone can drop tarballs that they want to be
> sustainable. The Galaxy Project was so generous and is funding the
> storage but we have plans to mirror and distribute the workload to
> universities and other institutes that want to help.
>
> The biggest problem we needed to solve was the access to the archive.
> Who can drop tarballs? How do we control access to prevent abuse
> of this
> system?
>
> We went ahead and the created the Cargo-Port:
> https://github.com/galaxyproject/cargo-port
> Access will be controlled by a community and via PR. Add your package
> and we will check the content (hopefully) automatically and the
> tarball
> will be mirrored to a storage server.
>
>
> RFC
> ---
>
> So far so good. This RFC is about the usage of Cargo-Port inside of
> Galaxy. I would like to propose a new action type that uses the
> Cargo-Port directly. It should replace ` sha256sum="6387238383883...">` and `` and
> offer a more transparent and user-friendly solution.
> The current state of the art is quite cumbersome since we need to
> generate manually the checksum, offer the correct link
> and get the same information into Cargo-Port. I would like to
> streamline
> this a little bit and use this as a good opportunity
> to fix and work on https://github.com/galaxyproject/galaxy/issues/896.
>
>
> Proposal ``:
>  * attribute for Id, Version, Platform, Architecture
>  * no URL, no checksum
>  * attribute for the URL to cargo-port/urls.tsv
>* default to the current github repo
>* configurable via galaxy.ini
>  * this action will more or less trigger this curl command: `$ curl
> https://raw.githubusercontent.com/galaxyproject/cargo-port/master/gsl.py
> | python - --package_id augustus_3_1`
>* which give us the freedom to change API, columns ... in
> Cargo-Port
> without updating Galaxy core
>* the only API that need to keep stable is `gsl`
>  * `gsl` will try to download from the original URL, specified in
> Cargo-Port. If this does not work we will download our archived one.
>  * Changing the current working dir? Is this what we want, e.g.
> automatically uncompress and change cwd like `download_by_url`.
>* We will need an attribute to not uncompress. A few tools need the
> tarballs uncompressed.
>
>
> Single Point of Failure - a small remark
> 
>
> Previously, Galaxy packages relied entirely on the kindness of
> upstream
> to maintain existing packages indefinitely. Obviously not a
> sustainable
> practice. Every time a tarball was moved, we had to hope one of us
> retained a copy so that we could ensure reproducibility. With the
> advent
> of the Cargo Port, we now maintain a complete, redundant copy of every
> upstream tarball used in IUC and devteam repositories, additionally
> adding sha256sums for every file to ensure download integrity. The
> community is welcome to request that files they use in their
> packages be
> added as well. We believe this will help combat the single point of
> failure by providing at least one level of duplication. The Cargo Port
> is considering plans to provide mirrors of itself to various
> universities and another layer of redundancy.
>
>
> Thanks for reading and we appreciate any comments.
>
> Eric, Nitesh & Bjoern
>
> -- https://gist.github.com/bgruening/48297c27cd72cbadea7a
>
>
> Maybe a question for Nitesh,
>
> Would this replace or coexist with related but narrower in scope
> Bioarchive project?
Different scope, coexist.

Bioarchive

  * Hosts only bioconductor packages
  * R package specific UI features.
  * may so

Re: [galaxy-dev] RFC: new

[Peter Cock]

On Friday, 1 January 2016, Björn Grüning  wrote:

> Hi Galaxy developers,
>
> this is a RFC to get the implementation details right for a new action
> type in `tool_dependencies.xml`.
>
> Since years we try to save a very crucial sustainability problem:
>   **Non-sustainable links**!
>
>
> A little bit of history
> 
>
> At first we tried to [mirror
> tarballs](https://github.com/bgruening/download_store) with sceptical
> sustainability,
> like BioC or random FTP servers.
> But over time we encountered many more places which we can not trust.
> Google-Code, SourceForge etc ...
> We tried to mirror the entire BioC history by tracking the SVN history
> down and creating tarball for every revision ...  a Herculean task ...
> but still limited in scope because there are so many other things that
> needs to be archived to make Galaxy and all tools sustainable.
>
> In the end we ended up with the simplest solution, provide a community
> archive where everyone can drop tarballs that they want to be
> sustainable. The Galaxy Project was so generous and is funding the
> storage but we have plans to mirror and distribute the workload to
> universities and other institutes that want to help.
>
> The biggest problem we needed to solve was the access to the archive.
> Who can drop tarballs? How do we control access to prevent abuse of this
> system?
>
> We went ahead and the created the Cargo-Port:
> https://github.com/galaxyproject/cargo-port
> Access will be controlled by a community and via PR. Add your package
> and we will check the content (hopefully) automatically and the tarball
> will be mirrored to a storage server.
>
>
> RFC
> ---
>
> So far so good. This RFC is about the usage of Cargo-Port inside of
> Galaxy. I would like to propose a new action type that uses the
> Cargo-Port directly. It should replace ` sha256sum="6387238383883...">` and `` and
> offer a more transparent and user-friendly solution.
> The current state of the art is quite cumbersome since we need to
> generate manually the checksum, offer the correct link
> and get the same information into Cargo-Port. I would like to streamline
> this a little bit and use this as a good opportunity
> to fix and work on https://github.com/galaxyproject/galaxy/issues/896.
>
>
> Proposal ``:
>  * attribute for Id, Version, Platform, Architecture
>  * no URL, no checksum
>  * attribute for the URL to cargo-port/urls.tsv
>* default to the current github repo
>* configurable via galaxy.ini
>  * this action will more or less trigger this curl command: `$ curl
> https://raw.githubusercontent.com/galaxyproject/cargo-port/master/gsl.py
> | python - --package_id augustus_3_1`
>* which give us the freedom to change API, columns ... in Cargo-Port
> without updating Galaxy core
>* the only API that need to keep stable is `gsl`
>  * `gsl` will try to download from the original URL, specified in
> Cargo-Port. If this does not work we will download our archived one.
>  * Changing the current working dir? Is this what we want, e.g.
> automatically uncompress and change cwd like `download_by_url`.
>* We will need an attribute to not uncompress. A few tools need the
> tarballs uncompressed.
>
>
> Single Point of Failure - a small remark
> 
>
> Previously, Galaxy packages relied entirely on the kindness of upstream
> to maintain existing packages indefinitely. Obviously not a sustainable
> practice. Every time a tarball was moved, we had to hope one of us
> retained a copy so that we could ensure reproducibility. With the advent
> of the Cargo Port, we now maintain a complete, redundant copy of every
> upstream tarball used in IUC and devteam repositories, additionally
> adding sha256sums for every file to ensure download integrity. The
> community is welcome to request that files they use in their packages be
> added as well. We believe this will help combat the single point of
> failure by providing at least one level of duplication. The Cargo Port
> is considering plans to provide mirrors of itself to various
> universities and another layer of redundancy.
>
>
> Thanks for reading and we appreciate any comments.
>
> Eric, Nitesh & Bjoern
>
> -- https://gist.github.com/bgruening/48297c27cd72cbadea7a
>
>
Maybe a question for Nitesh,

Would this replace or coexist with related but narrower in scope
Bioarchive project?

 https://bioarchive.galaxyproject.org/

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] RFC: new

[Björn Grüning]

Hi Galaxy developers,

this is a RFC to get the implementation details right for a new action
type in `tool_dependencies.xml`.

Since years we try to save a very crucial sustainability problem:
  **Non-sustainable links**!


A little bit of history


At first we tried to [mirror
tarballs](https://github.com/bgruening/download_store) with sceptical
sustainability,
like BioC or random FTP servers.
But over time we encountered many more places which we can not trust.
Google-Code, SourceForge etc ...
We tried to mirror the entire BioC history by tracking the SVN history
down and creating tarball for every revision ...  a Herculean task ...
but still limited in scope because there are so many other things that
needs to be archived to make Galaxy and all tools sustainable.

In the end we ended up with the simplest solution, provide a community
archive where everyone can drop tarballs that they want to be
sustainable. The Galaxy Project was so generous and is funding the
storage but we have plans to mirror and distribute the workload to
universities and other institutes that want to help.

The biggest problem we needed to solve was the access to the archive.
Who can drop tarballs? How do we control access to prevent abuse of this
system?

We went ahead and the created the Cargo-Port:
https://github.com/galaxyproject/cargo-port
Access will be controlled by a community and via PR. Add your package
and we will check the content (hopefully) automatically and the tarball
will be mirrored to a storage server.


RFC
---

So far so good. This RFC is about the usage of Cargo-Port inside of
Galaxy. I would like to propose a new action type that uses the
Cargo-Port directly. It should replace `` and `` and
offer a more transparent and user-friendly solution.
The current state of the art is quite cumbersome since we need to
generate manually the checksum, offer the correct link
and get the same information into Cargo-Port. I would like to streamline
this a little bit and use this as a good opportunity
to fix and work on https://github.com/galaxyproject/galaxy/issues/896.


Proposal ``:
 * attribute for Id, Version, Platform, Architecture
 * no URL, no checksum
 * attribute for the URL to cargo-port/urls.tsv
   * default to the current github repo
   * configurable via galaxy.ini
 * this action will more or less trigger this curl command: `$ curl
https://raw.githubusercontent.com/galaxyproject/cargo-port/master/gsl.py
| python - --package_id augustus_3_1`
   * which give us the freedom to change API, columns ... in Cargo-Port
without updating Galaxy core
   * the only API that need to keep stable is `gsl`
 * `gsl` will try to download from the original URL, specified in
Cargo-Port. If this does not work we will download our archived one.
 * Changing the current working dir? Is this what we want, e.g.
automatically uncompress and change cwd like `download_by_url`.
   * We will need an attribute to not uncompress. A few tools need the
tarballs uncompressed.


Single Point of Failure - a small remark


Previously, Galaxy packages relied entirely on the kindness of upstream
to maintain existing packages indefinitely. Obviously not a sustainable
practice. Every time a tarball was moved, we had to hope one of us
retained a copy so that we could ensure reproducibility. With the advent
of the Cargo Port, we now maintain a complete, redundant copy of every
upstream tarball used in IUC and devteam repositories, additionally
adding sha256sums for every file to ensure download integrity. The
community is welcome to request that files they use in their packages be
added as well. We believe this will help combat the single point of
failure by providing at least one level of duplication. The Cargo Port
is considering plans to provide mirrors of itself to various
universities and another layer of redundancy.


Thanks for reading and we appreciate any comments.

Eric, Nitesh & Bjoern

-- https://gist.github.com/bgruening/48297c27cd72cbadea7a

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: new param attribute for Galaxy tool XML

[Björn Grüning]

Am 04.03.2015 um 02:37 schrieb John Chilton:
> Peter,
> 
> Galaxy parameters should be case sensitive I think - they are used in
> plain dictionaries quite a bit and I have never seen any logic to make
> them not case sensitive.

I think so too, never tried it but I guess they are case sensitive and
no '-' is allowed.

> Bjoern,
> 
> Is this what you mean?
> 
> https://github.com/galaxyproject/galaxy/compare/dev...jmchilton:argument_name?expand=1

Pretty much yes!

> If it is what you want - I can open a pull request for it.

I was posting it here to get a small discussion started. If there are no
complains and every one is happy, please do so :)

> I guess I am a little skeptical this is useful. I would like to
> believe that Galaxy tool for the most part don't need to map to a
> single application underneath, but we don't have to rehash that
> argument. If the parameter name and help is not more useful than the
> underlying tool's parameter - why would the tool author change it in
> the first place? 

Can you elaborate here a little bit more?
My point is that the actual parameter _is_ important to map against the
original tool documentation, some pdfs etc. It's also important to map
some paper's method section back to a Galaxy tool. So I think it should
be available for the user. Currently, best practise is to put it into
help. This is good, but clutters the UI and it is unstructured.
Unstructured in a way, so that I can not easily get it, for example via
the API.

If we have this information as extra attribute we display it in a
special way in the new JS tool form, independent from the help text.
And I could get this information to do nasty things on a potential
GalaxyShell with it.

> Jen tells me the parameter name is important, you
> tell me the parameter name is important, etc... I respect you guys
> greatly and so I will defer to you - but it is frustrating we are
> breaking abstractions that the tool author set up (presumably for a
> reason).

Which abstraction do we break? Do you have an example? I think in most
of the cases a param maps one-to-one to an argument, isn't?
(There are special cases, but this does not go away and need to stay.)

I think this attribute is just an additional way to structure our params
and make it easier to map and document them.

Thanks, this kind of discussion I wanted to have :)
Cheers,
Bjoern

> -John
> 
> 
> On Tue, Mar 3, 2015 at 11:38 AM, Peter Cock  wrote:
>> Hi Björn,
>>
>> Command line arguments are often case sensitive
>> (e.g. samtools switches), but are Galaxy parameter
>> names?
>>
>> Peter
>>
>> On Sat, Feb 28, 2015 at 9:11 PM, Björn Grüning
>>  wrote:
>>> Hi all,
>>>
>>> we are planning to work on a project to implement a Galaxy fuse based
>>> shell. Probably starting with the work from Clare [1].
>>>
>>> Next to our Galaxy IPython integration it should attract
>>> more bioinformaticians and should offer a new way to interact with
>>> Galaxy. This includes moving, deleting datasets, but also executing
>>> tools and workflows. For the latter I would like to have some sort of
>>> bash auto-completion. Type in your tool/workflow and you will see all
>>> the parameters you can/should modify, in addition to your normal help page.
>>>
>>> Currently the parameter identifiers () do not need to
>>> be meaningful. In fact many tools invent their own unique names, to
>>> identify a parameter in the cheetah section. This name is always unique
>>> but it's hard to guess it's meaning from just the name and are also not
>>> mappable to the original parameter name. This makes tool execution from
>>> the API sometimes hard.
>>>
>>> I would like to propose a new attribute for all  tags. This
>>> should specify a unique value that 100% matches the original parameter
>>> name of the underlying tool.
>>>
>>> This attribute could be used to:
>>>  * automatically enhance the help text of each parameter. Currently best
>>> practise it to include this parameter in brackets at the end of each
>>> help text. We can do this now automatically, or only show it by mouse
>>> over etc ...
>>>  * In a Galaxy shell, the user could just type in a normal command (-i
>>> history1/foo.bam -o history1/bar.sam) and Galaxy shell would be able
>>> to map this parameter to the correct  tag in Galaxy
>>>
>>> I greatly appreciate any comments!
>>> Thanks,
>>> Bjoern
>>>
>>> [1]
>>> https://github.com/claresloggett/gvl_commandline_utilities/blob/master/galaxy-fuse.py
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail

Re: [galaxy-dev] RFC: new param attribute for Galaxy tool XML

[John Chilton]

Peter,

Galaxy parameters should be case sensitive I think - they are used in
plain dictionaries quite a bit and I have never seen any logic to make
them not case sensitive.

Bjoern,

Is this what you mean?

https://github.com/galaxyproject/galaxy/compare/dev...jmchilton:argument_name?expand=1

If it is what you want - I can open a pull request for it.

I guess I am a little skeptical this is useful. I would like to
believe that Galaxy tool for the most part don't need to map to a
single application underneath, but we don't have to rehash that
argument. If the parameter name and help is not more useful than the
underlying tool's parameter - why would the tool author change it in
the first place? Jen tells me the parameter name is important, you
tell me the parameter name is important, etc... I respect you guys
greatly and so I will defer to you - but it is frustrating we are
breaking abstractions that the tool author set up (presumably for a
reason).

-John


On Tue, Mar 3, 2015 at 11:38 AM, Peter Cock  wrote:
> Hi Björn,
>
> Command line arguments are often case sensitive
> (e.g. samtools switches), but are Galaxy parameter
> names?
>
> Peter
>
> On Sat, Feb 28, 2015 at 9:11 PM, Björn Grüning
>  wrote:
>> Hi all,
>>
>> we are planning to work on a project to implement a Galaxy fuse based
>> shell. Probably starting with the work from Clare [1].
>>
>> Next to our Galaxy IPython integration it should attract
>> more bioinformaticians and should offer a new way to interact with
>> Galaxy. This includes moving, deleting datasets, but also executing
>> tools and workflows. For the latter I would like to have some sort of
>> bash auto-completion. Type in your tool/workflow and you will see all
>> the parameters you can/should modify, in addition to your normal help page.
>>
>> Currently the parameter identifiers () do not need to
>> be meaningful. In fact many tools invent their own unique names, to
>> identify a parameter in the cheetah section. This name is always unique
>> but it's hard to guess it's meaning from just the name and are also not
>> mappable to the original parameter name. This makes tool execution from
>> the API sometimes hard.
>>
>> I would like to propose a new attribute for all  tags. This
>> should specify a unique value that 100% matches the original parameter
>> name of the underlying tool.
>>
>> This attribute could be used to:
>>  * automatically enhance the help text of each parameter. Currently best
>> practise it to include this parameter in brackets at the end of each
>> help text. We can do this now automatically, or only show it by mouse
>> over etc ...
>>  * In a Galaxy shell, the user could just type in a normal command (-i
>> history1/foo.bam -o history1/bar.sam) and Galaxy shell would be able
>> to map this parameter to the correct  tag in Galaxy
>>
>> I greatly appreciate any comments!
>> Thanks,
>> Bjoern
>>
>> [1]
>> https://github.com/claresloggett/gvl_commandline_utilities/blob/master/galaxy-fuse.py
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: new param attribute for Galaxy tool XML

[Peter Cock]

Hi Björn,

Command line arguments are often case sensitive
(e.g. samtools switches), but are Galaxy parameter
names?

Peter

On Sat, Feb 28, 2015 at 9:11 PM, Björn Grüning
 wrote:
> Hi all,
>
> we are planning to work on a project to implement a Galaxy fuse based
> shell. Probably starting with the work from Clare [1].
>
> Next to our Galaxy IPython integration it should attract
> more bioinformaticians and should offer a new way to interact with
> Galaxy. This includes moving, deleting datasets, but also executing
> tools and workflows. For the latter I would like to have some sort of
> bash auto-completion. Type in your tool/workflow and you will see all
> the parameters you can/should modify, in addition to your normal help page.
>
> Currently the parameter identifiers () do not need to
> be meaningful. In fact many tools invent their own unique names, to
> identify a parameter in the cheetah section. This name is always unique
> but it's hard to guess it's meaning from just the name and are also not
> mappable to the original parameter name. This makes tool execution from
> the API sometimes hard.
>
> I would like to propose a new attribute for all  tags. This
> should specify a unique value that 100% matches the original parameter
> name of the underlying tool.
>
> This attribute could be used to:
>  * automatically enhance the help text of each parameter. Currently best
> practise it to include this parameter in brackets at the end of each
> help text. We can do this now automatically, or only show it by mouse
> over etc ...
>  * In a Galaxy shell, the user could just type in a normal command (-i
> history1/foo.bam -o history1/bar.sam) and Galaxy shell would be able
> to map this parameter to the correct  tag in Galaxy
>
> I greatly appreciate any comments!
> Thanks,
> Bjoern
>
> [1]
> https://github.com/claresloggett/gvl_commandline_utilities/blob/master/galaxy-fuse.py
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] RFC: new param attribute for Galaxy tool XML

[Björn Grüning]

Hi all,

we are planning to work on a project to implement a Galaxy fuse based
shell. Probably starting with the work from Clare [1].

Next to our Galaxy IPython integration it should attract
more bioinformaticians and should offer a new way to interact with
Galaxy. This includes moving, deleting datasets, but also executing
tools and workflows. For the latter I would like to have some sort of
bash auto-completion. Type in your tool/workflow and you will see all
the parameters you can/should modify, in addition to your normal help page.

Currently the parameter identifiers () do not need to
be meaningful. In fact many tools invent their own unique names, to
identify a parameter in the cheetah section. This name is always unique
but it's hard to guess it's meaning from just the name and are also not
mappable to the original parameter name. This makes tool execution from
the API sometimes hard.

I would like to propose a new attribute for all  tags. This
should specify a unique value that 100% matches the original parameter
name of the underlying tool.

This attribute could be used to:
 * automatically enhance the help text of each parameter. Currently best
practise it to include this parameter in brackets at the end of each
help text. We can do this now automatically, or only show it by mouse
over etc ...
 * In a Galaxy shell, the user could just type in a normal command (-i
history1/foo.bam -o history1/bar.sam) and Galaxy shell would be able
to map this parameter to the correct  tag in Galaxy

I greatly appreciate any comments!
Thanks,
Bjoern

[1]
https://github.com/claresloggett/gvl_commandline_utilities/blob/master/galaxy-fuse.py
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/