[galaxy-dev] Login reguried causing nested main page loading

[Christian Brenninkmeijer]

Hi All,

I tried to turn on login required in config.ini

But now if you are not logged in it loads the main page where normally the 
welcome.html would go.
Which then recursively loads other main pages.

Which file am I missing?

Thanks
Christian
University of Manchester
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Login reguried causing nested main page loading

[Carl Eberhard]

Hi, Christian and Ryan

Which versions of Galaxy are you using when this happens?

Thanks,
Carl


On Mon, Jan 11, 2016 at 9:03 AM, Ryan G  wrote:

> I've seen this as well.  I find that restarting galaxy fixes the problem
> but I'd love to know the underlying cause.
>
> Please excuse any typos -- Sent from my iPhone
>
> On Jan 11, 2016, at 5:08 AM, Christian Brenninkmeijer <
> christian.brenninkmei...@manchester.ac.uk> wrote:
>
> Hi All,
>
> I tried to turn on login required in config.ini
>
> But now if you are not logged in it loads the main page where normally the
> welcome.html would go.
> Which then recursively loads other main pages.
>
> Which file am I missing?
>
> Thanks
> Christian
> University of Manchester
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] A Report Calc tool which provides a scripting language and interpreter for text mining and quality control

[Dooley, Damion]

Hi,

Thanks for quick reply, and good point about keeping math expressions
familiar to most users.   In this first round I settled for a simple
prefix "function(parameter1 parameter2 ...)".  All the python infix
operators like "a + b" have equivalent prefix functions "add(a b)" so the
latter are used.  

Yes, infix "a + b" parsing would be easier to read, and I was thinking we
could try that on our next iteration (and remain backward compatible) but
more opinion may shift that up!  (Some other issues to tackle too:
decisions about whether "/" should map to "div(a b)" or "truediv(a b)",
and how to avoid conflict with our namespace method of referring to
variables via a/b/c paths).

I'm definitely avoiding eval() since we do have to control exactly which
functions, conditionals, and loop constructs are executed.  Not trying to
provide the all-out iPython approach.
 
D.


On 2016-01-11, 12:03 PM, "Bob Harris"  wrote:

>On Jan 11, 2016, at 2:42 PM, Dooley, Damion wrote:
>> ... we're testing out a basic scripting language ... meant to provide
>>[folks] with
>> ways to [do something] without having to be programmers ...
>>  
>>   if( lt(/N50 20) set(report/job/status FAIL))
>> 
>> Math is accomplished by python built-in math functions ...
>
>
>It could well be that's the only way to accomplish what you want in
>whatever environment you're in.  But the use of prefix notation and a
>funny name, for an operator like "<" that non-programmers use familiarly
>as infix, would seem contrary to the stated goal that the user needn't be
>a programmer.
>
>If math can be accomplished via python, why not "<"?  By "math" do you
>only mean function calls, and not arithmetic operators?  Is it that
>python eval() can't be used because of security issues?
>
>Bob H
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] A Report Calc tool which provides a scripting language and interpreter for text mining and quality control

[Bob Harris]

> Yes, infix "a + b" parsing would be easier to read

If you are targeting non-programmers, I think the bigger point is that a + b is 
easier to write.  I do understand the motivation for prefix from the 
implementation standpoint.  The issue, I suppose, will be how early you want to 
include non-programmers as users.

Anyway, that's my 15.2 cents (inflation unadjusted).  I'm not your target 
audience in any case.

Bob H


On Jan 11, 2016, at 3:31 PM, Dooley, Damion wrote:

> Hi,
> 
> Thanks for quick reply, and good point about keeping math expressions
> familiar to most users.   In this first round I settled for a simple
> prefix "function(parameter1 parameter2 ...)".  All the python infix
> operators like "a + b" have equivalent prefix functions "add(a b)" so the
> latter are used.  
> 
> Yes, infix "a + b" parsing would be easier to read, and I was thinking we
> could try that on our next iteration (and remain backward compatible) but
> more opinion may shift that up!  (Some other issues to tackle too:
> decisions about whether "/" should map to "div(a b)" or "truediv(a b)",
> and how to avoid conflict with our namespace method of referring to
> variables via a/b/c paths).
> 
> I'm definitely avoiding eval() since we do have to control exactly which
> functions, conditionals, and loop constructs are executed.  Not trying to
> provide the all-out iPython approach.
> 
> D.
> 
> 
> On 2016-01-11, 12:03 PM, "Bob Harris"  wrote:
> 
>> On Jan 11, 2016, at 2:42 PM, Dooley, Damion wrote:
>>> ... we're testing out a basic scripting language ... meant to provide
>>> [folks] with
>>> ways to [do something] without having to be programmers ...
>>> 
>>>  if( lt(/N50 20) set(report/job/status FAIL))
>>> 
>>> Math is accomplished by python built-in math functions ...
>> 
>> 
>> It could well be that's the only way to accomplish what you want in
>> whatever environment you're in.  But the use of prefix notation and a
>> funny name, for an operator like "<" that non-programmers use familiarly
>> as infix, would seem contrary to the stated goal that the user needn't be
>> a programmer.
>> 
>> If math can be accomplished via python, why not "<"?  By "math" do you
>> only mean function calls, and not arithmetic operators?  Is it that
>> python eval() can't be used because of security issues?
>> 
>> Bob H
>> 
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Job metrics configuration

[Peter van Heusden]

Hi there

I'm trying to understand how the job metrics configuration works. Reading
through various examples, it looks like it works together with the dynamic
destination support.

First on the dynamic destination support, this works, as I understand it,
by importing modules (files ending in .py) from the directory set as the
rules_module, where rules_module is a parameter for the dynamic job plugin.
As in this snipped from the usegalaxy configuration:



usegalaxy.jobs.rules



So import usegalaxy.jobs.rules, find where this resolves to as a directory,
find all .py files in that directory. If rules_module is not specified, the
default is galaxy.jobs.rules (under galaxy/lib).

So these .py files are imported and within them are functions that are used
in dynamic destinations (those with runner="dynamic") and in turn tools are
directed to use these destinations.

Now the job metrics configuration supplies both a user interface and then a
set of keys that are made accessible to dynamic runners as key value pairs
on the job.parameters attribute (passed in as the third argument, i.e.
job). Specifically the job metrics configuration exists under a key
'__job_resource'.

Is this all correct? Is there a more straightforward way of moving from job
resource specification to job configuration than this?

Thanks,
Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] running in job_working_directory

[Jorrit Boekel]

To answer my own questions, apparently I missed the 
outputs_to_working_directory setting in config/galaxy.ini.

cheers,
— 
Jorrit Boekel
Proteomics systems developer
BILS / Lehtiö lab
Scilifelab Stockholm, Sweden



> On 29 Dec 2015, at 14:04, Jorrit Boekel  wrote:
> 
> Dear all,
> 
> I (finally) started transitioning our lab’s tools from using parallelism (the 
> multi-splitter) towards collections. In the parallelism scenario, tools were 
> run in their respective working dir, resulting from a chdir performed in 
> extract_parts.sh. That was nice. But I see when using collections that there 
> is a workdir around, but the tool is not actually run in it. I often need to 
> symlink to change input file extensions and can’t imagine noone else does 
> that, so I wondered if I was missing something. If I’m not, then here’s my 
> two questions:
> 
> - Is there an accepted place to symlink files in (and how do I pass the 
> working dir in a tool XML)? I don’t want them in the files/ dir, looks bad, 
> they’re not cleaned, etc etc.
> - Would it be a bad idea to make tools run inside a workdir, apart from 
> possibly breaking backwards compatibility?
> 
> cheers,
> — 
> Jorrit Boekel
> Proteomics systems developer
> BILS / Lehtiö lab
> Scilifelab Stockholm, Sweden
> 
> 
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Login reguried causing nested main page loading

[Ryan G]

I've seen this as well.  I find that restarting galaxy fixes the problem but 
I'd love to know the underlying cause.  

Please excuse any typos -- Sent from my iPhone

> On Jan 11, 2016, at 5:08 AM, Christian Brenninkmeijer 
>  wrote:
> 
> Hi All,
> 
> I tried to turn on login required in config.ini
> 
> But now if you are not logged in it loads the main page where normally the 
> welcome.html would go.
> Which then recursively loads other main pages.
> 
> Which file am I missing?
> 
> Thanks
> Christian
> University of Manchester
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] tools to reduce list of datasets.

[Christian Brenninkmeijer]

Hi All,

Can someone point me to documentations/ examples of how to write a tool to 
reduce a a list of datasets back into a single dataset.

==

What I want to do is process multiple pairs of reads.

1. load all the fastq files.

2. create a list of pairs

3. Run tools of each pair in the list (As a seperate job for each pair)

4. Run a single tool over all the results of 3 at the same time! (One job which 
takes all the files/ whole collection) as input.

thanks in advance
Christian




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Job metrics configuration

[Peter van Heusden]

Hi there John

Thanks for the clarification. My initial aim was, exactly, to expose those
parameters that, as you say, some people don't think should be exposed.

My use case is to allow jobs with different native parameters to be
scheduled on our clouster from our Galaxy server. Perhaps a better approach
is to use dynamic destinations that try and estimate appropriate values for
e.g. hvmem, but at present our users are used to setting such parameters
themselves.

Peter

On 11 January 2016 at 17:32, John Chilton  wrote:

> So job metrics are completely different than what you described here.
> Job metrics are enabled by default and can be configured fairly
> directly by modifying config/job_metrics_conf.xml. New plugins are
> relatively easy to write and can be added as python files to
> lib/galaxy/jobs/metrics/instrumenters/.
>
> What you described is what is used on usegalaxy.org to allow users to
> have some input on how jobs are scheduled. I call these job resources
> parameters - in job_conf.xml particular tools can be mapped to these
> parameters and - yes you are correct dynamic job destination functions
> can consume them.
>
> The dynamic job destinations definitely shouldn't be accessing these
> through job.parameters keyed on __job_resource. The dynamic
> destination function should just take in a parameter named
> resource_params and then the function will be passed in.
>
> Dynamic Destinations are documented here:
>
> https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping
>
> You are correct job resource parameters have not been documented*,
> there is an open issue here:
> https://github.com/galaxyproject/galaxy/issues/1266
>
> * It should be noted that while this a frequently requested feature
> and used on usegalaxy.org - the feature is somewhat controversial. The
> thinking among some is that is exposes a level of detail that Galaxy
> is supposed to hide. I don't share this particular concern, but the
> implementation does worry me - what it does to persisted job state and
> parameter handling is fairly hacky and I'm somewhat surprised we
> haven't encountered more problems. This is part of the reason not why
> this hasn't been documented, but why I feel less bad that it hasn't
> been documented than other stuff I have done that also hasn't been
> documented :|.
>
> -John
>
> On Mon, Jan 11, 2016 at 1:56 PM, Peter van Heusden 
> wrote:
> > Hi there
> >
> > I'm trying to understand how the job metrics configuration works. Reading
> > through various examples, it looks like it works together with the
> dynamic
> > destination support.
> >
> > First on the dynamic destination support, this works, as I understand
> it, by
> > importing modules (files ending in .py) from the directory set as the
> > rules_module, where rules_module is a parameter for the dynamic job
> plugin.
> > As in this snipped from the usegalaxy configuration:
> >
> > 
> > 
> > usegalaxy.jobs.rules
> > 
> >
> >
> > So import usegalaxy.jobs.rules, find where this resolves to as a
> directory,
> > find all .py files in that directory. If rules_module is not specified,
> the
> > default is galaxy.jobs.rules (under galaxy/lib).
> >
> > So these .py files are imported and within them are functions that are
> used
> > in dynamic destinations (those with runner="dynamic") and in turn tools
> are
> > directed to use these destinations.
> >
> > Now the job metrics configuration supplies both a user interface and
> then a
> > set of keys that are made accessible to dynamic runners as key value
> pairs
> > on the job.parameters attribute (passed in as the third argument, i.e.
> job).
> > Specifically the job metrics configuration exists under a key
> > '__job_resource'.
> >
> > Is this all correct? Is there a more straightforward way of moving from
> job
> > resource specification to job configuration than this?
> >
> > Thanks,
> > Peter
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] A Report Calc tool which provides a scripting language and interpreter for text mining and quality control

[Bob Harris]

On Jan 11, 2016, at 2:42 PM, Dooley, Damion wrote:
> ... we're testing out a basic scripting language ... meant to provide [folks] 
> with
> ways to [do something] without having to be programmers ...
>  
>   if( lt(/N50 20) set(report/job/status FAIL))
> 
> Math is accomplished by python built-in math functions ...


It could well be that's the only way to accomplish what you want in whatever 
environment you're in.  But the use of prefix notation and a funny name, for an 
operator like "<" that non-programmers use familiarly as infix, would seem 
contrary to the stated goal that the user needn't be a programmer.

If math can be accomplished via python, why not "<"?  By "math" do you only 
mean function calls, and not arithmetic operators?  Is it that  python eval() 
can't be used because of security issues?

Bob H

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] A Report Calc tool which provides a scripting language and interpreter for text mining and quality control

[Dooley, Damion]

Folks, we're testing out a basic scripting language and interpreter for
report writing and quality control that is meant to provide both
command-line programmers and Galaxy platform researchers and admins with
ways to tweak workflow quality control behaviour without having to be
programmers themselves.

I wanted to sound-out the community about whether or not there are any
basic objections to our approach, described briefly below?
 

Currently the interpreter provides all the built-in Python operator and
math functions (as well as some particular named group regular expression
functions for text mining) so that users can examine given input log or
data datasets for fields that need to be reported or compared with QC
metric rules.  

The Report Calc tool takes in a file of statements (each being a
function(parameter1 parameter2 ...) syntax) that look like this:

   set(4857000 report/contigs/reference_genome_size)
   set("serovar Typhimurium LT2" report/contigs/reference_genome)
   set(0.1 report/contigs/good_genome_size_ratio)


   set( statisticN(report/contigs/contig_lengths 50) report/contigs/N50)
   if( lt(/N50 20) set(report/job/status FAIL))

Math is accomplished by python built-in math functions (I.e. Ignore the
"/" - that's a namespace syntax character).

   set( 
  truediv( abs( sub( /sampleGenomeSize /referenceGenomeSize))
/referenceGenomeSize)



  report/contigs/sample_genome_size_ratio
   )

And writes any output as desired to a standard tool output folder:
  

   writeFile( 
  pageHtml( getHtml(report) "My Report Widget")
  report.html
   )


It allows users to build a ruleset (file containing statements like the
above), and process text, json, tabular datasets in their history, and it
can manipulate variables, arrays and dictionaries in the tools in-memory
temporary data structure namespace.  It doesn't touch Galaxy's inner
workings or interact with a workflow except by way of app exit codes.

One can even write little text-mining programs in it:

   iterate( readFileByName(contigs-all.fasta)
  if( eq( getitem(iterator/0/value 0) ">")
 append( 
regexp(iterator/0/value "length_(?P\\d+)_")
report/contigs/contig_lengths
 )
  )
   )


I'll release it shortly for review/play, with lots of documentation that
describes in detail what it can/can't do, and an argument for why one
might want to bother using/learning it.

There are one or two irrelevant python built-in functions we might filter
out (e.g. isCallable() ); so far we haven't spotted any security issues,
and we've limited the flow control to only accomplish loops by iterables
so there's no evident way to create infinite loops.  One only iterates
through files or in-memory arrays.

As well suggested functionality for the wish-list now accepted!

Regards,

Damion

 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/