Re: Defining Document Transformers in Solr Configuration

[Alexandre Rafalovitch]

I am not sure I fully understood the desired transformation, but
perhaps something from https://people.apache.org/~hossman/rev2016/
would help.

I am specifically thinking of:
*) f.person.qf example
*) ${people} example

Regards,
   Alex.

On 27 February 2018 at 15:20, simon  wrote:
> We do quite complex data pulls from a Solr index for subsequent analytics,
> currently using a home-grown Python API. Queries might include  a handful
> of pseudofields which this API rewrites to an aliased field invoking a
> Document Transformer in the 'fl' parameter list.
>
> For example 'numcites' is transformed to
>
> 'fl= ,numcites:[subquery]=pmid={!terms
> f=md_c_pmid v=$row.pmid}=10=q',...'
>
> What I'd ideally like to be able to do would be have this transformation
> defined in Solr configuration so that it's not tied  to one particular
> external API -  defining a macro, if you will, so that you could supply
> 'fl='a,b,c,%numcites%,...' in the request and have Solr do the expansion.
>
> Is there some way to do this that I've overlooked ? if not, I think it
> would be a useful new feature.
>
>
> -Simon

Re: Defining Document Transformers in Solr Configuration

[simon]

Thanks Mikhail:

I considered that, but not all queries would request that field, and there
are in fact a couple more similar DocTransformer-generated aliased fields
which we can optionally request, so it's not a general enough solution.

-Simon

On Wed, Feb 28, 2018 at 1:18 AM, Mikhail Khludnev  wrote:

> Hello, Simon.
>
> You can define a search handler where have 
> numcites:[subquery]=pmid={!terms
> f=md_c_pmid v=$row.pmid}=10=q
> 
> or something like that.
>
> On Tue, Feb 27, 2018 at 11:20 PM, simon  wrote:
>
> > We do quite complex data pulls from a Solr index for subsequent
> analytics,
> > currently using a home-grown Python API. Queries might include  a handful
> > of pseudofields which this API rewrites to an aliased field invoking a
> > Document Transformer in the 'fl' parameter list.
> >
> > For example 'numcites' is transformed to
> >
> > 'fl= ,numcites:[subquery]=pmid={!terms
> > f=md_c_pmid v=$row.pmid}=10=q',...'
> >
> > What I'd ideally like to be able to do would be have this transformation
> > defined in Solr configuration so that it's not tied  to one particular
> > external API -  defining a macro, if you will, so that you could supply
> > 'fl='a,b,c,%numcites%,...' in the request and have Solr do the expansion.
> >
> > Is there some way to do this that I've overlooked ? if not, I think it
> > would be a useful new feature.
> >
> >
> > -Simon
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: Defining Document Transformers in Solr Configuration

[Mikhail Khludnev]

Hello, Simon.

You can define a search handler where have 
numcites:[subquery]=pmid={!terms
f=md_c_pmid 
v=$row.pmid}=10=q
or something like that.

On Tue, Feb 27, 2018 at 11:20 PM, simon  wrote:

> We do quite complex data pulls from a Solr index for subsequent analytics,
> currently using a home-grown Python API. Queries might include  a handful
> of pseudofields which this API rewrites to an aliased field invoking a
> Document Transformer in the 'fl' parameter list.
>
> For example 'numcites' is transformed to
>
> 'fl= ,numcites:[subquery]=pmid={!terms
> f=md_c_pmid v=$row.pmid}=10=q',...'
>
> What I'd ideally like to be able to do would be have this transformation
> defined in Solr configuration so that it's not tied  to one particular
> external API -  defining a macro, if you will, so that you could supply
> 'fl='a,b,c,%numcites%,...' in the request and have Solr do the expansion.
>
> Is there some way to do this that I've overlooked ? if not, I think it
> would be a useful new feature.
>
>
> -Simon
>



-- 
Sincerely yours
Mikhail Khludnev

Re: Defining Document Transformers in Solr Configuration

[simon]

On Tue, Feb 27, 2018 at 5:34 PM, Diego Ceccarelli (BLOOMBERG/ LONDON) <
dceccarel...@bloomberg.net> wrote:

> I don't think you can define docTrasformer in the SolrConfig at the
> moment, I agree it would be a cool feature.
>
> Maybe one possibility could be to use the update request processors [1],
> and precompute the fields at index time, it would be more expensive in disk
> and index time,  but then it would simplify the fl logic and also
> performance at query time.
>

Wouldn't work for my use case - we are taking a field which contains the
IDs of documents which are cited  by the current document and 'inverting'
this to compute the number of documents which cite the current document in
the subquery. Anything precomputed could change as the index is updated.

>
> Cheers,
> Diego
>
> [1] https://lucene.apache.org/solr/guide/6_6/update-request-
> processors.html
>
> From: solr-user@lucene.apache.org At: 02/27/18 20:21:08To:
> solr-user@lucene.apache.org
> Subject: Defining Document Transformers in Solr Configuration
>
> We do quite complex data pulls from a Solr index for subsequent analytics,
> currently using a home-grown Python API. Queries might include  a handful
> of pseudofields which this API rewrites to an aliased field invoking a
> Document Transformer in the 'fl' parameter list.
>
> For example 'numcites' is transformed to
>
> 'fl= ,numcites:[subquery]=pmid={!terms
> f=md_c_pmid v=$row.pmid}=10=q',...'
>
> What I'd ideally like to be able to do would be have this transformation
> defined in Solr configuration so that it's not tied  to one particular
> external API -  defining a macro, if you will, so that you could supply
> 'fl='a,b,c,%numcites%,...' in the request and have Solr do the expansion.
>
> Is there some way to do this that I've overlooked ? if not, I think it
> would be a useful new feature.
>
>
> -Simon
>
>
>

Defining Document Transformers in Solr Configuration

[simon]

We do quite complex data pulls from a Solr index for subsequent analytics,
currently using a home-grown Python API. Queries might include  a handful
of pseudofields which this API rewrites to an aliased field invoking a
Document Transformer in the 'fl' parameter list.

For example 'numcites' is transformed to

'fl= ,numcites:[subquery]=pmid={!terms
f=md_c_pmid v=$row.pmid}=10=q',...'

What I'd ideally like to be able to do would be have this transformation
defined in Solr configuration so that it's not tied  to one particular
external API -  defining a macro, if you will, so that you could supply
'fl='a,b,c,%numcites%,...' in the request and have Solr do the expansion.

Is there some way to do this that I've overlooked ? if not, I think it
would be a useful new feature.


-Simon