Re: external file field and fl parameter
: reference to a few "special" values, such as "id" and "score". Neither : of them are technically "stored" fields either, but afaik you dont need : to use "field(id), field(score)" for those. Can you honestly say that : is consistent? Nope. I wasn't defending the quirks of the API, or trying give the impression that I thought it was consistent. My goal was simply to try and explain what the example Alan gave you was actually doing, and why/how it worked -- so that you weren't left with the string "fl=field(eff_field_name)" as some magical, inexplicable, black-box. -Hoss
Re: external file field and fl parameter
Chris, the confusion from my perspective is the general inconsistency and natural growth of the API which is somewhat expected based upon its history. Obviously this isnt sql, there is no ansi body defining the query language. I understand well the difference between stored, indexed etc. Going off of the apache wiki docs (which perhaps is not the correct place to go for documentation buts its what google gives me :-}) http://wiki.apache.org/solr/CommonQueryParameters The fl parameter doesnt actually mention stored. It actually gives reference to a few "special" values, such as "id" and "score". Neither of them are technically "stored" fields either, but afaik you dont need to use "field(id), field(score)" for those. Can you honestly say that is consistent? So On Jul 17, 2013, at 5:30 PM, Chris Hostetter wrote: > > : Yes that worked, thanks Alan. The consistency of this api is "challenging". > > It's important to understand what's happening here. > > fl, by default, only returns "stored" fields -- but you can also request > "psuedo-fields" such as the results of functions, or the result of a "Doc > Transformer" ... > > http://wiki.apache.org/solr/CommonQueryParameters#fl > > On the other side of things, the ExtenalFileField has some very special > behavior that allows it to be used as the input to a function, but it does > not act as a true stored or indexed field -- it's completley external > to the index... > > https://lucene.apache.org/solr/4_3_1/solr-core/org/apache/solr/schema/ExternalFileField.html > > The syntax Alan suggested tells the solr response writer to generate a > psuedo-field for each doc in the response which contains the results of a > function call -- that function just so happens to be a simple field() > function that returns the numeric value of the specified field name -- > which works for EFF since (as mentioned before) EFF can be used as the > input to any function. > > > -Hoss >
Re: external file field and fl parameter
: Yes that worked, thanks Alan. The consistency of this api is "challenging". It's important to understand what's happening here. fl, by default, only returns "stored" fields -- but you can also request "psuedo-fields" such as the results of functions, or the result of a "Doc Transformer" ... http://wiki.apache.org/solr/CommonQueryParameters#fl On the other side of things, the ExtenalFileField has some very special behavior that allows it to be used as the input to a function, but it does not act as a true stored or indexed field -- it's completley external to the index... https://lucene.apache.org/solr/4_3_1/solr-core/org/apache/solr/schema/ExternalFileField.html The syntax Alan suggested tells the solr response writer to generate a psuedo-field for each doc in the response which contains the results of a function call -- that function just so happens to be a simple field() function that returns the numeric value of the specified field name -- which works for EFF since (as mentioned before) EFF can be used as the input to any function. -Hoss
Re: external file field and fl parameter
Yes that worked, thanks Alan. The consistency of this api is "challenging".
C
On Jul 14, 2013, at 11:03 AM, Alan Woodward wrote:
> Hi Chris,
>
> Try wrapping the field name in a field() function in your fl parameter list,
> like so:
> fl=field(eff_field_name)
>
> Alan Woodward
> www.flax.co.uk
>
>
> On 14 Jul 2013, at 18:41, Chris Collins wrote:
>
>> Why would I be re-indexing an external file field? The whole purpose is that
>> its brought in at runtime and not part of the index?
>>
>> C
>> On Jul 14, 2013, at 10:13 AM, Shawn Heisey wrote:
>>
>>> On 7/14/2013 7:05 AM, Chris Collins wrote:
Yep I did switch on stored=true in the field type. I was able to confirm
a few ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling
the vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will
continue dissecting.
>>>
>>> Did you reindex when you changed the schema? Sorting uses indexed
>>> values, not stored values. The fl parameter requires the stored values.
>>> These are separate within the index, and one cannot substitute for the
>>> other. If you didn't reindex, then you won't have the stored values for
>>> existing documents.
>>>
>>> http://wiki.apache.org/solr/HowToReindex
>>>
>>> Thanks,
>>> Shawn
>>>
>>>
>>
>
Re: external file field and fl parameter
Hi Chris,
Try wrapping the field name in a field() function in your fl parameter list,
like so:
fl=field(eff_field_name)
Alan Woodward
www.flax.co.uk
On 14 Jul 2013, at 18:41, Chris Collins wrote:
> Why would I be re-indexing an external file field? The whole purpose is that
> its brought in at runtime and not part of the index?
>
> C
> On Jul 14, 2013, at 10:13 AM, Shawn Heisey wrote:
>
>> On 7/14/2013 7:05 AM, Chris Collins wrote:
>>> Yep I did switch on stored=true in the field type. I was able to confirm a
>>> few ways that there are values for the eff by two methods:
>>>
>>> 1) changing desc to asc produced drastically different results.
>>>
>>> 2) debugging FileFloatSource the following was getting triggered filling
>>> the vals array:
>>> while ((doc = docsEnum.nextDoc()) !=
>>> DocIdSetIterator.NO_MORE_DOCS)
>>> {
>>> vals[doc] = fval;
>>> }
>>>
>>> At least by you asking these questions I guess it should work. I will
>>> continue dissecting.
>>
>> Did you reindex when you changed the schema? Sorting uses indexed
>> values, not stored values. The fl parameter requires the stored values.
>> These are separate within the index, and one cannot substitute for the
>> other. If you didn't reindex, then you won't have the stored values for
>> existing documents.
>>
>> http://wiki.apache.org/solr/HowToReindex
>>
>> Thanks,
>> Shawn
>>
>>
>
Re: external file field and fl parameter
Why would I be re-indexing an external file field? The whole purpose is that
its brought in at runtime and not part of the index?
C
On Jul 14, 2013, at 10:13 AM, Shawn Heisey wrote:
> On 7/14/2013 7:05 AM, Chris Collins wrote:
>> Yep I did switch on stored=true in the field type. I was able to confirm a
>> few ways that there are values for the eff by two methods:
>>
>> 1) changing desc to asc produced drastically different results.
>>
>> 2) debugging FileFloatSource the following was getting triggered filling the
>> vals array:
>> while ((doc = docsEnum.nextDoc()) !=
>> DocIdSetIterator.NO_MORE_DOCS)
>>{
>>vals[doc] = fval;
>>}
>>
>> At least by you asking these questions I guess it should work. I will
>> continue dissecting.
>
> Did you reindex when you changed the schema? Sorting uses indexed
> values, not stored values. The fl parameter requires the stored values.
> These are separate within the index, and one cannot substitute for the
> other. If you didn't reindex, then you won't have the stored values for
> existing documents.
>
> http://wiki.apache.org/solr/HowToReindex
>
> Thanks,
> Shawn
>
>
Re: external file field and fl parameter
On 7/14/2013 7:05 AM, Chris Collins wrote:
> Yep I did switch on stored=true in the field type. I was able to confirm a
> few ways that there are values for the eff by two methods:
>
> 1) changing desc to asc produced drastically different results.
>
> 2) debugging FileFloatSource the following was getting triggered filling the
> vals array:
> while ((doc = docsEnum.nextDoc()) !=
> DocIdSetIterator.NO_MORE_DOCS)
> {
> vals[doc] = fval;
> }
>
> At least by you asking these questions I guess it should work. I will
> continue dissecting.
Did you reindex when you changed the schema? Sorting uses indexed
values, not stored values. The fl parameter requires the stored values.
These are separate within the index, and one cannot substitute for the
other. If you didn't reindex, then you won't have the stored values for
existing documents.
http://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
Re: external file field and fl parameter
Yep I did switch on stored=true in the field type. I was able to confirm a few
ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling the
vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will continue
dissecting.
Thanks Erick.
C
On Jul 14, 2013, at 5:16 AM, Erick Erickson wrote:
> Did you store the field? I.e. set stored="true"? And does the EFF contain
> values for the docs you're returning?
>
> Best
> Erick
>
> On Sun, Jul 14, 2013 at 3:32 AM, Chris Collins wrote:
>> I am playing with external file field for sorting. I created a dynamic
>> field using the ExternalFileField type.
>>
>> I naively assumed that the "fl" argument would allow me to return the value
>> the external field but doesnt seem to do so.
>>
>> For instance I have a defined a dynamic field:
>>
>> *_efloat
>>
>> then I used:
>>
>> sort=foo_efloat desc
>> fl=foo_efloat, score, description
>>
>> I get the score and description but the foo_efloat seems to be missing in
>> action.
>>
>>
>> Thoughts?
>>
>> C
>>
>
Re: external file field and fl parameter
Did you store the field? I.e. set stored="true"? And does the EFF contain values for the docs you're returning? Best Erick On Sun, Jul 14, 2013 at 3:32 AM, Chris Collins wrote: > I am playing with external file field for sorting. I created a dynamic field > using the ExternalFileField type. > > I naively assumed that the "fl" argument would allow me to return the value > the external field but doesnt seem to do so. > > For instance I have a defined a dynamic field: > > *_efloat > > then I used: > > sort=foo_efloat desc > fl=foo_efloat, score, description > > I get the score and description but the foo_efloat seems to be missing in > action. > > > Thoughts? > > C >
