Re: external file field and fl parameter
: Yes that worked, thanks Alan. The consistency of this api is challenging. It's important to understand what's happening here. fl, by default, only returns stored fields -- but you can also request psuedo-fields such as the results of functions, or the result of a Doc Transformer ... http://wiki.apache.org/solr/CommonQueryParameters#fl On the other side of things, the ExtenalFileField has some very special behavior that allows it to be used as the input to a function, but it does not act as a true stored or indexed field -- it's completley external to the index... https://lucene.apache.org/solr/4_3_1/solr-core/org/apache/solr/schema/ExternalFileField.html The syntax Alan suggested tells the solr response writer to generate a psuedo-field for each doc in the response which contains the results of a function call -- that function just so happens to be a simple field() function that returns the numeric value of the specified field name -- which works for EFF since (as mentioned before) EFF can be used as the input to any function. -Hoss
Re: external file field and fl parameter
Chris, the confusion from my perspective is the general inconsistency and natural growth of the API which is somewhat expected based upon its history. Obviously this isnt sql, there is no ansi body defining the query language. I understand well the difference between stored, indexed etc. Going off of the apache wiki docs (which perhaps is not the correct place to go for documentation buts its what google gives me :-}) http://wiki.apache.org/solr/CommonQueryParameters The fl parameter doesnt actually mention stored. It actually gives reference to a few special values, such as id and score. Neither of them are technically stored fields either, but afaik you dont need to use field(id), field(score) for those. Can you honestly say that is consistent? So On Jul 17, 2013, at 5:30 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : Yes that worked, thanks Alan. The consistency of this api is challenging. It's important to understand what's happening here. fl, by default, only returns stored fields -- but you can also request psuedo-fields such as the results of functions, or the result of a Doc Transformer ... http://wiki.apache.org/solr/CommonQueryParameters#fl On the other side of things, the ExtenalFileField has some very special behavior that allows it to be used as the input to a function, but it does not act as a true stored or indexed field -- it's completley external to the index... https://lucene.apache.org/solr/4_3_1/solr-core/org/apache/solr/schema/ExternalFileField.html The syntax Alan suggested tells the solr response writer to generate a psuedo-field for each doc in the response which contains the results of a function call -- that function just so happens to be a simple field() function that returns the numeric value of the specified field name -- which works for EFF since (as mentioned before) EFF can be used as the input to any function. -Hoss
Re: external file field and fl parameter
: reference to a few special values, such as id and score. Neither : of them are technically stored fields either, but afaik you dont need : to use field(id), field(score) for those. Can you honestly say that : is consistent? Nope. I wasn't defending the quirks of the API, or trying give the impression that I thought it was consistent. My goal was simply to try and explain what the example Alan gave you was actually doing, and why/how it worked -- so that you weren't left with the string fl=field(eff_field_name) as some magical, inexplicable, black-box. -Hoss
external file field and fl parameter
I am playing with external file field for sorting. I created a dynamic field using the ExternalFileField type. I naively assumed that the fl argument would allow me to return the value the external field but doesnt seem to do so. For instance I have a defined a dynamic field: *_efloat then I used: sort=foo_efloat desc fl=foo_efloat, score, description I get the score and description but the foo_efloat seems to be missing in action. Thoughts? C
Re: external file field and fl parameter
Did you store the field? I.e. set stored=true? And does the EFF contain values for the docs you're returning? Best Erick On Sun, Jul 14, 2013 at 3:32 AM, Chris Collins ch...@geekychris.com wrote: I am playing with external file field for sorting. I created a dynamic field using the ExternalFileField type. I naively assumed that the fl argument would allow me to return the value the external field but doesnt seem to do so. For instance I have a defined a dynamic field: *_efloat then I used: sort=foo_efloat desc fl=foo_efloat, score, description I get the score and description but the foo_efloat seems to be missing in action. Thoughts? C
Re: external file field and fl parameter
Yep I did switch on stored=true in the field type. I was able to confirm a few
ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling the
vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will continue
dissecting.
Thanks Erick.
C
On Jul 14, 2013, at 5:16 AM, Erick Erickson erickerick...@gmail.com wrote:
Did you store the field? I.e. set stored=true? And does the EFF contain
values for the docs you're returning?
Best
Erick
On Sun, Jul 14, 2013 at 3:32 AM, Chris Collins ch...@geekychris.com wrote:
I am playing with external file field for sorting. I created a dynamic
field using the ExternalFileField type.
I naively assumed that the fl argument would allow me to return the value
the external field but doesnt seem to do so.
For instance I have a defined a dynamic field:
*_efloat
then I used:
sort=foo_efloat desc
fl=foo_efloat, score, description
I get the score and description but the foo_efloat seems to be missing in
action.
Thoughts?
C
Re: external file field and fl parameter
On 7/14/2013 7:05 AM, Chris Collins wrote:
Yep I did switch on stored=true in the field type. I was able to confirm a
few ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling the
vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will
continue dissecting.
Did you reindex when you changed the schema? Sorting uses indexed
values, not stored values. The fl parameter requires the stored values.
These are separate within the index, and one cannot substitute for the
other. If you didn't reindex, then you won't have the stored values for
existing documents.
http://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
Re: external file field and fl parameter
Why would I be re-indexing an external file field? The whole purpose is that
its brought in at runtime and not part of the index?
C
On Jul 14, 2013, at 10:13 AM, Shawn Heisey s...@elyograg.org wrote:
On 7/14/2013 7:05 AM, Chris Collins wrote:
Yep I did switch on stored=true in the field type. I was able to confirm a
few ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling the
vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will
continue dissecting.
Did you reindex when you changed the schema? Sorting uses indexed
values, not stored values. The fl parameter requires the stored values.
These are separate within the index, and one cannot substitute for the
other. If you didn't reindex, then you won't have the stored values for
existing documents.
http://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
Re: external file field and fl parameter
Hi Chris,
Try wrapping the field name in a field() function in your fl parameter list,
like so:
fl=field(eff_field_name)
Alan Woodward
www.flax.co.uk
On 14 Jul 2013, at 18:41, Chris Collins wrote:
Why would I be re-indexing an external file field? The whole purpose is that
its brought in at runtime and not part of the index?
C
On Jul 14, 2013, at 10:13 AM, Shawn Heisey s...@elyograg.org wrote:
On 7/14/2013 7:05 AM, Chris Collins wrote:
Yep I did switch on stored=true in the field type. I was able to confirm a
few ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling
the vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will
continue dissecting.
Did you reindex when you changed the schema? Sorting uses indexed
values, not stored values. The fl parameter requires the stored values.
These are separate within the index, and one cannot substitute for the
other. If you didn't reindex, then you won't have the stored values for
existing documents.
http://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
Re: external file field and fl parameter
Yes that worked, thanks Alan. The consistency of this api is challenging.
C
On Jul 14, 2013, at 11:03 AM, Alan Woodward a...@flax.co.uk wrote:
Hi Chris,
Try wrapping the field name in a field() function in your fl parameter list,
like so:
fl=field(eff_field_name)
Alan Woodward
www.flax.co.uk
On 14 Jul 2013, at 18:41, Chris Collins wrote:
Why would I be re-indexing an external file field? The whole purpose is that
its brought in at runtime and not part of the index?
C
On Jul 14, 2013, at 10:13 AM, Shawn Heisey s...@elyograg.org wrote:
On 7/14/2013 7:05 AM, Chris Collins wrote:
Yep I did switch on stored=true in the field type. I was able to confirm
a few ways that there are values for the eff by two methods:
1) changing desc to asc produced drastically different results.
2) debugging FileFloatSource the following was getting triggered filling
the vals array:
while ((doc = docsEnum.nextDoc()) !=
DocIdSetIterator.NO_MORE_DOCS)
{
vals[doc] = fval;
}
At least by you asking these questions I guess it should work. I will
continue dissecting.
Did you reindex when you changed the schema? Sorting uses indexed
values, not stored values. The fl parameter requires the stored values.
These are separate within the index, and one cannot substitute for the
other. If you didn't reindex, then you won't have the stored values for
existing documents.
http://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
