Computing Results So That They are Returned in Search Results
I'd like to throw out a design question and see if its possible to solve this with Solr. I have a set of data that is computed that I'd like to make searchable. Ideally, I'd like to have all documents indexed and call it the day, but the nature of the data is such that it needs to be computed given a definition. I'm interested in searching on definitions and then creating results on the fly that are calculated based on something embedded in the definition. Is it possible to embed this calculation login into Solr's result handling process? I know this sounds exotic, but the nature of the data is such that I can't index these calculated documents because I don't know what the boundary is and specifiying an arbitrary number isn't ideal. Has anyone run across something like this? Thanks, Alejandr
Re: Computing Results So That They are Returned in Search Results
You could create a custom value source and then use it in a function query embedded in your return fields list (fl). So, the function query could use a function (value source) that takes a field, fetches its value, performs some arbitrary calculation, and then returns that value. fl=id,name,my-func(field1),my-func(field2) -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 10:10 AM To: solr-user@lucene.apache.org Subject: Computing Results So That They are Returned in Search Results I'd like to throw out a design question and see if its possible to solve this with Solr. I have a set of data that is computed that I'd like to make searchable. Ideally, I'd like to have all documents indexed and call it the day, but the nature of the data is such that it needs to be computed given a definition. I'm interested in searching on definitions and then creating results on the fly that are calculated based on something embedded in the definition. Is it possible to embed this calculation login into Solr's result handling process? I know this sounds exotic, but the nature of the data is such that I can't index these calculated documents because I don't know what the boundary is and specifiying an arbitrary number isn't ideal. Has anyone run across something like this? Thanks, Alejandr
Re: Computing Results So That They are Returned in Search Results
Sounds really close to what I'm looking for, but this sounds like it would result in a new field on a document (or a new value for a field defined to hold the result of a function). Would it be possible for a function query to produce a new document so that I can associate the computed value with it? Thanks, Alejandro On Wed, Oct 30, 2013 at 12:05 PM, Jack Krupansky j...@basetechnology.comwrote: You could create a custom value source and then use it in a function query embedded in your return fields list (fl). So, the function query could use a function (value source) that takes a field, fetches its value, performs some arbitrary calculation, and then returns that value. fl=id,name,my-func(field1),my-**func(field2) -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 10:10 AM To: solr-user@lucene.apache.org Subject: Computing Results So That They are Returned in Search Results I'd like to throw out a design question and see if its possible to solve this with Solr. I have a set of data that is computed that I'd like to make searchable. Ideally, I'd like to have all documents indexed and call it the day, but the nature of the data is such that it needs to be computed given a definition. I'm interested in searching on definitions and then creating results on the fly that are calculated based on something embedded in the definition. Is it possible to embed this calculation login into Solr's result handling process? I know this sounds exotic, but the nature of the data is such that I can't index these calculated documents because I don't know what the boundary is and specifiying an arbitrary number isn't ideal. Has anyone run across something like this? Thanks, Alejandr
Re: Computing Results So That They are Returned in Search Results
A function query is simply returning a calculated result based on existing data - no new fields required. Did you actually want to precompute a value, store it in the index, and then query on it? If so, you could do that indexing with a custom or scripted update processor. Flesh out an example of exactly what you want. -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 12:46 PM To: solr-user@lucene.apache.org Subject: Re: Computing Results So That They are Returned in Search Results Sounds really close to what I'm looking for, but this sounds like it would result in a new field on a document (or a new value for a field defined to hold the result of a function). Would it be possible for a function query to produce a new document so that I can associate the computed value with it? Thanks, Alejandro On Wed, Oct 30, 2013 at 12:05 PM, Jack Krupansky j...@basetechnology.comwrote: You could create a custom value source and then use it in a function query embedded in your return fields list (fl). So, the function query could use a function (value source) that takes a field, fetches its value, performs some arbitrary calculation, and then returns that value. fl=id,name,my-func(field1),my-**func(field2) -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 10:10 AM To: solr-user@lucene.apache.org Subject: Computing Results So That They are Returned in Search Results I'd like to throw out a design question and see if its possible to solve this with Solr. I have a set of data that is computed that I'd like to make searchable. Ideally, I'd like to have all documents indexed and call it the day, but the nature of the data is such that it needs to be computed given a definition. I'm interested in searching on definitions and then creating results on the fly that are calculated based on something embedded in the definition. Is it possible to embed this calculation login into Solr's result handling process? I know this sounds exotic, but the nature of the data is such that I can't index these calculated documents because I don't know what the boundary is and specifiying an arbitrary number isn't ideal. Has anyone run across something like this? Thanks, Alejandr
Re: Computing Results So That They are Returned in Search Results
Also note that function queries only return numbers (given their origin in scoring). They cannot be used to create virtual string or text fields. Upayavira On Wed, Oct 30, 2013, at 05:19 PM, Jack Krupansky wrote: A function query is simply returning a calculated result based on existing data - no new fields required. Did you actually want to precompute a value, store it in the index, and then query on it? If so, you could do that indexing with a custom or scripted update processor. Flesh out an example of exactly what you want. -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 12:46 PM To: solr-user@lucene.apache.org Subject: Re: Computing Results So That They are Returned in Search Results Sounds really close to what I'm looking for, but this sounds like it would result in a new field on a document (or a new value for a field defined to hold the result of a function). Would it be possible for a function query to produce a new document so that I can associate the computed value with it? Thanks, Alejandro On Wed, Oct 30, 2013 at 12:05 PM, Jack Krupansky j...@basetechnology.comwrote: You could create a custom value source and then use it in a function query embedded in your return fields list (fl). So, the function query could use a function (value source) that takes a field, fetches its value, performs some arbitrary calculation, and then returns that value. fl=id,name,my-func(field1),my-**func(field2) -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 10:10 AM To: solr-user@lucene.apache.org Subject: Computing Results So That They are Returned in Search Results I'd like to throw out a design question and see if its possible to solve this with Solr. I have a set of data that is computed that I'd like to make searchable. Ideally, I'd like to have all documents indexed and call it the day, but the nature of the data is such that it needs to be computed given a definition. I'm interested in searching on definitions and then creating results on the fly that are calculated based on something embedded in the definition. Is it possible to embed this calculation login into Solr's result handling process? I know this sounds exotic, but the nature of the data is such that I can't index these calculated documents because I don't know what the boundary is and specifiying an arbitrary number isn't ideal. Has anyone run across something like this? Thanks, Alejandr
Re: Computing Results So That They are Returned in Search Results
So here is my use case with a little more detail. I'm working with recurring events. Each event has an expression associated with it that defines its recurrence pattern. For example, monthly, daily, yearly... The event has metadata associated with it that is searchable. When a user performs a search, they can match on various metadata fields, but the query can also span a range of dates. If a match occurs, I'd like to unwind the expression into the instances specified by the pattern and return these virtual instances as results. Right now, I'm post processing data to hammer out the results that fit the window of time specified in the query, but this moves sorting and pagination out of the Solr tier. I'd like to see if I can get it to stay there :) Post processing also prohibits me from faceting which would be extremely useful. I'm trying to avoid heavy post processing if I can. Given the nature of the data, its not really feasible for me to pre-assemble instance data and index since I don't know the window of time a user will be looking at. Thanks, Alejandro On Wed, Oct 30, 2013 at 6:35 PM, Upayavira u...@odoko.co.uk wrote: Also note that function queries only return numbers (given their origin in scoring). They cannot be used to create virtual string or text fields. Upayavira On Wed, Oct 30, 2013, at 05:19 PM, Jack Krupansky wrote: A function query is simply returning a calculated result based on existing data - no new fields required. Did you actually want to precompute a value, store it in the index, and then query on it? If so, you could do that indexing with a custom or scripted update processor. Flesh out an example of exactly what you want. -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 12:46 PM To: solr-user@lucene.apache.org Subject: Re: Computing Results So That They are Returned in Search Results Sounds really close to what I'm looking for, but this sounds like it would result in a new field on a document (or a new value for a field defined to hold the result of a function). Would it be possible for a function query to produce a new document so that I can associate the computed value with it? Thanks, Alejandro On Wed, Oct 30, 2013 at 12:05 PM, Jack Krupansky j...@basetechnology.comwrote: You could create a custom value source and then use it in a function query embedded in your return fields list (fl). So, the function query could use a function (value source) that takes a field, fetches its value, performs some arbitrary calculation, and then returns that value. fl=id,name,my-func(field1),my-**func(field2) -- Jack Krupansky -Original Message- From: Alejandro Calbazana Sent: Wednesday, October 30, 2013 10:10 AM To: solr-user@lucene.apache.org Subject: Computing Results So That They are Returned in Search Results I'd like to throw out a design question and see if its possible to solve this with Solr. I have a set of data that is computed that I'd like to make searchable. Ideally, I'd like to have all documents indexed and call it the day, but the nature of the data is such that it needs to be computed given a definition. I'm interested in searching on definitions and then creating results on the fly that are calculated based on something embedded in the definition. Is it possible to embed this calculation login into Solr's result handling process? I know this sounds exotic, but the nature of the data is such that I can't index these calculated documents because I don't know what the boundary is and specifiying an arbitrary number isn't ideal. Has anyone run across something like this? Thanks, Alejandr
