Can I take Solr role (aggregator, searcher) into consideration at start-up?
Hello. My company uses Solr-4.10 in a distributed environment. I have written a SearchComponent which contains a cache which is loaded at start-up. The cache is only used on the searchers, never on the aggregators. Is there some way I can signal that the cache should be loaded only if the Solr instance in question is a searcher, not if it is an aggregator? At present the cache is loaded in the inform() function; is that the wrong place for it? Thanks in advance for your help. Jitka -- View this message in context: http://lucene.472066.n3.nabble.com/Can-I-take-Solr-role-aggregator-searcher-into-consideration-at-start-up-tp4266197.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: using data from external service in Solr: value source or auxiliary core?
Charlie, thank you for replying and for the link to the blog about XJoin. I am still concerned about performance and caching. Our environment is distributed and the external service would not be running on localhost. In our still experimental set-up, at least, the call to the service is much too slow to be a viable option without caching. I am new to Solr caching, hence all the questions in my original post. Thanks again, Jitka On Mon, Mar 14, 2016 at 2:07 AM, Charlie Hull-3 [via Lucene] < ml-node+s472066n4263572...@n3.nabble.com> wrote: > On 11/03/2016 17:36, Jitka wrote: > > Hello. Our company uses Solr-4.10 in a distributed environment. We are > > considering how best to customize results based on user preferences, > > information about which is obtained from an external service. At > present > > the preferences can be expressed as filters, but eventually we might > want to > > introduce boosts as well. > > You might want to consider XJoin as well: here's a blog post we wrote on > filtering using price discount data from an external source > > http://www.flax.co.uk/blog/2016/01/25/xjoin-solr-part-1-filtering-using-price-discount-data/ > > > > Cheers > > Charlie > > > > > The options we are considering are > > > > (1) a value source whose FunctionValues() function would make use of > data > > from the external service, and > > > > (2) a separate core containing docs whose primary key is user_id and > whose > > other fields represent data from the service. We would create filter > > queries based on joins with this core. > > > > Any suggestions would be most welcome. Are we missing an obvious > > alternative? > > > > I assume that we would want to cache data from the service (in option > (1)) > > or the join (in option (2)), and possibly also the values of the value > > source's FunctionValues() function in option (1). Would it make > > sense to use SolrCaches for this purpose and register them in > > solrconfig.xml? If so, or if not, how would we ensure that these > caches > > could be updated whenever the service is updated? How about ensuring > that > > the built-in caches don't prevent Solr from even looking at the custom > > caches? We could use the 'cache=false' option for the filterCache; > would we > > have to worry about other ones? > > > > I see how how a join on a second core could support a filter, but how > could > > we use a second core to support boosts? I suppose that if worse came to > > worst we could forgo the join and simply make a call to the second core > on > > localhost to retrieve data to be used in a value source. Is that a > > practical solution? > > > > Thanks in advance for your time and advice. > > > > Jitka > > > > > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/using-data-from-external-service-in-Solr-value-source-or-auxiliary-core-tp4263334.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > -- > Charlie Hull > Flax - Open Source Enterprise Search > > tel/fax: +44 (0)8700 118334 > mobile: +44 (0)7767 825828 > web: www.flax.co.uk > > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/using-data-from-external-service-in-Solr-value-source-or-auxiliary-core-tp4263334p4263572.html > To unsubscribe from using data from external service in Solr: value source > or auxiliary core?, click here > <http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code=4263334=aml0a2ExOEBnbWFpbC5jb218NDI2MzMzNHwyMTIwMTk5NzA1> > . > NAML > <http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/using-data-from-external-service-in-Solr-value-source-or-auxiliary-core-tp4263334p4263924.html Sent from the Solr - User mailing list archive at Nabble.com.
using data from external service in Solr: value source or auxiliary core?
Hello. Our company uses Solr-4.10 in a distributed environment. We are considering how best to customize results based on user preferences, information about which is obtained from an external service. At present the preferences can be expressed as filters, but eventually we might want to introduce boosts as well. The options we are considering are (1) a value source whose FunctionValues() function would make use of data from the external service, and (2) a separate core containing docs whose primary key is user_id and whose other fields represent data from the service. We would create filter queries based on joins with this core. Any suggestions would be most welcome. Are we missing an obvious alternative? I assume that we would want to cache data from the service (in option (1)) or the join (in option (2)), and possibly also the values of the value source's FunctionValues() function in option (1). Would it make sense to use SolrCaches for this purpose and register them in solrconfig.xml? If so, or if not, how would we ensure that these caches could be updated whenever the service is updated? How about ensuring that the built-in caches don't prevent Solr from even looking at the custom caches? We could use the 'cache=false' option for the filterCache; would we have to worry about other ones? I see how how a join on a second core could support a filter, but how could we use a second core to support boosts? I suppose that if worse came to worst we could forgo the join and simply make a call to the second core on localhost to retrieve data to be used in a value source. Is that a practical solution? Thanks in advance for your time and advice. Jitka -- View this message in context: http://lucene.472066.n3.nabble.com/using-data-from-external-service-in-Solr-value-source-or-auxiliary-core-tp4263334.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: search component needs access to results of previous component
Good question, Mikhail. I started by putting that logic in the prepare() function by sheer force of habit, but have since moved it first to distributedProcess() (my system is sharded) and now to handleResponses() (as in MoreLikeThisComponent.java, which I am mimicking without understanding). As of last night, the new query is being generated but I am not yet getting back responses for it. I have created a new ShardRequest.PURPOSE_XXX value for myself but don't know whether that was a good idea. -- View this message in context: http://lucene.472066.n3.nabble.com/search-component-needs-access-to-results-of-previous-component-tp4138335p4139593.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: search component needs access to results of previous component
Thanks for your reply. I'll check out that link. -- View this message in context: http://lucene.472066.n3.nabble.com/search-component-needs-access-to-results-of-previous-component-tp4138335p4139409.html Sent from the Solr - User mailing list archive at Nabble.com.
search component needs access to results of previous component
Hello and thanks for reading my question. If our high-level search handler doesn't get enough results back from a Solr query, it tweaks the query, re-sends to Solr, and combines the result sets from the two queries. I would like to modify our Solr set-up so that Solr itself could handle the deciding, tweaking, re-sending and combining within a single call. My current thought is to create a FollowUpQueryComponent that would live in the last-components section of the definition of our usual requestHandler in solrconfig.xml. The new component would take a look at the results of the default QueryComponent and decide whether or not to tweak the query and bring about another search. I tried to do the looking-and-tweaking at the aggregator level in the new component's prepare() function, but ran into two problems. 1) The new component's prepare() function is called before the default QueryComponent has done its work, so it doesn't have the information it needs to decide whether the new call will be needed or not, and 2) Even if I postpone the decision until after the default QueryComponent's results are in, as far as I can tell those results stay on the shards and are not available to the aggregator. Either I am going about this all wrong, or these must be standard problems for creators of custom QueryComponents. Are there standard solutions? Any feedback would be much appreciated. Thanks, Jitka -- View this message in context: http://lucene.472066.n3.nabble.com/search-component-needs-access-to-results-of-previous-component-tp4138335.html Sent from the Solr - User mailing list archive at Nabble.com.
