Re: SolrCloud - Service unavailable
Thanks Shawn, the links will be useful. I am still not sure if its related due to a timeout because the 503 error is coming from Tomcat, which means the requests are going through. I can access the Solr admin panel and I see a message saying the core was not initialized. Thanks, Indika On 18 September 2013 21:27, Shawn Heisey wrote: > On 9/18/2013 8:12 AM, Indika Tantrigoda wrote: > > I am using 3 Solr instances behind an Amazon ELB with 1 shared. Serving > > data via Solr works as expected, however I noticed a few times a 503 > error > > was poping up from the applications accessing Solr. Accessing Solr is > done > > via the AWS ELB. > > > > 3 Zookeeper instances also run on the same instances as Solr on a > separate > > disk. > > > > Solr version is 4.4. > > > > This issue seems to be a sporadic issue. Has anyone else observed this > kind > > of behavior ? > > What kind of session timeouts have you configured on the amazon load > balancer? I've never used amazon services, but hopefully this is > configurable. If the timeout is low enough, it could be just that the > request is taking longer than that to execute. You may need to increase > that timeout. > > Aside from general performance issues, one thing that can cause long > request times is stop-the-world Java garbage collections. This can be a > sign that your heap is too small, too large, or that your garbage > collection hasn't been properly tuned. > > http://wiki.apache.org/solr/SolrPerformanceProblems#GC_pause_problems > > http://wiki.apache.org/solr/SolrPerformanceProblems#How_much_heap_space_do_I_need.3F > > That same wiki page has another section about the OS disk cache. Not > having enough memory for this is the cause of a lot of performance issues: > > http://wiki.apache.org/solr/SolrPerformanceProblems#OS_Disk_Cache > > Thanks, > Shawn > >
SolrCloud - Service unavailable
Hi All, I am using 3 Solr instances behind an Amazon ELB with 1 shared. Serving data via Solr works as expected, however I noticed a few times a 503 error was poping up from the applications accessing Solr. Accessing Solr is done via the AWS ELB. 3 Zookeeper instances also run on the same instances as Solr on a separate disk. Solr version is 4.4. This issue seems to be a sporadic issue. Has anyone else observed this kind of behavior ? Thanks, Indika
Re: Solr Range Queries with Field value
Yes, the SQL statement is what I am trying to achieve. As for the merchant_end_of_day_in_utc_epoch, we map the time to start of epoch and convert that to UTC, so that all the merchants are in the same timezone which would make it easier to query for open ones. For the use case when we need to determine if a merchant is currently open now or in the future (within the same day). Therefore when converting the start/end times to epoch and UTC a session (i.e. start time to end time) might get spilled over past 12 midnight. After some research the following syntax worked start_time_utc_epoch:[1970-01-01T00:00:00Z TO _val_:"merchant_end_of_day_in_utc_epoch"]) Thanks, Indika On 1 May 2013 00:52, Arun Rangarajan wrote: > Erick, > > I believe Indika wants to do this SQL WHERE clause in Solr: > WHERE start_time_utc_epoch >= '1970-01-01T00:00:00Z' AND > start_time_utc_epoch > <= merchant_end_of_day_in_utc_epoch > > > On Tue, Apr 30, 2013 at 11:49 AM, Erick Erickson >wrote: > > > Could you define your use-case in some more detail? On the > > surface, this query doesn't really make a lot of sense. How > > would merchant_end_of_day_in_utc_epoch be determined? > > Presumably there are zillions of values across your index for > > this value, depending on the document. Which one should be > > used? > > > > Best > > Erick > > > > On Mon, Apr 29, 2013 at 8:34 PM, Indika Tantrigoda > > wrote: > > > Hi All, > > > > > > I'd like to know if its possible to use a field value in a Solr range > > query > > > ? Something similar to start_time_utc_epoch:[1970-01-01T00:00:00Z TO > > > merchant_end_of_day_in_utc_epoch]) > > > > > > merchant_end_of_day_in_utc_epoch is an indexed field. > > > > > > I am using Solr 4. > > > > > > Thanks, > > > Indika > > >
Re: Stored values and date math query
Thanks for the insights to the query Hoss. I am going to try out the
methods you highlighted.
Thanks,
Indika
On 3 March 2013 01:19, Chris Hostetter wrote:
>
> : sessionAvailableNowQuery = {!edismax}(start_time:[* TO
> : 1970-01-01T12:37:030Z] AND end_time:[1970-01-01T12:37:030Z +
> : (_val_:order_prep_time)MINUTES TO *] AND consumers:[1 TO *] AND
> : session_time_range_available:true)
>
> you can't embed the valuef of a field inside a query string like that (the
> "_val_" hook only lets you embed a function in a place where a query
> clause would normally be expected)
>
> : Is it possible to retrive an integer value from the index and pass it on
> it
> : a date math query ? Is there anything else that needs to be in the query
> ?
>
> not using the date match syntax, but you can use the function syntax to
> write a function that performs a math equation in which two indexed fields
> are the input - and there is an "ms()" function which can be used to get
> the milliseconds since epoch from a date value (either in an indexed field
> or a date match expression), or the diff in milliseconds between two date
> values.
>
> so you should be able to do something like this (untested) in a
> function which should return a positive value if "end_time" is at least
> "order_prep_time" minutes past your input date...
>
> sub(ms(end_time, 1970-01-01T12:37:030Z),
> product(order_prep_time,6))
>
> ..and then you can use that function in the frange parser to only match
> documents with positive values...
>
> fq={frange
> l=0}sub(ms(end_time,1970-01-01T12:37:030Z),product(order_prep_time,6))
>
>
>
> -Hoss
>
Re: Stored values and date math query
Hi Erick,
Thanks for the response. The the terms are indexed. I will need to dig
deeper and see there the issue is, it might be a correct syntax but I may
be using it incorrectly. In the meantime I have added a new column that
is end_time - order_prep_time and validate if a session is available
against that. Basically move the order_prep_time to the other side of the
equation.
Thanks,
Indika
On 28 February 2013 18:45, Erick Erickson wrote:
> Just to check, your order_prep_time is _indexed_ too, right? It's
> a bit confusing but anything you use in your function queries will
> be from indexed terms, not stored ones
>
> Best
> Erick
>
>
> On Tue, Feb 26, 2013 at 4:05 PM, Indika Tantrigoda >wrote:
>
> > Hi All,
> >
> > I am trying to use a stored value in the index and add it to a date
> > component as follows
> >
> > sessionAvailableNowQuery = {!edismax}(start_time:[* TO
> > 1970-01-01T12:37:030Z] AND end_time:[1970-01-01T12:37:030Z +
> > (_val_:order_prep_time)MINUTES TO *] AND consumers:[1 TO *] AND
> > session_time_range_available:true)
> >
> > (order_prep_time is the stored value)
> >
> > However, when doing so causes the query to return incorrect results. The
> > query is part of a if/else query present in the fields (fl) list
> >
> > _session_available_now_:if(exists(query($sessionAvailableNowQuery)), 100,
> > 0)
> >
> > Is it possible to retrive an integer value from the index and pass it on
> it
> > a date math query ? Is there anything else that needs to be in the query
> ?
> >
> > Thanks in advance.
> >
>
Re: Execute an independent query from the main query
Hi Otis, Yes, that seems like one solution, however I have multiple opening and closing hours, within the same day. Therefore it might become somewhat complicated to manage the index. For now I shifted the business logic to the client and a second query is made to get the additional data. Thanks for the suggestion. Indika On 20 November 2012 02:50, Otis Gospodnetic wrote: > Hi Indika, > > So my suggestion was to maybe consider changing the index structure and > pull open/close times into 1 or more fields in the main record, so you > don't have this problem all together. > > Otis > -- > Performance Monitoring - http://sematext.com/spm/index.html > Search Analytics - http://sematext.com/search-analytics/index.html > > > > > On Sun, Nov 18, 2012 at 10:39 PM, Indika Tantrigoda >wrote: > > > Hi Otis, > > > > Actually I maintain a separate document for each open/close time along > with > > the date (i.e. Sunday =1, Monday =2). I was thinking if it would be > > possible to query Solr asking, give the next day's (can be current_day > +1) > > minimum opening time as a response field. > > > > Thanks, > > Indika > > > > On 19 November 2012 04:50, Otis Gospodnetic > >wrote: > > > > > Hi, > > > > > > Maybe your index needs to have a separate field for each day open/close > > > time. No join or extra query needed then. > > > > > > Otis > > > -- > > > Performance Monitoring - http://sematext.com/spm > > > On Nov 18, 2012 5:35 PM, "Indika Tantrigoda" > wrote: > > > > > > > Thanks for the response. > > > > > > > > Erick, > > > > My use case is related to restaurant opening hours, In the same > request > > > to > > > > Solr I'd like to get the time when the restaurant opens the next > > > > day, preferably part of the fields returned, and this needs to be > > > > independent of the main queries search params. > > > > > > > > Yes, the Join wouldn't be suitable in this use case. > > > > > > > > Luis, > > > > I had thought of having the logic in the client side, but before > that I > > > > wanted to see if I could get the result from Solr itself. I > > > > am currently using SolrJ along with Spring. > > > > > > > > Thanks, > > > > Indika > > > > > > > > On 18 November 2012 21:49, Luis Cappa Banda > > wrote: > > > > > > > > > Hello! > > > > > > > > > > When queries become more and more complex and you need to apply one > > > > second > > > > > query with the resultant docs from the first one, or re-sort > results, > > > or > > > > > maybe add some promotional or special docs to the response, I > > recommend > > > > to > > > > > develop a Web App module that implements that complex business > logic > > > and > > > > > dispatches queries from your Client App to your Solr back-end. That > > > > module, > > > > > let's call Search Engine, lets you play with all those special use > > > cases. > > > > > If you are familiar with Java I suggest you to have a look at the > > > > > combination between SolrJ and Spring framework or Jersey. > > > > > > > > > > Regards, > > > > > > > > > > - Luis Cappa. > > > > > El 18/11/2012 15:15, "Indika Tantrigoda" > > > escribió: > > > > > > > > > > > Hi All, > > > > > > > > > > > > I would like to get results of an query that is different from > the > > > main > > > > > > query as a new field. This query needs to be independent from any > > > > filter > > > > > > queries applied to the main query. I was trying to achieve this > by > > > > > > fl=_external_query_result:query($myQuery), however that result > > seems > > > to > > > > > be > > > > > > governed by any filter queries applied to the main query ? Is it > > > > possible > > > > > > to have a completely separate query in the fl list and return its > > > > result > > > > > > along with the results (per results), or would I need to create a > > > > > separate > > > > > > query on the client side to get the results of the independent > > query > > > > > (based > > > > > > on the results from the first query) ? > > > > > > > > > > > > Thanks in advance, > > > > > > Indika > > > > > > > > > > > > > > > > > > > > >
Re: Date math query syntax
Hi Jack,
Thanks for the response. Let me explain my question better.
I have a filter query to limit the query results,
start_time:[* TO 1970-01-01T10:55:002Z] AND end_time:[1970-01-01T10:55:002Z
TO *] AND session_time_range_available:true
and I also use the same query as a function query in fl using the
if(exists(query), x, y) where query is,
{!edismax}(start_time:[* TO 1970-01-01T10:55:002Z] AND
end_time:[1970-01-01T10:55:002Z TO *] AND session_time_range_available:true)
I need to extend the end_time range query based on the logic, similar to
end_time:[1970-01-01T10:55:002Z+30MINUTUES TO *]
How much I need to offset the end_time range is defined as an int type in
the schema. I was trying to use the field value query (_val_:field) to
retrive the end_time offset, resulting in the query being
1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES
However the query seems to have syntax errors. Or is it allowed to use the
_val_:field in such instances ?
As an extension I'll also have two offsets resulting in something similar
to
1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES+(_val_:extend_time)MINUTES
Thanks,
Indika
On 19 November 2012 09:17, Jack Krupansky wrote:
> You can do numeric math in function queries, but not string operations,
> and there is no provision for doing string operations in a query using the
> result of an embedded function query.
>
> You haven't shown us a full query.
>
> Sounds like we have another candidate for "XY Problem". First you need to
> tell us what you are really trying to accomplish, THEN a solution can be
> pursued.
>
> Take a look at the "ms" function query for date calculations - which is
> separate from so-called "date-math" which is a constant offset from a base
> date literal value.
>
> -- Jack Krupansky
>
> -Original Message- From: Indika Tantrigoda
> Sent: Sunday, November 18, 2012 3:11 PM
> To: solr-user@lucene.apache.org
> Subject: Date math query syntax
>
>
> Hi All,
>
> I am trying to use date math along with a filed
> value. 1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES is the date math
> query I am using. prep_time is of type int. I am having trouble properly
> formatting the string resulting in exceptions. I am guessing that I am not
> properly formatting the query with quotes and/or parenthesis.
>
> Thanks,
> Indika
>
Re: Retrieve unique documents on a non id field
Hi, Thanks for the reply. Yes, I grouped the documents based on the restaurant_id and got 1 result per group. Setting the group.format to simple helped with the formatting. Thanks, Indika On 8 November 2012 12:10, Rafał Kuć wrote: > Hello! > > Look at the field collapsing functionality - > http://wiki.apache.org/solr/FieldCollapsing > > It allows you to group documents based on field value, query or > function query. > > -- > Regards, > Rafał Kuć > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > > > Hi All, > > > Currently I am using Solr for searching and filtering restaurants based > on > > certain criteria. For example I use Solr to obtain the list of > restaurants > > open in the day. > > > A restaurant can have sessions when its open, e.g. Breakfast, Lunch and > > Dinner, and the time information related to these sessions are stored in > > three different documents with a field to identify the restaurant > > (restaurant_id). > > > When I query for restaurants that are open at 11:00 AM using > (start_time:[* > > TO 1100] AND end_time:[1100 TO *]) and if sessions overlap (say Breakfast > > and Lunch) I would get both the Breakfast document and the Lunch > document. > > These are in fact two different documents and would have the same > > restaurant_id. > > > My question is, is there a way to retrive only one document if the > > restaurant_id repeated in the response. > > > Thanks, > > Indika > >
Re: Sorting by boolean function
Hi,
Yes, the new sort query worked!. Thank you for the quick response.
Thanks,
Indika
On 8 November 2012 04:31, Rafał Kuć wrote:
> Hello!
>
> Ahhh, sorry. I see now. The function query you are specifying for the
> sort is not proper. Solr doesn't understand that part:
>
> exists(start_time:[* TO 1721] AND end_time:[1721 TO *])
>
> try something like that:
>
>
> sort=if(exists(query($innerQuery)),100,0)+desc&innerQuery={!edismax}(start_time:[*+TO+1721]
> AND end_time:[1721+TO+*])
>
> However remember that exists function is only Solr 4.0.
>
> I don't have any example data with me right now, so I can't
> say if it will work for sure, but clean Solr 4.0 doesn't cry about
> errors now :)
>
> --
> Regards,
> Rafał Kuć
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch
>
> > Thanks for the response, I did append the sort order but the result was
> the
> > same,
>
> > Can't determine a Sort Order (asc or desc) in sort spec
> > 'if(exists(start_time:[* TO 1721] AND end_time:[1721 TO *]),100,0) asc',
> > pos=23
>
> > Thanks,
> > Indika
>
>
>
> > On 8 November 2012 04:02, Rafał Kuć wrote:
>
> >> Hello!
> >>
> >> From looking at your sort parameter it seems that you are missing the
> >> order of sorting. The function query will return a value on which
> >> Solr will sort and Solr needs to know if the documents should be
> >> sorted in descending or ascending order. Try something like that:
> >>
> >> sort=if(exists(start_time:[* TO 1721] AND end_time:[1721 TO
> >> *]),100,0)+desc
> >>
> >> --
> >> Regards,
> >> Rafał Kuć
> >> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
> ElasticSearch
> >>
> >> > Hi All,
> >>
> >> > I am trying to sort by a boolean function to sort the places that are
> >> open
> >> > and not open as given below
> >>
> >> > sort=if(exists(start_time:[* TO 1721] AND end_time:[1721 TO *]),100,0)
> >>
> >> > However I keep getting an error message saying
> >> > Can't determine a Sort Order (asc or desc) in sort spec
> >> > 'if(exists(start_time:[* TO 1721] AND end_time:[1721 TO *]),100,0)
> asc',
> >> > pos=23
> >>
> >> > I assume that boolean functions can be used to sort. Is there anything
> >> > missing from the sort function ?
> >>
> >> > Thanks,
> >> > Indika
> >>
> >>
>
>
Re: Sorting by boolean function
Thanks for the response, I did append the sort order but the result was the same, Can't determine a Sort Order (asc or desc) in sort spec 'if(exists(start_time:[* TO 1721] AND end_time:[1721 TO *]),100,0) asc', pos=23 Thanks, Indika On 8 November 2012 04:02, Rafał Kuć wrote: > Hello! > > From looking at your sort parameter it seems that you are missing the > order of sorting. The function query will return a value on which > Solr will sort and Solr needs to know if the documents should be > sorted in descending or ascending order. Try something like that: > > sort=if(exists(start_time:[* TO 1721] AND end_time:[1721 TO > *]),100,0)+desc > > -- > Regards, > Rafał Kuć > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > > > Hi All, > > > I am trying to sort by a boolean function to sort the places that are > open > > and not open as given below > > > sort=if(exists(start_time:[* TO 1721] AND end_time:[1721 TO *]),100,0) > > > However I keep getting an error message saying > > Can't determine a Sort Order (asc or desc) in sort spec > > 'if(exists(start_time:[* TO 1721] AND end_time:[1721 TO *]),100,0) asc', > > pos=23 > > > I assume that boolean functions can be used to sort. Is there anything > > missing from the sort function ? > > > Thanks, > > Indika > >
Re: SolrJ 4.0.0 addFilterQuery() issue ?
Just created the defect in Jira.
https://issues.apache.org/jira/browse/SOLR-4020
Thanks.
On 31 October 2012 10:47, Indika Tantrigoda wrote:
> Thanks for the reply Chris.
>
> Yes you are correct, SolrJ is serializing a String[] instead of the
> separate String values.
>
> Using solrQuery.add("fq", "your first filter"); and solrQuery.add("fq",
> "your second filter"); has the same effect. Because it calls the add()
> method in the ModifiableSolrParams.java class. (Similar to
> solrQuery.setFilterQueries()).
>
> Yes, I will open a Jira issue for this with more information.
>
> Thanks,
> Indika
>
>
> On 31 October 2012 05:08, Chris Hostetter wrote:
>
>>
>> : org.apache.solr.common.SolrException:
>> : org.apache.lucene.queryparser.classic.ParseException: Cannot parse
>> : '[Ljava.lang.String;@1ec278b5': Encountered "" at line 1, column
>> 28.
>>
>> Hmmm.. that looks like a pretty anoying bug -- somehwere SolrJ is
>> serializing a String[] instead of sending the individual String values.
>>
>> can you please open a jira for this with these details?
>>
>> : Is there a new/alternate way in SolrJ 4 that this is done ?
>>
>> I would say that one possible workarround may be to
>> use...
>> solrQuery.add("fq", "your first filter");
>> solrQuery.add("fq", "your second filter");
>>
>> ...but i don't know where the bug is to know if that will actally work.
>> if you could try that also and mention the results in a comment in the
>> Jira you open that would be helpful.
>>
>> -Hoss
>>
>
>
Re: How to change the boost of fields in edismx at runtime
Hi Saroj, You could use the boost function in a FunctionQuery. Something similar to, _val_:Title^10 and _val_:Keyword^2 _val_:Title^2 and _val_:Keyword^10 See http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documentson how to use the boost function. Thanks, Indika On 31 October 2012 10:48, roz dev wrote: > Hi All > > I am wondering if there is any way in which we can change the relative > boost of different fields in edismax, at the run-time. > > For example, I can define 2 fields in my edismax query > > -Title - boosted to 10 > -Keyword - Boosted to 2 > > > Then, If I find that results are not of my liking then I would like to > change the boost as following > > - Title - boosted to 2 > -Keyword - boosted to 10 > > Is there any way to change this boost, at run-time, without having to > restart solr with new boosts in edismax? > > Any thoughts are much appreciated. > > Thanks > Saroj >
Re: SolrJ 4.0.0 addFilterQuery() issue ?
Thanks for the reply Chris.
Yes you are correct, SolrJ is serializing a String[] instead of the
separate String values.
Using solrQuery.add("fq", "your first filter"); and solrQuery.add("fq",
"your second filter"); has the same effect. Because it calls the add()
method in the ModifiableSolrParams.java class. (Similar to
solrQuery.setFilterQueries()).
Yes, I will open a Jira issue for this with more information.
Thanks,
Indika
On 31 October 2012 05:08, Chris Hostetter wrote:
>
> : org.apache.solr.common.SolrException:
> : org.apache.lucene.queryparser.classic.ParseException: Cannot parse
> : '[Ljava.lang.String;@1ec278b5': Encountered "" at line 1, column
> 28.
>
> Hmmm.. that looks like a pretty anoying bug -- somehwere SolrJ is
> serializing a String[] instead of sending the individual String values.
>
> can you please open a jira for this with these details?
>
> : Is there a new/alternate way in SolrJ 4 that this is done ?
>
> I would say that one possible workarround may be to
> use...
> solrQuery.add("fq", "your first filter");
> solrQuery.add("fq", "your second filter");
>
> ...but i don't know where the bug is to know if that will actally work.
> if you could try that also and mention the results in a comment in the
> Jira you open that would be helpful.
>
> -Hoss
>
SolrJ 4.0.0 addFilterQuery() issue ?
Hi All,
I am using Solr 4.0 and SolrJ 4.0.0 to access Solr from a Spring web
application, and I seem to have an issue when querying Solr. If the
SolrQuery contains more than one addFilterQuery() I get an exception
stating,
org.apache.solr.common.SolrException:
org.apache.lucene.queryparser.classic.ParseException: Cannot parse
'[Ljava.lang.String;@1ec278b5': Encountered "" at line 1, column 28.
Was expecting one of:
"TO" ...
...
...
I am using the addFilterQuery() in the following manner:
solrQuery.addFilterQuery("{!field f=facet_state}CA");
solrQuery.addFilterQuery("{!field f=facet_city}Test City");
The issue only comes up if there are more than one addFilterQuery() or if
multiple parameters are passed to setFilterQueries() or addFilterQuery() -
solrQuery.setFilterQueries("{!field f=facet_state}CA", "{!field
f=facet_city}Test City");
solrQuery.addFilterQuery("{!field f=facet_state}CA", "{!field
f=facet_city}Test City");
Is there a new/alternate way in SolrJ 4 that this is done ?
Thanks in advance.
Indika
Re: Solr edismax clarification
Hi, Thanks for the response. Below is the field type and schema definition. This is the request handler. edismax explicit *:* 0.01 title^2 description^1.0 pow(100,boost_factor) Here is the output from debugQuery=on BoostedQuery(boost(+(((title:boston^2.0 | description:boston)~0.01 (title:public^2.0 | description:public)~0.01 (title:library^2.0 | description:library)~0.01)~3),pow(const(100.0),int(boost_factor stopwords_en.txt is inlcuded for stopworods. i.e. the list of stopwords that come with Solr. The full query string is http://localhost:/solr/select/?fl=title&rows=13&qt=general_search&q=boston+public+library&start=0&debugQuery=on I use SolrJ in my application to connect to Solr. As per the query string shown above if I search for Boston public library I do not get a document that has the title Boston. Thanks. On 17 February 2012 15:55, O. Klein wrote: > > Indika Tantrigoda wrote > > > > Hi All, > > > > I am using edismax SearchHandler in my search and I have some issues in > > the > > search results. As I understand if the "defaultOperator" is set to OR the > > search query will be passed as -> The OR quick OR brown OR fox > > implicitly. > > > > > > Did you also remove "mm"? If not "defaultOperator" is ignored and it > follows "mm" settings. > > http://wiki.apache.org/solr/DisMaxQParserPlugin#mm_.28Minimum_.27Should.27_Match.29 > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-edismax-clarification-tp3751013p3753260.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Solr edismax clarification
Hi All, I am using edismax SearchHandler in my search and I have some issues in the search results. As I understand if the "defaultOperator" is set to OR the search query will be passed as -> The OR quick OR brown OR fox implicitly. However if I search for The quick brown fox, I get lesser results than explicitly adding the OR. Another issue is that if I search for The quick brown fox other documents that contain the word fox is not in the search results. Thanks.
Re: Boosting based on field values
Hi,
{!boost b=pow(1,featured_listing)} is the boost function I used.
Got the results as expected.
Thanks.
Regards,
Indika
On 3 July 2010 21:10, Indika Tantrigoda wrote:
> Thanks for the info. I'll try this out.
>
> Regards,
> Indika
>
>
> On 3 July 2010 20:48, Ahmet Arslan wrote:
>
>> > I'd like to know if its possible to boost the score of
>> > documents based on a
>> > field value. Ex. My schema has a field called isFeatured, and if the
>> > value of the field
>> > is true or "1" I'd like to
>> > have these documents come first in a query result
>> > regardless of the score.
>>
>> Yes it is possible. http://wiki.apache.org/solr/FunctionQuery
>>
>> http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents
>>
>> Something like:
>> &q={!boost b=pow(x,abs(sub(isFeatured,1)),0.5)}yourQuery
>>
>>
>>
>>
>>
>
Re: Boosting based on field values
Thanks for the info. I'll try this out.
Regards,
Indika
On 3 July 2010 20:48, Ahmet Arslan wrote:
> > I'd like to know if its possible to boost the score of
> > documents based on a
> > field value. Ex. My schema has a field called isFeatured, and if the
> > value of the field
> > is true or "1" I'd like to
> > have these documents come first in a query result
> > regardless of the score.
>
> Yes it is possible. http://wiki.apache.org/solr/FunctionQuery
>
> http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents
>
> Something like:
> &q={!boost b=pow(x,abs(sub(isFeatured,1)),0.5)}yourQuery
>
>
>
>
>
Boosting based on field values
Hello all, I'd like to know if its possible to boost the score of documents based on a field value. Ex. My schema has a field called isFeatured, and if the value of the field is true or "1" I'd like to have these documents come first in a query result regardless of the score. Thanks in advance. Regards, Indika
Re: NGramFilterFactory usage
Hello, I got the problem solved. The issue was I was setting the wrong field name in the qf parameter of my request handler. Thanks. Regards, Indika On 27 June 2010 08:52, Indika Tantrigoda wrote: > Hello, > > > Applying the NGramFilterFactory for analyzer type="query" didnt solve the > issue. > From the examples I've seen it is only necesssary to have the > NGramFilterFactory at index time right ? > > Regards, > Indika > > On 27 June 2010 01:14, Indika Tantrigoda wrote: > >> Hi all, >> >> I've been working with Solr for while and the search components work as >> expected. >> Recently I've had the requirement to do searching on partial words and I >> setup the NGramFilterFactory. >> >> My schema.xml is as follows : >> >> > positionIncrementGap="100" stored="false" multiValued="true"> >> >> >> >> > maxGramSize="15"/> >> >> >> >> >> >> >> >> > multiValued="false"/> >> > multiValued="true"/> >> >> >> Furthermore I am using the dismax query hanlder and have set a boost on >> the nGram_text field. >> >> If I do a *:* on the Solr administration interface it shows the nGram_text >> field to be populated. >> However if I search for plan (Assume I indexed the word Plane) no results >> are shown. >> Is there any other configurations that needs to be done ? >> >> Thanks in advance, >> >> Regards, >> Indika >> > >
Re: NGramFilterFactory usage
Hello, Applying the NGramFilterFactory for analyzer type="query" didnt solve the issue. >From the examples I've seen it is only necesssary to have the NGramFilterFactory at index time right ? Regards, Indika On 27 June 2010 01:14, Indika Tantrigoda wrote: > Hi all, > > I've been working with Solr for while and the search components work as > expected. > Recently I've had the requirement to do searching on partial words and I > setup the NGramFilterFactory. > > My schema.xml is as follows : > > positionIncrementGap="100" stored="false" multiValued="true"> > > > >maxGramSize="15"/> > > > > > > > > multiValued="false"/> > multiValued="true"/> > > > Furthermore I am using the dismax query hanlder and have set a boost on the > nGram_text field. > > If I do a *:* on the Solr administration interface it shows the nGram_text > field to be populated. > However if I search for plan (Assume I indexed the word Plane) no results > are shown. > Is there any other configurations that needs to be done ? > > Thanks in advance, > > Regards, > Indika >
NGramFilterFactory usage
Hi all, I've been working with Solr for while and the search components work as expected. Recently I've had the requirement to do searching on partial words and I setup the NGramFilterFactory. My schema.xml is as follows : Furthermore I am using the dismax query hanlder and have set a boost on the nGram_text field. If I do a *:* on the Solr administration interface it shows the nGram_text field to be populated. However if I search for plan (Assume I indexed the word Plane) no results are shown. Is there any other configurations that needs to be done ? Thanks in advance, Regards, Indika
Solr work at Drupal
Hi all, Came across this, http://groups.drupal.org/node/66608 Regards, Indika
Re: Solr commit issue
Thanks for the reply. Here is another thread I found similar to this http://www.mail-archive.com/solr-user@lucene.apache.org/msg28236.html >From what I understand the IndexReaders get reopened after a commit. Regards, Indika On 2 May 2010 00:29, Erick Erickson wrote: > The underlying IndexReader must be reopened. If you're > searching for a document with a searcher that was opened > before the document was indexed, it won't show up on the > search results. > > I'm guessing that your statement that when you search > for it with some test is coincidence, but that's just a guess. > > HTH > Erick > > On Sat, May 1, 2010 at 1:07 PM, Indika Tantrigoda >wrote: > > > Hi all, > > > > I've been working with Solr for a few weeks and have gotten SolrJ > > to connect to it, index, search documents. > > > > However I am having an issue when a document is committed. > > When a document is committed it does not show in the search results if I > do > > a *:* search, > > but if I search for it with some text then it is shown in the results. > > Only when another document is committed, the previous document is found > > when > > I do a *:* search > > > > Is this because of the SolrJ client or do I have to pass additional > > parameters to commit() ? > > > > Thanks in advance. > > > > Regards, > > Indika > > >
Solr commit issue
Hi all, I've been working with Solr for a few weeks and have gotten SolrJ to connect to it, index, search documents. However I am having an issue when a document is committed. When a document is committed it does not show in the search results if I do a *:* search, but if I search for it with some text then it is shown in the results. Only when another document is committed, the previous document is found when I do a *:* search Is this because of the SolrJ client or do I have to pass additional parameters to commit() ? Thanks in advance. Regards, Indika
Re: Filter query with special character using SolrJ client
Thank you very much for the explanation.
Regards,
Indika
On 29 March 2010 22:28, Chris Hostetter wrote:
>
> : It works, thanks. Just implemented the code...:):):)
> :
> : Could you explain what "{!field f=yourStringField}Cameras & Photos" does.
>
> {!field} says that the string should be parsed using the FIeldQParser.
> the FieldQParser takes an 'f' local param telling it what field you want
> to use, and the rest of the string is the exact value you want to
> passed to the analyzer for thet field 'f' ... it's a query parser that
> supports no markup of any kind, and only produces basic
> PhraseQueries or TermQueries (there's also the "raw" QParser for when you
> absolutely know you only want TermQueries) ...
>
>
> http://wiki.apache.org/solr/SolrQuerySyntax#Other_built-in_useful_query_parsers
>
> http://lucene.apache.org/solr/api/org/apache/solr/search/FieldQParserPlugin.html
>
>
> -Hoss
>
>
Re: Filter query with special character using SolrJ client
It works, thanks. Just implemented the code...:):):)
Could you explain what "{!field f=yourStringField}Cameras & Photos" does.
Regards,
Indika
On 29 March 2010 21:55, Chris Hostetter wrote:
>
> : Since the names of the string fields are not predefined I might have to
> : find a method to do this automatically.
>
> if the fields are strings, and you are only looking for "exact" matches
> (ie: you don't need any special query parser syntax) use the "field"
> QParser
>
> : > SolrQuery.addFilterQuery("yourStringField:Cameras\\ \\&\\ Photos")
>
> solrQuery.addFilterQuery("{!field f=yourStringField}Cameras & Photos")
>
>
> -Hoss
>
>
Re: Drill down a solr result set by facets
Hi Dhanushka, Have you tried to use the filter query parameter. Check out this article, the Applying Constraints section should be helpful to you. http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Faceted-Search-Solr Solr Wiki link to filter query parameter http://wiki.apache.org/solr/CommonQueryParameters#fq I am at the moment implementing a similar system where the user needs to drill down to the data. What I am doing now is say if the user selects "Chemistry" from the facet I request a query with the filter query applied to fDepartmentName and when the user selects "US Cancer/Diabetic Research Institute" from the fSponsor facet I will apply filter querying to both the fDepartmentName and fSponsor. Hope this helps. Regards, Indika On 29 March 2010 21:19, Dhanushka Samarakoon wrote: > Hi, > > I'm trying to perform a search based on keywords and then reduce the result > set based on facets that user selects. > First query for a search would look like this. > > > http://localhost:8983/solr/select/?q=cancer+stem&version=2.2&wt=php&start=&rows=10&indent=on&qt=dismax&facet=on&facet.mincount=1&facet.field=fDepartmentName&facet.field=fInvestigatorName&facet.field=fSponsor&facet.date=DateAwarded&facet.date.start=2009-01-01T00:00:00Z&facet.date.end=2010-01-01T00:00:00Z&facet.date.gap=%2B1MONTH > > In the above query (as per dismax on the solr config file) it searches > multiple fields such as GrantTitle, DepartmentName, InvestigatorName, > etc... > > Then if user select 'Chemistry' from the facet field 'fDepartmentName' and > 'US Cancer/Diabetic Research Institute' from 'fSponsor' I need to reduce > the > result set above to only records from where fDepartmentName is 'Chemistry' > and 'fSponsor' is 'US Cancer/Diabetic Research Institute' > The following query is not working. > select/?q=cancer+stem+fDepartmentName:Chemistry+fSponsor:US Cancer/Diabetic > Research Institute&version=2.2& > > Fields starting with 'f' are defined in the schema.xml as copy fields. >multiValued="true" /> >multiValued="true" /> > > > Any ideas on the correct syntax? > > Thanks, > Dhanushka. >
Re: Filter query with special character using SolrJ client
Thanks for the info. I'll have a try and see how the results are.
Since the names of the string fields are not predefined I might have to
find a method to do this automatically.
I'll try a few more workarounds and see what the results turn out to be.
Regards,
Indika
On 29 March 2010 06:57, Ahmet Arslan wrote:
>
> > I am trying to get filter querying to work on search.
> > When the filter query contains no special character or is a
> > single word the
> > filter query parameter returns the result as expected.
> > However when I run filter query where special characters
> > are present no
> > results are results.
> >
> > e.g Cameras & Photos, I removed the & character as
> > well (Cameras and Photos)
> > and still no results are returned.
> >
> > However for a string like Electronics the results are
> > returned as expected.
> >
> > I am using the string field to store and index the field
> > that I want to
> > filter and SolrJ as the client.
> >
> > From the documentation I read solrJ automatically escapes
> > special
> > characters.
>
> With SolrJ you don't need to urlencode. However you still need to escape
> special query syntax parameters. Can you try this:
>
> SolrQuery.addFilterQuery("yourStringField:Cameras\\ \\&\\ Photos")
>
>
>
>
Filter query with special character using SolrJ client
Hello, I am trying to get filter querying to work on search. When the filter query contains no special character or is a single word the filter query parameter returns the result as expected. However when I run filter query where special characters are present no results are results. e.g Cameras & Photos, I removed the & character as well (Cameras and Photos) and still no results are returned. However for a string like Electronics the results are returned as expected. I am using the string field to store and index the field that I want to filter and SolrJ as the client. >From the documentation I read solrJ automatically escapes special characters. Any help regarding this is much appreciated. Regards, Indika
Re: SolrJ and HTMLStripCharFilterFactory
Hi Erick, Thank you very much for the explanation. The example you gave made things clear. I ran some queries with my existing index and the results were as expected. Regards, Indika On 27 March 2010 17:09, Erick Erickson wrote: > I think you're getting confused by the difference between indexing and > storing. These are orthogonal operations for all they occur in the same > definition. > > When you index something, the input is put through your analyzer chain, and > the resulting tokens are stored after all appropriate transformations, > which > is what you're seeing when you look at your index through the admin panel > and report the html is stripped. This is what's searched. > > But when you fetch a field that has been stored, the original raw text is > returned. This is never searched, just kept around for retrieval. > > The idea here is to be able to have your index contain some displayable > text. Think about the title of a book, for instance "The Grapes of Wrath". > You want to search it after it's been lower-cased, stop words removed, etc. > But if you wanted to present it to a user, you sure wouldn't want to > display > "grapes wrath" which might be the tokens after lowercasing and removing > stopwords.. > > HTH > Erick > > On Sat, Mar 27, 2010 at 1:13 AM, Indika Tantrigoda >wrote: > > > Hello to all, > > > > I've been working with Solr for a few weeks and I have gotten indexing > and > > searching to work. > > However I am having trouble with indexing HTML content and using > > HTMLStripCharFilterFactory. > > > > My schema.xml looks like this > > > > positionIncrementGap="100"> > > > > > > -- > > /> > > > > and I am indexing the HTML content using SolrJ as the client (with Spring > > being the framework). > > > > However when I do a search for all documents, the HTML content is also in > > my > > text field. > > > > But when I did an analysis using the Solr admin panel with HTML content > it > > shows the tokens extracted > > properly with HTML tags removed. > > > > I found a similar issue at > > http://www.mail-archive.com/solr-user@lucene.apache.org/msg28736.html > > but I am still unable to get it working. I am using Solr 1.4 > > > > Any help regarding this is this much appreciated. > > > > Thanks in advance. > > > > Regards, > > Indika > > >
SolrJ and HTMLStripCharFilterFactory
Hello to all, I've been working with Solr for a few weeks and I have gotten indexing and searching to work. However I am having trouble with indexing HTML content and using HTMLStripCharFilterFactory. My schema.xml looks like this -- /> and I am indexing the HTML content using SolrJ as the client (with Spring being the framework). However when I do a search for all documents, the HTML content is also in my text field. But when I did an analysis using the Solr admin panel with HTML content it shows the tokens extracted properly with HTML tags removed. I found a similar issue at http://www.mail-archive.com/solr-user@lucene.apache.org/msg28736.html but I am still unable to get it working. I am using Solr 1.4 Any help regarding this is this much appreciated. Thanks in advance. Regards, Indika
