Multiple groups of boolean queries in a single query.
Hello friends, I have a schema which contains various types of records of three different categories for ease of management and for making a single query to fetch all the data. The fields are grouped into three different types of records. For example: fields type 1: field name=x_date type=tdate indexed=true stored=true/ field name=x_name type=tdate indexed=true stored=true/ field name=x_type type=tdate indexed=true stored=true/ fields type 2: field name=y_date type=tdate indexed=true stored=true/ field name=y_name type=string indexed=true stored=true/ field name=y_phone type=string indexed=true stored=true/ fields type 3: field name=z_date type=tdate indexed=true stored=true/ field name=z_type type=string indexed=true stored=true/ common partition field which identifies the category of the data record field name=xyz_category type=string indexed=true stored=true/ What should I do to fetch all these records in the form: (+x_date:[2011-01-01T00:00:00Z TO *] +x_type:(1 OR 2 OR 3 OR 4) +xyz_category:X) OR (+y_date:[2012-06-01T00:00:00Z TO *] +y_name:sam~ +xyz_category:Y) OR (+z_date:[2013-03-01T00:00:00Z TO *] +xyz_category:Z) Can we construct a query like this? Or is it even possible? Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-groups-of-boolean-queries-in-a-single-query-tp4074294.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multiple groups of boolean queries in a single query.
My entire concern is to be able to make a single query to fetch all the types of records. If I had to create three different cores for this different types of data, I would have to make 3 calls to solr to fetch the entire set of data. And I will be having approx 15 such types in real. Also, at any given record, either the section 1 fields are filled up or section 2's or section 3's. At no point, will we have all these fields populated in a single record. Only field that will have data for all records is xyz_category to allow us to partition the data set. Any suggestions in writing a single query to fetch all the data we need will be highly appreciated. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-groups-of-boolean-queries-in-a-single-query-tp4074294p4074296.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: filter query from external list of Solr unique IDs
Does anything exists already in solr 4.3 to meet this usecase scenario? -- View this message in context: http://lucene.472066.n3.nabble.com/filter-query-from-external-list-of-Solr-unique-IDs-tp1709060p4070874.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Faceting doesn't return values.
Hello,
I have a field defined in my schema.xml like so:
field name=sa_site_city type=string indexed=true stored=true/
string is a type :
fieldType name=string class=solr.StrField sortMissingLast=true /
When I run the query for faceting data by the city:
http://XX.XX.XX.XX/solr/collection1/select?q=mm_state_codewt=jsonindent=truefacet=truefacet.field=sa_site_city
I get empty result like so:
{
responseHeader:{
status:0,
QTime:1,
params:{
facet:true,
indent:true,
q:mm_state_code,
facet.field:sa_site_city,
wt:json}},
response:{numFound:0,start:0,docs:[]
},
facet_counts:{
facet_queries:{},
facet_fields:{
sa_site_city:[]},
facet_dates:{},
facet_ranges:{}}}
I wonder what am I doing wrong?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
Ok after I added debug=all to the query, I get:
{
responseHeader:{
status:0,
QTime:11,
params:{
facet:true,
indent:true,
q:mm_state_code,
debug:all,
facet.field:sa_site_city,
wt:json}},
response:{numFound:0,start:0,docs:[]
},
facet_counts:{
facet_queries:{},
facet_fields:{
sa_site_city:[]},
facet_dates:{},
facet_ranges:{}},
debug:{
rawquerystring:mm_state_code,
querystring:mm_state_code,
parsedquery:sa_property_id:mm_state_code,
parsedquery_toString:sa_property_id:mm_state_code,
explain:{},
QParser:LuceneQParser,
timing:{
time:4.0,
prepare:{
time:2.0,
query:{
time:0.0},
facet:{
time:0.0},
mlt:{
time:0.0},
highlight:{
time:0.0},
stats:{
time:0.0},
debug:{
time:0.0}},
process:{
time:1.0,
query:{
time:0.0},
facet:{
time:0.0},
mlt:{
time:0.0},
highlight:{
time:0.0},
stats:{
time:0.0},
debug:{
time:1.0}
I have not defined any default facet filed, in the handler in solrconfig.xml
file.
Also, there is plenty of data available and the field sa_site_city
What I am trying to understand is this:
parsedquery:sa_property_id:mm_state_code
I have a field sa_property_id in the schema but i have not defined it in the
query nor in solrconfig.xml, but why is it still evaluated? Any help in
solving this problem will be greatly appreciated.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065294.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
Ok my bad. I do have a default field defined in the /select handler in the config file. lst name=defaults str name=echoParamsexplicit/str int name=rows10/int str name=dfsa_property_id/str /lst But then how do I change my query now? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065298.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
Thanks for your reply. I have my request url modified like this: http://xx.xx.xx.xx/solr/collection1/select?q=TXdf=mm_state_codewt=xmlindent=truefacet=truefacet.field=sa_site_citydebug=all Facet Filed = sa_site_city ( city wise facet) Default Filed = mm_state_code Query= TX When I run this query, I get something like this: ?xml version=1.0 encoding=UTF-8? response lst name=responseHeader int name=status0/int int name=QTime3/int lst name=params str name=facettrue/str str name=dfsa_site_city/str str name=indenttrue/str str name=qTX/str str name=_1369238921109/str str name=debugall/str str name=facet.fieldsa_site_city/str str name=wtxml/str /lst /lst result name=response numFound=0 start=0 /result lst name=facet_counts lst name=facet_queries/ lst name=facet_fields lst name=sa_site_city/ /lst lst name=facet_dates/ lst name=facet_ranges/ /lst lst name=debug str name=rawquerystringTX/str str name=querystringTX/str str name=parsedquerysa_site_city:TX/str str name=parsedquery_toStringsa_site_city:TX/str lst name=explain/ str name=QParserLuceneQParser/str lst name=timing double name=time2.0/double lst name=prepare double name=time0.0/double lst name=query double name=time0.0/double /lst lst name=facet double name=time0.0/double /lst lst name=mlt double name=time0.0/double /lst lst name=highlight double name=time0.0/double /lst lst name=stats double name=time0.0/double /lst lst name=debug double name=time0.0/double /lst /lst lst name=process double name=time2.0/double lst name=query double name=time1.0/double /lst lst name=facet double name=time1.0/double /lst lst name=mlt double name=time0.0/double /lst lst name=highlight double name=time0.0/double /lst lst name=stats double name=time0.0/double /lst lst name=debug double name=time0.0/double /lst /lst /lst /lst /response I do have the data in my index and that I verified by running other queries. I can't figure out what I am missing. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065360.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
When I use your query, I get :
?xml version=1.0 encoding=UTF-8?
response
lst name=responseHeader
int name=status400/int
int name=QTime12/int
lst name=params
str name=facettrue/str
str name=dfmm_state_code/str
str name=indenttrue/str
str name=q*mm_state_code:(**TX)*/str
str name=_1369244078714/str
str name=debugall/str
str name=facet.fieldsa_site_city/str
str name=wtxml/str
/lst
/lst
lst name=error
str name=msgorg.apache.solr.search.SyntaxError: Cannot parse
'*mm_state_code:(**TX)*': Encountered : : at line 1, column 14.
Was expecting one of:
EOF
AND ...
OR ...
NOT ...
+ ...
- ...
BAREOPER ...
( ...
* ...
^ ...
QUOTED ...
TERM ...
FUZZY_SLOP ...
PREFIXTERM ...
WILDTERM ...
REGEXPTERM ...
[ ...
{ ...
LPARAMS ...
NUMBER ...
/str
int name=code400/int
/lst
/response
Not sure why the data wont show up. Almost all the records has the field
sa_site_city has data and is also indexed. :(
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065406.html
Sent from the Solr - User mailing list archive at Nabble.com.
Best way to design a story and comments schema.
Hi, I wish to know how to best design a schema to store comments in stories / articles posted. I have a set of fields: / lt;field name=quot;subjectquot; type=quot;text_generalquot; indexed=quot;truequot; stored=quot;truequot;/gt; lt;field name=quot;keywordsquot; type=quot;text_generalquot; indexed=quot;truequot; stored=quot;truequot;/gt; lt;field name=quot;categoryquot; type=quot;text_generalquot; indexed=quot;truequot; stored=quot;truequot;/gt; lt;field name=quot;contentquot; type=quot;text_generalquot; indexed=quot;falsequot; stored=quot;truequot; /gt; / Users can post their comments on a post and I should be able to retrieve these comments and show it along side the original post. I only need to show the last 3 comments and show a facet of the remaining comments which user can click and see the rest of the comments ( something like facebook does ). One alternative, I could think of, was adding a dynamic field for all comments : /lt;dynamicField name=quot;comment_*quot; type=quot;stringquot; indexed=quot;falsequot; stored=quot;truequot;/gt;/ So, to store each comments, I would send a text to solr of the form - For Field Name: /comment_n/ Value:/[Commenter Name]:[Commenter ID]:[Actual Comment Text]/ And to keep the count of those comments, I could use another field like so :/lt;field name=quot;comment_countquot; type=quot;intquot; indexed=quot;truequot; stored=quot;truequot;/gt;/ With this approach, I will have to do some calculation when a comment is deleted by the user but I still can manage to show the comments right. My idea is to find the best solution for this scenario which will be fast and also be simple. Kindly suggest. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-design-a-story-and-comments-schema-tp4062867.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Best way to design a story and comments schema.
Thanks for your reply. I generally get confused by a collection and a core. But just FYI, I do have two cores at the moment - one for the users and another for the Stories. Initially I thought of adding an extra core for the Comments too but realized that it would mean multiple HTTP calls to fetch both the story and the comments. Also, when a story is deleted, so should be its comments. Having that spread across two cores might cause issues with transaction when I delete the story and try to delete the respective comments? Or when I delete the User and all hos stories and comments? I really wish to understand how that works. Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-design-a-story-and-comments-schema-tp4062867p4062913.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Best way to design a story and comments schema.
I think I got your point. So, what I will create are three cores (or collections) - one for the users, one for the stories and the last one for comments. When I need to find all the stories posted by a single user, I first need to search the stories core with a unique userid in the filter and then run another query to fetch the collection of comments. Correct? Also, I have no such requirement to search through the comments and its mostly a storage filed for me. So, do you think I should shift that into a DB from where I may query the comments? Or will it be too costly for Solr to just plain store that data in a core? Which would be the best option here? Also, the idea of custom search component sounds great. But as you said, I will first try this out with a simple possible setup and then go from there. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-design-a-story-and-comments-schema-tp4062867p4062929.html Sent from the Solr - User mailing list archive at Nabble.com.
Is cache useful for my scenario?
Hi, I am new in Solr and wish to use version 4.2.x for my app in production. I want to show hundreds and thousands of markers on a map with contents coming from Solr. As the user moves around the map and pans, the browser will fetch data/markers using a BBOX filter (based on the maps' viewport boundary). There will be a lot of data that will be indexed in Solr. My question is, does caching help in my case? As the filter queries will vary for almost all users ( because the viewport latitude/longitude would vary), in what ways can I use Caching to increase performance. Should I completely turn off caching? If you can suggest by your experience, it would be really nice. Thanks Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Is-cache-useful-for-my-scenario-tp4056250.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr - Spatial Search for Specif Areas on Map
David, Thanks for such a detailed response. The data volume I mentioned is the total set of records we have - but we would never ever need to search the entire base in one query; we would divide the data by region or zip code. So, in that case I assume that for a single region, we would not have more than 200M records (this is real , we have a region with that many records). So, I can assume that I can create shards based on regions and the requests would get distributed among these region servers, right? You also mentioned about ~20 concurrent queries per shard - do you have links to some benchmarks? I am very interested to know about the hardware sizing details for such a setup. About setting up Solr for a single shard, I think I will go by your advice. Will see how much a single shard can handle in a decent machine :) The reason why I came up with that figure was, I have a user base of 500k and theres a lot of activity which would happen on the map - every time someone moves the tiles, zooms in/out, scrolls, we are going to send a server side request to fetch some data ( I agree we can benefit much using caching but I believe Solr itself has its own local cache). I might be a bit unrealistic with my 10K rps projections but I have read about 9K rps to map servers from some sources on the internet. And, NO, I don't work for Google :) But who knows we might be building something that can get so much traffic to us in a while. :D BTW, my question still remains - can we do search on polygonal areas on the map? If so, do you have any link where i can get more details? Bounding Box thing wont work for me I guess :( Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Spatial-Search-for-Specif-Areas-on-Map-tp3995051p3995209.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr - Spatial Search for Specif Areas on Map
Hi, I am new to Solr Spatial Search and would like to understand if Solr can be used successfully for very large data sets in the range of 4Billion records. I need to search some filtered data based on a region - maybe a set of lat/lons or polygon area. is that possible in solr? How fast is it with such data size? Will it be able to handle the load for 1 req/sec? If so, how? Do you think solr can beat the performance of PostGIS? As I am about to choose the right technology for my new project, I need some expert comments from the community. Regards Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Spatial-Search-for-Specif-Areas-on-Map-tp3995051.html Sent from the Solr - User mailing list archive at Nabble.com.
