How to handle special characters in fuzzy search query
So my solr query is implemented in two parts,first query does an exact
search if there are no results found for exact then it goes to the second
query that does a fuzzy search.
every things works fine but in situations like--A user enters burg +
So in exact search no records will come,so second query is called to do a
fuzzy search.Now comes the problem my fuzzy query does not understand
special characters like +,-* which throws and error.If i dont pass special
characters it works fine. But in real world a user can put characters with
their search,which will throw an error.
Now iam stuck in this and dont know how to resolve this issue.
This is how my exact search query looks like
$query1=(business_name:$data*^100 OR city_name:$data*^1 OR
locality_name:$data*^6 OR business_search_tag_name:$data*^8 OR
type_name:$data*^7) AND (business_active_flag:1) AND
(business_visible_flag:1) AND (delete_status_businessmasters:0);
This is how my fuzzy query looks like
$query2='(_query_:%20{!complexphrase%20qf=business_name^100+type_name^0.4+locality_name^6%27}%20'.$url_new.')AND(business_active_flag:1)AND(business_point:[1.5
TO 2.0])q.op=ANDwt=jsonindent=true';
Iam new to solr and dont know how to tackle this situation.
Details
Solrphpclient
php
solr 4.9
--
Regards
Madhav Bahuguna
How to do fuzzy search on phrases
HI Iam using solr 4.7 and solr php client. So heres the issue ,i have data indexed in solr eg mist spa I have applied fuzzy to my search and If i search myst or mysty i get the correct result i get mist spa in result. But if i write mysty spa or must spa i do not get ant results.Iam not able to implement fuzzy search on more than one word. Can any one advise me or help me regarding this. The query iam passing using solr php client is $querynew=(business_name:$data~N); -- Regards Madhav Bahuguna
How to use less than and greater than in data-config file of solr
I have two tables and i want to link them using greater than and less than
condition.They have nothing in common,the only way i can link them is using
range values.Iam able to do this in Mysql but how do i do this in solr in
my data-config.xml file This is how my data-config file looks
entity name=business_colors query=SELECT business_colors_id,
business_rating_from,business_rating_to,business_text,hex_colors,rgb_colors,business_colors_modify
from business_colors where business_rating_from gt;=
'${businessmasters.business_point}' AND
business_rating_to lt; '${businessmasters.business_point}'
deltaQuery=select business_colors_id from business_colors where
business_colors_modify
'${dih.last_index_time}'
parentDeltaQuery=select business_id from businessmasters where
business_point lt;
${business_colors.business_rating_from} AND business_point gt;=
${business_colors.business_rating_from}
field column=business_colors_id name=id/
field column=business_rating_from name=business_rating_from
indexed=true stored=true /
field column=business_rating_to name=business_rating_to
indexed=true stored=true /
field column=business_text name=business_text indexed=true
stored=true /
field column=hex_colors name=hex_colors indexed=true stored=true /
field column=rgb_colors name=rgb_colors indexed=true stored=true /
field column=business_colors_modify name=business_colors_modify
indexed=true
stored=true/
When i click full indexing data does not get index and no error is shown.
What is wrong with this,Can any one help and advise.What i have seen is
that if i replace AND with OR it works fine or just use one condition
instead of both it works fine . Can any one advise and help How do i
achieve what i want to do.
I have also posted this question in stackoverflow
http://stackoverflow.com/questions/26397084/how-use-less-than-and-greater-than-in-data-config-file-of-solr
--
Regards
Madhav Bahuguna
How to link tables based on range values solr data-config
Hi ,
Businessmasters
Business_id Business_point
13.4
22.8
38.0
Business_Colors
business_colors_id business_rating_from business_rating_to rating
1 2 5 OK
2 5 10 GOOD
310 15
Excellent
I want to link the two tables based business_rating_from and
business_rating_to like
SELECT business_colors_id,business_rating_from,business_rating_to,rating
where business_rating_from = 2 AND business_rating_to 5;
Now i want to index them into solr.This is how my data-config file looks
entity name=business_colors query=SELECT business_colors_id,
business_rating_from,business_rating_to,business_text,hex_colors,rgb_colors,business_colors_modify
from business_colors where business_rating_from gt;=
'${businessmasters.business_point}' AND
business_rating_to lt; '${businessmasters.business_point}'
deltaQuery=select business_colors_id from business_colors where
business_colors_modify'${dih.last_index_time}'
parentDeltaQuery=select business_id from businessmasters where
business_point lt; ${business_colors.business_rating_from} AND
business_point gt;= ${business_colors.business_rating_from}
field column=business_colors_id name=id/
field column=business_rating_from name=business_rating_from
indexed=true stored=true /
field column=business_rating_to name=business_rating_to
indexed=true stored=true /
field column=business_text name=business_text indexed=true
stored=true /
field column=hex_colors name=hex_colors indexed=true
stored=true /
field column=rgb_colors name=rgb_colors indexed=true
stored=true /
field column=business_colors_modify name=business_colors_modify
indexed=true
stored=true/
When i click full indexing data does not get index and no error is shown.
What is wrong with this,Can any one help and advise. How do i achieve what
i want to do
I also have this question posted on stack over flow
http://stackoverflow.com/questions/26256344/how-to-link-tables-based-on-range-values-solr-data-config
--
Regards
Madhav Bahuguna
Solr stops in between indexing
Hi, I have solr configured on google cloud server. When ever i try to index it ,it stops in between and shows and error connection lost,connection timeout. I have 2200 records some time it stops full indexing at 917 sometime 1385 sometime 2185. I have apache2 running on google cloud on debian OS. Earlier it was working fine,it has started giving this error recently only Please advise and help. -- Regards Madhav Bahuguna
How to stop Solr delta import from creating a log file
I have solr installed on Debian and every time delta import takes place a file gets created in my root directory. The files that get created look like this dataimport?command=delta-import.1 dataimport?command=delta-import.2 . . . dataimport?command=delta-import.30 Every time there is a delta import a file gets created , i opened the file in vi editor and its an xml file. Why are these files getting created and how do i stop solr from creating them. To start solr i use this command Java -jar start.jar According to this command no log files should be created. Please advise and help iam new to solr. -- Regards Madhav Bahuguna
Sort not working in solr
Iam trying to sort my records but the result i get is not correct My url query-- http://localhost:8983/solr/select/?q=*:*fl=business_pointsort=business_point+desc Iam trying to sort my records by business_points but the result i get is in like this 9 8 7 6 5 45 4 4 10 1 Whys am i getting my results in the wrong order my schema looks like this field name=business_point type=text_general indexed=true stored=true required=false multiValued=false/ -- Regards Madhav Bahuguna
delta import to detect deleted values
Hi, How do i make delta import detect deleted values. I do have a timestamp column to detect changes ,but the requirement is such that rows will be deleted from the table.Every time i run delta import the index still shows the deleted value. How do i make delta import detect deleted values so that it does not show deleted values. -- Regards Madhav Bahuguna
delta import not working properly
I have 8 tables in my solr data-config files and all are joined since i
need data from all of them.
But out of those 8 tables i have three tables that have common fields
,which i can use to link.But the issues is that the common fields in the
three table that i have are such that they are repeating values.
eg:-
First table(businessmasters) has 2 columns
Business_id(pk)Business_name Mod1(on update timestamp)
1ABC
2XYZ
3KOT
Second table(search_tag) has 3 columns
Search_tag_id(pk) Business_id Search_tag_name Mod2(on update timestamp)
1 1hair
2 1trimming
3 2massage
4 1facial
5 2makeup
Now i want to join the two tables and i do that on the basis of
Business_id.But since iam joining on Business_id which is a non key in
second table i just write that business_id is a key(because if i cant use
Search_tag_id as its not common in both tables and if i take Search_tag_id
as pk and join the tables on Business_id solr gives an error during delta
import
deltaQuery has no column to resolve to declared primary key).
But when i say that Business_id is pk and join tables on the basis of that
it works fine and delta import works.
But the issue comes when i try to change the Business_id in second table
eg i want to change the Business_id of Search_tag_id 3 to 1 and i run
delta import.The update adds the search tag to Business_id 1 but also
doesn't remove the search tag from Business_ID 2.The value of search tags
in Business_id 2 remains there ,where as it should go away if iam changing
Business_id from 2 to 1.Since now Business_id is not linked to any search
tag.
Iam not able to figure out how to resolve this issue.This is how i write it
in data-config file.
entity name=businessmasters pk=business_id query=SELECT Business_id,
Business_name,Mod1 from businessmasters
deltaImportQuery=SELECT Business_id,
Business_name,Mod1 from businessmasters where
business_id='${dih.delta.business_id}
deltaQuery=select business_id from businessmasters where modified
'${dih.last_index_time}'
field column=business_id name=id/
field column=business_name name=business_name indexed=true
stored=true /
field column=Mod1 name=Mod1 indexed=true stored=true /
entity name=search_tag pk=business_id query=SELECT
Search_tag_id,Search_tag_name from search_tag where
business_id='${businessmasters.business_id}'
deltaQuery=select business_id from search_tag where Mod2
'${dih.last_index_time}'
parentDeltaQuery=select business_id from businessmasters where
business_id=${search_tag.business_id}
field column=business_id name=id /
field column=Search_tag_id name=Search_tag_id indexed=true
stored=true /
field column=Search_tag_name name=Search_tag_name indexed=true
stored=true /
field column=Mod2 name=Mod2 indexed=true stored=true /
/entity
/entity
Thanks in advance iam new to solr and iam stuck on this issue.
--
Regards
Madhav Bahuguna
How to setup solr on the server
Hi, I have solr ready and configured and is running on my local machine.but now i want to upload my solr to the server. How do i do that and what are the things i need to keep in mind.Plus i also need to know how to configure a cron job i searched on the net and came up with this command,will this work fine on a linux server. 'http://localhost:8983/solr/db/dataimport?command=delta-import' The server that iam trying to upload solr to is a linux machine(debian). -- Regards Madhav Bahuguna
solr indexing not working when i try to insert 1000000 rows but works fine when i try to index 400000 rows or below
fine. Iam new to solr and Iam not able to figure out what the issues is. Please advise and help Regards Madhav Bahuguna
Re: solr indexing not working when i try to insert 1000000 rows but works fine when i try to index 400000 rows or below
Hi Ahmet, Thanks a ton. You were absolutely right the moment i added the line batchsize=-1 it worked . Thank you so much its been 7 days and i just could not figure out what the issue was. Its working like a charm now. Thanks Again regards Madhav Bahuguna On Tue, Jun 3, 2014 at 5:28 PM, Ahmet Arslan iori...@yahoo.com wrote: Hi Madhav, Just a guess, try using batchSize=-1 dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver batchSize=-1/ Ahmet On Tuesday, June 3, 2014 12:48 PM, madhav bahuguna madhav.bahug...@gmail.com wrote: HI iam using solr 4.7.1 and trying to do a full import.My data source is a table in mysql. It has 1000 rows and 20 columns. Whenever iam trying to do a full import solr stops responding. But when i try to do a import with a limit of 40 or less it works fine. If i try to import more than this solr wont index the result it either stops responding or will show indexing failed. In the error log it says Unable to execute query.But i dont understand how is the query running fine for lesser number of records but fails when i run more number of records My system config are follows CPU-i7 Ram -6Gb OS-64 bit windows 7 I am not able to figure out what the problem is ,i have tried increasing the max_allowed_packet to 1000M and even java heap size. please help thanks in advance *This is the error code* `Exception while processing: playername document : SolrInputDocument(fields: []):org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: SELECT player_id,firstname,lastname,value1,value2,value3,value4,value5,value6, value7,value8,value9,value10, value11,value18,value19,value20, country_id, playername_modtime,player_flag from playername WHERE 'true' != 'false' OR playername.playername_modtime '2014-05-23 10:38:56' Processing Document # 1 at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:71) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:281) at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:238) at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:42) at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59) at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:477) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:331) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:239) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:464) Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet successfully received from the server was 130,037 milliseconds ago. The last packet sent successfully to the server was 130,038 milliseconds ago. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at com.mysql.jdbc.Util.handleNewInstance(Util.java:409) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127) at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2288) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:2044) at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3549) at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:489) at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3240) at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2411) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2834) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2781) at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:908) at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:788) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:274) ... 12 more Caused by: java.io.EOFException: Can not read response from server. Expected to read 6 bytes, read 4 bytes before connection was unexpectedly lost. at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3161) at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2269) ... 23 more 5/23/2014 8:32:18 PM ERROR DataImporter Full Import
