Re: need advice on parent child mulitple category
hello any advice please -- View this message in context: http://lucene.472066.n3.nabble.com/need-advice-on-parent-child-mulitple-category-tp4214140p4214602.html Sent from the Solr - User mailing list archive at Nabble.com.
need advice on parent child mulitple category
hello all i need some best practice advice on nested categories i have a atom file which i am going to index in solr. The file has the below categorization as you can see below. the first four category comparison test, honda , kia and sedan are parent level Then the next two Civic is a child of Honda and Rio is a child of kia Then the next two 2015 is child of Civic and 2015 is a child of Rio comparison road test between honda 2015 and kia rio 2015 i was planning to store in solr the above categorization as comparison road test between honda 2015 and kia rio 2015 Comparison Test, Honda, Sedan, Kia Civic, Rio 2015 so far so good but i came across the following issue suppose if i have one more feed 2015 Honda Accord Maintenance and for this my solr doc is 2015 Honda Accord Maintenance Maintenance, Honda, Sedan Accord 2015 when i display the facets at cat1 level i see Comparison Test(1) Honda(2) Sedan (2) Kia(1) Maintenance(1) Once a user clicks Honda then our query will be cat1:Honda and facets on cat2 will show Civic Rio Accord as you can see Rio is not correct the fact which makes it complicated is many to many category mappings. any idea how to solve this. -- View this message in context: http://lucene.472066.n3.nabble.com/need-advice-on-parent-child-mulitple-category-tp4214140.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: filter on solr pivot data
Thanks for the reply but a filter query like -[* TO *] will give me vins which dont have a photo, it might qualify a dealer to show up but what if that dealer has other vin which has photo my requirement is i want to show the dealer only if all vin have no photos -- View this message in context: http://lucene.472066.n3.nabble.com/filter-on-solr-pivot-data-tp4178451p4179011.html Sent from the Solr - User mailing list archive at Nabble.com.
filter on solr pivot data
Hello i need to know how can i filter on solr pivot data. For exampel we have a dealer which might have many cars in his lot and car has photos, i need to find out a dealer which has cars which has no photos so i have dealer1 -> has 20 cars -> all of them has photos dealer2 -> has 20 cars -> some of them have photos dealer3 -> has 20 cars -> none of them have photos in the results i want to see only dealers which has no photos ie dealer3, i managed to do pivot and get a breakdown by vin and photo exists now i want to apply filter and get only those dealer who has all vin which have photo exists as 0 lst name="facet_pivot"> vin 1N4AA5AP0EC908535 1 mappings_|photo_exist| 1 1 vin 1N4AA5AP1EC470625 1 mappings_|photo_exist| 1 1 is it possible -- View this message in context: http://lucene.472066.n3.nabble.com/filter-on-solr-pivot-data-tp4178451.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Fetching document by comparing date to today date
Thanks for the quick answer i was able to solve my problem with the addition of two new fields and if todays date is april14 my query is (liveStartDate:[* TO 2014-04-14T00:00:00Z] AND liveEndDate:[2014-04-14T00:00:00Z TO *]) and its fetches me the correct document guess my initial question does solr provide out of the box functionality if i have the below three documents set, how can i get only doc2 if i assume todays date is 14-APRIL-2014 for simplicity i gave the liveDate as dates beginning at the start of each month but in real life these dates can be anything. -- View this message in context: http://lucene.472066.n3.nabble.com/Fetching-document-by-comparing-date-to-today-date-tp4130802p4130807.html Sent from the Solr - User mailing list archive at Nabble.com.
Fetching document by comparing date to today date
Hello i have come across many threads where people have asked how to fetch doc based on date comparison, my problem is pretty much on the same line. based on todays date i want to fetch documents which are live For example i have three doc below i want to select only one doc based on todays date, hence if today is 14-APR and if i run query liveDate:[* TO 14-APR-2014] its getting two doc i want to get only the latest one which is doc2. Is there out of the box method which can solve my issue. In order to fix this issue i proposed to have doc liveStartDate and liveEndDate Hence if today is 14-APR-2014 can i run a query where i can give a condition something like currentDate>liveStartDate AND currentDatehttp://lucene.472066.n3.nabble.com/Fetching-document-by-comparing-date-to-today-date-tp4130802.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Rounding date indexing to minute
Just to clarify when people mention rounding date till minute they mean to store seconds as 00 hence there is nothing like storing date in below format, or am i wrong. 2014-03-11T07:00Z Date are always stored in below format and by rounding people mean to store seconds as 00 so taht there are fewer terms -03-11Thh:mm:ssZ -- View this message in context: http://lucene.472066.n3.nabble.com/Rounding-date-indexing-to-minute-tp4129482p4129483.html Sent from the Solr - User mailing list archive at Nabble.com.
Rounding date indexing to minute
Hello can someone please tell me how to make sure in solr to store date only till minute level, since i am having issues with date range query performance. i read in forums to reduce date precision so that the queries become faster. As of now its storing date till seconds. 2014-03-11T07:00:00Z i am only concerned till minute granularity. Also the i am using solr.trieDateField is there a provision for this Please let me know thanks darniz -- View this message in context: http://lucene.472066.n3.nabble.com/Rounding-date-indexing-to-minute-tp4129482.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Filter query not working for time range
Hello is there a fix for the NOW rounding Otherwise i have to get current date and crreate a range query like * TO -MM-ddThh:mm:ssZ -- View this message in context: http://lucene.472066.n3.nabble.com/Filter-query-not-working-for-time-range-tp4122441p4122723.html Sent from the Solr - User mailing list archive at Nabble.com.
Filter query not working for time range
Hello
i have a field in solr
i need to get all documents whose cdStartDAte is null and also if the
cdStartDate is present then it should be from * TO NOW
i added the fq condition ((*:* AND -(cdStartDate:[* TO *])) OR
(cdStartDate:[* TO NOW]))
the issue its working fine except its only filtering document by date and
not considering time.
For example if today is March 9 2pm i get both the doc returns as you can
see the second document date is March 9, 8m yet it comes up in search
results any advice
{
"id": "9-MAR AT 7AM",
"title": "testCD valid from March 9 8PM",
"cdStartDate": "2014-03-09T07:00:00Z"
},
{
"id": "9-MAR AT 8PM",
"title": "testCD valid from March 9 8PM",
"cdStartDate": "2014-03-09T20:00:00Z"
}
--
View this message in context:
http://lucene.472066.n3.nabble.com/Filter-query-not-working-for-time-range-tp4122441.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
Thanks eric i think thats the way to go This leads to one more question. since now that i have two doc with the same content path i want to get distinct content path with max date. 1 /editorial-updates Sept content : Honda is releasing the car this month 2 /editorial-updates Oct content : Honda is releasing the car this month For example if a user searches for car text:car AND live_dt:[* TO NOW] then both the doc are returned. i want the max latest doc to come in the above case id=2 and the other document should not come, Just to add, i use dismax handler where we have boosting on specific fields and till now doc where returned by natural scoring order by dismax handler. if i add &sort=live_dt desc does it order doc purely by live_dt or also respect relevancy. since also have some other doc which dont have live_dt. any thoughts -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4095321.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
Thanks Eric Ok if we go by that proposal of copying all date fields into on bag_of_dates field Hence now we have a field and it will look something like this. 2013-09-01T00:00:00Z 2013-12-01T00:00:00Z Sept content : Honda is releasing the car this month Dec content : Toyota is releasing the car this month and i also agree now we can make a range query where bag_of_dates:[* TO NOW] AND text:Toyota but still how are we going to make sure the document should not get returned since toyota is only searchable from 1-DEC-2013 i hope i am able to explain it properly ON our website, when we render data we dont show this line "Dec content : Toyota is releasing the car this month" on the page since todays date is not 1-DEC-2013 yet. hence we dont want this doc to be shown in search result as well when we query solr -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4093961.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
Thanks Eric. i hope i understood correctly, but my main concern is i have to tie specific content indexed to a specific time range. and make that document come up in search results only for that time. As i have mentioned in my previous example we have multiple data-string structures which makes a bit more complicated, on top of that i don't know what will be the exact date. Hence if someone searches for toyota and if today is 6-OCT-2013 this doc should not come in search results since the keyword toyota should be searched only after 1-DEC-2013. 2013-09-01T00:00:00Z Sept content : Honda is releasing the car this month 2013-12-01T00:00:00Z Dec content : Toyota is releasing the car this month i dont know using a copy field might solve this or correct me if i am wrong. may be we are pursuing something which is not meant for Solr. Thanks Rashid -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4093790.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
i am assuming there is no solution or i have to handle it at index time. Any solr experts please -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4093414.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
Thanks Eric When i did solr in 2010 i thought now they might have evolved and allow doing query by providing wildcard in field name, but looks like i have to provide a concrete dynamic field name to query. Anyway will look in the catch all fields. Do you have any examples on how a catch all fields will help with this, or how my doc will look like and how can i query. darniz -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4092989.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
Hello i just wanted to make sure can we query dynamic fields using wildcard well if not then i dont think this solution might work, since i dont know the exact concrete name of the field. -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4092830.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Doing time sensitive search in solr
Thanks for the quick answers. i have gone thru the presentation and thats what i was tilting towards using dynamic fields i just want to run down an example so thats its clear about how to approach this issue. Sept content : Honda is releasing the car this month Dec content : Toyota is releasing the car this month After adding dynamic fields like *_entryDate and *_entryText my solr doc will look something like this. 2013-09-01T00:00:00Z Sept content : Honda is releasing the car this month 2013-12-01T00:00:00Z Dec content : Toyota is releasing the car this month if someone searches for a query something like *_entryDate:[* TO NOW] AND *_entryText:Toyota the results wont show up toyota in the search results. the only disadvantage we have with this approach is we might end up with a lot of runtime fields since we have thousands of entries which might be time bound in our cms. i might also do some more investigation to see if we can handle this at index time to index data as time comes some scheduler of something, because the above approach might solve the issue but may make the queries very slow. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273p4092763.html Sent from the Solr - User mailing list archive at Nabble.com.
Doing time sensitive search in solr
hello Users, i have a requirement where my content should be search based upon time. For example below is our content in our cms. Sept content : Honda is releasing the car this month Dec content : Toyota is releasing the car this month On the website based upon time we display the content. On the solr side, until now we were indexing all entries element in Solr in text field. Now after we introduced time sensitive information in our cms, i need to know if someone queries for word "Toyota" it should NOT come up in my search results since that content is going live in dec. The solr text field looks something like Honda is releasing the car this month Toyota is releasing this month is there a way we can search the text field or append any meta data to the text field based on date. i hope i have made the issue clear. i kind of don't agree with this kind of practice but our requirement is pretty peculiar since we don't want to reindex data again and again. -- View this message in context: http://lucene.472066.n3.nabble.com/Doing-time-sensitive-search-in-solr-tp4092273.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: error while doing partial update using curl
Thanks for the reply i checked the solr version and we do use exact version 4.0.0.2012.10.06.03.04.33 and also i have posted my solconfig.xml file snippet, and your assumption was correct, it did used XmlUpdateRequestHandler. Do you know whats the correct way to provide xml in order to do partial update. This wiki here suggested me to do partial updating, which is not working for me http://wiki.apache.org/solr/UpdateXmlMessages#Optional_attributes_for_.22field.22 -- View this message in context: http://lucene.472066.n3.nabble.com/error-while-doing-partial-update-using-curl-tp4022313p4022589.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: error while doing partial update using curl
i tried issuing a command using curl with xml syntax and it turns out that it replace my whole documents rather than updating a specific field this is what i gave, i got an impression providing update=set will only changes that field rather than reindexing the entire document. Any idea how to issue partial updates curl http://host:8080/solr/update?commit=true -H 'Content-type:text/xml' -d '63481697male' -- View this message in context: http://lucene.472066.n3.nabble.com/error-while-doing-partial-update-using-curl-tp4022313p4022416.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: error while doing partial update using curl
Sorry for urgency, but i tried many different things i would appreciate if anyone can provide solution for this. -- View this message in context: http://lucene.472066.n3.nabble.com/error-while-doing-partial-update-using-curl-tp4022313p4022408.html Sent from the Solr - User mailing list archive at Nabble.com.
error while doing partial update using curl
Hello
i am trying to update a filed in my solr doc using curl, dont know why its
giving me this error when i try to do this statment
curl 'myhostname:8080/solr/update?commit=true' -H
'Content-type:application/json' -d
'[{"id":"contentId#63481697","price":{"set":16595}}]'
i am getting this error
4000Unexpected
character '[' (code 91) in prolog; expected '<'
at [row,col {unknown-source}]: [1,1]400
any help will be appreciated.
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/error-while-doing-partial-update-using-curl-tp4022313.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting 411 Length required when adding docs
Hello
can anyone has any advice
This is the code i am using
server =new CommonsHttpSolrServer("http://www.mysolrserver.com/solr";);
Credentials def = new UsernamePasswordCredentials("xxx","xxx");
server.getHttpClient().getState().setCredentials(AuthScope.ANY,def);
server.getHttpClient().getParams().setAuthenticationPreemptive(true);
for (Obj obj:objects){
server.addBean(obj);
}
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/Getting-411-Length-required-when-adding-docs-tp3508372p3511501.html
Sent from the Solr - User mailing list archive at Nabble.com.
Inserting documents using get method
Hello All,
i am trying to insert document using the server.addBean(obj) method. somehow
i am getting HTTP error 411 Length required. After trying a lot i decided to
change my method from post to get.
if i open a browser and execute this query
mysolrserver/solr/update?stream.body=testTestL
it works which gives me a sense that the get request is working.Aslo i am
able to retreive documents
My question is whether the server.addBean(obj) uses post request or get
request behind the scene. If it uses post method isthere a way i can
configure CommonsHttpSolrServerit to insert document using get method.
Right now this is the below code which is failing
server =new CommonsHttpSolrServer("http://www.mysolrserver.com/solr";);
Credentials def = new UsernamePasswordCredentials("xxx","xxx");
server.getHttpClient().getState().setCredentials(AuthScope.ANY,def);
server.getHttpClient().getParams().setAuthenticationPreemptive(true);
for (Obj obj:objects){
server.addBean(obj);
}
Thanks
darniz
--
View this message in context:
http://lucene.472066.n3.nabble.com/Inserting-documents-using-get-method-tp3511499p3511499.html
Sent from the Solr - User mailing list archive at Nabble.com.
Getting 411 Length required when adding docs
Hello All, i am this strange issue of http 411 Length required error. My Solr is hosted on third party hosting company and it was working fine all these while. i really don't understand why this happened. Attached is the stack trace any help will be appreciated org.apache.solr.common.SolrException: Length Required Length Required request: http://www.listing-social.com/solr/update?wt=javabin&version=1 at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:424) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:243) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:64) at org.apache.solr.client.solrj.SolrServer.addBean(SolrServer.java:68) at com.listings.solr.service.impl.BulkIndexingServiceImpl.startBulkIndexing(BulkIndexingServiceImpl.java:55) at com.listings.action.BulkIndexingAction.execute(BulkIndexingAction.java:42) at org.apache.struts.chain.commands.servlet.ExecuteAction.execute(ExecuteAction.java:53) at org.apache.struts.chain.commands.AbstractExecuteAction.execute(AbstractExecuteAction.java:64) at org.apache.struts.chain.commands.ActionCommandBase.execute(ActionCommandBase.java:48) at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:190) at org.apache.commons.chain.generic.LookupCommand.execute(LookupCommand.java:304) at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:190) at org.apache.struts.chain.ComposableRequestProcessor.process(ComposableRequestProcessor.java:280) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1858) at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:446) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-411-Length-required-when-adding-docs-tp3508372p3508372.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: configure httpclient to access solr with user credential on third party host
thanks exaclty i asked my domain hosting provider and he provided me with some other port i am wondering can i specify credentials without the port i mean when i open the browser and i type www.mydomainmame/solr i get the tomcat auth login screen. in the same way can i configure the http client so that i dont have to specify the port Thanks darniz -- View this message in context: http://lucene.472066.n3.nabble.com/configure-httpclient-to-access-solr-with-user-credential-on-third-party-host-tp2360364p2364190.html Sent from the Solr - User mailing list archive at Nabble.com.
configure httpclient to access solr with user credential on third party host
Hello,
i uploaded solr.war file on my hosting provider and added security
constraint in web.xml file on my solr war so that only specific user with a
certain role can issue get and post request. When i open browser and type
www.maydomainname.com/solr i get a dialog box to enter userid and password.
No issues until now.
Now the issue is that i have one more app on the same tomcat container
which will index document into solr. In order for this app to issue post
request it has to configure the http client credentials. I checked with my
hosting service and they told me at tomcat is running on port 8834 since
apache is sitting in the front, the below is the code snipped i use to set
http credentials.
CommonsHttpSolrServer server = new
CommonsHttpSolrServer("http://localhost:8834/solr";);
Credentials defaultcreds = new
UsernamePasswordCredentials("solr","solr");
server.getHttpClient().getState().setCredentials(new
AuthScope("localhost",8834,AuthScope.ANY_REALM),
defaultcreds);
i am getting the following error, any help will be appreciated.
ERROR TP-Processor9 org.apache.jk.common.MsgAjp - BAD packet signature 20559
ERROR TP-Processor9 org.apache.jk.common.ChannelSocket - Error, processing
connection
java.lang.IndexOutOfBoundsException
at java.io.BufferedInputStream.read(BufferedInputStream.java:310)
at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:621)
at
org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:578)
at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:686)
at
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:891)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:619)
ERROR TP-Processor9 org.apache.jk.common.MsgAjp - BAD packet signature 20559
ERROR TP-Processor9 org.apache.jk.common.ChannelSocket - Error, processing
connection
java.lang.IndexOutOfBoundsException
at java.io.BufferedInputStream.read(BufferedInputStream.java:310)
at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:621)
at
org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:578)
at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:686)
at
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:891)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:619)
--
View this message in context:
http://lucene.472066.n3.nabble.com/configure-httpclient-to-access-solr-with-user-credential-on-third-party-host-tp2360364p2360364.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr on third party hosting account throws java.lang.RuntimeException: Cannot create directory: /usr/local/tomcat/a
Ok got it
The problem was very simple, thanks to post i just have to comment the
following line in solrconfig.xml
${solr.data.dir:./solr/data}
and that seems to do the trick
Thanks guys
--
View this message in context:
http://lucene.472066.n3.nabble.com/solr-on-third-party-hosting-account-throws-java-lang-RuntimeException-Cannot-create-directory-usr-loa-tp2204009p2207265.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr on third party hosting account throws java.lang.RuntimeException: Cannot create directory: /usr/local/tomcat/a
Thanks thats the bottle neck i am strugging with. i have configured the jndi lookup for solr home many times on my pc no issues, but just a bit clueless on how to go with a remote machine. here is te exact snipped i have in my web.xml please have a look and tell me is there something wrong with the way i declare. solr/home /home/myaccount/solr-home java.lang.String then i also treid this java:comp/env/solr/home /home/myaccount/solr-home java.lang.String but still no luck -- View this message in context: http://lucene.472066.n3.nabble.com/solr-on-third-party-hosting-account-throws-java-lang-RuntimeException-Cannot-create-directory-usr-loa-tp2204009p2207211.html Sent from the Solr - User mailing list archive at Nabble.com.
solr on third party hosting account throws java.lang.RuntimeException: Cannot create directory: /usr/local/tomcat/a
Hi Guys, I am using third party hosting i uploaded the solr war file. Stopped my tomcat, changed the web.xml and set the solr home manually there to /usr/myaccountname/solr-home As said in doc, i copied the entire contents under solr_dist/example/solr under my solr-home folder. i bounced the server and when i try to access solr, i get the following error, does this has something to do with my hosting account permission or i have set some environment variable using export command. Any help will be appreciated. HTTP Status 500 - Severe errors in solr configuration. Check your log files for more detailed information on what may be wrong. If you want solr to continue after configuration errors, change: false in null - java.lang.RuntimeException: Cannot create directory: /usr/local/tomcat/apache-tomcat-6.0.26/bin/./solr/data/index at org.apache.lucene.store.NativeFSLockFactory.acquireTestLock(NativeFSLockFactory.java:74) at org.apache.lucene.store.NativeFSLockFactory.makeLock(NativeFSLockFactory.java:137) at org.apache.lucene.store.Directory.makeLock(Directory.java:131) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1563) at org.apache.lucene.index.IndexWriter.(IndexWriter.java:1421) at org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:191) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:392) at org.apache.solr.core.SolrCore.(SolrCore.java:545) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:137) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:295) at -- View this message in context: http://lucene.472066.n3.nabble.com/solr-on-third-party-hosting-account-throws-java-lang-RuntimeException-Cannot-create-directory-usr-loa-tp2204009p2204009.html Sent from the Solr - User mailing list archive at Nabble.com.
Do we need index analyzer for query elevation component
Hello, In order for query elevation we define a type. do we really need index time analyzer for query elevation type. Let say we have some document already indexed and i added only the query time analyzer, looks like solr reads the words in elevate.xml and map words to the respective document. in that case why would we need index time analyzers, unless i am missing something. Please let me know darniz -- View this message in context: http://lucene.472066.n3.nabble.com/Do-we-need-index-analyzer-for-query-elevation-component-tp1077130p1077130.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Spell check returns strange suggestion
Adding on top of this there are business requirement we need to meet. Let me add we are having a huge content with make, model and their description. Right now we are copying make,model and description to our spelll check field. So our spellcheck fields have words like honda, civic etc.. We are also copying description to the spell text field So if the description is like this honda city is a left hand drive then words like hand, drive alos gets indexed in the spell check index. The issue is if someone types handa then in our context we would have liked to get spell suggesetions like honda instead solr returns back suggestions as hand since hand is a valid word in our dictionary. we could have blocked hand in stop words from being indexed to our fiedl but is this correct way to do things. i mean there might be many more terms which might resemble to other model names etc.. Does anyone has this kind of issue Thanks darniz darniz wrote: > > Thanks for the prompt reply > > i added the parameter > 0.7 > > to my config and this seems to take care of it. > Works which are present very close to the misspelled words seems to come > back now. > > darniz > > Markus Jelsma - Buyways B.V. wrote: >> >> darniz said: >>> >>> Hello All >>> Please reply to this ASAP >>> I am using indexbasedSpellchecker right now i copy only model, and make >>> names and some other fields to my spellcheck field. >>> Hence my spell check field consists of only 120 words. >>> >>> The issue is if i type hond i get back honda which is fine. But when i >>> type term like true i get back suggestion like ram. >> >> I'm not quite sure what you're telling us now but you are using >> onlyMorePopular=true which will almost always return suggestions except >> if >> the specified term is actually the most popular. >> >> Another good practice, only show the user suggestions if the >> correctlySpelled flag is really false. Many users seem to depend on the >> condition of availability of a collation. >> >> Try turning onlyMorePopular off or rely on the correctlySpelled flag you >> have. >> >> >>> >>> I read there are some configuration to make for distance measure. >>> Right now This is my spell check configuration >>> >>> default >>> searchSpellText >>> true >>> true >>> >> name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance >>> ./edmundsSpellcheckerDataIndex >>> >>> >>> and here is my query >>> q=true&version=2.2&start=0&rows=10&indent=on&spellcheck=true&spellcheck.field=edmundsSearchSpellText&spellcheck.collate=true&spellcheck.extendedResults=true&spellcheck.onlyMorePopular=true >>> >>> thanks >>> darniz >>> -- >>> View this message in context: >>> http://old.nabble.com/Spell-check-returns-strange-suggestion-tp27693520p27693520.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >> >> >> >> >> > > -- View this message in context: http://old.nabble.com/Spell-check-returns-strange-suggestion-tp27693520p27714312.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multiple SpellCheckRequestHandlers
I have the same issue i have two spell checker defined one is default which is index based and the other is file which is file based can i give something like in url where i specify both dictionary. &spellcheck=true&spellcheck.dictionary=file&spellcheck.dictionary=default Any advice will be appreciated. darniz Otis Gospodnetic wrote: > > I don't think so. I just prefer shorter (cleaner?) URLs. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > - Original Message >> From: solr_user >> To: solr-user@lucene.apache.org >> Sent: Tuesday, May 6, 2008 3:35:43 PM >> Subject: Re: Multiple SpellCheckRequestHandlers >> >> >> Thanks Otis, >> >> Actually, I am planning to make use of the qt parameter to specify >> which >> handler should be used for the query. Would there be any downside to >> that? >> >> >> >> Otis Gospodnetic wrote: >> > >> > Hello, >> > >> > If you configured "/sc1" and "/sc2", then use something like >> > http://../sc1?. for the first one and http://./sc2? for >> > the second one. >> > >> > Otis >> > -- >> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> > >> > - Original Message >> >> From: solr_user >> >> To: solr-user@lucene.apache.org >> >> Sent: Tuesday, May 6, 2008 1:57:17 PM >> >> Subject: Re: Multiple SpellCheckRequestHandlers >> >> >> >> >> >> And how do I specify in the query which requesthandler to use? >> >> >> >> >> >> >> >> Otis Gospodnetic wrote: >> >> > >> >> > Yes, just define two instances (with two distinct names) in >> >> solrconfig.xml >> >> > and point each of them to a different index. >> >> > >> >> > Otis >> >> > -- >> >> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> >> > >> >> > - Original Message >> >> >> From: solr_user >> >> >> To: solr-user@lucene.apache.org >> >> >> Sent: Tuesday, May 6, 2008 12:16:07 AM >> >> >> Subject: Multiple SpellCheckRequestHandlers >> >> >> >> >> >> >> >> >> Hi all, >> >> >> >> >> >> Is it possible in Solr to have multiple >> SpellCheckRequestHandlers. >> >> In >> >> >> my >> >> >> application I have got two different spell check indexes. I want >> the >> >> >> spell >> >> >> checker to check for a spelling suggestion in the first index and >> if >> >> it >> >> >> fails to get any suggestion from the first index only then it >> should >> >> try >> >> >> to >> >> >> get a suggestion from the second index. >> >> >> >> >> >> Is it possible to have a separate SpellCheckRequestHandler one >> for >> >> each >> >> >> index? >> >> >> >> >> >> Solr-User >> >> >> >> >> >> >> >> >> -- >> >> >> View this message in context: >> >> >> >> >> >> http://www.nabble.com/Multiple-SpellCheckRequestHandlers-tp17071568p17071568.html >> >> >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> >> >> >> >> >> >> > >> >> > >> >> > >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/Multiple-SpellCheckRequestHandlers-tp17071568p17088834.html >> >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> >> >> >> > >> > >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/Multiple-SpellCheckRequestHandlers-tp17071568p17090642.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > > > -- View this message in context: http://old.nabble.com/Multiple-SpellCheckRequestHandlers-tp17071568p27709265.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: 0.70 || Spell Checker Issue !!
Is there a way you can override accuracy with the html url parameters for
testing.
i dont want to change config file for each change and bounce server.
Thanks
darniz
Grant Ingersoll-6 wrote:
>
> Hmm, as far as I can tell, neither of those should give suggestions,
> as they both have edit distances less than 0.70.
>
> In the Lucene code, I ran the following test against the Levenshtein
> Distance:
> public void testJava() throws Exception {
> float d1 = sd.getDistance("jav", "java");
> System.out.println("di: " + d1);
> d1 = sd.getDistance("jav", "javas");
> System.out.println("di: " + d1);
>}
>
> It prints:
> di: 0.666
> di: 0.333
>
>
> Is "javas"/"java" the only entry in your spellings?
>
> I presume it does give you the expected suggestion or is it giving you
> something else?
>
> Also, what does your query look like?
>
> -Grant
>
>
>
> On Jan 14, 2009, at 12:33 PM, Navdeep wrote:
>
>>
>> Hi all
>>
>> The search with an accuracy defined by user in SOLRCONFIG.XML is not
>> working
>> for me. I have provided an example below.
>> Please let me know your valuable suggestion.
>>
>> For example:
>> Accuracy tag is used in Solrconfig.xml file in below manner.
>>
>>
>>textSpell
>>
>> default
>> solr.spelling.FileBasedSpellChecker
>> ./spellings.txt
>> UTF-8
>> ./spellcheckerFile
>> 0.70
>>
>>
>> As per above description, the accuracy set to 0.70 which means to 70%
>> The keyword ( java ) is stored in spellings.txt file which is also
>> mentioned
>> above.
>>
>> Now if I try to make search for "jav" word, there is no result found
>> (This
>> is the problem, why there is no result ??)
>>
>> where as
>> If I replace (java) keyword with (javas) in spellings.txt file and
>> keep the
>> accuracy 0.70 ...
>>
>> Now if I try to make search for "jav" word, I start getting the
>> results ...
>>
>> What is this behaviour ?? Does anyone knows what is the reason ...
>>
>> Regards,
>> Navdeep
>> --
>> View this message in context:
>> http://www.nabble.com/%3Cstr-name%3D%22accuracy%22%3E0.70%3C-str%3E-%7C%7C-Spell-Checker-Issue-%21%21-tp21460978p21460978.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
> --
> Grant Ingersoll
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
>
>
>
>
>
>
--
View this message in context:
http://old.nabble.com/%3Cstr-name%3D%22accuracy%22%3E0.70%3C-str%3E-%7C%7C-Spell-Checker-Issue-%21%21-tp21460978p27707896.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Spell check returns strange suggestion
Thanks for the prompt reply i added the parameter 0.7 to my config and this seems to take care of it. Works which are present very close to the misspelled words seems to come back now. darniz Markus Jelsma - Buyways B.V. wrote: > > darniz said: >> >> Hello All >> Please reply to this ASAP >> I am using indexbasedSpellchecker right now i copy only model, and make >> names and some other fields to my spellcheck field. >> Hence my spell check field consists of only 120 words. >> >> The issue is if i type hond i get back honda which is fine. But when i >> type term like true i get back suggestion like ram. > > I'm not quite sure what you're telling us now but you are using > onlyMorePopular=true which will almost always return suggestions except if > the specified term is actually the most popular. > > Another good practice, only show the user suggestions if the > correctlySpelled flag is really false. Many users seem to depend on the > condition of availability of a collation. > > Try turning onlyMorePopular off or rely on the correctlySpelled flag you > have. > > >> >> I read there are some configuration to make for distance measure. >> Right now This is my spell check configuration >> >> default >> searchSpellText >> true >> true >> > name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance >> ./edmundsSpellcheckerDataIndex >> >> >> and here is my query >> q=true&version=2.2&start=0&rows=10&indent=on&spellcheck=true&spellcheck.field=edmundsSearchSpellText&spellcheck.collate=true&spellcheck.extendedResults=true&spellcheck.onlyMorePopular=true >> >> thanks >> darniz >> -- >> View this message in context: >> http://old.nabble.com/Spell-check-returns-strange-suggestion-tp27693520p27693520.html >> Sent from the Solr - User mailing list archive at Nabble.com. > > > > > -- View this message in context: http://old.nabble.com/Spell-check-returns-strange-suggestion-tp27693520p27694636.html Sent from the Solr - User mailing list archive at Nabble.com.
Spell check returns strange suggestion
Hello All Please reply to this ASAP I am using indexbasedSpellchecker right now i copy only model, and make names and some other fields to my spellcheck field. Hence my spell check field consists of only 120 words. The issue is if i type hond i get back honda which is fine. But when i type term like true i get back suggestion like ram. I read there are some configuration to make for distance measure. Right now This is my spell check configuration default searchSpellText true true org.apache.lucene.search.spell.JaroWinklerDistance ./edmundsSpellcheckerDataIndex and here is my query q=true&version=2.2&start=0&rows=10&indent=on&spellcheck=true&spellcheck.field=edmundsSearchSpellText&spellcheck.collate=true&spellcheck.extendedResults=true&spellcheck.onlyMorePopular=true thanks darniz -- View this message in context: http://old.nabble.com/Spell-check-returns-strange-suggestion-tp27693520p27693520.html Sent from the Solr - User mailing list archive at Nabble.com.
Spell check returns strange suggestion
Hello All Please reply to this ASAP I am using indexbasedSpellchecker right now i copy only model, and make names and some other fields to my spellcheck field. Hence my spell check field consists of only 120 words. The issue is if i type hond i get back honda which is fine. But when i type term like true i get back suggestion like ram. I read there are some configuration to make for distance measure. Right now This is my spell check configuration default searchSpellText true true org.apache.lucene.search.spell.JaroWinklerDistance ./edmundsSpellcheckerDataIndex and here is my query q=true&version=2.2&start=0&rows=10&indent=on&spellcheck=true&spellcheck.field=edmundsSearchSpellText&spellcheck.collate=true&spellcheck.extendedResults=true&spellcheck.onlyMorePopular=true thanks darniz -- View this message in context: http://old.nabble.com/Spell-check-returns-strange-suggestion-tp27693522p27693522.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: spellcheck.build=true has no effect
Hello Can someone please correct me or acknowlege me is this the correct behaviour. Thanksdarniz darniz wrote: > > Hello All. > After doing a lot of research i came to this conclusion please correct me > if i am wrong. > i noticed that if you have buildonCommit and buildOnOptimize as true in > your spell check component, then the spell check builds whenever a commit > or optimze happens. which is the desired behaviour and correct. > please read on. > > I am using Index based spell checker and i am copying make and model to my > spellcheck field. i index some document and the make and model are being > copied to spellcheck field when i commit. > Now i stopped my solr server and > I added one more filed bodytype to be copied to my spellcheck field. > i dont want to reindex data so i issued a http request to rebuild my > spellchecker > &spellcheck=true&spellcheck.build=true&spellcheck.dictionary=default. > Looks like the above command has no effect, the bodyType is not being > copied to spellcheck field. > > The only time the spellcheck filed has bodyType value copied into it is > when i have to do again reindex document and do a commmit. > > Is this the desired behaviour. > Adding buildOncommit and buildOnOptimize will force the spellchecker to > rebuild only if a commit or optimize happens > Please let me know if there are some configurable parameters so that i can > issue the http command rather than indexing data again and again. > > > thanks > darniz > > -- View this message in context: http://old.nabble.com/spellcheck.build%3Dtrue-has-no-effect-tp27648346p27661847.html Sent from the Solr - User mailing list archive at Nabble.com.
spellcheck.build=true has no effect
Hello All. After doing a lot of research i came to this conclusion please correct me if i am wrong. i noticed that if you have buildonCommit and buildOnOptimize as true in your spell check component, then the spell check builds whenever a commit or optimze happens. which is the desired behaviour and correct. please read on. I am using Index based spell checker and i am copying make and model to my spellcheck field. i index some document and the make and model are being copied to spellcheck field when i commit. Now i stopped my solr server and I added one more filed bodytype to be copied to my spellcheck field. i dont want to reindex data so i issued a http request to rebuild my spellchecker &spellcheck=true&spellcheck.build=true&spellcheck.dictionary=default. Looks like the above command has no effect, the bodyType is not being copied to spellcheck field. The only time the spellcheck filed has bodyType value copied into it is when i have to do again reindex document and do a commmit. Is this the desired behaviour. Adding buildOncommit and buildOnOptimize will force the spellchecker to rebuild only if a commit or optimize happens Please let me know if there are some configurable parameters thanks darniz -- View this message in context: http://old.nabble.com/spellcheck.build%3Dtrue-has-no-effect-tp27648346p27648346.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting spelll checker index
Thanks If this is really the case, i declared a new filed called mySpellTextDup and retired the original field. Now i have a new field which powers my dictionary with no words in it and now i am free to index which ever term i want. This is not the best of solution but i cant think of a reasonable workaround Thanks darniz Lance Norskog-2 wrote: > > This is a quirk of Lucene - when you delete a document, the indexed > terms for the document are not deleted. That is, if 2 documents have > the word 'frampton' in an indexed field, the term dictionary contains > the entry 'frampton' and pointers to those two documents. When you > delete those two documents, the index contains the entry 'frampton' > with an empty list of pointers. So, the terms are still there even > when you delete all of the documents. > > Facets and the spellchecking dictionary build from this term > dictionary, not from the text string that are 'stored' and returned > when you search for the documents. > > The command throws away these remnant terms. > > http://www.lucidimagination.com/blog/2009/03/18/exploring-lucenes-indexing-code-part-2/ > > On Wed, Feb 17, 2010 at 12:24 PM, darniz wrote: >> >> Please bear with me on the limitted understanding. >> i deleted all documents and i made a rebuild of my spell checker  using >> the >> command >> spellcheck=true&spellcheck.build=true&spellcheck.dictionary=default >> >> After this i went to the schema browser and i saw that mySpellText still >> has >> around 2000 values. >> How can i make sure that i clean up that field. >> We had the same issue with facets too, even though we delete all the >> documents, and if we do a facet on make we still see facets but we can >> filter out facets by saying facet.mincount>0. >> >> Again coming back to my question how can i make mySpellText fields get >> rid >> of all previous terms >> >> Thanks a lot >> darniz >> >> >> >> hossman wrote: >>> >>> : But still i cant stop thinking about this. >>> : i deleted my entire index and now i have 0 documents. >>> : >>> : Now if i make a query with accrd i still get a suggestion of accord >>> even >>> : though there are no document returned since i deleted my entire index. >>> i >>> : hope it also clear the spell check index field. >>> >>> there are two Lucene indexes when you use spell checking. >>> >>> there is the "main" index which is goverend by your schema.xml and is >>> what >>> you add your own documents to, and what searches are run agains for the >>> result section of solr responses. >>> >>> There is also the "spell" index which has only two fields and in >>> which each "document" corrisponds to a "word" that might be returend as >>> a >>> spelling suggestion, and the other fields contain various >>> start/end/middle >>> ngrams that represent possible misspellings. >>> >>> When you use the spellchecker component it builds the "spell" index >>> makinga document out of every word it finds in whatever field name you >>> configure it to use. >>> >>> deleting your entire "main" index won't automaticly delete the "spell" >>> index (allthough you should be able rebuild the "spell" index using the >>> *empty* "main" index, that should work). >>> >>> : i am copying both fields to a field called >>> : >>> : >>> >>> ..at this point your "main" index has a field named mySpellText, and for >>> ever document it contains a copy of make and model. >>> >>> :     >>> :       default >>> :       mySpellText >>> :       true >>> :       true >>> >>> ...so whenever you commit or optimize your "main" index it will take >>> every >>> word from the mySpellText and use them all as individual documents in >>> the >>> "spell" index. >>> >>> In your previous email you said you changed hte copyField declaration, >>> and >>> then triggered a commit -- that rebuilt your "spell" index, but the data >>> was still all there in the mySpellText field of the "main" index, so the >>> rebuilt "spell" index was exactly the same. >>> >>> : i have buildOnOPtmize and buildOnCommit as true so when
Re: Deleting spelll checker index
Please bear with me on the limitted understanding. i deleted all documents and i made a rebuild of my spell checker using the command spellcheck=true&spellcheck.build=true&spellcheck.dictionary=default After this i went to the schema browser and i saw that mySpellText still has around 2000 values. How can i make sure that i clean up that field. We had the same issue with facets too, even though we delete all the documents, and if we do a facet on make we still see facets but we can filter out facets by saying facet.mincount>0. Again coming back to my question how can i make mySpellText fields get rid of all previous terms Thanks a lot darniz hossman wrote: > > : But still i cant stop thinking about this. > : i deleted my entire index and now i have 0 documents. > : > : Now if i make a query with accrd i still get a suggestion of accord even > : though there are no document returned since i deleted my entire index. i > : hope it also clear the spell check index field. > > there are two Lucene indexes when you use spell checking. > > there is the "main" index which is goverend by your schema.xml and is what > you add your own documents to, and what searches are run agains for the > result section of solr responses. > > There is also the "spell" index which has only two fields and in > which each "document" corrisponds to a "word" that might be returend as a > spelling suggestion, and the other fields contain various start/end/middle > ngrams that represent possible misspellings. > > When you use the spellchecker component it builds the "spell" index > makinga document out of every word it finds in whatever field name you > configure it to use. > > deleting your entire "main" index won't automaticly delete the "spell" > index (allthough you should be able rebuild the "spell" index using the > *empty* "main" index, that should work). > > : i am copying both fields to a field called > : > : > > ..at this point your "main" index has a field named mySpellText, and for > ever document it contains a copy of make and model. > > : > : default > : mySpellText > : true > : true > > ...so whenever you commit or optimize your "main" index it will take every > word from the mySpellText and use them all as individual documents in the > "spell" index. > > In your previous email you said you changed hte copyField declaration, and > then triggered a commit -- that rebuilt your "spell" index, but the data > was still all there in the mySpellText field of the "main" index, so the > rebuilt "spell" index was exactly the same. > > : i have buildOnOPtmize and buildOnCommit as true so when i index new > document > : i want my dictionary to be created but how can i make sure i remove the > : preivious indexed terms. > > everytime the spellchecker component "builds" it will create a completley > new "spell" index .. but if the old data is still in the "main" index then > it will also be in the "spell" index. > > The only reason i can think of why you'd be seeing words in your "spell" > index after deleting documents from your "main" index is that even if you > delete documents, the Terms are still there in the underlying index untill > the segments are merged ... so if you do an optimize that will force them > to be expunged --- but i honestly have no idea if that is what's causing > your problem, because quite frankly i really don't understand what your > problem is ... you have to provide specifics: reproducible steps anyone > can take using a clean install of solr to see the the behavior you are > seeing that seems incorrect. (ie: modifications to the example schema, > and commands to execute against hte demo port to see the bug) > > if you can provide details like that then it's possible to understand what > is going wrong for you -- which is a prereq to providing useful help. > > > > -Hoss > > > -- View this message in context: http://old.nabble.com/Deleting-spelll-checker-index-tp27376823p27629740.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting spelll checker index
Thanks Hoss Apology for flooding the post. But still i cant stop thinking about this. i deleted my entire index and now i have 0 documents. Now if i make a query with accrd i still get a suggestion of accord even though there are no document returned since i deleted my entire index. i hope it also clear the spell check index field. Let me give some history about what i am doing. i want my spellchecker to be indexed by make and model name both fields are of type string. i am copying both fields to a field called definition of field and field type is in the request handler dismax text 100 spellcheck and here is my spell check component default decleration textSpell default mySpellText true true org.apache.lucene.search.spell.JaroWinklerDistance ./mySpellcheckerDataIndex i have buildOnOPtmize and buildOnCommit as true so when i index new document i want my dictionary to be created but how can i make sure i remove the preivious indexed terms. Thanks darniz How can i reset my dictionary is there away to do it. hossman wrote: > > > : Any update on this > > Patience my friend ... 5 hours after you send an email isn't long enough > to wait before asking for "any update on this" -- it's just increasing the > volume of mail everyone gets and distracting people from actual > bugs/issues. > > FWIW: this doesn't really seem directly related to the thread you > initially started about Deleting the spell checker index -- what you're > asking about now is rebuilding the spellchecker index... > > : > I stop the sorl server removed the copy filed for model. now i only > copy > : > make to the spellText field and started solr server. > : > i refreshed the dictiaonry by issuring the following command. > : > spellcheck.build=true&spellcheck.dictionary=default > : > So i hope it should rebuild by dictionary, bu the strange thing is > that it > : > still gives a suggestion for accrd. > > that's because removing the copyField declaration doens't change anything > about the values that have already been copied to the "spellText" field > -- rebuilding your spellcheker index is just re-reading the same > indexed values from that field. > > : > How can i create the dictionary again by changing my schema and > issuing a > : > command > : > spellcheck.build=true&spellcheck.dictionary=default > > it's just not possible. a schema change like that doesn't magicly > undo all of the values that were already copied. > > > > -Hoss > > > -- View this message in context: http://old.nabble.com/Deleting-spelll-checker-index-tp27376823p27615354.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting spelll checker index
Any update on this Do you guys want to rephrase my question, if its not clear. Thanks darniz darniz wrote: > > HI Guys > Opening this thread again. > I need to get around this issue. > i have a spellcheck field defined and i am copying two fileds make and > model to this field > > > i have buildoncommit and buildonoptimize set to true hence when i index > data and try to search for a work accod i get back suggestion accord since > model is also being copied. > I stop the sorl server removed the copy filed for model. now i only copy > make to the spellText field and started solr server. > i refreshed the dictiaonry by issuring the following command. > spellcheck.build=true&spellcheck.dictionary=default > So i hope it should rebuild by dictionary, bu the strange thing is that it > still gives a suggestion for accrd. > I have to reindex data again and then it wont offer me suggestion which is > the correct behavour. > > How can i create the dictionary again by changing my schema and issuing a > command > spellcheck.build=true&spellcheck.dictionary=default > > i cant afford to reindex data everytime. > > Any answer ASAP will be appreciated > > Thanks > darniz > > > > > > > > > > darniz wrote: >> >> Then i assume the easiest way is to delete the directory itself. >> >> darniz >> >> >> hossman wrote: >>> >>> >>> : We are using Index based spell checker. >>> : i was wondering with the help of any url parameters can we delete the >>> spell >>> : check index directory. >>> >>> I don't think so. >>> >>> You might be able to configure two differnet spell check components that >>> point at the same directory -- one hat builds off of a real field, and >>> one >>> that builds off of an (empty) text field (using FileBasedSpellChecker) >>> .. >>> then you could trigger a rebuild of an empty spell checking index using >>> the second component. >>> >>> But i've never tried it so i have no idea if it would work. >>> >>> >>> -Hoss >>> >>> >>> >> >> > > -- View this message in context: http://old.nabble.com/Deleting-spelll-checker-index-tp27376823p27570613.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting spelll checker index
HI Guys Opening this thread again. I need to get around this issue. i have a spellcheck field defined and i am copying two fileds make and model to this field i have buildoncommit and buildonoptimize set to true hence when i index data and try to search for a work accod i get back suggestion accord since model is also being copied. I stop the sorl server removed the copy filed for model. now i only copy make to the spellText field and started solr server. i refreshed the dictiaonry by issuring the following command. spellcheck.build=true&spellcheck.dictionary=default So i hope it should rebuild by dictionary, bu the strange thing is that it still gives a suggestion for accrd. I have to reindex data again and then it wont offer me suggestion which is the correct behavour. How can i create the dictionary again by changing my schema and issuing a command spellcheck.build=true&spellcheck.dictionary=default i cant afford to reindex data everytime. Any answer ASAP will be appreciated Thanks darniz darniz wrote: > > Then i assume the easiest way is to delete the directory itself. > > darniz > > > hossman wrote: >> >> >> : We are using Index based spell checker. >> : i was wondering with the help of any url parameters can we delete the >> spell >> : check index directory. >> >> I don't think so. >> >> You might be able to configure two differnet spell check components that >> point at the same directory -- one hat builds off of a real field, and >> one >> that builds off of an (empty) text field (using FileBasedSpellChecker) .. >> then you could trigger a rebuild of an empty spell checking index using >> the second component. >> >> But i've never tried it so i have no idea if it would work. >> >> >> -Hoss >> >> >> > > -- View this message in context: http://old.nabble.com/Deleting-spelll-checker-index-tp27376823p27567465.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting spelll checker index
Then i assume the easiest way is to delete the directory itself. darniz hossman wrote: > > > : We are using Index based spell checker. > : i was wondering with the help of any url parameters can we delete the > spell > : check index directory. > > I don't think so. > > You might be able to configure two differnet spell check components that > point at the same directory -- one hat builds off of a real field, and one > that builds off of an (empty) text field (using FileBasedSpellChecker) .. > then you could trigger a rebuild of an empty spell checking index using > the second component. > > But i've never tried it so i have no idea if it would work. > > > -Hoss > > > -- View this message in context: http://old.nabble.com/Deleting-spelll-checker-index-tp27376823p27381620.html Sent from the Solr - User mailing list archive at Nabble.com.
Deleting spelll checker index
Hello all, We are using Index based spell checker. i was wondering with the help of any url parameters can we delete the spell check index directory. please let me know thans darniz -- View this message in context: http://old.nabble.com/Deleting-spelll-checker-index-tp27376823p27376823.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: matching exact/whole phrase
Extending this thread. Is it safe to say in order to do exact matches the field should be a string. Let say for example i have two fields on is caption which is of type string and the other is regular text. So if i index caption as "my car is the best car in the world" it will be stored and i copy the caption to the text field. Since text has all anylysers defined so lets assume only the following words are indexed after stop words and other filters "my", "car","best","world" Now in my dismax handler if i have the qf defined as text field and run a phrase search on text field "my car is the best car in the world" i dont get back any results. looking with debugQuery=on this is the parsedQuery text:"my tire pressure warning light came my honda civic" This will not work since text was indexed by removing all stop words. But if i remove the double quotes it matches that document. Now if i add extra query field &qf=caption and then do a phrase search i get back that document since caption is of type string and it maintains all the stop words and other stuff. Is my assumption correct. After i get a response i will put some more questions. Thanks darniz Sandeep Shetty-2 wrote: > > That was the answer I was looking for, I will try that one out > > Thanks Daniel > > -Original Message- > From: Daniel Papasian [mailto:daniel.papas...@chronicle.com] > Sent: 01 April 2008 16:03 > To: solr-user@lucene.apache.org > Subject: Re: matching exact/whole phrase > > Sandeep Shetty wrote: >> Hi people, >> >> I am looking to provide exact phrase match, along with the full text >> search with solr. I want to achieve the same effect in solr rather >> than use a separate SQL query. I want to do the following as an >> example >> >> The indexed field has the text "car repair" (without the double >> quotes) for a document and I want this document to come in the >> search result only if someone searches for "car repair". The document >> should not show up for "repair" and "car" searches. >> >> Is it possible to do this type of exact phrase matching if needed >> with solr itself? > > It sounds like you want to do an exact string match, and not a text > match, so I don't think there's anything complex you'd need to do... > just store the field with "car repair" as type="string" and do all of > the literal searches you want. > > But if you are working off a field that contains something beyond the > exact match of what you want to search for, you'll just need to define a > new field type and use only the analysis filters that you need, and > you'll have to think more about what you need if that's the case. > > Daniel > > Sandeep Shetty > Technical Development Manager > > Touch Local > 89 Albert Embankment, London, SE1 7TP, UK > D: 020 7840 4335 > E: sandeep.she...@touchlocal.com > T: 020 7840 4300 > F: 020 7840 4301 > > This email is confidential and may also be privileged. If you are not the > intended recipient please notify us immediately by calling 020 7840 4300 > or email postmas...@touchlocal.com. You should not copy it or use it for > any purpose nor disclose its contents to any other person. Touch Local Ltd > cannot accept liability for statements made which are clearly the sender's > own and are not made on behalf of the firm. > Registered in England and Wales. Registration Number: 2885607 VAT Number: > GB896112114 > > Help to save some trees. Print e-mails only if you really need to. > > -- View this message in context: http://old.nabble.com/matching-exact-whole-phrase-tp16424969p27329651.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: XmlUpdateRequestHandler with HTMLStripCharFilterFactory
no problem Erick Erickson wrote: > > Ah, I read your post too fast and ignored the title. Sorry 'bout that. > > Erick > > On Mon, Jan 11, 2010 at 2:55 PM, darniz wrote: > >> >> Well thats the whole discussion we are talking about. >> I had the impression that the html tags are filtered and then the field >> is >> stored without tags. But looks like the html tags are removed and terms >> are >> indexed purely for indexing, and the actual text is stored in raw format. >> >> Lets say for example if i enter a field like >> honda car road review >> When i do analysis on the body field the html filter removes the tag >> and >> indexed works honda, car, road, review. But when i fetch body field to >> display in my document it returns honda car road review >> >> I hope i make sense. >> thanks >> darniz >> >> >> >> Erick Erickson wrote: >> > >> > This page: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters >> > <http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters>shows you >> > many >> > of the SOLR analyzers and filters. Would one of >> > the various *HTMLStrip* stuff work? >> > >> > HTH >> > ERick >> > >> > On Mon, Jan 11, 2010 at 2:44 PM, darniz >> wrote: >> > >> >> >> >> Thanks we were having the saem issue. >> >> We are trying to store article content and we are strong a field like >> >> This article is for blah . >> >> Wheni see the analysis.jsp page it does strip out the tags and is >> >> indexed. but when we fetch the document it returns the field with the >> >> >> tags. >> >> From solr point of view, its correct but our issue is that this kind >> of >> >> html >> >> tags is screwing up our display of our page. Is there an easy way to >> >> esure >> >> how to strip out hte html tags, or do we have to take care of >> manually. >> >> >> >> Thanks >> >> Rashid >> >> >> >> >> >> aseem cheema wrote: >> >> > >> >> > Alright. It turns out that escapedTags is not for what I thought it >> is >> >> > for. >> >> > The problem that I am having with HTMLStripCharFilterFactory is that >> >> > it strips the html while indexing the field, but not while storing >> the >> >> > field. That is why what is see in analysis.jsp, which is index >> >> > analysis, does not match what gets stored... because.. well HTML is >> >> > stripped only for indexing. Makes so much sense. >> >> > >> >> > Thanks to Ryan McKinley for clarifying this. >> >> > Aseem >> >> > >> >> > On Wed, Nov 11, 2009 at 9:50 AM, aseem cheema >> >> >> > wrote: >> >> >> I am trying to post a document with the following content using >> SolrJ: >> >> >> content >> >> >> I need the xml/html tags to be ignored. Even though this works fine >> in >> >> >> analysis.jsp, this does not work with SolrJ, as the client escapes >> the >> >> >> < and > with < and > and HTMLStripCharFilterFactory does not >> >> >> strip those escaped tags. How can I achieve this? Any ideas will be >> >> >> highly appreciated. >> >> >> >> >> >> There is escapedTags in HTMLStripCharFilterFactory constructor. Is >> >> >> there a way to get that to work? >> >> >> Thanks >> >> >> -- >> >> >> Aseem >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Aseem >> >> > >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://old.nabble.com/XmlUpdateRequestHandler-with-HTMLStripCharFilterFactory-tp26305561p27116434.html >> >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> >> >> >> > >> > >> >> -- >> View this message in context: >> http://old.nabble.com/XmlUpdateRequestHandler-with-HTMLStripCharFilterFactory-tp26305561p27116601.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://old.nabble.com/XmlUpdateRequestHandler-with-HTMLStripCharFilterFactory-tp26305561p27118304.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: XmlUpdateRequestHandler with HTMLStripCharFilterFactory
Well thats the whole discussion we are talking about. I had the impression that the html tags are filtered and then the field is stored without tags. But looks like the html tags are removed and terms are indexed purely for indexing, and the actual text is stored in raw format. Lets say for example if i enter a field like honda car road review When i do analysis on the body field the html filter removes the tag and indexed works honda, car, road, review. But when i fetch body field to display in my document it returns honda car road review I hope i make sense. thanks darniz Erick Erickson wrote: > > This page: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters > <http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters>shows you > many > of the SOLR analyzers and filters. Would one of > the various *HTMLStrip* stuff work? > > HTH > ERick > > On Mon, Jan 11, 2010 at 2:44 PM, darniz wrote: > >> >> Thanks we were having the saem issue. >> We are trying to store article content and we are strong a field like >> This article is for blah . >> Wheni see the analysis.jsp page it does strip out the tags and is >> indexed. but when we fetch the document it returns the field with the >> tags. >> From solr point of view, its correct but our issue is that this kind of >> html >> tags is screwing up our display of our page. Is there an easy way to >> esure >> how to strip out hte html tags, or do we have to take care of manually. >> >> Thanks >> Rashid >> >> >> aseem cheema wrote: >> > >> > Alright. It turns out that escapedTags is not for what I thought it is >> > for. >> > The problem that I am having with HTMLStripCharFilterFactory is that >> > it strips the html while indexing the field, but not while storing the >> > field. That is why what is see in analysis.jsp, which is index >> > analysis, does not match what gets stored... because.. well HTML is >> > stripped only for indexing. Makes so much sense. >> > >> > Thanks to Ryan McKinley for clarifying this. >> > Aseem >> > >> > On Wed, Nov 11, 2009 at 9:50 AM, aseem cheema >> > wrote: >> >> I am trying to post a document with the following content using SolrJ: >> >> content >> >> I need the xml/html tags to be ignored. Even though this works fine in >> >> analysis.jsp, this does not work with SolrJ, as the client escapes the >> >> < and > with < and > and HTMLStripCharFilterFactory does not >> >> strip those escaped tags. How can I achieve this? Any ideas will be >> >> highly appreciated. >> >> >> >> There is escapedTags in HTMLStripCharFilterFactory constructor. Is >> >> there a way to get that to work? >> >> Thanks >> >> -- >> >> Aseem >> >> >> > >> > >> > >> > -- >> > Aseem >> > >> > >> >> -- >> View this message in context: >> http://old.nabble.com/XmlUpdateRequestHandler-with-HTMLStripCharFilterFactory-tp26305561p27116434.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://old.nabble.com/XmlUpdateRequestHandler-with-HTMLStripCharFilterFactory-tp26305561p27116601.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: XmlUpdateRequestHandler with HTMLStripCharFilterFactory
Thanks we were having the saem issue. We are trying to store article content and we are strong a field like This article is for blah . Wheni see the analysis.jsp page it does strip out the tags and is indexed. but when we fetch the document it returns the field with the tags. >From solr point of view, its correct but our issue is that this kind of html tags is screwing up our display of our page. Is there an easy way to esure how to strip out hte html tags, or do we have to take care of manually. Thanks Rashid aseem cheema wrote: > > Alright. It turns out that escapedTags is not for what I thought it is > for. > The problem that I am having with HTMLStripCharFilterFactory is that > it strips the html while indexing the field, but not while storing the > field. That is why what is see in analysis.jsp, which is index > analysis, does not match what gets stored... because.. well HTML is > stripped only for indexing. Makes so much sense. > > Thanks to Ryan McKinley for clarifying this. > Aseem > > On Wed, Nov 11, 2009 at 9:50 AM, aseem cheema > wrote: >> I am trying to post a document with the following content using SolrJ: >> content >> I need the xml/html tags to be ignored. Even though this works fine in >> analysis.jsp, this does not work with SolrJ, as the client escapes the >> < and > with < and > and HTMLStripCharFilterFactory does not >> strip those escaped tags. How can I achieve this? Any ideas will be >> highly appreciated. >> >> There is escapedTags in HTMLStripCharFilterFactory constructor. Is >> there a way to get that to work? >> Thanks >> -- >> Aseem >> > > > > -- > Aseem > > -- View this message in context: http://old.nabble.com/XmlUpdateRequestHandler-with-HTMLStripCharFilterFactory-tp26305561p27116434.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: NOT highlighting synonym
Thanks Unfortunately thats not the case. We are using the same field to do search on and display that text. So looks like in this case this is not possible Am i correct We have a custom field type with synonyms defined at query time. Erik Hatcher-4 wrote: > > > On Dec 23, 2009, at 2:26 PM, darniz wrote: >> i have a requirement where we dont want to hightlight synonym matches. >> for example i search for caddy and i dont want to highlight matched >> synonym >> like cadillac. >> Looking at highlighting parameters i didn't find any support for this. >> anyone can offer any advice. > > You can control what gets highlighted by which analyzer is used. You > may need a different field for highlighting than you use for searching > in this case - but you can just create another field type without the > synonym filter in it and use that for highlighting. > > Erik > > > -- View this message in context: http://old.nabble.com/NOT-highlighting-synonym-tp26906321p26945921.html Sent from the Solr - User mailing list archive at Nabble.com.
NOT highlighting synonym
Hi Guys. i have a requirement where we dont want to hightlight synonym matches. for example i search for caddy and i dont want to highlight matched synonym like cadillac. Looking at highlighting parameters i didn't find any support for this. anyone can offer any advice. darniz -- View this message in context: http://old.nabble.com/NOT-highlighting-synonym-tp26906321p26906321.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Synonym questions
i have a similar use case where i want to preserve original words forexamplei have one way mapping at index time monsoon,sony,levinson => audio system when i index document for example with word sony i want ot index audio system as well as the brand name ie sony. so if some one types category:sony i want highlighted resutls. if user types category:audio system i want the same document too. i initially thought there might be some property like preserveOriginal for synonym filter too like we had for worddelimiter tokeniser. please let me know is there any alternative. Thanks darniz hossman wrote: > > > : 1) For some of my synonyms, it seems to make senses to simply replace > the > : original word with the other (e.g. "theatre" => "theater", so searches > for > : either will find either). For others, I want to add an alternate term > while > : preserving the original (e.g. "cirque" => "circus", so searches for > "circus" > : find Cirque du Soleil, but searches for "cirque" only match "cirque", > not > : "circus". > > the simplest way i can think of to achieve something like this off the top > of my head is to do synonym injecting when idexing and don't use the > synonym filter at all when querying... > > theatre, theater => theatre, theater > circus => circus, cirque > > (but i haven't tested this to be sure it does what i think it does, and i > haven't thought about SynonymFilter very hard for a while now so i may be > forgetting something subtle) > > > > -Hoss > > > -- View this message in context: http://old.nabble.com/Synonym-questions-tp12078428p26896098.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Can we build complex filter queries in SOLR
Anyway i wrote a lucene syntax for this +((+category:audio +brand:sony)|(+category:video +brand:sony)) Thanks darniz Shalin Shekhar Mangar wrote: > > On Thu, Dec 10, 2009 at 2:50 AM, darniz wrote: > >> >> Can you please let me know how to describe that condition. >> >> >> For example lets say i want to give the following condition >> >> ((category:audio or category:video) AND (brand:sony OR brand:samsung OR >> brand:sanyo)) >> How would you represent this condition in fq paramenter of dismax >> >> > Are you saying that the above syntax does not work in an fq? Note, the > "or" > should be in capitals. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://old.nabble.com/Can-we-build-complex-filter-queries-in-SOLR-tp12735426p26731738.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: does fq parameter effects boosting
Thanks Yonik The question i was asking was that since filter queries are cached, if i change the relevancy model the cached queries wont be returned. correct me if i am wrong. Yonik Seeley-2 wrote: > > fq's are filters and have no effect on the relevancy scores generated > for documents. They only affect which documents are matched. > > -Yonik > http://www.lucidimagination.com > > > On Wed, Dec 9, 2009 at 5:00 PM, darniz wrote: >> >> Hello >> can somone please answer this. >> someone told me that using fq parameter in the dismax handler might cuase >> some relevancy and weighting issues. >> I haven't read this anywhere. >> Please let me know if this is the case. >> >> Thanks >> darniz >> -- >> View this message in context: >> http://old.nabble.com/does-fq-parameter-effects-boosting-tp26718016p26718016.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://old.nabble.com/does-fq-parameter-effects-boosting-tp26718016p26719680.html Sent from the Solr - User mailing list archive at Nabble.com.
does fq parameter effects boosting
Hello can somone please answer this. someone told me that using fq parameter in the dismax handler might cuase some relevancy and weighting issues. I haven't read this anywhere. Please let me know if this is the case. Thanks darniz -- View this message in context: http://old.nabble.com/does-fq-parameter-effects-boosting-tp26718016p26718016.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Can we build complex filter queries in SOLR
Can you please let me know how to describe that condition. For example lets say i want to give the following condition ((category:audio or category:video) AND (brand:sony OR brand:samsung OR brand:sanyo)) How would you represent this condition in fq paramenter of dismax condition goes here is it represented in lucene syntax. Please let me know darniz Alessandro Ferrucci-3 wrote: > > yeah that is possible, I just tried on one of my solr instances..let's say > you have an index of player names: > > (first-name:Tim AND last-name:Anderson) OR (first-name:Anwar AND > last-name:Johnson) OR (conference:Mountain West) > > will give you the results that logically match this query.. > > HTH. > > Alessandro Ferrucci :) > > On 9/17/07, Dilip.TS wrote: >> >> Hi, >> >> I would like to know if we can build a complex filter queryString in SOLR >> using the following condition. >> (Field1 = "abc" AND Field2 = "def") OR (Field3 = "abcd" AND >> Field4 >> = "defgh" AND (...)). >> so on... >> >> Thanks in advance >> >> Regards, >> Dilip TS >> >> > > -- View this message in context: http://old.nabble.com/Can-we-build-complex-filter-queries-in-SOLR-tp12735426p26717914.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: KStem download
: org.apache.solr.util.plugin.ResourceLoaderAware at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) ... 55 more Dec 8, 2009 10:24:11 AM org.apache.solr.common.SolrException log SEVERE: java.lang.NoClassDefFoundError: org/apache/solr/util/plugin/ResourceLoaderAware at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:621) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124) at java.net.URLClassLoader.defineClass(URLClassLoader.java:260) at java.net.URLClassLoader.access$000(URLClassLoader.java:56) Jay Hill wrote: > > The two jar files are all you should need, and the configuration is > correct. > However I noticed that you are on Solr 1.3. I haven't tested the Lucid > KStemmer on a non-Lucid-certified distribution of 1.3. I have tested it on > recent versions of 1.4 and it works fine (just tested with the most recent > nightly build). > > So there are two options, but I don't know if either will work for you: > 1. Move up to Solr 1.4, copy over the jars and configure. > 2. Get the free Lucid certified distribution of 1.3 which already has the > Lucid KStemmer (and other fixes which are an improvement over the standard > 1.3). > > -Jay > http://www.lucidimagination.com > > > On Mon, Sep 14, 2009 at 6:09 PM, darniz wrote: > >> >> i was able to declare a field type when the i use the lucid distribution >> of >> solr >> >> >> >>> class="com.lucidimagination.solrworks.analysis.LucidKStemFilterFactory" >> protected="protwords.txt" /> >> >> >> >> But if i copy the two jars and put it in lib directory of apache solr >> distribution it still gives me the following error. >> >> SEVERE: java.lang.NoClassDefFoundError: >> org/apache/solr/util/plugin/ResourceLoaderAware >>at java.lang.ClassLoader.defineClass1(Native Method) >>at java.lang.ClassLoader.defineClass(ClassLoader.java:621) >>at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124) >>at java.net.URLClassLoader.defineClass(URLClassLoader.java:260) >>at java.net.URLClassLoader.access$000(URLClassLoader.java:56) >>at java.net.URLClassLoader$1.run(URLClassLoader.java:195) >>at java.security.AccessController.doPrivileged(Native Method) >>at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >>at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >>at java.lang.ClassLoader.loadClass(ClassLoader.java:252) >>at >> >> org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:375) >>at >> >> org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:337) >>at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) >>at java.lang.Class.forName0(Native Method) >>at java.lang.Class.forName(Class.java:247) >>at >> >> org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:257) >>at >> >> org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:278) >>at >> >> org.apache.solr.util.plugin.AbstractPluginLoader.create(AbstractPluginLoader.java:83) >>at >> >> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140) >>at >> org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:781) >>at >> org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:56) >>at >> org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:413) >>at >> org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:431) >>at >> >> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140) >>at >> org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:440) >>at org.apache.solr.schema.IndexSchema.(IndexSchema.java:92) >>at org.apache.solr.core.SolrCore.(SolrCore.java:412) >>at >> >> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:119) >>at >> org.apache.solr.servlet.S
Re: java.lang.NumberFormatException: For input string: ""
its strange i had a dismaxhandler and it had an empty value for ps field i added a default value like 100 and the error disappeared. markrmiller wrote: > > Can you share the config files? > > darniz wrote: >> Hello All, >> >> i am getting this exception when i start solr. >> when i use hte original schema file and config file it is fine, but when >> we >> put our own schema file it gives the error. >> i made sure we dont have any documents in our index. >> Still we get this error, any idea >> Cant figure out which field is causing the problem >> >> SEVERE: java.lang.NumberFormatException: For input string: "" >> at >> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) >> at java.lang.Integer.parseInt(Integer.java:468) >> at java.lang.Integer.valueOf(Integer.java:553) >> at >> org.apache.solr.common.util.DOMUtil.addToNamedList(DOMUtil.java:132) >> at >> org.apache.solr.common.util.DOMUtil.nodesToNamedList(DOMUtil.java:98) >> at >> org.apache.solr.common.util.DOMUtil.childNodesToNamedList(DOMUtil.java:88) >> at >> org.apache.solr.common.util.DOMUtil.addToNamedList(DOMUtil.java:142) >> at >> org.apache.solr.common.util.DOMUtil.nodesToNamedList(DOMUtil.java:98) >> at >> org.apache.solr.common.util.DOMUtil.childNodesToNamedList(DOMUtil.java:88) >> at org.apache.solr.core.PluginInfo.(PluginInfo.java:54) >> at >> org.apache.solr.core.SolrConfig.readPluginInfos(SolrConfig.java:220) >> at >> org.apache.solr.core.SolrConfig.loadPluginInfo(SolrConfig.java:212) >> at org.apache.solr.core.SolrConfig.(SolrConfig.java:184) >> at >> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:134) >> at >> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83) >> at >> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) >> at >> org.mortbay.jetty.servlet.Context.startContext(Context.java:139) >> at >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >> at >> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) >> at org.mortbay.jetty.Server.doStart(Server.java:210) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.mortbay.start.Main.invokeMain(Main.java:183) >> at org.mortbay.start.Main.start(Main.java:497) >> at org.mortbay.start.Main.main(Main.java:115) >> > > > -- View this message in context: http://old.nabble.com/java.lang.NumberFormatException%3A-For-input-string%3A-%22%22-tp26631247p26632600.html Sent from the Solr - User mailing list archive at Nabble.com.
java.lang.NumberFormatException: For input string: ""
Hello All, i am getting this exception when i start solr. when i use hte original schema file and config file it is fine, but when we put our own schema file it gives the error. i made sure we dont have any documents in our index. Still we get this error, any idea Cant figure out which field is causing the problem SEVERE: java.lang.NumberFormatException: For input string: "" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:468) at java.lang.Integer.valueOf(Integer.java:553) at org.apache.solr.common.util.DOMUtil.addToNamedList(DOMUtil.java:132) at org.apache.solr.common.util.DOMUtil.nodesToNamedList(DOMUtil.java:98) at org.apache.solr.common.util.DOMUtil.childNodesToNamedList(DOMUtil.java:88) at org.apache.solr.common.util.DOMUtil.addToNamedList(DOMUtil.java:142) at org.apache.solr.common.util.DOMUtil.nodesToNamedList(DOMUtil.java:98) at org.apache.solr.common.util.DOMUtil.childNodesToNamedList(DOMUtil.java:88) at org.apache.solr.core.PluginInfo.(PluginInfo.java:54) at org.apache.solr.core.SolrConfig.readPluginInfos(SolrConfig.java:220) at org.apache.solr.core.SolrConfig.loadPluginInfo(SolrConfig.java:212) at org.apache.solr.core.SolrConfig.(SolrConfig.java:184) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:134) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) -- View this message in context: http://old.nabble.com/java.lang.NumberFormatException%3A-For-input-string%3A-%22%22-tp26631247p26631247.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Implementing phrase autopop up
can anybody update me if its possible that a word within a phrase is match, that phrase can be displayed. darniz darniz wrote: > > Thanks for your input > You made a valid point, if we are using field type as text to get > autocomplete it wont work because it goes through tokenizer. > Hence looks like for my use case i need to have a field which uses ngram > and copy. Here is what i did > > i created a filed as same as the lucid blog says. > > omitNorms="true" omitTermFreqAndPositions="true"/> > > with the following field configurtion > > positionIncrementGap="100"> > − > > > > maxGramSize="25"/> > > − > > > > > > > Now when i query i get the correct phrases for example if search for > autocomp:"how to" i get all the correct phrases like > > How to find a car > How to find a mechanic > How to choose the right insurance company > > etc... which is good. > > Now I have two question. > 1) Is it necessary to give the query in quote. My gut feeling is yes, > since if you dont give quote i get phrases beginning with How followed by > some other words like How can etc... > > 2)if i search for word for example choose, it gives me nothing > I was expecting to see a result considering there is a word "choose" in > the phrase > How to choose the right insurance company > > i might look more at documentation but do you have anything to advice. > > darniz > > > > > > > > > > Shalin Shekhar Mangar wrote: >> >> On Tue, Nov 24, 2009 at 10:12 AM, darniz wrote: >> >>> >>> hello all >>> Let me first explain the task i am trying to do. >>> i have article with title for example >>> >>> >Car Insurance for Teenage Drivers >>> >>> − >>> >>> A Total Loss? >>> >>> If a user begins to type car insu i want the autopop to show up with the >>> entire phrase. >>> There are two ways to implement this. >>> First is to use the termcomponent and the other is to use a field with >>> field >>> type which uses solr.EdgeNGramFilterFactor filter. >>> >>> I started with using with Term component and i declared a term request >>> handler and gave the following query >>> >>> http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car >>> The issue is that its not giving the entire pharse, it gives me back >>> results >>> like car, caravan, carbon. Now i know using terms.prefix will only give >>> me >>> results where the sentence start with car. On top of this i also want if >>> there is word like car somewhere in between the title that should also >>> show >>> up in autopop very much similar like google where a word is not >>> necessarily >>> start at the beginning but it could be present anywhere in the middle of >>> the >>> title. >>> The question is does TermComponent is a good candidate or using a >>> custom >>> field lets the name is autoPopupText with field type configured with all >>> filter and EdgeNGramFilterFactor defined and copying the title to the >>> autoPopupText field and using it to power autopopup. >>> >>> The other thing is that using EdgeNGramFilterFactor is more from index >>> point of view when you index document you need to know which fields you >>> want >>> to copy to autoPopupText field where as using Term component is more >>> like >>> you can define at query time what fields you want to use to fetch >>> autocomplete from. >>> >>> Any idea whats the best and why the Term component is not giving me an >>> entire phrase which i mentioned earlier. >>> FYI >>> my title field is of type text. >>> >> >> >> You are using a tokenized field type with TermsComponent therefore each >> word >> in your phrase gets indexed as a separate token. You should use a >> non-tokenized type (such as a string type) with TermsComponent. However, >> this will only let you search by prefix and not by words in between the >> phrase. >> >> Your best bet here would be to use EdgeNGramFilterFactory. If your index >> is >> very large, you can consider doing a prefix search on shingles too. >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- View this message in context: http://old.nabble.com/Implementing-phrase-autopop-up-tp26490419p26506470.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Implementing phrase autopop up
Thanks for your input You made a valid point, if we are using field type as text to get autocomplete it wont work because it goes through tokenizer. Hence looks like for my use case i need to have a field which uses ngram and copy. Here is what i did i created a filed as same as the lucid blog says. with the following field configurtion − − Now when i query i get the correct phrases for example if search for autocomp:"how to" i get all the correct phrases like How to find a car How to find a mechanic How to choose the right insurance company etc... which is good. Now I have two question. 1) Is it necessary to give the query in quote. My gut feeling is yes, since if you dont give quote i get phrases beginning with How followed by some other words like How can etc... 2)if i search for word for example choose, it gives me nothing I was expecting to see a result considering there is a word "choose" in the phrase How to choose the right insurance company i might look more at documentation but do you have anything to advice. darniz Shalin Shekhar Mangar wrote: > > On Tue, Nov 24, 2009 at 10:12 AM, darniz wrote: > >> >> hello all >> Let me first explain the task i am trying to do. >> i have article with title for example >> >> >Car Insurance for Teenage Drivers >> >> − >> >> A Total Loss? >> >> If a user begins to type car insu i want the autopop to show up with the >> entire phrase. >> There are two ways to implement this. >> First is to use the termcomponent and the other is to use a field with >> field >> type which uses solr.EdgeNGramFilterFactor filter. >> >> I started with using with Term component and i declared a term request >> handler and gave the following query >> >> http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car >> The issue is that its not giving the entire pharse, it gives me back >> results >> like car, caravan, carbon. Now i know using terms.prefix will only give >> me >> results where the sentence start with car. On top of this i also want if >> there is word like car somewhere in between the title that should also >> show >> up in autopop very much similar like google where a word is not >> necessarily >> start at the beginning but it could be present anywhere in the middle of >> the >> title. >> The question is does TermComponent is a good candidate or using a custom >> field lets the name is autoPopupText with field type configured with all >> filter and EdgeNGramFilterFactor defined and copying the title to the >> autoPopupText field and using it to power autopopup. >> >> The other thing is that using EdgeNGramFilterFactor is more from index >> point of view when you index document you need to know which fields you >> want >> to copy to autoPopupText field where as using Term component is more like >> you can define at query time what fields you want to use to fetch >> autocomplete from. >> >> Any idea whats the best and why the Term component is not giving me an >> entire phrase which i mentioned earlier. >> FYI >> my title field is of type text. >> > > > You are using a tokenized field type with TermsComponent therefore each > word > in your phrase gets indexed as a separate token. You should use a > non-tokenized type (such as a string type) with TermsComponent. However, > this will only let you search by prefix and not by words in between the > phrase. > > Your best bet here would be to use EdgeNGramFilterFactory. If your index > is > very large, you can consider doing a prefix search on shingles too. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://old.nabble.com/Implementing-phrase-autopop-up-tp26490419p26499912.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: initiate reindexing in solr for field type changes
thanks darniz Shalin Shekhar Mangar wrote: > > On Thu, Nov 19, 2009 at 4:50 AM, darniz wrote: > >> >> Thanks >> Could you elaborate what is compatible schema change. >> Do you mean schema change which deals only with query time. >> >> > A compatible schema change would be addition of new fields. Removal of > fields may also be called compatible as long as your application does not > try to index or query them. > > Modifying the field type of an existing field or adding/removing/modifying > tokenizers or filters on a field type is usually an incompatible change > and > needs re-indexing of affected documents. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26499804.html Sent from the Solr - User mailing list archive at Nabble.com.
Implementing phrase autopop up
hello all Let me first explain the task i am trying to do. i have article with title for example >Car Insurance for Teenage Drivers − A Total Loss? If a user begins to type car insu i want the autopop to show up with the entire phrase. There are two ways to implement this. First is to use the termcomponent and the other is to use a field with field type which uses solr.EdgeNGramFilterFactor filter. I started with using with Term component and i declared a term request handler and gave the following query http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car The issue is that its not giving the entire pharse, it gives me back results like car, caravan, carbon. Now i know using terms.prefix will only give me results where the sentence start with car. On top of this i also want if there is word like car somewhere in between the title that should also show up in autopop very much similar like google where a word is not necessarily start at the beginning but it could be present anywhere in the middle of the title. The question is does TermComponent is a good candidate or using a custom field lets the name is autoPopupText with field type configured with all filter and EdgeNGramFilterFactor defined and copying the title to the autoPopupText field and using it to power autopopup. The other thing is that using EdgeNGramFilterFactor is more from index point of view when you index document you need to know which fields you want to copy to autoPopupText field where as using Term component is more like you can define at query time what fields you want to use to fetch autocomplete from. Any idea whats the best and why the Term component is not giving me an entire phrase which i mentioned earlier. FYI my title field is of type text. Thanks darniz -- View this message in context: http://old.nabble.com/Implementing-phrase-autopop-up-tp26490419p26490419.html Sent from the Solr - User mailing list archive at Nabble.com.
Embedded solr with third party libraries
Hi We are having issue running our test cases with third party library for embedded solr. For exampel we are using kstem library which is not a part of solr distirbution. When we run test cases our schema.xml has defintion for lucid kstem and it throws ClassNotFound Exception. We declared the depency for the two jars lucid-kstem.jar and lucid-solr-kstem.jar but still it throws an error. Till now whenever we have to run embedded solr we manually copy all the config files like schema.xml, solrConfig.xml to a temp directory which is considered solr home and its generally under the user home directory which will copy all config files to user_home/solr/conf/directory. below is the example C:\DOCUME~1\username\LOCALS~1\Temp\solr-all\0.8194571792905493\solr\conf\schema.xml Now in order for the jar to be loaded should i copy the two jars to solr/lib directory. is that the default location embedded solr looks into for some default jars. Any advice. -- View this message in context: http://old.nabble.com/Embedded-solr-with-third-party-libraries-tp26452534p26452534.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: initiate reindexing in solr for field type changes
Thanks Could you elaborate what is compatible schema change. Do you mean schema change which deals only with query time. darniz Otis Gospodnetic wrote: > > Darniz, > > Yes, if there is an incompatible schema change, you need to reindex your > documents. > > Otis > P.S. > Please include the copy of the response when replying, so the > context/background of your question is easy to figure out. > -- > Sematext is hiring -- http://sematext.com/about/jobs.html?mls > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR > > > > - Original Message >> From: darniz >> To: solr-user@lucene.apache.org >> Sent: Wed, November 18, 2009 1:30:06 PM >> Subject: Re: initiate reindexing in solr for field type changes >> >> >> Thanks >> So going by you reply, can i assume that if there is a configuration >> change >> to my schema I have to again index documents, >> There is no short cut of updating the index. >> Because we cant afford to index 2 million documents again and again. >> There should be some utility or command line which does this things in >> the >> background. >> >> i hope i make sense. >> >> darniz >> -- >> View this message in context: >> http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26413172.html >> Sent from the Solr - User mailing list archive at Nabble.com. > > > -- View this message in context: http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26417644.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: initiate reindexing in solr for field type changes
Thanks So going by you reply, can i assume that if there is a configuration change to my schema I have to again index documents, There is no short cut of updating the index. Because we cant afford to index 2 million documents again and again. There should be some utility or command line which does this things in the background. i hope i make sense. darniz -- View this message in context: http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26413172.html Sent from the Solr - User mailing list archive at Nabble.com.
initiate reindexing in solr for field type changes
Hi users, i wanted to know is there a way we can initialte solr indexing. I mean for example i have a field which was of type string and i indexed 100 documents. When i change the field to text i dont want to load the document again, i should be able to just run a command line and the documents should be indexed again with the updated field type. The same applies with stemmer. for example we were using Porter stemmer and we indexed like 2 million docs. Now we want to change the configuration to Kstemmer and for this to take effect we have to again bulk load all 2 million docs. Is there a way to run the indexer again. Please let me know. Thanks darniz -- View this message in context: http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26397067.html Sent from the Solr - User mailing list archive at Nabble.com.
using different field for search and boosting
hello i wanted to know if its possible to search on one field and provide boosting relevancy on other fields. For example if i have fields like make, model, description etc and all are copied to text field. So can i define a handler where i do a search on text field but can define relevancy models on make,model and description ie make^4 model^2 Any advice. -- View this message in context: http://old.nabble.com/using-different-field-for-search-and-boosting-tp26260479p26260479.html Sent from the Solr - User mailing list archive at Nabble.com.
lucid kstem group and artifact id to put in POM
Hello Right now we are using lucid Kstemmer and it works fine and the two jars required "lucid-kstem.jar" and "lucid-solr-kstem.jar" are present in our web app. i am trying to get hold of groupid and artifact so that i can plug it into maven to download this two files from POM. i searched the maven repo at http://repo2.maven.org/maven2 and can find the Kstem definition Any advice darniz -- View this message in context: http://old.nabble.com/lucid-kstem-group-and-artifact-id-to-put-in-POM-tp26163608p26163608.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Spell check suggestion and correct way of implementation and some Questions
Thanks i included the buildOncommit and buildOnOptimize as true and indexed some documents and it automatically builds the dictionary. Are there any performance issues we should be aware of, with this approach. Rashid -- View this message in context: http://old.nabble.com/Spell-check-suggestion-and-correct-way-of-implementation-and-some-Questions-tp26096664p26162724.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Spell check suggestion and correct way of implementation and some Questions
Hello everybody i am able to use spell checker but i have some questions if someone can answer this if i search free text word waranty then i get back suggestion warranty which is fine. but if do a search on field for example description:waranty the output collation element is description:warranty which i dont want i want to get back only the text ie warranty. We are using collation to return back the results since if a user types three words then we use collation in the response element to display the spelling suggestion. Any advice darniz -- View this message in context: http://old.nabble.com/Spell-check-suggestion-and-correct-way-of-implementation-and-some-Questions-tp26096664p26157895.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Spell check suggestion and correct way of implementation and some Questions
Hello everybody i am able to use spell checker but i have some questions if someone can answer this if i search free text word waranty then i get back suggestion warranty which is fine. but if do a search on field for example description:waranty the output collation element is description:warranty which i dont want i want to get back only the text ie warranty. We are using collation to return back the results since if a user types three words then we use collation in the response element to display the spelling suggestion. Any advice darniz -- View this message in context: http://old.nabble.com/Spell-check-suggestion-and-correct-way-of-implementation-and-some-Questions-tp26096664p26157893.html Sent from the Solr - User mailing list archive at Nabble.com.
Spell check suggestion and correct way of implementation and some Questions
Hi Solr masters, I just wanted to confirm what i read is the best and optimal way of doing spell check correction. i am using index based spell check. For this as the doc says Create a field ex. my_text with a fieldType my_text_fl which has all the filters defined. Copy all the othe fields data to the field my_text using copy features. In the solr config file in the searchComponent which is spellcheck add the below code mySpellChecker spell_text org.apache.lucene.search.spell.JaroWinklerDistance ./mySpellCheckerDataIndex build the dictionary using the get command Question. Should i build the dictionlary only once and after that as new words are indexed the dictionary will be updated. Or i to do that manually over certain interval. add the spellcheck component to the handler in my case as of now standard requets handler. I might also start adding some more dismax handlers depending on my requirement explicit spellcheck run the query with parameter spell.check=true, and also specify against which dictionary you want to run spell check again in my case my spellcheck.dictionary parameter is mySpellChecker. Please let know if this the best way since i have to start implementing it. thanks darniz -- View this message in context: http://www.nabble.com/Spell-check-suggestion-and-correct-way-of-implementation-and-some-Questions-tp26096664p26096664.html Sent from the Solr - User mailing list archive at Nabble.com.
Question regarding proximity search
Hi I would appreciate if someone can throw some light on the following point regarding proximity search. i have a search box and if a use comes and type in "honda car" WITHOUT any double quotes, i want to get all documents with matches, and also they should be ranked based on proximity. i.e. the more the two terms are nearer the more is the rank. >From the admin looks like in order to test proximity i have to always give the word in double quote and a slop value http://localhost:8983/solr/select/?q="honda+car"~12&version=2.2&start=0&rows=10&indent=on Hence looks like from admin point of view in order to do proximity i have to always give it in double quotes. My questions is in order to do proximity search we always have to pass the query as a phrase ie. in double quotes. The next question is that i thought using dismax handler i could do a search on a field and i can specify the ps value in order to define proximity. Here is my request handler decleration dismax text 6 text * true false true true and here are my documents, i am loading just three documents in my index to test this Honda:1 Honda Civic Honda civic is a nice car. this car won the award at LA auto show Honda civic is a nice car. Honda:2 Honda CRV Honda is a nice car. This car won the best award at Detroit car show Honda CRV is a midsize suv and this comes in the category of car. Honda:3 Honda Pilot nothing Honda Pilot car. No matter where you go, on any terrain a four wheel honda pilot will give you the most amazing driving experience and this is the query i am giving and i get back no results. any advice where i am going wrong http://localhost:8983/solr/proxTest/?q="honda car" Thanks darniz -- View this message in context: http://www.nabble.com/Question-regarding-proximity-search-tp25829639p25829639.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Question regarding synonym
yes that's what we decided to expand these terms while indexing. if we have bayrische motoren werke => bmw and i have a document which has bmw in it, searching for text:bayrische does not give me results. i have to give text:"bayrische motoren werke" then it actually takes the synonym and gets me the document. Now if i change the synonym mapping to bayrische motoren werke , bmw with expand parameter to true and also use this file at indexing. now at the time i index this document along with "bmw" i also index the following words "bayrische" "motoren" "werke" any text query like text:motoren or text:bayrische will give me results now. Please correct me if my assumption is wrong. Thanks darniz Christian Zambrano wrote: > > > > On 10/02/2009 06:02 PM, darniz wrote: >> Thanks >> As i said it even works by giving double quotes too. >> like carDescription:"austin martin" >> >> So is that the conclusion that in order to map two word synonym i have to >> always enclose in double quotes, so that it doen not split the words >> >> >> >> > Yes, but there are things you need to keep in mind. > > From the solr wiki: > > Keep in mind that while the SynonymFilter will happily work with > *synonyms* containing multiple words (ie: > "sea biscuit, sea biscit, seabiscuit") The recommended approach for > dealing with *synonyms* like this, is to expand the synonym when > indexing. This is because there are two potential issues that can arrise > at query time: > >1. > > The Lucene QueryParser tokenizes on white space before giving any > text to the Analyzer, so if a person searches for the words > sea biscit the analyzer will be given the words "sea" and "biscit" > seperately, and will not know that they match a synonym. > >2. > > Phrase searching (ie: "sea biscit") will cause the QueryParser to > pass the entire string to the analyzer, but if the SynonymFilter > is configured to expand the *synonyms*, then when the QueryParser > gets the resulting list of tokens back from the Analyzer, it will > construct a MultiPhraseQuery that will not have the desired > effect. This is because of the limited mechanism available for the > Analyzer to indicate that two terms occupy the same position: > there is no way to indicate that a "phrase" occupies the same > position as a term. For our example the resulting MultiPhraseQuery > would be "(sea | sea | seabiscuit) (biscuit | biscit)" which would > not match the simple case of "seabisuit" occuring in a document > > >> >> >> >> >> >> >> >> Christian Zambrano wrote: >> >>> When you use a field qualifier(fieldName:valueToLookFor) it only applies >>> to the word right after the semicolon. If you look at the debug >>> infomation you will notice that for the second word it is using the >>> default field. >>> >>> carDescription:austin >>> *text*:martin >>> >>> the following should word: >>> >>> carDescription:(austin martin) >>> >>> >>> On 10/02/2009 05:46 PM, darniz wrote: >>> >>>> This is not working when i search documents i have a document which >>>> contains >>>> text aston martin >>>> >>>> when i search carDescription:"austin martin" i get a match but when i >>>> dont >>>> give double quotes >>>> >>>> like carDescription:austin martin >>>> there is no match >>>> >>>> in the analyser if i give austin martin with out quotes, when it passes >>>> through synonym filter it matches aston martin , >>>> may be by default analyser treats it as a phrase "austin martin" but >>>> when >>>> i >>>> try to do a query by typing >>>> carDescription:austin martin i get 0 documents. the following is the >>>> debug >>>> node info with debugQuery=on >>>> >>>> carDescription:austin martin >>>> carDescription:austin martin >>>> carDescription:austin text:martin >>>> carDescription:austin >>>> text:martin >>>> >>>> dont know why it breaks the word, may be its a desired behaviour >>>> when i give carDescription:"austin martin" of course in this its able >>>> to >>>> map >>>> to synonym and i get the desired result >>>> >>>> Any opinion >>>> >>>> darniz >>>> >>>> >>>> >>>> Ensdorf Ken wrote: >>>> >>>> >>>>> >>>>> >>>>>> Hi >>>>>> i have a question regarding synonymfilter >>>>>> i have a one way mapping defined >>>>>> austin martin, astonmartin => aston martin >>>>>> >>>>>> >>>>>> >>>>> ... >>>>> >>>>> >>>>>> Can anybody please explain if my observation is correct. This is a >>>>>> very >>>>>> critical aspect for my work. >>>>>> >>>>>> >>>>> That is correct - the synonym filter can recognize multi-token >>>>> synonyms >>>>> from consecutive tokens in a stream. >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> > > -- View this message in context: http://www.nabble.com/Question-regarding-synonym-tp25720572p25754288.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Question regarding synonym
Thanks As i said it even works by giving double quotes too. like carDescription:"austin martin" So is that the conclusion that in order to map two word synonym i have to always enclose in double quotes, so that it doen not split the words Christian Zambrano wrote: > > When you use a field qualifier(fieldName:valueToLookFor) it only applies > to the word right after the semicolon. If you look at the debug > infomation you will notice that for the second word it is using the > default field. > > carDescription:austin *text*:martin > > the following should word: > > carDescription:(austin martin) > > > On 10/02/2009 05:46 PM, darniz wrote: >> This is not working when i search documents i have a document which >> contains >> text aston martin >> >> when i search carDescription:"austin martin" i get a match but when i >> dont >> give double quotes >> >> like carDescription:austin martin >> there is no match >> >> in the analyser if i give austin martin with out quotes, when it passes >> through synonym filter it matches aston martin , >> may be by default analyser treats it as a phrase "austin martin" but when >> i >> try to do a query by typing >> carDescription:austin martin i get 0 documents. the following is the >> debug >> node info with debugQuery=on >> >> carDescription:austin martin >> carDescription:austin martin >> carDescription:austin text:martin >> carDescription:austin text:martin >> >> dont know why it breaks the word, may be its a desired behaviour >> when i give carDescription:"austin martin" of course in this its able to >> map >> to synonym and i get the desired result >> >> Any opinion >> >> darniz >> >> >> >> Ensdorf Ken wrote: >> >>> >>>> Hi >>>> i have a question regarding synonymfilter >>>> i have a one way mapping defined >>>> austin martin, astonmartin => aston martin >>>> >>>> >>> ... >>> >>>> Can anybody please explain if my observation is correct. This is a very >>>> critical aspect for my work. >>>> >>> That is correct - the synonym filter can recognize multi-token synonyms >>> from consecutive tokens in a stream. >>> >>> >>> >>> >> > > -- View this message in context: http://www.nabble.com/Question-regarding-synonym-tp25720572p25723980.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Question regarding synonym
This is not working when i search documents i have a document which contains text aston martin when i search carDescription:"austin martin" i get a match but when i dont give double quotes like carDescription:austin martin there is no match in the analyser if i give austin martin with out quotes, when it passes through synonym filter it matches aston martin , may be by default analyser treats it as a phrase "austin martin" but when i try to do a query by typing carDescription:austin martin i get 0 documents. the following is the debug node info with debugQuery=on carDescription:austin martin carDescription:austin martin carDescription:austin text:martin carDescription:austin text:martin dont know why it breaks the word, may be its a desired behaviour when i give carDescription:"austin martin" of course in this its able to map to synonym and i get the desired result Any opinion darniz Ensdorf Ken wrote: > >> Hi >> i have a question regarding synonymfilter >> i have a one way mapping defined >> austin martin, astonmartin => aston martin >> > ... >> >> Can anybody please explain if my observation is correct. This is a very >> critical aspect for my work. > > That is correct - the synonym filter can recognize multi-token synonyms > from consecutive tokens in a stream. > > > -- View this message in context: http://www.nabble.com/Question-regarding-synonym-tp25720572p25723829.html Sent from the Solr - User mailing list archive at Nabble.com.
Question regarding synonym
Hi i have a question regarding synonymfilter i have a one way mapping defined austin martin, astonmartin => aston martin what baffling me is that if i give at query time the word austin martin it first goes through white space and generate two words in analysis page "austin" and "martin" then after synonym filter it replace it with words aston martin Thats good and thats what i want but i am wodering sicne it went to white space tokeniser first and split the word in to two different word "austin" and "martin" how come it was able to map the entire synonym and replace it. If i give only austin the after passing thruough synonym filter it does not replace it with aston. That leads me to conclude that even though "austin martin" went thru whitespace tokenizer factory and got split into two the word ordering is still preserved to find a synonym match. Can anybody please explain if my observation is correct. This is a very critical aspect for my work. Thanks darniz -- View this message in context: http://www.nabble.com/Question-regarding-synonym-tp25720572p25720572.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: KStem download
i checked jar and the presence of ResourceLoaderAware interface is there. Looks like its a versioning issue. darniz wrote: > > Ok i downlaod the lucid imaginationversion of Solr. > > From the lib directory i copied the two jars > lucid-kstem.jar and lucid-solr-kstem.jar > > and put in my local solr instance > at > C:\solr\apache-solr-1.3.0\lib > > When i declare a field type like this > > > > protected="protwords.txt" /> > > > > its throwing class not found exception. > > Is there some other files which i am missing. > > Please let me know thanks > > darniz > > Yonik Seeley-2 wrote: >> >> On Mon, Sep 14, 2009 at 1:56 PM, darniz wrote: >>> Pascal Dimassimo wrote: >>>> >>>> Hi, >>>> >>>> I want to try KStem. I'm following the instructions on this page: >>>> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem >>>> >>>> ... but the download link doesn't work. >>>> >>>> Is anyone know the new location to download KStem? >>>> >>> I am stuck with the same issue >>> its link is not working for a long time >>> >>> >>> is there any alternate link >>> Please let us know >> >> *shrug* - looks like they changed their download structure (or just >> took it down). I searched around their site a bit but couldn't find >> another one (and google wasn't able to find it either). >> >> The one from Lucid is functionally identical, free, and much, much >> faster though - I'd just use that. >> >> -Yonik >> http://www.lucidimagination.com >> >> > > -- View this message in context: http://www.nabble.com/KStem-download-tp24375856p25446263.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: KStem download
i was able to declare a field type when the i use the lucid distribution of solr But if i copy the two jars and put it in lib directory of apache solr distribution it still gives me the following error. SEVERE: java.lang.NoClassDefFoundError: org/apache/solr/util/plugin/ResourceLoaderAware at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:621) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124) at java.net.URLClassLoader.defineClass(URLClassLoader.java:260) at java.net.URLClassLoader.access$000(URLClassLoader.java:56) at java.net.URLClassLoader$1.run(URLClassLoader.java:195) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:375) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:337) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:257) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:278) at org.apache.solr.util.plugin.AbstractPluginLoader.create(AbstractPluginLoader.java:83) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140) at org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:781) at org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:56) at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:413) at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:431) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:440) at org.apache.solr.schema.IndexSchema.(IndexSchema.java:92) at org.apache.solr.core.SolrCore.(SolrCore.java:412) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:119) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) Caused by: java.lang.ClassNotFoundException: org.apache.solr.util.plugin.ResourceLoaderAware at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) ... 53 more Even though i chec
Re: KStem download
Ok i downlaod the lucid imaginationversion of Solr. >From the lib directory i copied the two jars lucid-kstem.jar and lucid-solr-kstem.jar and put in my local solr instance at C:\solr\apache-solr-1.3.0\lib When i declare a field type like this its throwing class not found exception. Is there some other files which i am missing. Please let me know thanks darniz Yonik Seeley-2 wrote: > > On Mon, Sep 14, 2009 at 1:56 PM, darniz wrote: >> Pascal Dimassimo wrote: >>> >>> Hi, >>> >>> I want to try KStem. I'm following the instructions on this page: >>> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem >>> >>> ... but the download link doesn't work. >>> >>> Is anyone know the new location to download KStem? >>> >> I am stuck with the same issue >> its link is not working for a long time >> >> >> is there any alternate link >> Please let us know > > *shrug* - looks like they changed their download structure (or just > took it down). I searched around their site a bit but couldn't find > another one (and google wasn't able to find it either). > > The one from Lucid is functionally identical, free, and much, much > faster though - I'd just use that. > > -Yonik > http://www.lucidimagination.com > > -- View this message in context: http://www.nabble.com/KStem-download-tp24375856p25440692.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: KStem download
Ok i downlaod the lucid imaginationversion of Solr. >From the lib directory i copied the two jars lucid-kstem.jar and lucid-solr-kstem.jar and put in my local solr instance at C:\solr\apache-solr-1.3.0\lib When i declare a field type like this its throwing class not found exception. Is there some other files which i am missing. Please let me know thanks Rashid Yonik Seeley-2 wrote: > > On Mon, Sep 14, 2009 at 1:56 PM, darniz wrote: >> Pascal Dimassimo wrote: >>> >>> Hi, >>> >>> I want to try KStem. I'm following the instructions on this page: >>> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem >>> >>> ... but the download link doesn't work. >>> >>> Is anyone know the new location to download KStem? >>> >> I am stuck with the same issue >> its link is not working for a long time >> >> >> is there any alternate link >> Please let us know > > *shrug* - looks like they changed their download structure (or just > took it down). I searched around their site a bit but couldn't find > another one (and google wasn't able to find it either). > > The one from Lucid is functionally identical, free, and much, much > faster though - I'd just use that. > > -Yonik > http://www.lucidimagination.com > > -- View this message in context: http://www.nabble.com/KStem-download-tp24375856p25440690.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: KStem download
Pascal Dimassimo wrote: > > Hi, > > I want to try KStem. I'm following the instructions on this page: > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem > > ... but the download link doesn't work. > > Is anyone know the new location to download KStem? > I am stuck with the same issue its link is not working for a long time is there any alternate link Please let us know darniz -- View this message in context: http://www.nabble.com/KStem-download-tp24375856p25440432.html Sent from the Solr - User mailing list archive at Nabble.com.
Question regarding Stemmer
Hi i want to get some answers to some of my questions. Going by the Solr Wiki There are three approaches for Stemming Porter or Reduction Algorithm As far as i know there is "solr.EnglishPorterFilterFactory" and there is "solr.SnowballPorterFilterFactory" Both uses the same stemming algorithm. Hence i assume the only difference is that SnowBall allows you to specify a language. is my asssumption correct? The other thing is that the wiki talks about "solr.PorterStemFilterFactory" and that uses PorterStemming Algorithm but doesn't have any example specifying how to declare a field type with that kind of stemmer. Could any body give some snippet. Expanssion Stemming By using SynonymFilterFactory no comments Kstem which is less aggressive stemmer When i click the download the jar it doesn't work for the past two days. Dont know exactlty when it will be working. Thanks darniz -- View this message in context: http://www.nabble.com/Question-regarding-Stemmer-tp25409688p25409688.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SnowballPorterFilterFactory stemming word question
The link to download kstem is not working. Any other link please Yonik Seeley-2 wrote: > > On Mon, Sep 7, 2009 at 2:49 AM, darniz wrote: >> Does solr provide any implementation for dictionary stemmer, please let >> me >> know > > The Krovetz stemmer is dictionary based (english only): > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem > > But from your original question, maybe you are concerned when the > stemmer doesn't return real words? For normal search, don't be. > During index time, words are stemmed, and then later the query is > stemmed. If the results match up, you're good. For example, a > document containing the word "machines" may stem to "machin" and then > a query of "machined" will stem to "machin" and thus match the > document. > > > -Yonik > http://www.lucidimagination.com > > -- View this message in context: http://www.nabble.com/SnowballPorterFilterFactory-stemming-word-question-tp25180310p25404615.html Sent from the Solr - User mailing list archive at Nabble.com.
Using EnglishPorterFilterFactory in code
hello i have a task where my user is giving me 20 words of english dictionary and i have to run a program and generate a report with all stemmed words. I have to use EnglishPorterFilterFactory and SnowballPorterFilterFactory to check which one is faster and gets the best results Should i write a java module and use the library which comes with solr. is there any code snipped which i can use Is there any utiltiy which solr provides. If i can get a faint idea of how to do it is to create EnglishPorterFilter from EnglishPorterFilterFactory by passing a tokenizer etc... i will appreciate if some one can give me a hint on this. thanks darniz -- View this message in context: http://www.nabble.com/Using-EnglishPorterFilterFactory-in-code-tp25393325p25393325.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SnowballPorterFilterFactory stemming word question
Thanks Yonik i have a task where my user is giving me 20 words of english dictionary and i have to run a program and generate a report with all stemmed words. I have to use EnglishPorterFilterFactory and SnowballPorterFilterFactory to check which one is faster and gets the best results Should i write a java module and use the library which comes with solr. is there any code snipped which i can use If i can get a faint idea of how to do it is to create EnglishPorterFilter from EnglishPorterFilterFactory by passing a tokenizer etc... i will appreciate if some one can give me a hint on this. thanks darniz Yonik Seeley-2 wrote: > > On Mon, Sep 7, 2009 at 2:49 AM, darniz wrote: >> Does solr provide any implementation for dictionary stemmer, please let >> me >> know > > The Krovetz stemmer is dictionary based (english only): > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem > > But from your original question, maybe you are concerned when the > stemmer doesn't return real words? For normal search, don't be. > During index time, words are stemmed, and then later the query is > stemmed. If the results match up, you're good. For example, a > document containing the word "machines" may stem to "machin" and then > a query of "machined" will stem to "machin" and thus match the > document. > > > -Yonik > http://www.lucidimagination.com > > -- View this message in context: http://www.nabble.com/SnowballPorterFilterFactory-stemming-word-question-tp25180310p25393323.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SnowballPorterFilterFactory stemming word question
Thanks Hoss Could you please provide with any example Does solr provide any implementation for dictionary stemmer, please let me know Thanks Rashid hossman wrote: > > > : If i give "machine" why is that it stems to "machin", now from where > does > : this word come from > : If i give "revolutionary" it stems to "revolutionari", i thought it > should > : stem to revolution. > : > : How does stemming work? > > the porter stemmer (and all of the stemmers provided with solr) are > programtic stemmers ... they don't actually know the root of any words the > use an aproximate algorithm to compute a *token* from a word based on a > set of rules ... these tokens aren't neccessarily real words (and most of > the time they aren't words) but the same token tends to be produced from > words with similar roots. > > if you want to see the actaul root word, you'll have to use a dictionary > based stemmer. > > > -Hoss > > > -- View this message in context: http://www.nabble.com/SnowballPorterFilterFactory-stemming-word-question-tp25180310p25325738.html Sent from the Solr - User mailing list archive at Nabble.com.
SnowballPorterFilterFactory stemming word question
i have a field defined in my schema.xml file If i analyse this field type in analysis.jsp, the follwoing are the results if i give "running" its stems word to run which is fine If i give "machine" why is that it stems to "machin", now from where does this word come from If i give "revolutionary" it stems to "revolutionari", i thought it should stem to revolution. How does stemming work? Does it reduces adverb to verb etc..., or we have to customize it. Please let me know Thanks -- View this message in context: http://www.nabble.com/SnowballPorterFilterFactory-stemming-word-question-tp25180310p25180310.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr.StopFilterFactory not filtering words
Thanks Yonik So its basically how the field is indexed and not stored. So i give "the elephant is an animal" and try to get back the document it should see the entire string, only the index is done on elephant and animal. i was of the impression that when solr loads that document it strips out those words in order to check that i did a q on "the elephant" and when i look at the output "the elephant" "the elephant" +DisjunctionMaxQuery((onlynoun:elephant)) () Is this the correct way of evaluating your index are correctly done. One last doubt about this is that when i declared the field i gave analyser as index. what is the difference if i give the analyzer as query. -- View this message in context: http://www.nabble.com/solr.StopFilterFactory-not-filtering-words-tp25123903p25139099.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr.StopFilterFactory not filtering words
Thanks Yonik So the stopFilter works is that if i give a string like "the elephant is an animal", and when i retrieve the document the stored value will always be the same, only the index will be done on "elephant" and "animal". I was of the impression that Solr automatically takes out that words when it is even storing the value. How can i confirm that those are the only two terms indexex. AS you said i gave the following query to check is that the way it was indexed so if i give q on "animal" it should return me that document, but i did a q on "the animal" it should not return me that document since "the" was removed while indexing. Please let me know my assumption is correct, if this is the case then even i did a q on "the elephant" it still returns me that document. q="the%20elphant"&version=2.2&start=0&rows=10&indent=on&qf=onlynoun^5 And this the field how entered in to Solr an elphant is an animal Thanks -- View this message in context: http://www.nabble.com/solr.StopFilterFactory-not-filtering-words-tp25123903p25138502.html Sent from the Solr - User mailing list archive at Nabble.com.
solr.StopFilterFactory not filtering words
HI i dont understand whats wrong here. i am trying to implement solr.StopFilterFactory here is my field type definition and i have field which is when i try to insert a doc like an elphant is an animal it does not remove an and is from the phrase. Any idea why Thanks -- View this message in context: http://www.nabble.com/solr.StopFilterFactory-not-filtering-words-tp25123903p25123903.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: defining qf in your own request handler
Continuing on this i am having a use case where i have to strip out single quote for certain fields for example for testing i added teh following fieldType in schema.xml file and then i declared a field like this. So i am assuming if i index a document with a field like this can't be the case i am expecting this should be index as "this can t be the case" But that doesn't seem to happen Any quick input on this Thanks -- View this message in context: http://www.nabble.com/defining-qf-in-your-own-request-handler-tp25119913p25123890.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: defining qf in your own request handler
Thanks it working now. Moving on the same topic i need two more answers 1) is there a parameter where i can flush out the browser cache. Most of the time i see cached content in brower and i have to delete the temp files folder. 2) the other thing i wanted to know how to do stem search. Does dismax allows you to do it. for example if i say q=thriller i should get all documents which have text like thrilling, thrillls etc... Sorry i could have given a better example cant think of any. but is there a stem search option in this. 3)Lastly could you please explain what the following clauses in dismax decleration means true false true true Thanks -- View this message in context: http://www.nabble.com/defining-qf-in-your-own-request-handler-tp25119913p25120889.html Sent from the Solr - User mailing list archive at Nabble.com.
defining qf in your own request handler
Hello i created a custom request handler and i want it to do a search on features and make field by default. i added the following decleration in my solrconfig.xml file 12 explicit * model desc features,make as you can see in the qf i defined features and make to be the default search fields, the issue is that if i give http://localhost:8983/solr/def-features-make/?q=antenna&version=2.2&start=0&rows=10&indent=on it gets me all the document which have antenna in the featuers list. but when i give http://localhost:8983/solr/def-features-make/?q=Ford&version=2.2&start=0&rows=10&indent=on i am expecting to see all documents whose make is Ford. but it returns no documents Please let me know if i am doing something wrong here. Thanks -- View this message in context: http://www.nabble.com/defining-qf-in-your-own-request-handler-tp25119913p25119913.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Common Solr Question
Coming back to Question one This is how my standard handler looks like 12 explicit make,model i added fl parameter which return me those fields described, for this also i have to delete my browser cache and after that by defatul only those two fields are returned. but still the rows are ten when i issue a regular query like make:Honda from solr admin interface, by default it return rows 10 which i can see in the browser url http://localhost:8983/solr/select/?q=make%3AHonda&version=2.2&start=0&rows=10&indent=on but when i remove the rows in the browser url and hit again i get 12 rows this time. I know this is not a big thing to discuss upon, but since i am preparing a seminar on this i want my audience to be absolutely clear about this. Thanks -- View this message in context: http://www.nabble.com/Common-Solr-Question-tp25068160p25083956.html Sent from the Solr - User mailing list archive at Nabble.com.
Common Solr Question
Hi i have some basic yet important question about solr and that too with terminology. I want to be crystal clear about it. Please answer the following questions. Question 1 1) "Incoming queries are processed by the appropriate SolrRequestHandler. For the purposes of this discussion, you will use the StandardRequestHandler" So i assume all request which we make like --For select http://dl1rap711-epe.media.edmunds.com:8983/solr/select/?q=make%3AHonda&version=2.2&start=0&rows=10&indent=on the question is where is it defined in solrconfig.xml. if i have to change the default size for my result set from 10 to for example say 50 where should i change it. i tried to do this explicit 55 - But did not work. Question 2 2) When we issue an update command something like this http://localhost:8983/solr/update?stream.body=2007HyundaiSonata The following request handler will be used which is mentioned in the solrconfig.xml file Is this correct. Question 3 3) To upload CSV data we need to use /update/csv handler. I would appreciate how to specify this in the url if i have to upload a csv file. Question 4 3)If this is the case, every url request is mapped to a request handler. For load csv file use /update/csv which is implemented by solr.CSVRequestHandler For analysis file use /analysis which is implement by solr.AnalysisRequestHandler For now this is it. More to follow Thanks -- View this message in context: http://www.nabble.com/Common-Solr-Question-tp25068160p25068160.html Sent from the Solr - User mailing list archive at Nabble.com.
Writing own request handler tutorial
could anybody provide me with a complete data import handler example with oracle if there is any. thanks rashid -- View this message in context: http://www.nabble.com/Writing-own-request-handler-tutorial-tp24943849p24943849.html Sent from the Solr - User mailing list archive at Nabble.com.
getting all rows from SOLRJ client using setRows method
Hello is there a way you can get all the results back from SOLR when querying solrJ client my gut feeling was that this might work query.setRows(-1) The way is to change the configuration xml file, but that like hard coding the configuration, and there also i have to set some valid number, i cant say return all rows. Is there a way to done through query. Thanks rashid -- View this message in context: http://www.nabble.com/getting-all-rows-from-SOLRJ-client-using-setRows-method-tp23662668p23662668.html Sent from the Solr - User mailing list archive at Nabble.com.
