Re: solr distributed search example - exception
Hi Mark, i actually got this error coz i was using an old version of java. now the problem is solved Thanks anyways Raakhi On Tue, Jun 9, 2009 at 11:17 AM, Rakhi Khatwani wrote: > Hi Mark, > yea i would like to open a JIRA issue for it. how do i go about > that? > > Regards, > Raakhi > > > > On Mon, Jun 8, 2009 at 7:58 PM, Mark Miller wrote: > >> That is a very odd cast exception to get. Do you want to open a JIRA issue >> for this? >> >> It looks like an odd exception because the call is: >> >> NodeList nodes = (NodeList)solrConfig.evaluate(configPath, >> XPathConstants.NODESET); // cast exception is we get an ArrayList rather >> than NodeList >> >> Which leads to: >> >> Object o = xpath.evaluate(xstr, doc, type); >> >> where type = XPathConstants.NODESET >> >> So you get back an Object based on the XPathConstant passed. There does >> not appear to be a value that would return an ArrayList. >> Using XPathConstants.NODESET gets you a NodeList according to the XPath >> API. >> >> I'm not sure what could cause this to happen. >> >> - Mark >> >> >> Rakhi Khatwani wrote: >> >>> Hi, >>> I was executing a simple example which demonstrates >>> DistributedSearch. >>> example provided in the following link: >>> >>> http://wiki.apache.org/solr/DistributedSearch >>> >>> however, when i startup the server in both port nos: 8983 and 7574, i get >>> the following exception: >>> >>> SEVERE: Could not start SOLR. Check solr/home property >>> java.lang.ClassCastException: java.util.ArrayList cannot be cast to >>> org.w3c.dom.NodeList >>> at >>> >>> org.apache.solr.search.CacheConfig.getMultipleConfigs(CacheConfig.java:61) >>> at org.apache.solr.core.SolrConfig.(SolrConfig.java:131) >>> at org.apache.solr.core.SolrConfig.(SolrConfig.java:70) >>> at >>> >>> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117) >>> at >>> >>> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) >>> at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> >>> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) >>> at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) >>> at >>> >>> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) >>> at >>> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) >>> at >>> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> >>> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >>> at >>> >>> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> >>> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) >>> at org.mortbay.jetty.Server.doStart(Server.java:210) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) >>> at java.lang.reflect.Method.invoke(libgcj.so.7rh) >>> at org.mortbay.start.Main.invokeMain(Main.java:183) >>> at org.mortbay.start.Main.start(Main.java:497) >>> at org.mortbay.start.Main.main(Main.java:115) >>> 2009-06-08 18:36:28.016::WARN: failed SolrRequestFilter >>> java.lang.NoClassDefFoundError: org.apache.solr.core.SolrCore >>> at java.lang.Class.initializeClass(libgcj.so.7rh) >>> at >>> >>> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:77) >>> at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> >>> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) >>> at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) >>> at >>> >>> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) >>> at >>> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) >>> at >>> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> >>> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >>> at >>> >>> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>> at >>> >>> org.mortbay.jetty.handler.HandlerCollection.doS
Re: When searching for !...@#$%^&*() all documents are matched incorrectly
On Monday 01 June 2009 16:50, Sam Michaels wrote: > So the fix for this problem would be > > 1. Stop using WordDelimiterFilter for queries (what is the alternative) OR > 2. Not allow any search strings without any alphanumeric characters.. We ran into this same problem while replacing all characters using a PatternReplaceFilter. I've been working around this bug by using a LengthFilter to filter out tokens of zero length. .øs > Yonik Seeley-2 wrote: > > OK, here's the deal: > > > > -features:foo features:(\...@#$%\^&\*\(\)) > > -features:foo features:(\...@#$%\^&\*\(\)) > > -features:foo > > -features:foo > > > > The text analysis is throwing away non alphanumeric chars (probably > > the WordDelimiterFilter). The Lucene (and Solr) query parser throws > > away term queries when the token is zero length (after analysis). > > Solr then interprets the left over "-features:foo" as "all documents > > not containing foo in the features field", so you get a bunch of > > matches. > > > > -Yonik > > http://www.lucidimagination.com > > > > On Mon, Jun 1, 2009 at 10:15 AM, Sam Michaels wrote: > >> Walter, > >> > >> The analysis link does not produce any matches for either @ or > >> !...@#$%^&*() strings when I try to match against bathing. I'm worried that > >> this might be > >> the symptom of another problem (which has not revealed itself yet) and > >> want > >> to get to the bottom of this... > >> > >> Thank you. > >> sm > >> > >> Walter Underwood wrote: > >>> Use the [analysis] link on the Solr admin UI to get more info on > >>> how this is being interpreted. > >>> > >>> However, I am curious about why this is important. Do users enter > >>> this query often? If not, maybe it is not something to spend time on. > >>> > >>> wunder > >>> > >>> On 5/31/09 2:56 PM, "Sam Michaels" wrote: > Here is the output from the debug query when I'm trying to match the > String @ > against Bathing (should not match) > > > 3.2689073 = (MATCH) weight(activity_type:NAME in 0), product of: > 0.9994 = queryWeight(activity_type:NAME), product of: > 3.2689075 = idf(docFreq=153, numDocs=1489) > 0.30591258 = queryNorm > 3.2689075 = (MATCH) fieldWeight(activity_type:NAME in 0), product > of: 1.0 = tf(termFreq(activity_type:NAME)=1) > 3.2689075 = idf(docFreq=153, numDocs=1489) > 1.0 = fieldNorm(field=activity_type, doc=0) > > > Looks like the AND clause in the search string is ignored... > > SM. > > ryantxu wrote: > > two key things to try (for anyone ever wondering why a query matches > > documents) > > > > 1. add &debugQuery=true and look at the explain text below -- > > anything that contributed to the score is listed there > > 2. check /admin/analysis.jsp -- this will let you see how analyzers > > break text up into tokens. > > > > Not sure off hand, but I'm guessing the WordDelimiterFilterFactory > > has something to do with it... > > > > > > On Sat, May 30, 2009 at 5:59 PM, Sam Michaels > > > > wrote: > >> Hi, > >> > >> I'm running Solr 1.3/Java 1.6. > >> > >> When I run a query like - (activity_type:NAME) AND > >> title:(\...@#$%\^&\*\(\)) > >> all the documents are returned even though there is not a single > >> match. > >> There is no title that matches the string (which has been escaped). > >> > >> My document structure is as follows > >> > >> > >> NAME > >> Bathing > >> > >> > >> > >> > >> The title field is of type text_title which is described below. > >> > >> >> positionIncrementGap="100"> > >> > >> > >> > >> >> generateWordParts="1" generateNumberParts="1" catenateWords="1" > >> catenateNumbers="1" catenateAll="1" splitOnCaseChange="1"/> > >> > >> > >> > >> > >> > >> >> synonyms="synonyms.txt" > >> ignoreCase="true" expand="true"/> > >> >> generateWordParts="1" generateNumberParts="1" catenateWords="1" > >> catenateNumbers="1" catenateAll="1" splitOnCaseChange="1"/> > >> > >> > >> > >> > >> > >> > >> When I run the query against Luke, no results are returned. Any > >> suggestions > >> are appreciated. > >> > >> > >> -- > >> View this message in context: > >> http://www.nabble.com/When-searching-for-%21%40-%24-%5E-*%28%29-all- > >>document s-are-matched-incorrectly-tp23797731p23797731.html > >> Sent from the Solr - User mailing list archive at Nabble.com. > >> > >> -- > >> View this message in context: > >> http://www.nabble.com/When-searching-for-%21%40-%24-%5E-*%28%29-all-docu > >>ments-are-matched-incorrectly-tp23797731p23815688.html Sent from the Solr > >> - User mailing list archive at Nabble.com
Solr Multiple Queries?
Hi all, I just get started looking at using Solr as my search web service. But I don't know does Solr have some features for multiple queries: - Startswith - Exact Match - Contain - Doesn't Contain - In the range Could anyone guide me how to implement those features in Solr? Cheers, Samnang
Re: solr distributed search example - exception
Hi Mark, yea i would like to open a JIRA issue for it. how do i go about that? Regards, Raakhi On Mon, Jun 8, 2009 at 7:58 PM, Mark Miller wrote: > That is a very odd cast exception to get. Do you want to open a JIRA issue > for this? > > It looks like an odd exception because the call is: > > NodeList nodes = (NodeList)solrConfig.evaluate(configPath, > XPathConstants.NODESET); // cast exception is we get an ArrayList rather > than NodeList > > Which leads to: > > Object o = xpath.evaluate(xstr, doc, type); > > where type = XPathConstants.NODESET > > So you get back an Object based on the XPathConstant passed. There does not > appear to be a value that would return an ArrayList. > Using XPathConstants.NODESET gets you a NodeList according to the XPath > API. > > I'm not sure what could cause this to happen. > > - Mark > > > Rakhi Khatwani wrote: > >> Hi, >> I was executing a simple example which demonstrates DistributedSearch. >> example provided in the following link: >> >> http://wiki.apache.org/solr/DistributedSearch >> >> however, when i startup the server in both port nos: 8983 and 7574, i get >> the following exception: >> >> SEVERE: Could not start SOLR. Check solr/home property >> java.lang.ClassCastException: java.util.ArrayList cannot be cast to >> org.w3c.dom.NodeList >> at >> org.apache.solr.search.CacheConfig.getMultipleConfigs(CacheConfig.java:61) >> at org.apache.solr.core.SolrConfig.(SolrConfig.java:131) >> at org.apache.solr.core.SolrConfig.(SolrConfig.java:70) >> at >> >> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117) >> at >> >> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) >> at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) >> at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) >> at >> >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >> at >> >> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) >> at org.mortbay.jetty.Server.doStart(Server.java:210) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) >> at java.lang.reflect.Method.invoke(libgcj.so.7rh) >> at org.mortbay.start.Main.invokeMain(Main.java:183) >> at org.mortbay.start.Main.start(Main.java:497) >> at org.mortbay.start.Main.main(Main.java:115) >> 2009-06-08 18:36:28.016::WARN: failed SolrRequestFilter >> java.lang.NoClassDefFoundError: org.apache.solr.core.SolrCore >> at java.lang.Class.initializeClass(libgcj.so.7rh) >> at >> >> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:77) >> at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) >> at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) >> at >> >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >> at >> >> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >> at >> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) >> at org.mortbay.jetty.Server.doStart(Server.java:210) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >>
spellcheck /too many open files
Hi , 1)Does the spell check component support all languages? 2) I have a scnenario where i have abt 20 webapps in a single container.We get too many open files at index time /while restarting tomcat. The mergefactor is at default. If i reduce the merge factor to 2 and optimize the index ,will the open files be closed automatically or would i have to reindex to close the open files or how do i close the already opened files.This is on linux with solr 1.3 and tomcat 5.5 Regards Revas
Solr relevancy score - conversion
Hi, I am using solr to inxdex some of the legal documents, where i need the solr search engine to return relevancy ranking score for each search results. As of now i am getting score like 3.12, 1.23, 0.23 so on. Would need an more proportionate score like rounded to 100% (95% relevant, 80 % relevant and so on). Is there a way to make solr returns such scores of such relevance. Any other approach to arrive at this scores also be appreciated thanks vijay -- View this message in context: http://www.nabble.com/Solr-relevancy-score---conversion-tp23936413p23936413.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Use the same SQL Field in Dataimporthandler twice?
On Tue, Jun 9, 2009 at 12:41 AM, gateway0 wrote:
>
> Thanks for your answer.
>
> "${db.tableA.id}" that specifies the sql query that the Dataimporthandler
> should Use the sql field "id" in table "tableA" located in Database "db".
The naming convention does not work like that.
if the entity name is 'tableA' then the field 'id' is addressed as 'tableA.id'
As I said earlier, if you could privide mw with the entire
data-config.xml it would be more helpful
>
> like in the example from the Solr Wiki:
> "
>
> "
>
> It´s strange I know but when I use something other than "id" as the foreign
> key for the query everything works!
>
> like:
> "${db.tableA.anotherid}"
>
>
>
> Noble Paul നോബിള് नोब्ळ्-2 wrote:
>>
>> what is ${db.tableA.id} ?
>>
>> I think there is something extra in that
>>
>> can you paste the whole data-config.xml?
>>
>> can you paste
>>
>> On Sun, Jun 7, 2009 at 1:09 AM, gateway0 wrote:
>>>
>>> Hi,
>>>
>>> I tried to do the following:
>>>
>>> "
>>>
>>>
>>>
>>>
>>>
>>> "
>>>
>>> So I use the SQL Table Field "id" twice once for "db_id" in my index and
>>> for
>>> the sql query as "fid=id".
>>>
>>> That doesn´t work!
>>>
>>> But when I change the query from "fid=id" to like "fid=otherkey" it does
>>> work!
>>> Like:
>>> "
>>>
>>>
>>>
>>>
>>>
>>> "
>>>
>>> Is there any other kind of a workaround so I can use the SQL Field "id"
>>> twice as I wanted to? Thanks
>>>
>>> kind regards, Sebastian
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23904968.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> -
>> Noble Paul | Principal Engineer| AOL | http://aol.com
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23930286.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Collapse Results
both CollapseComponent and QueryComponent
execute searches and add results to the Response.
To get only Collapse results, remove the QueryComponent from the
requestHandler.
In solrconfig.xml, I added:
>
>class="org.apache.solr.handler.component.CollapseComponent" />
>
>
>
>
>
> explicit
>
>
>
>collapse
>
>
On Mon, Jun 8, 2009 at 5:38 PM, Nirkhe, Chandra wrote:
> Hi,
> I am using Solr 1.3 with collapse patch. In the results it returns
> standard response as well as collapse response. How do I get just
> collapse response?
>
> Regards
> Chandra
>
> "responseHeader":{
> "status":0,
> "QTime":25,
> "params":{
>"collapse":"true",
>"wt":"json",
>"collapse.field":"resource_id",
>"indent":"on",
>"fl":"resource_id",
>"q":"George",
>"qt":"provider",
>"version":"2.2"}},
> "response":{"numFound":5364,"start":0,"docs":[
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001},
>{
> "resource_id":1001}]
> },
> "spellcheck":{
> "suggestions":[]},
> "collapse_counts":[
> "field","resource_id",
> "doc",[
>"10001",1628,
>"64308",88,
>"66931",148,
>"79374",23,
>"107152",621,
>"135179",87,
>"242091",154,
>"254375",192,
>"264802",143,
>"388030",352],
> "count",[
>"1001",1628,
>"1060",88,
>"1064",148,
>"1084",23,
>"1127",621,
>"1141",87,
>"1899",154,
>"1913",192,
>"1924",143,
>"2024",352],
> "debug",[
>"Docset type","HashDocSet(16)",
>"Total collapsing time(ms)",10,
>"Create uncollapsed docset(ms)",8,
>"Collapsing normal time(ms)",1,
>"Creating collapseinfo time(ms)",1,
>"Convert to bitset time(ms)",0,
>"Create collapsed docset time(ms)",0]],
> "response":{"numFound":16,"start":0,"docs":[
>{
> "resource_id":1001},
>{
> "resource_id":1060},
>{
> "resource_id":1064},
>{
> "resource_id":1084},
>{
> "resource_id":1127},
>{
> "resource_id":1141},
>{
> "resource_id":1899},
>{
> "resource_id":1913},
>{
> "resource_id":1924},
>{
> "resource_id":2024}]
> }}
>
>
>
>
Schema vs Dynamic Fields
On the wiki, it says: > One of the powerful features of Lucene is that you don't have to > pre-define every field when you first create your index. Even though > Solr provides strong datatyping for fields, it still preserves that > flexibility using "Dynamic Fields". Is the use of a predefined schema primarily a "type safety" feature? We're considering using Solr for a data set that is very free-form; will we get much slower results if the majority of our data is in a dynamic field such as: I'm a little unclear on the trade-offs involved and would appreciate a hint. Phil Hagelberg http://technomancy.us
User Credentials for Solr Data Dir
Hi, I am currently using solr 1.3 and runnign the sole as NT service. I need to store data indexes on a Remote Filer machine. the Filer needs user credentials inorder to access the same.. Is there a solr configuration which I can use to pass these credentials? I was reading some blogs and they suggested to run the NT service with user who can access the resource needed. Since I need to use existing build and deploy tools in the company, and they always run the NT serviec "LOCAL System" which cannot access other resource. Thats why i am trying to explore if its possible to pass these credentials via JNDI/System variables? Is it possible? Thanks Vaibhav _ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/india/windows/windowslive/
Re: Questions regarding IT search solution
Hi Jeff, Thanks for the link. You are my lifesaver :)This is exactly simillar to what I am looking for. Thanks,Surfer --- On Fri, 6/5/09, Jeff Hammerbacher wrote: From: Jeff Hammerbacher Subject: Re: Questions regarding IT search solution To: solr-user@lucene.apache.org, silentsurfe...@yahoo.com Date: Friday, June 5, 2009, 12:15 AM Hey, Your system sounds similar to the work don by Stu Hood at Rackspace in their Mailtrust unit. See http://highscalability.com/how-rackspace-now-uses-mapreduce-and-hadoop-query-terabytes-datafor more details and inspiration. Regards, Jeff On Thu, Jun 4, 2009 at 4:58 PM, wrote: > Hi, > This is encouraging to know that solr/lucene solution may work. > Can anyone using solr/lucene for such scenario can confirm that the > solution is used and working fine? That would be really helpful, as I just > started looking into the solr/lucene solution only couple of days back and > might be difficult to be 100% confident before proposing the solution > approach in next couple of days. > Thanks,Surfer > > --- On Thu, 6/4/09, Otis Gospodnetic wrote: > > From: Otis Gospodnetic > Subject: Re: Questions regarding IT search solution > To: > solr-user@lucene.apache.org > Date: Thursday, June 4, 2009, 10:26 PM > > > My guess is Solr/Lucene would work. Not sure how well/fast, but it would, > esp. if you avoid range queries (or use tdate), and esp. if you > shard/segment indices smartly, so that at query time you send (or distribute > if you have to) the query to only those shards that have the data (if your > query is for a limited time period). > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: Silent Surfer > > To: solr-user@lucene.apache.org > > Sent: Thursday, June 4, 2009 5:52:21 PM > > Subject: Re: > Questions regarding IT search solution > > > > Hi, > > As Alex correctly pointed out my main intention is to figure out whether > > Solr/lucene offer functionalities to replicate what Splunk is doing in > terms of > > building indexes etc for enabling search capabilities. > > We evaluated Splunk, but it is not very cost effective solution for us as > we may > > have logs running into few GBs per day as there can be around 25-20 > servers > > running, and Splunk licensing model is based of size of logs per day that > too, > > the license valid for only 1 year. > > With this back ground, any further inputs on this are greatly > appreciated. > > Thanks,Surfer > > > > --- On Thu, 6/4/09, Alexandre Rafalovitch wrote: > > > > From: Alexandre Rafalovitch > > Subject: Re: Questions regarding IT search solution > > To: solr-user@lucene.apache.org > > Date: Thursday, June 4, 2009, 9:27 PM > > > > I would also be interested to know what other existing solutions exist. > > > > Splunk's advantage is that it does extraction of the fields with > > advanced searching functionality (it has lexers/parsers for multiple > > content types). I believe that's the Solr's function desired in > > original posting. At the time they came out (2004), I was not aware of > > any good open source solutions to do what they did. And I would have > > loved one, as I was analyzing multi-gigabite logs. > > > > Hadoop might be a way to process the files, but what would do the > > indexing and searching? > > > > Regards, > > Alex. > > > > On Thu, Jun 4, 2009 at 11:56 AM, Walter Underwoodwrote: > > > Why build one? Don't those already exist? > > > > > > Personally, I'd start with Hadoop instead of Solr. Putting > logs in a > > > search index is guaranteed to not scale. People were already trying > > > different approaches ten years ago. > > > > > > wunder > > > > > > On 6/4/09 8:41 AM, "Silent Surfer" wrote: > > > > > >> Hi, > > >> Any help/pointers on the following message would really help me.. > > >> Thanks,Surfer > > >> > > >> --- On Tue, 6/2/09, Silent Surfer wrote: > > >> > > >> From: Silent Surfer > > >> Subject: Questions regarding IT search solution > > >> To: solr-user@lucene.apache.org > > >> Date: Tuesday, June 2, 2009, 5:45 PM > > >> > > >> Hi, > > >> I am new to Lucene forum and it is my first question.I need a > clarification > > >> from you. > > >> Requirement:--1. Build a IT search tool for logs > similar to > > >> that of Splunk(Only wrt searching logs but not in terms of reporting, > graphs > > >> etc) using > solr/lucene. The log files are mainly the server logs like JBoss, > > >> Custom application server logs (May or may not be log4j logs) and the > files > > >> size can go potentially upto 100 MB2. The logs are spread across > multiple > > >> servers (25 to 30 servers)2. Capability to be do search almost > realtime3. > > >> Support distributed search > > >> > > >> Our search criterion can be based on a keyword or timestamp or IP > address > > etc. > > >> Can anyone throw some light if solr/lucene is right solution for this > ? > > >> Appreciate any quick help in this regard. > > >> Thanks,Surfer > > > >
Collapse Results
Hi,
I am using Solr 1.3 with collapse patch. In the results it returns
standard response as well as collapse response. How do I get just
collapse response?
Regards
Chandra
"responseHeader":{
"status":0,
"QTime":25,
"params":{
"collapse":"true",
"wt":"json",
"collapse.field":"resource_id",
"indent":"on",
"fl":"resource_id",
"q":"George",
"qt":"provider",
"version":"2.2"}},
"response":{"numFound":5364,"start":0,"docs":[
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001},
{
"resource_id":1001}]
},
"spellcheck":{
"suggestions":[]},
"collapse_counts":[
"field","resource_id",
"doc",[
"10001",1628,
"64308",88,
"66931",148,
"79374",23,
"107152",621,
"135179",87,
"242091",154,
"254375",192,
"264802",143,
"388030",352],
"count",[
"1001",1628,
"1060",88,
"1064",148,
"1084",23,
"1127",621,
"1141",87,
"1899",154,
"1913",192,
"1924",143,
"2024",352],
"debug",[
"Docset type","HashDocSet(16)",
"Total collapsing time(ms)",10,
"Create uncollapsed docset(ms)",8,
"Collapsing normal time(ms)",1,
"Creating collapseinfo time(ms)",1,
"Convert to bitset time(ms)",0,
"Create collapsed docset time(ms)",0]],
"response":{"numFound":16,"start":0,"docs":[
{
"resource_id":1001},
{
"resource_id":1060},
{
"resource_id":1064},
{
"resource_id":1084},
{
"resource_id":1127},
{
"resource_id":1141},
{
"resource_id":1899},
{
"resource_id":1913},
{
"resource_id":1924},
{
"resource_id":2024}]
}}
Re: Query faceting
In order to get the the values you want for the service field you will need to change the fieldType definition in schema.xml for "service" to use something that doesn't alter your original values. Try the "string" fieldType to start and look at the fieldType definition for "string". I'm guessing you have it set to "text" or something else with a chain of filters during analysis. If you don't want back facets with a count of 0 set this param: &facet.mincount=1 Have a look at all the values you can set on facets: http://wiki.apache.org/solr/SimpleFacetParameters -Jay On Mon, Jun 8, 2009 at 2:09 PM, siping liu wrote: > > Hi, > > I have a field called "service" with following values: > > - Shuttle Services > - Senior Discounts > - Laundry Rooms > > - ... > > > > When I conduct query with "facet=true&facet.field=service&facet.limit=-1", > I get something like this back: > > - shuttle 2 > > - service 3 > > - senior 0 > > - laundry 0 > > - room 3 > > - ... > > > > Questions: > > - How not to break up fields values in words, so I can get something like > "Shuttle Services 2" back? > > - How to tell Solr not to return facet with 0 value? The query takes long > time to finish, seemingly because of the long list of items with 0 count. > > > > thanks for any advice. > > _ > Insert movie times and more without leaving Hotmail®. > > http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009 >
Query faceting
Hi, I have a field called "service" with following values: - Shuttle Services - Senior Discounts - Laundry Rooms - ... When I conduct query with "facet=true&facet.field=service&facet.limit=-1", I get something like this back: - shuttle 2 - service 3 - senior 0 - laundry 0 - room 3 - ... Questions: - How not to break up fields values in words, so I can get something like "Shuttle Services 2" back? - How to tell Solr not to return facet with 0 value? The query takes long time to finish, seemingly because of the long list of items with 0 count. thanks for any advice. _ Insert movie times and more without leaving Hotmail®. http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009
RE: Fetching Dynamic Fields
No I am looking for specific scenario. I want to query for the dynamic fields along with few normal fields. Thanks, Kalyan Manepalli -Original Message- From: Avlesh Singh [mailto:avl...@gmail.com] Sent: Monday, June 08, 2009 1:22 PM To: solr-user@lucene.apache.org Subject: Re: Fetching Dynamic Fields Hope this helps - http://wiki.apache.org/solr/CommonQueryParameters#head-db2785986af2355759faaaca53dc8fd0b012d1ab Cheers Avlesh On Mon, Jun 8, 2009 at 9:10 PM, Manepalli, Kalyan < kalyan.manepa...@orbitz.com> wrote: > Hi all, >Is there a way to select all the dynamic fields in the fl field > without using *. Here is what I am looking for. > Fields in the schema, locationName_*, locationId,description,content. > I want to select just the locationName_* and locationId. How can I do this > without using fl=*, coz I don't want to fetch all the other fields. > > Any suggestions in this regard will be helpful. > > Thanks, > Kalyan Manepalli > >
Re: Use the same SQL Field in Dataimporthandler twice?
Thanks for your answer.
"${db.tableA.id}" that specifies the sql query that the Dataimporthandler
should Use the sql field "id" in table "tableA" located in Database "db".
like in the example from the Solr Wiki:
"
"
It´s strange I know but when I use something other than "id" as the foreign
key for the query everything works!
like:
"${db.tableA.anotherid}"
Noble Paul നോബിള് नोब्ळ्-2 wrote:
>
> what is ${db.tableA.id} ?
>
> I think there is something extra in that
>
> can you paste the whole data-config.xml?
>
> can you paste
>
> On Sun, Jun 7, 2009 at 1:09 AM, gateway0 wrote:
>>
>> Hi,
>>
>> I tried to do the following:
>>
>> "
>>
>>
>>
>>
>>
>> "
>>
>> So I use the SQL Table Field "id" twice once for "db_id" in my index and
>> for
>> the sql query as "fid=id".
>>
>> That doesn´t work!
>>
>> But when I change the query from "fid=id" to like "fid=otherkey" it does
>> work!
>> Like:
>> "
>>
>>
>>
>>
>>
>> "
>>
>> Is there any other kind of a workaround so I can use the SQL Field "id"
>> twice as I wanted to? Thanks
>>
>> kind regards, Sebastian
>> --
>> View this message in context:
>> http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23904968.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> -
> Noble Paul | Principal Engineer| AOL | http://aol.com
>
>
--
View this message in context:
http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23930286.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: searchcomponent howto ...
sorry. my 'tab' character is linked to 'send'. i meant to include this bit
of source
@Override
public void process(ResponseBuilder rb) throws IOException {
SolrQueryRequest request = rb.req;
SolrParams params = request.getParams();
String qs = rb.getQueryString();
String defType = params.get(QueryParsing.DEFTYPE);
defType = defType==null ? QParserPlugin.DEFAULT_QTYPE :
defType;
try {
QParser parser =
QParser.getParser(encoder.process(rb.getQueryString()), defType, request);
rb.setQuery( parser.getQuery() );
rb.setQparser(parser);
} catch (ParseException e) {
throw new
SolrException(SolrException.ErrorCode.BAD_REQUEST, e);
}
}
and then ask,
- how can i set the value of query so that it is reflected in the 'q'
node of the search results e.g. solr.
the example 'process' method above works, but the original query is still
written to the search results page.
On Mon, Jun 8, 2009 at 9:08 AM, Grant Ingersoll wrote:
> Is there a question here?
>
>
> On Jun 7, 2009, at 2:50 PM, David Palaitis wrote:
>
> I'm trying to write a very simple SearchComponent that performs a simple
>> operation on the query string before passing through to QueryComponent.
>>
>> The EncodingSearchComponent will perform something like the following
>> Base64(MD5(querystring))
>>
>
> --
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
Trie Patches- Backportable?
Hi, I am still using Solr 1.2 with the Lucene 2.2 that came with that version of Solr. I am interested in taking advantage of the trie filtering to alleviate some performance problems and was wondering how back-portable these patches are? I am also trying to understand how the Trie algorithm cuts down the number of term queries compared to a normal range query. I was at the recent Bay Area lucene/solr meetup where this was covered but missed some of the details. I know the ideal case is to upgrade to a newer Solr/Lucene but we are resource constrained and can't devote the time right now to test and upgrade our production systems to a newer Solr. Thanks! Amit
Re: Fetching Dynamic Fields
Hope this helps - http://wiki.apache.org/solr/CommonQueryParameters#head-db2785986af2355759faaaca53dc8fd0b012d1ab Cheers Avlesh On Mon, Jun 8, 2009 at 9:10 PM, Manepalli, Kalyan < kalyan.manepa...@orbitz.com> wrote: > Hi all, >Is there a way to select all the dynamic fields in the fl field > without using *. Here is what I am looking for. > Fields in the schema, locationName_*, locationId,description,content. > I want to select just the locationName_* and locationId. How can I do this > without using fl=*, coz I don't want to fetch all the other fields. > > Any suggestions in this regard will be helpful. > > Thanks, > Kalyan Manepalli > >
Re: questions about optimizations SOLR-1111,SOLR-1169,SOLR-1179
Marc Sturlese wrote: > > Hey there, > I have a few questions about the new solr speed and memory imporvements. > They look like at pretty low level but I would like to understand some > stuff... > Sorry if something I say sounds odd. I am trying to optimize couple of > custom components with these new features but not sure if I am in the > right way- > > 1.-SOLR-111 > I have seen there where some memory problems with FieldCache and older > nightly builds. I use it in stats component, field collapsing and for a > custom sort. Is FieldCache working alrite in the last nightly? > > 2.-SOLR-1169 > I have seen that HashDocSet has been replaced by SortedIntDocSet. Is it > already that way in the last nighlty? I am asking because I am testing > field collapsing patch too. A HashDocsSet is being used in there... would > be a good idea with the last nigthly to replace it with a SortedIntDocSet? > > 3.-SOLR-1179 > In my custom component I have a DocList with all docs that match a query > sorted by relevance. I have to iterate over that DocList and its not > really good in performance... Iterate over a DocSet it's much faster but I > will not have the elements sorted by relevance anymore. Is it possible to > get them sorted by relevance with the new SortedIntDocSet? > > Thanks in advance > > > -- View this message in context: http://www.nabble.com/questions-about-optimizations-SOLR-%2CSOLR-1169%2CSOLR-1179-tp23889097p23929482.html Sent from the Solr - User mailing list archive at Nabble.com.
Example folder - can we change it?
When I install solr , by default it will install it under /opt/apache-solr-1.3.0/ The bin , config file and data is under /opt/apache-solr-1.3.0/example/solr Is there anyway that we change the example to something else? Because "example" is can be interpreted wrong ( like sample, so it's not real) Francis
Re: query issue /special character and case
On Sat, Jun 6, 2009 at 11:40 AM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Sat, May 30, 2009 at 9:48 AM, revas wrote: > > > Hi , > > > > When i give a query like the following ,why does it become a phrase query > > as shown below? > > The field type is the default text field in the schema. > > > > volker-blanz > > PhraseQuery(content:"volker blanz") > > > > What is the query that was sent to Solr? The query is content:volker-blanz and this is a default text field > > > > > Also when i have special characters in the query as SCHÖLKOPF , i am not > > able to convert the "o" with spl character to lower case on my unix > os/it > > works fine on windows xp OS .Also if i have a spl character in my query > ,i > > would like to search for it wihtout the special character as SCHOLKOPF > > ,this works fine in windows with strtr (string translate php fucntion) > ,but > > again not in windows OS. > > > > Hmm, not sure. If you are using Tomcat, have you enabled UTF-8? > > > http://wiki.apache.org/solr/SolrTomcat#head-20147ee4d9dd5ca83ed264898280ab60457847c4 > > You can try using the analysis.jsp on the text field with this token and > see > how it is being analyzed. See if that gives some hints. Yes i am using tomcat and have enabled utf -8 in tomcat. > > > -- > Regards, > Shalin Shekhar Mangar. >
Re: query issue /special character and case
Regarding being able to search SCHOLKOPF (o with no umlaut) and match SCHÖLKOPF (with umlaut) try using the ISOLatin1AccentFilterFactory in your analysis chain: This filter removes accented chars and replaces them with non-accented versions. As always, make sure to add it to the for both type "index" and type "query". -Jay On Fri, Jun 5, 2009 at 11:10 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Sat, May 30, 2009 at 9:48 AM, revas wrote: > > > Hi , > > > > When i give a query like the following ,why does it become a phrase query > > as shown below? > > The field type is the default text field in the schema. > > > > volker-blanz > > PhraseQuery(content:"volker blanz") > > > > What is the query that was sent to Solr? > > > > Also when i have special characters in the query as SCHÖLKOPF , i am not > > able to convert the "o" with spl character to lower case on my unix > os/it > > works fine on windows xp OS .Also if i have a spl character in my query > ,i > > would like to search for it wihtout the special character as SCHOLKOPF > > ,this works fine in windows with strtr (string translate php fucntion) > ,but > > again not in windows OS. > > > > Hmm, not sure. If you are using Tomcat, have you enabled UTF-8? > > > http://wiki.apache.org/solr/SolrTomcat#head-20147ee4d9dd5ca83ed264898280ab60457847c4 > > You can try using the analysis.jsp on the text field with this token and > see > how it is being analyzed. See if that gives some hints. > > -- > Regards, > Shalin Shekhar Mangar. >
Re: Configure Collection Distribution in Solr 1.3
You'll find everything you need in the Wiki. http://wiki.apache.org/solr/SolrCollectionDistributionOperationsOutline http://wiki.apache.org/solr/SolrCollectionDistributionScripts If things are still uncertain I've written a guide for when we used the solr distribution scrips on our lucene index earlier. You can read that guide here: http://www.integrasco.no/index.php?option=com_content&view=article&id=51:lucene-index-replication&catid=35:blog&Itemid=53 Cheers, Aleksander On Mon, 08 Jun 2009 18:22:01 +0200, MaheshR wrote: Hi, we configured multi-core solr 1.3 server in Tomcat 6.0.18 servlet container. Its working great. Now I need to configure collection Distribution to replicate indexing data between master and 2 slaves. Please provide me step by step instructions to configure collection distribution between master and slaves would be helpful. Thanks in advance. Thanks Mahesh. -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Configure Collection Distribution in Solr 1.3
Hi, we configured multi-core solr 1.3 server in Tomcat 6.0.18 servlet container. Its working great. Now I need to configure collection Distribution to replicate indexing data between master and 2 slaves. Please provide me step by step instructions to configure collection distribution between master and slaves would be helpful. Thanks in advance. Thanks Mahesh. -- View this message in context: http://www.nabble.com/Configure-Collection-Distribution-in-Solr-1.3-tp23927332p23927332.html Sent from the Solr - User mailing list archive at Nabble.com.
Fetching Dynamic Fields
Hi all, Is there a way to select all the dynamic fields in the fl field without using *. Here is what I am looking for. Fields in the schema, locationName_*, locationId,description,content. I want to select just the locationName_* and locationId. How can I do this without using fl=*, coz I don't want to fetch all the other fields. Any suggestions in this regard will be helpful. Thanks, Kalyan Manepalli
Re: Terms Component
You can try out the nightly build of solr (which is the solr 1.4 dev version) containing all the new nice and shiny features of Solr 1.4:) To use Terms Component you simply need to configure the handler as explained in the documentation / wiki. Cheers, Aleksander On Mon, 08 Jun 2009 14:22:15 +0200, Anshuman Manur wrote: while on the subject, can anybody tell me when Solr 1.4 might come out? Thanks Anshuman Manur On Mon, Jun 8, 2009 at 5:37 PM, Anshuman Manur wrote: I'm using Solr 1.3 apparently.and Solr 1.4 is not out yet. Sorry..My mistake! On Mon, Jun 8, 2009 at 5:18 PM, Anshuman Manur < anshuman_ma...@stragure.com> wrote: Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: solr distributed search example - exception
That is a very odd cast exception to get. Do you want to open a JIRA issue for this? It looks like an odd exception because the call is: NodeList nodes = (NodeList)solrConfig.evaluate(configPath, XPathConstants.NODESET); // cast exception is we get an ArrayList rather than NodeList Which leads to: Object o = xpath.evaluate(xstr, doc, type); where type = XPathConstants.NODESET So you get back an Object based on the XPathConstant passed. There does not appear to be a value that would return an ArrayList. Using XPathConstants.NODESET gets you a NodeList according to the XPath API. I'm not sure what could cause this to happen. - Mark Rakhi Khatwani wrote: Hi, I was executing a simple example which demonstrates DistributedSearch. example provided in the following link: http://wiki.apache.org/solr/DistributedSearch however, when i startup the server in both port nos: 8983 and 7574, i get the following exception: SEVERE: Could not start SOLR. Check solr/home property java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.w3c.dom.NodeList at org.apache.solr.search.CacheConfig.getMultipleConfigs(CacheConfig.java:61) at org.apache.solr.core.SolrConfig.(SolrConfig.java:131) at org.apache.solr.core.SolrConfig.(SolrConfig.java:70) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) 2009-06-08 18:36:28.016::WARN: failed SolrRequestFilter java.lang.NoClassDefFoundError: org.apache.solr.core.SolrCore at java.lang.Class.initializeClass(libgcj.so.7rh) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:77) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) Caused by: java.lang.ClassNotFoundException: org.apache.solr.core.JmxMonitoredMap not found in StartLoader[file:/home/ithurs/apache-solr-1.3.0/example7574/, file:/home/ithurs/ap
StreamingUpdateSolrServer recommendations?
Hi all, I guess this questions i mainly aimed to you, Ryan. I've been trying out your StreamingUpdateSolrServer implementation for indexin, and clearly see the improvements in indexing-times compared to the CommonsHttpSolrServer :) Great work! My question is, do you have any recommendations as to what values I should use / have you found a "sweet-spot"? What are the trade-offs? Thread count is obvious with regard to the number of cpus available, but what about the queue size? Any thoughts? I tried 20 / 3 as you have posted in the issue thread, and get averages of about 80 documents / sec (and I have not optimized the document processing etc, which takes the larger part of the time). Anyways, I was just curious on what others are using (and what times you are getting at) Keep up the good work! Aleks -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: Does Solr know what host and port it is running on?
how about putting it in the tag? On Mon, Jun 8, 2009 at 6:39 PM, Grant Ingersoll wrote: > All good points, Paul. I've settled on putting it into the config, as much > as I don't want to. > > > On Jun 6, 2009, at 4:59 PM, Paul Libbrecht wrote: > >> It is common knowledge that a web-application cannot know it's port and >> host-name: >> - many possible cnames to host-name >> - potential tunnels >> - most important and most used: proxies, typically an apache mod_proxy or >> mod_webapp on a port different than the servlet container port. >> >> >> Le 06-juin-09 à 09:38, Noble Paul നോബിള് नोब्ळ् a écrit : >>> >>> can it be added into the solrconfig.xml or solr.xml ? >> >> My experience is that it is simple as that: you have to configure >> something which, in ActiveMath, we have called the "advertizedURL". >> >> Making it non-configurable really often creates issues and requires the >> deployer to dig deep into the servlet container configuration (I never was >> fully able with mod_proxy and tomcat). Defaulting to something such as >> request.getServerPort etc is probably right though. >> >> hope it helps. >> >> paul >> >>> On Sat, Jun 6, 2009 at 10:55 AM, Shalin Shekhar >>> Mangar wrote: On Sat, Jun 6, 2009 at 9:58 AM, Grant Ingersoll wrote: > Does Solr know what host and port it is running on and, if so, how do I > get > access to it programmatically? I feel like I'm missing something > obvious. > No, it does not. You can only get this on the first request. There's an issue open at https://issues.apache.org/jira/browse/SOLR-727 > > -- > Grant Ingersoll > http://www.lucidimagination.com/ > > Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using > Solr/Lucene: > http://www.lucidimagination.com/search > > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
solr distributed search example - exception
Hi, I was executing a simple example which demonstrates DistributedSearch. example provided in the following link: http://wiki.apache.org/solr/DistributedSearch however, when i startup the server in both port nos: 8983 and 7574, i get the following exception: SEVERE: Could not start SOLR. Check solr/home property java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.w3c.dom.NodeList at org.apache.solr.search.CacheConfig.getMultipleConfigs(CacheConfig.java:61) at org.apache.solr.core.SolrConfig.(SolrConfig.java:131) at org.apache.solr.core.SolrConfig.(SolrConfig.java:70) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) 2009-06-08 18:36:28.016::WARN: failed SolrRequestFilter java.lang.NoClassDefFoundError: org.apache.solr.core.SolrCore at java.lang.Class.initializeClass(libgcj.so.7rh) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:77) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) Caused by: java.lang.ClassNotFoundException: org.apache.solr.core.JmxMonitoredMap not found in StartLoader[file:/home/ithurs/apache-solr-1.3.0/example7574/, file:/home/ithurs/apache-solr-1.3.0/example7574/lib/jetty-6.1.3.jar, file:/home/ithurs/apache-solr-1.3.0/example7574/lib/jetty-util-6.1.3.jar, file:/home/ithurs/apache-solr-1.3.0/example7574/lib/servlet-api-2.5-6.1.3.jar] at java.net.URLClassLoader.findClass(libgcj.so.7rh) at java.lang.ClassLoader.loadClass(libgcj.so.7rh) at java.lang.ClassLoader.loadClass(libgcj.so.7rh) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:375) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:337) at java.lang.Class.forName(libgcj.so.7rh) at java.lang.Class.initializeClass(libgcj.so.7rh) ...22 more 2009-06-08 18:36:28.031::WARN:
Re: Does Solr know what host and port it is running on?
All good points, Paul. I've settled on putting it into the config, as much as I don't want to. On Jun 6, 2009, at 4:59 PM, Paul Libbrecht wrote: It is common knowledge that a web-application cannot know it's port and host-name: - many possible cnames to host-name - potential tunnels - most important and most used: proxies, typically an apache mod_proxy or mod_webapp on a port different than the servlet container port. Le 06-juin-09 à 09:38, Noble Paul നോബിള് नोब्ळ् a écrit : can it be added into the solrconfig.xml or solr.xml ? My experience is that it is simple as that: you have to configure something which, in ActiveMath, we have called the "advertizedURL". Making it non-configurable really often creates issues and requires the deployer to dig deep into the servlet container configuration (I never was fully able with mod_proxy and tomcat). Defaulting to something such as request.getServerPort etc is probably right though. hope it helps. paul On Sat, Jun 6, 2009 at 10:55 AM, Shalin Shekhar Mangar wrote: On Sat, Jun 6, 2009 at 9:58 AM, Grant Ingersoll wrote: Does Solr know what host and port it is running on and, if so, how do I get access to it programmatically? I feel like I'm missing something obvious. No, it does not. You can only get this on the first request. There's an issue open at https://issues.apache.org/jira/browse/SOLR-727 -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: searchcomponent howto ...
Is there a question here? On Jun 7, 2009, at 2:50 PM, David Palaitis wrote: I'm trying to write a very simple SearchComponent that performs a simple operation on the query string before passing through to QueryComponent. The EncodingSearchComponent will perform something like the following Base64(MD5(querystring)) -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: Terms Component
It seems like your /terms handler is likely expecting a q parameter due to the QueryComponent being one of the Handlers SearchComponents. What does your configuration for /terms look like? -Grant On Jun 8, 2009, at 7:48 AM, Anshuman Manur wrote: Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org .apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java: 104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org .apache .solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 290) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 235) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 233) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 175) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java: 568) at org .ofbiz .catalina .container .CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 286) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 844) at org.apache.coyote.http11.Http11Protocol $Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint $Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: Terms Component
while on the subject, can anybody tell me when Solr 1.4 might come out? Thanks Anshuman Manur On Mon, Jun 8, 2009 at 5:37 PM, Anshuman Manur wrote: > I'm using Solr 1.3 apparently.and Solr 1.4 is not out yet. > Sorry..My mistake! > > > On Mon, Jun 8, 2009 at 5:18 PM, Anshuman Manur < > anshuman_ma...@stragure.com> wrote: > >> Hello, >> >> I want to use the terms component in Solr 1.4: But >> >> http://localhost:8983/solr/terms?terms.fl=name >> >> >> But, I get the following error with the above query: >> >> java.lang.NullPointerException >> at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) >> at >> org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) >> at org.apache.solr.search.QParser.getQuery(QParser.java:88) >> >> >> at >> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) >> at >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) >> >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) >> at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) >> >> >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) >> >> >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >> >> >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> >> >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> at >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) >> at >> org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) >> >> >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) >> at >> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) >> at >> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) >> >> >> at >> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) >> at java.lang.Thread.run(Thread.java:619) >> >> >> Any help would be great. >> >> Thanks >> Anshuman Manur >> > >
Re: Terms Component
I'm using Solr 1.3 apparently.and Solr 1.4 is not out yet. Sorry..My mistake! On Mon, Jun 8, 2009 at 5:18 PM, Anshuman Manur wrote: > Hello, > > I want to use the terms component in Solr 1.4: But > > http://localhost:8983/solr/terms?terms.fl=name > > > But, I get the following error with the above query: > > java.lang.NullPointerException > at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) > at > org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) > at org.apache.solr.search.QParser.getQuery(QParser.java:88) > > at > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) > at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) > at > org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) > > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) > at java.lang.Thread.run(Thread.java:619) > > > Any help would be great. > > Thanks > Anshuman Manur >
Terms Component
Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur
How to make wordDelimiterFilter[pulled from Solr nighly] to not break non-english words in a wrong way in lucene indexing/searching?
Hi All,
I'm trying to index some indian web page content which are basically a mix
of indian and say 5% of english content in the same page itself. For all
this I can not use standard or simple analyzer as they break the non-english
words in a wrong places say[because the isLetter(ch) happens to be false for
them, even if they are part of a word]. So I wrote/extended the anayzer that
does the following,
public class IndicAnalyzer extends Analyzer {
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream ts = new WhitespaceTokenizer(reader);
//ts = new WordDelimiterFilter(ts, 1, 1, 1, 1, 0);
ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS);
ts = new LowerCaseFilter(ts);
ts = new PorterStemFilter(ts);
return ts;
}
}
This is working fine to some extent when the line commented above is left as
such, but its not able to give me the results when the documtnt contains a
string say "he...@how.com" and the searched query is hello, this is expected
as the above code doesnot do any word delimiting around these basic
characters [like @ . , etc ].
Now the problem is when I'm trying to use wordDelimiterFilter[hte commented
out line, this filter I got from solr] it is breaking say hindi words around
some characters which are actually part of a word. After going through the
code for this filter I found that it is using the isLetter() standard
fuction of java which I think returns false for those hindi characters
around which it is breaking the words. As per javadoc isLetter() is Unicode
compliant, right? so can't we say that it is aware of the above characters
that they are word delimiters, then why is this breaking around those
characters. I'm stuck and dont know how to get rid of the problem. And
because of this problem when I search for say a hindi word "helo" , assuming
its hindi, it highlights this word but alognwith that it also highlights the
letters of this word h/e/l/o whereever it finds it which it should not do,
right?
I request both Solr and Lucene users to guide me in fixing this issue. BTW,
do we need to do some sort of normalization for the content before sending
it to lucene indexer? just a thought, i don know whats the way out?
Query on date fields
Hi, I have two date attributes in my Indexes: DisplayStartDate_dt DisplayEndDate_dt I need to fetch results where today's date lies between displayStartDate and dislayEndDate. However i cannot send hardcoded displayStartdate and displayEndDate date in query as there are 1000 different dates in indexes Please suggest the query. Thanks, Prerna -- View this message in context: http://www.nabble.com/Query-on-date-fields-tp23919342p23919342.html Sent from the Solr - User mailing list archive at Nabble.com.
