Re: Slight issue with classloading and DataImportHandler
We plan to use SolrResourceLoader (in the next patch) . That is the best way to go. But we still prefer the usage of DIH package classes without any prefix. type=HttpDataSource instead of type=solr.HttpDataSource But users must be able to load their classes using the solr.classname format --Noble On Thu, Jun 19, 2008 at 10:57 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : aah!. We always assumed that people put the custom jars in the : WEB-INF/lib folder of solr webapp and hence they are automatically in : the classpath we shall make the necessary changes . It would be better to use the classloader from the SolrResourceLoader ... that should be safe for anyone with any setup. DIH does not load class using the SolrResourceLoader. It tries a Class.forName() with the name you provide if it fails it prepends org.apache.solr.handler.dataimport. and retries. ... The reason for doing so is that we do not use any of the 'solr.' packages in DIH. All our implementations fall into the default package and we can directly use them w/o the package name. FWIW: there isn't relaly a solr. package ... solr. can be used as an short form alias for the likely package when Solr resolves classes, where the likely package varies by context and there can be multiple options that it tries in order DIH could do the same thing, letting short form solr. signify that Transformers, Evaluators, etc are in the o.a.s.handler.dataimport package. the advantage of this over what it sounds like DIH currently does is that if there is an o.a.s.handler.dataimport.WizWatTransformer but someone wants to write their own (package less) WizWatTransformer they can and refer to it simply as WizWatTransformer (whereas to use the one that ships with DIH they would specify solr.WizWatTransformer). There's no ambiguity as to which one someone means unless they create a package called solr ... but then they'ed just be looking for trouble :) -Hoss -- --Noble Paul
Problem with searching using the DisMaxHandler
Hello all, I have defined a DisMax handler. It should search in the following fields: content1, content2 and id(doc uid). I would like to beable to specify a query like the following: (search terms) AND ( id1 OR id2 .. idn) My intent is to retrieve only the docs in which hits for the search terms occur and that the docs have one of the specified ids. Unfortunately, I get not document matches. Can any one shed some light on the What I am doing wrong? Thanks, Brian
Re[2]: Seeking suggestions - keyword related site promotion
Hello Stephen, I'm trying to understand what you are suggesting. Yes, I have a fixed set of keywords for each good site, although they occasionally get edited. Not sure what you mean by index these keywords into a field for each site. Did you mean index these keywords for all pages (that contain those keywords) of each site? So that I can boost the field weight in dismax query in order to boost these sites? I think this is a way of doing this. Problem the right way. The downside is that when I need to edit the keyword list, I'll need to re-index all pages that contain those keywords. But there's probably no easy way around it. As far as I know, Nutch doesn't do site specific boost. Thanks, Jack Wednesday, June 18, 2008, 10:52:33 PM, you wrote: Is there a fixed set of keywords? If so, I suppose you could simply index these keywords into a field for each site (either through some kind of automatic parser or manually - from personal experience I would recommend manually unless you have tens of thousands of these things), and then search that field with each word in the query (with or). Any site that had one of these keywords would match it if it were used in the query... If there is no list here and you're just indexing all the content of all these sites... isn't that what Nutch is designed for? -- Steve On Jun 18, 2008, at 11:05 PM, JLIST wrote: Hi all, This is what I'm trying to do: since some sources (say, some web sites) are more authoritative than other sources on certain subjects, I'd like to promote those sites when the query contains certain keywords. I'm not sure what is the best way to implement this. I suppose I can index the keywords in a field for all pages from that site but this isn't very efficient, and any changes in the keyword list would require re-indexing all pages of that site. I wonder if there is a more efficient way that can dynamically promote sites from a domain that is considered more related to the queries. Any suggestion is welcome.
Re: Problem with searching using the DisMaxHandler
The dismax query parser only accepts limited special syntax. From the wiki: http://wiki.apache.org/solr/DisMaxRequestHandler?highlight=%28CategorySolrRequestHandler%29%7C%28%28CategorySolrRequestHandler%29%29 ...is designed to process simple user entered phrases (without heavy syntax) It does not support AND, OR, or parens. But it does support a negation (minus sign in front of a term). One option is to separate the id's from the search terms, and build up a filter query (fq parameter) that OR's them all together. The default query parser for fq parameters is the standard query parser, not dismax. In terms of troubleshooting query parsing woes, don't forget that Solr supports debugQuery=true parameter. The response will show the parsed query, which will often shed light on what is going on. Erik On Jun 19, 2008, at 4:01 AM, Brian Carmalt wrote: Hello all, I have defined a DisMax handler. It should search in the following fields: content1, content2 and id(doc uid). I would like to beable to specify a query like the following: (search terms) AND ( id1 OR id2 .. idn) My intent is to retrieve only the docs in which hits for the search terms occur and that the docs have one of the specified ids. Unfortunately, I get not document matches. Can any one shed some light on the What I am doing wrong? Thanks, Brian
http://localhost:8983/solr/core0/update
Hi, I have created 2 cores ( core0 and core1 ) now I want to update post.sh to update data for core0 . I changed URL=http://localhost:8983/solr/update to URL=http://localhost:8983/solr/core0/update in post.sh but while posting it gives error curl: (7) couldn't connect Any help would be greatly appreciated with regards Rohit Arora
Re: http://localhost:8983/solr/core0/update
Hi, Sounds like nothing is running on port 8983. Maybe check port 8080? The port being used by the container probably gets printed on startup (at least Jetty does that). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: rohit arora [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Thursday, June 19, 2008 5:23:51 AM Subject: http://localhost:8983/solr/core0/update Hi, I have created 2 cores ( core0 and core1 ) now I want to update post.sh to update data for core0 . I changed URL=http://localhost:8983/solr/update to URL=http://localhost:8983/solr/core0/update in post.sh but while posting it gives error curl: (7) couldn't connect Any help would be greatly appreciated with regards Rohit Arora
snapshooter configuration
Hello, In my solrconfig I have the entry: !-- A postCommit event is fired after every commit or optimize command listener event=postCommit class=solr.RunExecutableListener str name=exesolr/bin/snapshooter/str str name=dir./str bool name=waittrue/bool arr name=args strarg1/str strarg2/str /arr arr name=env strMYVAR=val1/str /arr /listener -- I am unable to get this wokring - the catalin.out is unable to find snapshooter. Do others give the full path to snapshooter? Why do the template docs not say /full/path/to/snapshooter Thanks, John
Re: Slight issue with classloading and DataImportHandler
Hi, Are you using DataImportHandler in a production environment ? When it will be available in the nightly builds ? Thanks, William. On Thu, Jun 19, 2008 at 3:12 AM, Noble Paul നോബിള് नोब्ळ् [EMAIL PROTECTED] wrote: We plan to use SolrResourceLoader (in the next patch) . That is the best way to go. But we still prefer the usage of DIH package classes without any prefix. type=HttpDataSource instead of type=solr.HttpDataSource But users must be able to load their classes using the solr.classname format --Noble On Thu, Jun 19, 2008 at 10:57 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : aah!. We always assumed that people put the custom jars in the : WEB-INF/lib folder of solr webapp and hence they are automatically in : the classpath we shall make the necessary changes . It would be better to use the classloader from the SolrResourceLoader ... that should be safe for anyone with any setup. DIH does not load class using the SolrResourceLoader. It tries a Class.forName() with the name you provide if it fails it prepends org.apache.solr.handler.dataimport. and retries. ... The reason for doing so is that we do not use any of the 'solr.' packages in DIH. All our implementations fall into the default package and we can directly use them w/o the package name. FWIW: there isn't relaly a solr. package ... solr. can be used as an short form alias for the likely package when Solr resolves classes, where the likely package varies by context and there can be multiple options that it tries in order DIH could do the same thing, letting short form solr. signify that Transformers, Evaluators, etc are in the o.a.s.handler.dataimport package. the advantage of this over what it sounds like DIH currently does is that if there is an o.a.s.handler.dataimport.WizWatTransformer but someone wants to write their own (package less) WizWatTransformer they can and refer to it simply as WizWatTransformer (whereas to use the one that ships with DIH they would specify solr.WizWatTransformer). There's no ambiguity as to which one someone means unless they create a package called solr ... but then they'ed just be looking for trouble :) -Hoss -- --Noble Paul
solrj.embedded.JettySolrRunner and logging to file instead of STDERR
Hi, I've modified a copy of ./src/test/org/apache/solr/TestDistributedSearch.java for my own build process. I can compile fine but running the test always logs to STDERR INFO: Logging to STDERR via org.mortbay.log.StdErrLog This method appears deprecated? //public JettySolrRunner( String context, String home, String dataDir, int port, boolean log ) How can I log to a file instead of STDERR Many thanks, Dan
Re: solrj.embedded.JettySolrRunner and logging to file instead of STDERR
Solr uses JDK logging system, Configuring this will make you happy. []s, LUcas Lucas Frare A. Teixeira [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Tel: +55 11 3660.1622 - R3018 Daniel Rosher escreveu: Hi, I've modified a copy of ./src/test/org/apache/solr/TestDistributedSearch.java for my own build process. I can compile fine but running the test always logs to STDERR INFO: Logging to STDERR via org.mortbay.log.StdErrLog This method appears deprecated? //public JettySolrRunner( String context, String home, String dataDir, int port, boolean log ) How can I log to a file instead of STDERR Many thanks, Dan
xpath over update xml
I’m new to solr (using the 1.3 nightly at the moment) and trying to configure it to accept a third-party xml schema at the /update interface. I would like to define transformations like those of the DataImportHandler which use xpath syntax. The DataImportHandler ‘pulls’ data from a url. I would like to push data into the index using the standard /update api but be able to use xpath expressions to define the source of the my fields. Any help appreciated. Thanks. Mike -- View this message in context: http://www.nabble.com/xpath-over-update-xml-tp18012869p18012869.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: [Update] Solr can be started from jetty but not tomcat
The error messages due to this problem are very misleading. After a lot of trial and error I got solr to work with tomcat by adding the xalan.jar to the libs directory and rebuilding the war file. Vinci wrote: Hi all, after several hour I make the solr works a little bit: the jetty version works, but the tomcat version doesn't. Enviroment: JRE 1.6, tomcat 5.5, ubuntu 7.10. Solr nightly (8 Mar 08) Look like the multicore.xml cause the problem...the Solr die at the time of Config? In the localhost log: org.apache.catalina.core.StandardContext filterStart SEVERE: Exception starting filter SolrRequestFilter java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.core.SolrConfig at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:114) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:221) at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:302) at org.apache.catalina.core.ApplicationFilterConfig.init(ApplicationFilterConfig.java:78) at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3635) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4222) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:760) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:740) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:544) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:626) at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:553) at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:488) at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1138) at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311) at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:120) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1022) at org.apache.catalina.core.StandardHost.start(StandardHost.java:736) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) at org.apache.catalina.core.StandardService.start(StandardService.java:448) at org.apache.catalina.core.StandardServer.start(StandardServer.java:700) at org.apache.catalina.startup.Catalina.start(Catalina.java:552) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433) Catalina log: org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: Using JNDI solr.home: /var/webapps/solr org.apache.solr.servlet.SolrDispatchFilter init INFO: looking for multicore.xml: /var/webapps/solr/multicore.xml org.apache.solr.servlet.SolrDispatchFilter init SEVERE: Could not start SOLR. Check solr/home property java.lang.ExceptionInInitializerError at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:104) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:221) at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:302) at org.apache.catalina.core.ApplicationFilterConfig.init(ApplicationFilterConfig.java:78) at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3635) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4222) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:760) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:740) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:544) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:626) at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:553) at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:488) at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1138) at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311) at
Re: Did you mean functionality
Is there any work being done on getting this into SolrJ at the moment? Thanks! Matthew Runo Software Developer Zappos.com 702.943.7833 On Jun 18, 2008, at 3:09 AM, Lucas F. A. Teixeira wrote: Yeah, i read it. Thanks a lot, I`m waiting for it! []s, Lucas Lucas Frare A. Teixeira [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Tel: +55 11 3660.1622 - R3018 Grant Ingersoll escreveu: Also see http://wiki.apache.org/solr/SpellCheckComponent I expect to commit fairly soon. On Jun 17, 2008, at 5:46 PM, Otis Gospodnetic wrote: Hi Lucas, Have a look at (the patch in) SOLR-572, lots of work happening there as we speak. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Lucas F. A. Teixeira [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Tuesday, June 17, 2008 4:30:12 PM Subject: Did you mean functionality Hello everybody, I need to integrate the Lucene SpellChecker Contrib lib in my applycation, but I`m using the EmbeededSolrServer to access all indexes. I want to know what should I do (if someone have any step-by- step, link, tutorial or smoke signal) of what I need to do during indexing, and of course to search through this words generated by this API. I can use the lib itself to search the suggestions, w/out using solr, but I`m confused about how may I proceed when indexing this docs. Thanks a lot, []s, -- Lucas Frare A. Teixeira [EMAIL PROTECTED] Tel: +55 11 3660.1622 - R3018
Re: Did you mean functionality
FYI, it's now committed. Umm, SolrJ, not yet, but that sounds like a logical next step. Patches welcome! I don't have time for it at this particular moment, but I would be more than happy to commit one if someone else writes it up. -Grant On Jun 19, 2008, at 2:07 PM, Matthew Runo wrote: Is there any work being done on getting this into SolrJ at the moment? Thanks! Matthew Runo Software Developer Zappos.com 702.943.7833 On Jun 18, 2008, at 3:09 AM, Lucas F. A. Teixeira wrote: Yeah, i read it. Thanks a lot, I`m waiting for it! []s, Lucas Lucas Frare A. Teixeira [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Tel: +55 11 3660.1622 - R3018 Grant Ingersoll escreveu: Also see http://wiki.apache.org/solr/SpellCheckComponent I expect to commit fairly soon. On Jun 17, 2008, at 5:46 PM, Otis Gospodnetic wrote: Hi Lucas, Have a look at (the patch in) SOLR-572, lots of work happening there as we speak. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Lucas F. A. Teixeira [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Tuesday, June 17, 2008 4:30:12 PM Subject: Did you mean functionality Hello everybody, I need to integrate the Lucene SpellChecker Contrib lib in my applycation, but I`m using the EmbeededSolrServer to access all indexes. I want to know what should I do (if someone have any step-by- step, link, tutorial or smoke signal) of what I need to do during indexing, and of course to search through this words generated by this API. I can use the lib itself to search the suggestions, w/out using solr, but I`m confused about how may I proceed when indexing this docs. Thanks a lot, []s, -- Lucas Frare A. Teixeira [EMAIL PROTECTED] Tel: +55 11 3660.1622 - R3018 -- Grant Ingersoll http://www.lucidimagination.com Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ
Re: Did you mean functionality
On Thu, Jun 19, 2008 at 2:07 PM, Matthew Runo [EMAIL PROTECTED] wrote: Is there any work being done on getting this into SolrJ at the moment? Just a note to those who may be new to SolrJ: you can still access new or custom functionality in a generic way via getResponse() w/o explicit SolrJ support. -Yonik
Re: Did you mean functionality
Hmmm, good point. I had completely forgotten about that route. Thanks! Matthew Runo Software Developer Zappos.com 702.943.7833 On Jun 19, 2008, at 1:31 PM, Yonik Seeley wrote: On Thu, Jun 19, 2008 at 2:07 PM, Matthew Runo [EMAIL PROTECTED] wrote: Is there any work being done on getting this into SolrJ at the moment? Just a note to those who may be new to SolrJ: you can still access new or custom functionality in a generic way via getResponse() w/o explicit SolrJ support. -Yonik
Solr performance issues
Hi, I've been using solr for a little without worrying too much about how it works but now it's becoming a bottleneck in my application. I have a couple issues with it: 1. My index always gets slower and slower when commiting/optimizing for some obscure reason. It goes from 1 second with a new index to 45 seconds with an index with the same amount of data but used for a few days. Restarting solr doesn't fix it. The only way I found to fix that is to delete the whole index completely by deleting the index folder. Then when I rebuild the index everything goes back to normal and fast... and then performance slowly deteriorates again. So, the amount of data is not a factor because rebuilding the index from scratch fixes the problem and I am sending optimize once in a while... even maybe too often. 2. I use acts_as_solr and by default they only make post requests, even for /select. With that setup the response time for most queries, simple or complex ones, were ranging from 150ms to 600ms, with an average of 250ms. I changed the select request to use get requests instead and now the response time is down to 10ms to 60ms. Did someone seen that before? Why is it doing it? Thanks in advance, Sebastien
Re: Slight issue with classloading and DataImportHandler
We are using DIH in production . It is a planned feature for 1.3 realease. Let us see when it happens --Noble On Thu, Jun 19, 2008 at 7:35 PM, William Silva [EMAIL PROTECTED] wrote: Hi, Are you using DataImportHandler in a production environment ? When it will be available in the nightly builds ? Thanks, William. On Thu, Jun 19, 2008 at 3:12 AM, Noble Paul നോബിള് नोब्ळ् [EMAIL PROTECTED] wrote: We plan to use SolrResourceLoader (in the next patch) . That is the best way to go. But we still prefer the usage of DIH package classes without any prefix. type=HttpDataSource instead of type=solr.HttpDataSource But users must be able to load their classes using the solr.classname format --Noble On Thu, Jun 19, 2008 at 10:57 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : aah!. We always assumed that people put the custom jars in the : WEB-INF/lib folder of solr webapp and hence they are automatically in : the classpath we shall make the necessary changes . It would be better to use the classloader from the SolrResourceLoader ... that should be safe for anyone with any setup. DIH does not load class using the SolrResourceLoader. It tries a Class.forName() with the name you provide if it fails it prepends org.apache.solr.handler.dataimport. and retries. ... The reason for doing so is that we do not use any of the 'solr.' packages in DIH. All our implementations fall into the default package and we can directly use them w/o the package name. FWIW: there isn't relaly a solr. package ... solr. can be used as an short form alias for the likely package when Solr resolves classes, where the likely package varies by context and there can be multiple options that it tries in order DIH could do the same thing, letting short form solr. signify that Transformers, Evaluators, etc are in the o.a.s.handler.dataimport package. the advantage of this over what it sounds like DIH currently does is that if there is an o.a.s.handler.dataimport.WizWatTransformer but someone wants to write their own (package less) WizWatTransformer they can and refer to it simply as WizWatTransformer (whereas to use the one that ships with DIH they would specify solr.WizWatTransformer). There's no ambiguity as to which one someone means unless they create a package called solr ... but then they'ed just be looking for trouble :) -Hoss -- --Noble Paul -- --Noble Paul
Re: xpath over update xml
This means you may need to write your own RequestHandler. If you wish to push data, write it to a directory and use DIH with FileDataSource --Noble On Thu, Jun 19, 2008 at 9:58 PM, segv [EMAIL PROTECTED] wrote: I'm new to solr (using the 1.3 nightly at the moment) and trying to configure it to accept a third-party xml schema at the /update interface. I would like to define transformations like those of the DataImportHandler which use xpath syntax. The DataImportHandler 'pulls' data from a url. I would like to push data into the index using the standard /update api but be able to use xpath expressions to define the source of the my fields. Any help appreciated. Thanks. Mike -- View this message in context: http://www.nabble.com/xpath-over-update-xml-tp18012869p18012869.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul
Re: Did you mean functionality
I can take it up. But should we wait for the feature to 'stabilize' before adding it to SolrJ? Till then the approach suggested by Yonik (getResponse()) should be fine --Noble On Fri, Jun 20, 2008 at 2:06 AM, Matthew Runo [EMAIL PROTECTED] wrote: Hmmm, good point. I had completely forgotten about that route. Thanks! Matthew Runo Software Developer Zappos.com 702.943.7833 On Jun 19, 2008, at 1:31 PM, Yonik Seeley wrote: On Thu, Jun 19, 2008 at 2:07 PM, Matthew Runo [EMAIL PROTECTED] wrote: Is there any work being done on getting this into SolrJ at the moment? Just a note to those who may be new to SolrJ: you can still access new or custom functionality in a generic way via getResponse() w/o explicit SolrJ support. -Yonik -- --Noble Paul
How to use SOLR1.2
Hi, I am new in SOLR 1.2, configured Admin GUI. Facing problem in using this. could you pls help me out to configure the nex. Regards, Ranjeet
