Re: Specifying multiple documents in DataImportHandler dataConfig
You can only have one document tag and the entities must be nested within that. >From the wiki, if you issue a simple "/dataimport?command=full-import" all top level entities will be processed. >Maybe I should be more clear: I have multiple tables in my DB that I >need to save to my Solr index. In my app code I have logic to persist >each table, which maps to an application model to Solr. This is fine. >I am just trying to speed up indexing time by using DIH instead of >going through my application. From what I understand of DIH I can >specify one dataSource element and then a series of document/entity >sets, for each of my models. But like I said before, DIH only appears >to want to index the first document declared under the dataSource tag. > >-Rupert > >On Tue, Sep 8, 2009 at 4:05 PM, Rupert Fiasco wrote: >> I am using the DataImportHandler with a JDBC datasource. From my >> understanding of DIH, for each of my "content types" e.g. Blog posts, >> Mesh Categories, etc I would construct a series of document/entity >> sets, like >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Solr parses this just fine and allows me to issue a >> /dataimport?command=full-import and it runs, but it only runs against >> the "first" document (blog_entries). It doesnt run against the 2nd >> document (mesh_categories). >> >> If I remove the 2 document elements and wrap both entity sets in just >> one document tag, then both sets get indexed, which seemingly achieves >> my goal. This just doesnt make sense from my understanding of how DIH >> works. My 2 content types are indeed separate so they logically >> represent two document types, not one. >> >> Is this correct? What am I missing here? >> >> Thanks >> -Rupert >> -- === Fergus McMenemie Email:fer...@twig.me.uk Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer ===
Very Urjent
Iam new to solr, My requirement is that,i need to have Autocompletion text box in my blog application,i need to know how to implement it with solr 1.4. I have gone through TermsComponent,but TermsComponent is not available in solr 1.4 which i have downloaded. Can any one please help out ,how to do autosuggest using solr 1.4,and provide me the code along with schema.xml and solrConfig.xml.So that it will be useful for me to know how to configure it. Waiting for your reply. Regards, Rekha. -- View this message in context: http://www.nabble.com/Very-Urjent-tp25359244p25359244.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SnowballPorterFilterFactory stemming word question
On Mon, Sep 7, 2009 at 2:49 AM, darniz wrote: > Does solr provide any implementation for dictionary stemmer, please let me > know The Krovetz stemmer is dictionary based (english only): http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/Kstem But from your original question, maybe you are concerned when the stemmer doesn't return real words? For normal search, don't be. During index time, words are stemmed, and then later the query is stemmed. If the results match up, you're good. For example, a document containing the word "machines" may stem to "machin" and then a query of "machined" will stem to "machin" and thus match the document. -Yonik http://www.lucidimagination.com
Re: Whitespace in query
: Subject: Whitespace in query : : Hi all, : : I'm trying to filter by a 'document name' which has spaces in it. : (E.g. "docname:Struts 1" ) 1) if you want to make a string with field with a space in it, that typically means using quotes -- either to query a literaly string field exactly or to use a phrase query... docname:"Struts 1" 2) your error message has nothing to do with having whitespace in your query... : [08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - << "HTTP/1.1 505 HTTP Version Not Supported[\r][\n]" 505 is usually thrown by HTTP daemons that don't understand HTTP/1.1, but from what i can tell Coyote should support 1.1, so i'm not sure why you owuld be getting this error ... do *any* requests work for you? did you try executing a search from the browser? have you considered using SolrJ? -Hoss
Re: Passing FuntionQuery string parameters
did you implement your own ValueSourceParser . the
FunctionQParser#parseArg() method supports strings
On Wed, Sep 9, 2009 at 12:10 AM, wojtekpia wrote:
>
> Hi,
>
> I'm writing a function query to score documents based on Levenshtein
> distance from a string. I want my function calls to look like:
>
> lev(myFieldName, 'my string to match')
>
> I'm running into trouble parsing the string I want to match ('my string to
> match' above). It looks like all the built in support is for parsing field
> names and numeric values. Am I missing the string parsing support, or is it
> not there, and if not, why?
>
> Thanks,
>
> Wojtek
> --
> View this message in context:
> http://www.nabble.com/Passing-FuntionQuery-string-parameters-tp25351825p25351825.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Backups using Replication
which version of Solr are you using? the "backupAfter" name was introduced recently On Tue, Sep 8, 2009 at 10:26 PM, wojtekpia wrote: > > I'm trying to create data backups using the ReplicationHandler's built in > functionality. I've configured my master as > http://wiki.apache.org/solr/SolrReplication documented : > > > > ... > optimize > ... > > > > > but I don't see any backups created on the master. Do I need the snapshooter > script available? I did not deploy it on my master, I assumed it was part of > the 'old' way of doing replication. > > If I invoke the backup command over HTTP > (http://master_host:port/solr/replication?command=backup) then it seems to > work - I get directories like "snapshot.20090908094423". > > Thanks, > > Wojtek > -- > View this message in context: > http://www.nabble.com/Backups-using-Replication-tp25350083p25350083.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: SnowballPorterFilterFactory stemming word question
: Thanks Hoss : Could you please provide with any example : : Does solr provide any implementation for dictionary stemmer, please let me As mentioned on the wiki... http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters Expansion stemming -- Takes a root word and 'expands' it to all of its various forms -- can be used either at insertion time or at query time. One way to approach this is by using the SynonymFilterFactory -Hoss
Re: solrj DocumentObjectBinder prints to stdout
I just removed it and checked it in.
Thanks
On Wed, Sep 9, 2009 at 8:12 AM, Ilan Rabinovitch wrote:
> Hello,
>
> We recently noticed that the inject method of DocumentObjectBinder [1] in
> solr4j logs to standard out using System.out.println if any variables in the
> returned result have null value.
>
>
> The system.out.println statement was added on 2009-07-14 as part of
> SOLR-1129.
>
> This can get quite spammy on the console, and as its a System.out statement
> there is no method by which to configure it currently.
>
> Should this be logging at the INFO/DEBUG level using slf4j instead?
> I've opened SOLR-1417 to track this.
>
>
> Thanks,
> Ilan
>
> [1]
> src/solrj/org/apache/solr/client/solrj/beans/DocumentObjectBinder.java:316
> (r794144)
>
> void inject(T obj, SolrDocument sdoc) {
> Object val = getFieldValue(sdoc);
> if(val == null) { System.out.println("val null for "+ name); return; }
>
>
>
> --
> Ilan Rabinovitch
> i...@fonz.net
>
> ---
> SCALE 8x: 2010 Southern California Linux Expo
> Feb 19-21, 2010
> Los Angeles, CA
> http://www.socallinuxexpo.org
>
>
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
Re: abortOnConfigurationError=false not taking effect in solr 1.3
Hi, I tried the false in solr.xml but it does not seem to be working for me. I manually modified solrconfig.xml for core1 to have an invalid path in datadir. And when i restarted my app server (jboss), it threw following error which is what is expected. There were other cores which loaded just fine. 20:23:26,997 ERROR [STDERR] Sep 8, 2009 8:23:26 PM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: java.io.IOException: The filename, directory name, or volume label syntax is incorrect at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:302) Once the server got started completely, i tried to access the other core's admin page (which does not have any problem) using the url http://localhost:8080/solr/core0/admin/ but it returned me a blank page and following error on server side. But when i correct the path for datadir in solrconfig.xml for core1 and after server restart, everything works fine. Am I missing something here? Please let me know if my testcase is incorrect. 20:30:01,736 ERROR [CoyoteAdapter] An exception or error occurred in the container during the request processing java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:680) at org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:419) at org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor.java:1588) at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:934) at org.apache.coyote.Response.action(Response.java:183) at org.apache.coyote.Response.sendHeaders(Response.java:379) at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305) at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:273) at org.apache.catalina.connector.Response.finishResponse(Response.java:486) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:257) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:580) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Thanks, Dharmveer djain101 wrote: > > Thanks a lot. I will try this. > > > > markrmiller wrote: >> >> >> >> >> >> false >> >> >> >> >> >> djain101 wrote: >>> Thanks !!! Can you please give an example to put it in solr.xml? How is >>> the >>> xml structure look like for enabling this property? >>> >>> >>> markrmiller wrote: >>> Yes - try adding it to solr.xml - thats where the CoreContainer looks for it. -- - Mark http://www.lucidimagination.com djain101 wrote: > Any help is much appreciated. > > Thanks, > Dharmveer > > > djain101 wrote: > > >> Hi, >> >> We have set >> false >> in solrconfig.xml for all the cores but still whenever any >> configuration >> error occurs search does not work in any of the cores. Is there any >> other >> configuration we have to set so as to not affect search in the cores >> which >> are configured correctly? >> >> For one of the core, we got following error today but then none of >> the >> core is working after that. The below message suggests to add >> abortOnConfigurationError in solr.xml but we have it added in >> solrconfig.xml for all the cores. Do we need to add in solr.xml? If >> yes, >> where? >> >> Please suggest what is the right way to configure so that if one core >> fails due to configuration errors, all other cores remain unaffected? >> >> * >> Check your log files for more detailed information on what may be >> wrong. >> >> If you want solr to continue after configuration errors, change: >> >> >>false >> >> in solr.xml >> >> java.lang.RuntimeException: java.io.IOException: Cannot create >> directory: >> /usr/local/app/data/search/core09/index >> *** >> >> >> > > >>> >>> >> >> >> -- >> - Mark >> >> http://www.lucidimagination.com >> >> >> >> >> > > -- View this message in context: http://www.nabble.com/abortOnConfigurationError%3Dfalse-not-taking-effect-in-solr-1.3-tp25332254p25358007.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Specifying multiple documents in DataImportHandler dataConfig
DIH allows only tag. you may have multiple root tags and you may invoke them by name(s). When no name is passed all root entities are invoked one after another. On Wed, Sep 9, 2009 at 5:12 AM, Rupert Fiasco wrote: > Maybe I should be more clear: I have multiple tables in my DB that I > need to save to my Solr index. In my app code I have logic to persist > each table, which maps to an application model to Solr. This is fine. > I am just trying to speed up indexing time by using DIH instead of > going through my application. From what I understand of DIH I can > specify one dataSource element and then a series of document/entity > sets, for each of my models. But like I said before, DIH only appears > to want to index the first document declared under the dataSource tag. > > -Rupert > > On Tue, Sep 8, 2009 at 4:05 PM, Rupert Fiasco wrote: >> I am using the DataImportHandler with a JDBC datasource. From my >> understanding of DIH, for each of my "content types" e.g. Blog posts, >> Mesh Categories, etc I would construct a series of document/entity >> sets, like >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Solr parses this just fine and allows me to issue a >> /dataimport?command=full-import and it runs, but it only runs against >> the "first" document (blog_entries). It doesnt run against the 2nd >> document (mesh_categories). >> >> If I remove the 2 document elements and wrap both entity sets in just >> one document tag, then both sets get indexed, which seemingly achieves >> my goal. This just doesnt make sense from my understanding of how DIH >> works. My 2 content types are indeed separate so they logically >> represent two document types, not one. >> >> Is this correct? What am I missing here? >> >> Thanks >> -Rupert >> > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Results from 2 core
I have a multi core Solr setup. Is it possible to return results from the second core, if the search on the first core, does not return any results. Or if its possible to return, the results from both the cores, in one response Both the core, Have the different schema, one is getting its data from database another is getting the payload from the Nutch crawl. Thanks/Regards, Parvez
solrj DocumentObjectBinder prints to stdout
Hello,
We recently noticed that the inject method of DocumentObjectBinder [1]
in solr4j logs to standard out using System.out.println if any variables
in the returned result have null value.
The system.out.println statement was added on 2009-07-14 as part of
SOLR-1129.
This can get quite spammy on the console, and as its a System.out
statement there is no method by which to configure it currently.
Should this be logging at the INFO/DEBUG level using slf4j instead?
I've opened SOLR-1417 to track this.
Thanks,
Ilan
[1]
src/solrj/org/apache/solr/client/solrj/beans/DocumentObjectBinder.java:316
(r794144)
void inject(T obj, SolrDocument sdoc) {
Object val = getFieldValue(sdoc);
if(val == null) { System.out.println("val null for "+ name); return; }
--
Ilan Rabinovitch
i...@fonz.net
---
SCALE 8x: 2010 Southern California Linux Expo
Feb 19-21, 2010
Los Angeles, CA
http://www.socallinuxexpo.org
Re: date field
Hi Gérard, Concerning the issue with the ":" character you can use the ClientUtils.escapeQueryChars() method to handle special characters that are part of the query syntax. The complete list of special characters is in the source code. check out the following resources org/apache/solr/client/solrj/util/ClientUtils.java http://lucene.apache.org/java/2_3_2/queryparsersyntax.html#Escaping%20Special%20Characters 2009/9/8 Gérard Dupont > Hi all, > > I'm currently facing a little difficulty to index and search on date field. > The indexing is done in the right way (I guess) and I can find valid date > in > the field like "2009-05-01T12:45:32Z". However when I'm searching the user > don't always give an exact date. for instance they give "2008-05-01" to get > all documents related to that day. I can do a trick using wildcard but is > there another way to do it ? Moreover if they give the full date string (or > if I hack the query parser) I can have the full syntax, but then the ":" > annoy me because the Lucene parser does not allow it without quotes. Any > ideas ? > > -- > Gérard Dupont > Information Processing Control and Cognition (IPCC) - EADS DS > http://weblab.forge.ow2.org > > Document & Learning team - LITIS Laboratory > -- "Good Enough" is not good enough. To give anything less than your best is to sacrifice the gift. Quality First. Measure Twice. Cut Once.
Re: Misleading log messages while deploying solr
: But the log message that is getting print in the server console, in my case : jboss, is showing status as error. : Why is this showing as ERROR, even though things are working fine. Solr is not declaring that those messages are ERRORs, solr is just logging informational messages (hence then "INFO" lines) using the java logging framework. My guess: since the logs are getting prefixed with "ERROR [STDERR]" something about the way your jboss container is configured is probably causing those log messages to be written to STDERR, and then jboss is capturing the STDERR and assuming that if it went there it mist be an "ERROR" of some kind and logging it to the console (using it's own log format, hence the touble timestamps per line message) In short: jboss is doing this in response to normal logging from solr. you should investigate your options for configuriring jboss and how it deals with log messages from applications. : 11:41:19,030 INFO [TomcatDeployer] deploy, ctxPath=/solr, : warUrl=.../tmp/deploy/tmp43266solr-exp.war/ : 11:41:19,948 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM : org.apache.solr.servlet.SolrDispatchFilter init : INFO: SolrDispatchFilter.init() : 11:41:19,975 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM : org.apache.solr.core.SolrResourceLoader locateInstanceDir : INFO: No /solr/home in JNDI : 11:41:19,976 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM : org.apache.solr.core.SolrResourceLoader locateInstanceDir : INFO: using system property solr.solr.home: C:\app\Search : 11:41:19,984 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM : org.apache.solr.core.CoreContainer$Initializer initialize : INFO: looking for solr.xml: C:\app\Search\solr.xml : 11:41:20,084 ERROR [STDERR] 8 Sep, 2009 11:41:20 AM : org.apache.solr.core.SolrResourceLoader : INFO: Solr home set to 'C:\app\Search' : 11:41:20,142 ERROR [STDERR] 8 Sep, 2009 11:41:20 AM : org.apache.solr.core.SolrResourceLoader createClassLoader : INFO: Adding : 'file:/C:/app/Search/lib/apache-solr-dataimporthandler-1.3.0.jar' to Solr : classloader : 11:41:20,144 ERROR [STDERR] 8 Sep, 2009 11:41:20 AM : org.apache.solr.core.SolrResourceLoader createClassLoader : INFO: Adding 'file:/C:/app/Search/lib/jsp-2.1/' to Solr classloader : : ... : INFO: Reusing parent classloader : 11:41:21,870 ERROR [STDERR] 8 Sep, 2009 11:41:21 AM : org.apache.solr.core.SolrConfig : INFO: Loaded SolrConfig: solrconfig.xml : 11:41:21,909 ERROR [STDERR] 8 Sep, 2009 11:41:21 AM : org.apache.solr.schema.IndexSchema readSchema : INFO: Reading Solr Schema : 11:41:22,092 ERROR [STDERR] 8 Sep, 2009 11:41:22 AM : org.apache.solr.schema.IndexSchema readSchema : INFO: Schema name=contacts schema : 11:41:22,121 ERROR [STDERR] 8 Sep, 2009 11:41:22 AM : org.apache.solr.util.plugin.AbstractPluginLoader load : INFO: created string: org.apache.solr.schema.StrField : : . : -- : View this message in context: http://www.nabble.com/Misleading-log-messages-while-deploying-solr-tp25354654p25354654.html : Sent from the Solr - User mailing list archive at Nabble.com. : -Hoss
Re: Does the default operator affect phrase searching?
: My problem was that somewhere along the way I picked up the mis-understanding : that I could do a phrase search like Text:"david pdf" and Solr/lucene would pick up : all documents that hit the field Text with either 'david' or 'pdf'. Now, I'm just doing the you were probably thinking of this... Text:(david pdf) -Hoss
Re: Date Faceting and Double Counting
: I ran into that problem as well but the solution was provided to me by : this very list :) See : http://www.nabble.com/Range-queries-td24057317.html It's not the : cleanest solution, but as long as you know what you're doing it's not : that bad. Hmmm... yeah, that's a total hack. one of these days we really need to fix the lucene query parser grammer so inclusive/exclusive can be different for hte upper/lower bounds... datefield:[NOW/DAY TO NOW/DAY+1DAY} -Hoss
Re: Drill down into hierarchical facet : how to?
: I know I can make a facet query : something like Europe/France/* to search all POI in France : but how can I show the facet level under France (Paris, Lyon etc ...) ??? i don't fully understand your question, so i'm not sure if you mean "filter query" (fq) instead of "facet query" (facet.query) but if you are using facet=location, then facet.prefix can help you ensure that you only return values that start with Europe/France you may also be interested in this thread... http://www.nabble.com/Re%3A-Hierarchical-Faceting-p20176326.html -Hoss
Re: date field
: the field like "2009-05-01T12:45:32Z". However when I'm searching the user : don't always give an exact date. for instance they give "2008-05-01" to get whatever date your users give you, you'll need to format it in the full ISO format in order to query on it (ie: 2008-05-01 into 2008-05-01T00:00:00Z) but once you do that, you can append date math syntax to query for a range some range arround your value yourField:[2008-05-01T00:00:00Z/DAY TO 2008-05-01T00:00:00Z/DAY+1DAY-1MILLISECOND] http://wiki.apache.org/solr/SolrQuerySyntax -Hoss
Re: Specifying multiple documents in DataImportHandler dataConfig
Maybe I should be more clear: I have multiple tables in my DB that I need to save to my Solr index. In my app code I have logic to persist each table, which maps to an application model to Solr. This is fine. I am just trying to speed up indexing time by using DIH instead of going through my application. From what I understand of DIH I can specify one dataSource element and then a series of document/entity sets, for each of my models. But like I said before, DIH only appears to want to index the first document declared under the dataSource tag. -Rupert On Tue, Sep 8, 2009 at 4:05 PM, Rupert Fiasco wrote: > I am using the DataImportHandler with a JDBC datasource. From my > understanding of DIH, for each of my "content types" e.g. Blog posts, > Mesh Categories, etc I would construct a series of document/entity > sets, like > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Solr parses this just fine and allows me to issue a > /dataimport?command=full-import and it runs, but it only runs against > the "first" document (blog_entries). It doesnt run against the 2nd > document (mesh_categories). > > If I remove the 2 document elements and wrap both entity sets in just > one document tag, then both sets get indexed, which seemingly achieves > my goal. This just doesnt make sense from my understanding of how DIH > works. My 2 content types are indeed separate so they logically > represent two document types, not one. > > Is this correct? What am I missing here? > > Thanks > -Rupert >
Re: Concept Expansion
: Subject: Concept Expansion : References: : <847931a70909041446x79f7b600g27ea556c85691...@mail.gmail.com> : <847931a70909051050x41d0b8b2jd6424312c3ee5...@mail.gmail.com> http://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mailing list, please do not reply to an existing message, instead start a fresh email. Even if you change the subject line of your email, other mail headers still track which thread you replied to and your question is "hidden" in that thread and gets less attention. It makes following discussions in the mailing list archives particularly difficult. See Also: http://en.wikipedia.org/wiki/Thread_hijacking -Hoss
Re: Dynamically building the value of a field upon indexing
: Subject: Dynamically building the value of a field upon indexing : References: <25332254.p...@talk.nabble.com> <25354786.p...@talk.nabble.com> http://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mailing list, please do not reply to an existing message, instead start a fresh email. Even if you change the subject line of your email, other mail headers still track which thread you replied to and your question is "hidden" in that thread and gets less attention. It makes following discussions in the mailing list archives particularly difficult. See Also: http://en.wikipedia.org/wiki/Thread_hijacking -Hoss
Specifying multiple documents in DataImportHandler dataConfig
I am using the DataImportHandler with a JDBC datasource. From my understanding of DIH, for each of my "content types" e.g. Blog posts, Mesh Categories, etc I would construct a series of document/entity sets, like Solr parses this just fine and allows me to issue a /dataimport?command=full-import and it runs, but it only runs against the "first" document (blog_entries). It doesnt run against the 2nd document (mesh_categories). If I remove the 2 document elements and wrap both entity sets in just one document tag, then both sets get indexed, which seemingly achieves my goal. This just doesnt make sense from my understanding of how DIH works. My 2 content types are indeed separate so they logically represent two document types, not one. Is this correct? What am I missing here? Thanks -Rupert
Re: abortOnConfigurationError=false not taking effect in solr 1.3
Thanks a lot. I will try this. markrmiller wrote: > > > > > > false > > > > > > djain101 wrote: >> Thanks !!! Can you please give an example to put it in solr.xml? How is >> the >> xml structure look like for enabling this property? >> >> >> markrmiller wrote: >> >>> Yes - try adding it to solr.xml - thats where the CoreContainer looks >>> for it. >>> >>> -- >>> - Mark >>> >>> http://www.lucidimagination.com >>> >>> >>> >>> djain101 wrote: >>> Any help is much appreciated. Thanks, Dharmveer djain101 wrote: > Hi, > > We have set > false > in solrconfig.xml for all the cores but still whenever any > configuration > error occurs search does not work in any of the cores. Is there any > other > configuration we have to set so as to not affect search in the cores > which > are configured correctly? > > For one of the core, we got following error today but then none of the > core is working after that. The below message suggests to add > abortOnConfigurationError in solr.xml but we have it added in > solrconfig.xml for all the cores. Do we need to add in solr.xml? If > yes, > where? > > Please suggest what is the right way to configure so that if one core > fails due to configuration errors, all other cores remain unaffected? > > * > Check your log files for more detailed information on what may be > wrong. > > If you want solr to continue after configuration errors, change: > > >false > > in solr.xml > > java.lang.RuntimeException: java.io.IOException: Cannot create > directory: > /usr/local/app/data/search/core09/index > *** > > > >>> >>> >>> >>> >>> >> >> > > > -- > - Mark > > http://www.lucidimagination.com > > > > > -- View this message in context: http://www.nabble.com/abortOnConfigurationError%3Dfalse-not-taking-effect-in-solr-1.3-tp25332254p25355662.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: abortOnConfigurationError=false not taking effect in solr 1.3
false djain101 wrote: > Thanks !!! Can you please give an example to put it in solr.xml? How is the > xml structure look like for enabling this property? > > > markrmiller wrote: > >> Yes - try adding it to solr.xml - thats where the CoreContainer looks >> for it. >> >> -- >> - Mark >> >> http://www.lucidimagination.com >> >> >> >> djain101 wrote: >> >>> Any help is much appreciated. >>> >>> Thanks, >>> Dharmveer >>> >>> >>> djain101 wrote: >>> >>> Hi, We have set false in solrconfig.xml for all the cores but still whenever any configuration error occurs search does not work in any of the cores. Is there any other configuration we have to set so as to not affect search in the cores which are configured correctly? For one of the core, we got following error today but then none of the core is working after that. The below message suggests to add abortOnConfigurationError in solr.xml but we have it added in solrconfig.xml for all the cores. Do we need to add in solr.xml? If yes, where? Please suggest what is the right way to configure so that if one core fails due to configuration errors, all other cores remain unaffected? * Check your log files for more detailed information on what may be wrong. If you want solr to continue after configuration errors, change:false in solr.xml java.lang.RuntimeException: java.io.IOException: Cannot create directory: /usr/local/app/data/search/core09/index *** >>> >>> >> >> >> >> >> > > -- - Mark http://www.lucidimagination.com
Re: abortOnConfigurationError=false not taking effect in solr 1.3
Thanks !!! Can you please give an example to put it in solr.xml? How is the xml structure look like for enabling this property? markrmiller wrote: > > Yes - try adding it to solr.xml - thats where the CoreContainer looks > for it. > > -- > - Mark > > http://www.lucidimagination.com > > > > djain101 wrote: >> Any help is much appreciated. >> >> Thanks, >> Dharmveer >> >> >> djain101 wrote: >> >>> Hi, >>> >>> We have set false >>> in solrconfig.xml for all the cores but still whenever any configuration >>> error occurs search does not work in any of the cores. Is there any >>> other >>> configuration we have to set so as to not affect search in the cores >>> which >>> are configured correctly? >>> >>> For one of the core, we got following error today but then none of the >>> core is working after that. The below message suggests to add >>> abortOnConfigurationError in solr.xml but we have it added in >>> solrconfig.xml for all the cores. Do we need to add in solr.xml? If yes, >>> where? >>> >>> Please suggest what is the right way to configure so that if one core >>> fails due to configuration errors, all other cores remain unaffected? >>> >>> * >>> Check your log files for more detailed information on what may be wrong. >>> >>> If you want solr to continue after configuration errors, change: >>> >>> >>>false >>> >>> in solr.xml >>> >>> java.lang.RuntimeException: java.io.IOException: Cannot create >>> directory: >>> /usr/local/app/data/search/core09/index >>> *** >>> >>> >> >> > > > > > > -- View this message in context: http://www.nabble.com/abortOnConfigurationError%3Dfalse-not-taking-effect-in-solr-1.3-tp25332254p25355527.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: abortOnConfigurationError=false not taking effect in solr 1.3
Yes - try adding it to solr.xml - thats where the CoreContainer looks for it. -- - Mark http://www.lucidimagination.com djain101 wrote: > Any help is much appreciated. > > Thanks, > Dharmveer > > > djain101 wrote: > >> Hi, >> >> We have set false >> in solrconfig.xml for all the cores but still whenever any configuration >> error occurs search does not work in any of the cores. Is there any other >> configuration we have to set so as to not affect search in the cores which >> are configured correctly? >> >> For one of the core, we got following error today but then none of the >> core is working after that. The below message suggests to add >> abortOnConfigurationError in solr.xml but we have it added in >> solrconfig.xml for all the cores. Do we need to add in solr.xml? If yes, >> where? >> >> Please suggest what is the right way to configure so that if one core >> fails due to configuration errors, all other cores remain unaffected? >> >> * >> Check your log files for more detailed information on what may be wrong. >> >> If you want solr to continue after configuration errors, change: >> >>false >> >> in solr.xml >> >> java.lang.RuntimeException: java.io.IOException: Cannot create directory: >> /usr/local/app/data/search/core09/index >> *** >> >> > >
Dynamically building the value of a field upon indexing
I would like to build the value of a field based on the value of multiple other fields at submission time. I.e. I would like to submit a document such as; foo baa And would like SOLR to store the document as foo baa foo:baa Just to complicate matters I would like the aggregated field to be the unique key. Is this possible? Thanks, Gert. Please help Logica to respect the environment by not printing this email / Pour contribuer comme Logica au respect de l'environnement, merci de ne pas imprimer ce mail / Bitte drucken Sie diese Nachricht nicht aus und helfen Sie so Logica dabei, die Umwelt zu schützen. / Por favor ajude a Logica a respeitar o ambiente nao imprimindo este correio electronico. This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Re: abortOnConfigurationError=false not taking effect in solr 1.3
Any help is much appreciated. Thanks, Dharmveer djain101 wrote: > > Hi, > > We have set false > in solrconfig.xml for all the cores but still whenever any configuration > error occurs search does not work in any of the cores. Is there any other > configuration we have to set so as to not affect search in the cores which > are configured correctly? > > For one of the core, we got following error today but then none of the > core is working after that. The below message suggests to add > abortOnConfigurationError in solr.xml but we have it added in > solrconfig.xml for all the cores. Do we need to add in solr.xml? If yes, > where? > > Please suggest what is the right way to configure so that if one core > fails due to configuration errors, all other cores remain unaffected? > > * > Check your log files for more detailed information on what may be wrong. > > If you want solr to continue after configuration errors, change: > >false > > in solr.xml > > java.lang.RuntimeException: java.io.IOException: Cannot create directory: > /usr/local/app/data/search/core09/index > *** > -- View this message in context: http://www.nabble.com/abortOnConfigurationError%3Dfalse-not-taking-effect-in-solr-1.3-tp25332254p25354786.html Sent from the Solr - User mailing list archive at Nabble.com.
Misleading log messages while deploying solr
Hi I am deploying solr in jboss and is indexing data from db. Everything works fine and i am getting correct response. But the log message that is getting print in the server console, in my case jboss, is showing status as error. Why is this showing as ERROR, even though things are working fine. Since in the actual production environment this causes lot of mess up with other logs, and also misleads with this message, how can i hide this ERROR message(if it is not really error). or can i set the log level to INFO. 11:41:19,030 INFO [TomcatDeployer] deploy, ctxPath=/solr, warUrl=.../tmp/deploy/tmp43266solr-exp.war/ 11:41:19,948 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() 11:41:19,975 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: No /solr/home in JNDI 11:41:19,976 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: using system property solr.solr.home: C:\app\Search 11:41:19,984 ERROR [STDERR] 8 Sep, 2009 11:41:19 AM org.apache.solr.core.CoreContainer$Initializer initialize INFO: looking for solr.xml: C:\app\Search\solr.xml 11:41:20,084 ERROR [STDERR] 8 Sep, 2009 11:41:20 AM org.apache.solr.core.SolrResourceLoader INFO: Solr home set to 'C:\app\Search' 11:41:20,142 ERROR [STDERR] 8 Sep, 2009 11:41:20 AM org.apache.solr.core.SolrResourceLoader createClassLoader INFO: Adding 'file:/C:/app/Search/lib/apache-solr-dataimporthandler-1.3.0.jar' to Solr classloader 11:41:20,144 ERROR [STDERR] 8 Sep, 2009 11:41:20 AM org.apache.solr.core.SolrResourceLoader createClassLoader INFO: Adding 'file:/C:/app/Search/lib/jsp-2.1/' to Solr classloader ... INFO: Reusing parent classloader 11:41:21,870 ERROR [STDERR] 8 Sep, 2009 11:41:21 AM org.apache.solr.core.SolrConfig INFO: Loaded SolrConfig: solrconfig.xml 11:41:21,909 ERROR [STDERR] 8 Sep, 2009 11:41:21 AM org.apache.solr.schema.IndexSchema readSchema INFO: Reading Solr Schema 11:41:22,092 ERROR [STDERR] 8 Sep, 2009 11:41:22 AM org.apache.solr.schema.IndexSchema readSchema INFO: Schema name=contacts schema 11:41:22,121 ERROR [STDERR] 8 Sep, 2009 11:41:22 AM org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created string: org.apache.solr.schema.StrField . -- View this message in context: http://www.nabble.com/Misleading-log-messages-while-deploying-solr-tp25354654p25354654.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR-769 clustering
Hi, It seems like the problem can be on two layers: 1) getting the right contents of stop* files for Carrot2, 2) making sure Solr picks up the changes. I tried your quick and dirty hack too. It didn't work also. phase like > "Carbon Atoms in the Group" with "in" still appear in my clustering labels. > Here most probably layer 1) applies: if you add "in" to stopwords, the Lingo algorithm (Carrot2's default) will still create labels with "in" inside, but will not create labels starting / ending in "in". If you'd like to eliminate "in" completely, you'd need to put an appropriate regexp in stoplabels.*. For more details, please see Carrot2 manual: http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-words http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-regexps The easiest way to tune the stopwords and see their impact on clusters is to use Carrot2 Document Clustering Workbench (see http://wiki.apache.org/solr/ClusteringComponent). > What i did is, > > 1. use "java uf carrot2-mini.jar stoplabels.en" command to replace the > stoplabel.en file. > 2. apply clustering patch. re-complie the solr with the new > carrot2-mini.jar. > 3. deploy the new apache-solr-1.4-dev.war to tomcat. > Once you make sure the changes to stopwords.* and stoplabels.* have the desired effect on clusters, the above procedure should do the trick. You can also put the modified files in WEB-INF/classes of the WAR, if that's any easier. For your reference, I've updated http://wiki.apache.org/solr/ClusteringComponent to contain a procedure working with the Jetty starter distributed in Solr's examples folder. > class="org.apache.solr.handler.clustering.ClusteringComponent" > name="clustering"> > >default > > name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm >20 >0.150 > name="carrot.lingo.threshold.candidateClusterThreshold">0.775 > Not really related to your issue, but the above file looks a little outdated -- the two parameters:"carrot.lingo.threshold.clusterAssignment" and "carrot.lingo.threshold.candidateClusterThreshold" are not there anymore (but there are many others: http://download.carrot2.org/stable/manual/#section.component.lingo). For most up to date examples, please see http://wiki.apache.org/solr/ClusteringComponent and solrconfig.xml in contrib\clustering\example\conf. Cheers, Staszek
Re: Highlighting... is highlighting too many fields
> I currently have highlighting working, but when I search for > Query: "smith~ category_id:(1 OR 2 OR 3)" > Results: "name: Mr. John Smith, > addresses: 1 Main St, NYC, > NY, 552666" > Why does it show highlights on the addresses, when I > specifically sent in a > query for category_id? When I set > hl.requireFieldMatch and > hl.usePhraseHighlighter to true, I get 0 results > highlighted. Althougth hl.usePhraseHighlighter is about PhraseQuery (and SpanQuery) hl.requireFieldMatch=true should work for your case. When you set hl.requireFieldMatch to true, do you get result returned? But without highlighted? If yes I think your default operator is set to OR. Those results without highlights are coming from category_id:(1 OR 2 OR 3) part of your query. Try "smith~ AND category_id:(1 OR 2 OR 3)" or alternatively you can use filter queries for structured fields (integer, string) like q=smith~&fq=category_id:(1 OR 2 OR 3) Hope this helps.
RE: An issue with using Solr Cell and multiple files
seems to be an error with curl Kevin Miller-17 wrote: > > I am getting the same error message. I am running Solr on a Windows > machine. Is the commit command a curl command or is it a Solr command? > > > Kevin Miller > Web Services > > -Original Message- > From: Grant Ingersoll [mailto:gsing...@apache.org] > Sent: Tuesday, September 08, 2009 12:52 PM > To: solr-user@lucene.apache.org > Subject: Re: An issue with using Solr Cell and multiple files > > solr/examples/exampledocs/post.sh does: > curl $URL --data-binary '' -H 'Content-type:text/xml; > charset=utf-8' > > Not sure if that helps or how it compares to the book. > > On Sep 8, 2009, at 1:48 PM, Kevin Miller wrote: > >> I am using the Solr nightly build from 8/11/2009. I am able to index >> my documents using the Solr Cell but when I attempt to send the commit > >> command I get an error. I am using the example found in the Solr 1.4 >> Enterprise Search Server book (recently released) found on page 84. >> It >> shows to commit the changes as follows (I am showing where my files >> are located not the example in the book): >> c:\curl\bin\curl http://echo12:8983/solr/update/ -H "Content-Type: >> text/xml" --data-binary '' >> >> this give me this error: The system cannot find the file specified. >> >> I get the same error when I modify it to look like the following: >> c:\curl\bin\curl http://echo12:8983/solr/update/ '> waitFlush="false"/>' c:\curl\bin\curl "http://echo12:8983/solr/update/"; -H "Content-Type: >> text/xml" --data-binary '' c:\curl\bin\curl http://echo12:8983/solr/update/ '' c:\curl\bin\curl "http://echo12:8983/solr/update/"; '' >> >> I am using the example configuration in Solr so my documents are found > >> in the exampledocs folder also my curl program in located in the root >> directory which is the reason for the way the curl command is being >> executed. >> >> I would appreciate any information on where to look or how to get the >> commit command to execute after indexing multiple files. >> >> Kevin Miller >> Oklahoma Tax Commission >> Web Services > > -- > Grant Ingersoll > http://www.lucidimagination.com/ > > Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) > using Solr/Lucene: > http://www.lucidimagination.com/search > > > -- View this message in context: http://www.nabble.com/An-issue-with-%3Ccommit-%3E-using-Solr-Cell-and-multiple-files-tp25350995p25352122.html Sent from the Solr - User mailing list archive at Nabble.com.
Passing FuntionQuery string parameters
Hi,
I'm writing a function query to score documents based on Levenshtein
distance from a string. I want my function calls to look like:
lev(myFieldName, 'my string to match')
I'm running into trouble parsing the string I want to match ('my string to
match' above). It looks like all the built in support is for parsing field
names and numeric values. Am I missing the string parsing support, or is it
not there, and if not, why?
Thanks,
Wojtek
--
View this message in context:
http://www.nabble.com/Passing-FuntionQuery-string-parameters-tp25351825p25351825.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR-769 clustering
Hi Staszek, I tried your quick and dirty hack too. It didn't work also. phase like "Carbon Atoms in the Group" with "in" still appear in my clustering labels. What i did is, 1. use "java uf carrot2-mini.jar stoplabels.en" command to replace the stoplabel.en file. 2. apply clustering patch. re-complie the solr with the new carrot2-mini.jar. 3. deploy the new apache-solr-1.4-dev.war to tomcat. I am using the nightly build version of the solr. following is clustering setting in solrconfig.xml , pretty standard: * explicit default true name abstract id true false default org.carrot2.clustering.lingo.LingoClusteringAlgorithm 20 0.150 0.775 *I am wondering is there any extra setting that i need to configure in my solrconfig.xml or schema.xml? or any special parameters that i need to enable in the solrconfig.xml?* thanks -GC * On Tue, Sep 8, 2009 at 11:04 PM, Stanislaw Osinski wrote: > Hi there, > > I try to apply the stoplabels with the instructions that you given in the > > solr clustering Wiki. But it didn't work. > > > > I am runing the patched solr on tomcat. So to enable the stop label. I > add > > "-cp " in to my system's CATALINA_OPTS. > I > > tried to change the file name from stoplabels.txt to stoplabel.en also . > It > > didn't work too. > > > > Then I also find out that in carrot manual page > > ( > > > > > http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-words > > ). > > It suggested to edit the stopwords files inside the carrot2-core.jar. I > > tried this but it didn't work too. > > > > I am not sure what is wrong with my set up. will it be caused by any sort > > of > > caching? > > > > A quick and dirty hack would be to simply replace the corresponding files > (stoplabels.*) in carrot2-mini.jar. > > I know the packaging of the clustering contrib has changed a bit, so let me > see how it currently works and correct the wiki if needed. > > Thanks, > > Staszek >
RE: An issue with using Solr Cell and multiple files
I am getting the same error message. I am running Solr on a Windows machine. Is the commit command a curl command or is it a Solr command? Kevin Miller Web Services -Original Message- From: Grant Ingersoll [mailto:gsing...@apache.org] Sent: Tuesday, September 08, 2009 12:52 PM To: solr-user@lucene.apache.org Subject: Re: An issue with using Solr Cell and multiple files solr/examples/exampledocs/post.sh does: curl $URL --data-binary '' -H 'Content-type:text/xml; charset=utf-8' Not sure if that helps or how it compares to the book. On Sep 8, 2009, at 1:48 PM, Kevin Miller wrote: > I am using the Solr nightly build from 8/11/2009. I am able to index > my documents using the Solr Cell but when I attempt to send the commit > command I get an error. I am using the example found in the Solr 1.4 > Enterprise Search Server book (recently released) found on page 84. > It > shows to commit the changes as follows (I am showing where my files > are located not the example in the book): > >>> c:\curl\bin\curl http://echo12:8983/solr/update/ -H "Content-Type: > text/xml" --data-binary '' > > this give me this error: The system cannot find the file specified. > > I get the same error when I modify it to look like the following: > >>> c:\curl\bin\curl http://echo12:8983/solr/update/ ' waitFlush="false"/>' >>> c:\curl\bin\curl "http://echo12:8983/solr/update/"; -H "Content-Type: > text/xml" --data-binary '' >>> c:\curl\bin\curl http://echo12:8983/solr/update/ '' >>> c:\curl\bin\curl "http://echo12:8983/solr/update/"; '' > > I am using the example configuration in Solr so my documents are found > in the exampledocs folder also my curl program in located in the root > directory which is the reason for the way the curl command is being > executed. > > I would appreciate any information on where to look or how to get the > commit command to execute after indexing multiple files. > > Kevin Miller > Oklahoma Tax Commission > Web Services -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: SOLR-769 clustering
On Tue, Sep 8, 2009 at 9:56 PM, Grant Ingersoll wrote: > > On Sep 8, 2009, at 5:11 AM, Wang Guangchen wrote: > > Hi Staszek, >> >> I try to apply the stoplabels with the instructions that you given in the >> solr clustering Wiki. But it didn't work. >> >> I am runing the patched solr on tomcat. So to enable the stop label. I add >> "-cp " in to my system's CATALINA_OPTS. >> I >> tried to change the file name from stoplabels.txt to stoplabel.en also . >> It >> didn't work too. >> > > > Does it work if you add them to the Solr Home lib directory, which is where > the other clustering files get loaded from? I haven't tried it. > Hi, Thanks for your suggestions, but I put the stoplabels.en file into the solr home lib direcotry , it didn't work also. I tried botht he solr's lib directory and also the "../webapp/solr/WEB-INF/lib/". > > >> Then I also find out that in carrot manual page >> ( >> >> http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-words >> ). >> It suggested to edit the stopwords files inside the carrot2-core.jar. I >> tried this but it didn't work too. >> >> I am not sure what is wrong with my set up. will it be caused by any sort >> of >> caching? Please help. >> Thanks in advance. >> >> -GC >> >> >> On Fri, Apr 24, 2009 at 4:31 PM, Stanislaw Osinski > >wrote: >> >> How would we enable people via SOLR-769 to do this? >>> >>> >>> Good point, Grant! To apply the modified stopwords.* and stoplabels.* >>> files >>> to Solr, simply make them available in the classpath. For the example >>> Solr >>> runner scripts that would be something like: >>> >>> java -cp >>> -Dsolr.solr.home=./clustering/solr -jar start.jar >>> >>> I've documented the whole tuning procedure on the Wiki: >>> >>> http://wiki.apache.org/solr/ClusteringComponent >>> >>> Cheers, >>> >>> S. >>> >>> > -- > Grant Ingersoll > http://www.lucidimagination.com/ > > Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using > Solr/Lucene: > http://www.lucidimagination.com/search > >
Highlighting... is highlighting too many fields
I currently have highlighting working, but when I search for Query: "smith~ category_id:(1 OR 2 OR 3)" Results: "name: Mr. John Smith, addresses: 1 Main St, NYC, NY, 552666" Why does it show highlights on the addresses, when I specifically sent in a query for category_id? When I set hl.requireFieldMatch and hl.usePhraseHighlighter to true, I get 0 results highlighted. Any help greatly appreciated. John
Re: An issue with using Solr Cell and multiple files
solr/examples/exampledocs/post.sh does: curl $URL --data-binary '' -H 'Content-type:text/xml; charset=utf-8' Not sure if that helps or how it compares to the book. On Sep 8, 2009, at 1:48 PM, Kevin Miller wrote: I am using the Solr nightly build from 8/11/2009. I am able to index my documents using the Solr Cell but when I attempt to send the commit command I get an error. I am using the example found in the Solr 1.4 Enterprise Search Server book (recently released) found on page 84. It shows to commit the changes as follows (I am showing where my files are located not the example in the book): c:\curl\bin\curl http://echo12:8983/solr/update/ -H "Content-Type: text/xml" --data-binary '' this give me this error: The system cannot find the file specified. I get the same error when I modify it to look like the following: c:\curl\bin\curl http://echo12:8983/solr/update/ ' waitFlush="false"/>' c:\curl\bin\curl "http://echo12:8983/solr/update/"; -H "Content-Type: text/xml" --data-binary '' c:\curl\bin\curl http://echo12:8983/solr/update/ '' c:\curl\bin\curl "http://echo12:8983/solr/update/"; '' I am using the example configuration in Solr so my documents are found in the exampledocs folder also my curl program in located in the root directory which is the reason for the way the curl command is being executed. I would appreciate any information on where to look or how to get the commit command to execute after indexing multiple files. Kevin Miller Oklahoma Tax Commission Web Services -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
An issue with using Solr Cell and multiple files
I am using the Solr nightly build from 8/11/2009. I am able to index my documents using the Solr Cell but when I attempt to send the commit command I get an error. I am using the example found in the Solr 1.4 Enterprise Search Server book (recently released) found on page 84. It shows to commit the changes as follows (I am showing where my files are located not the example in the book): >> c:\curl\bin\curl http://echo12:8983/solr/update/ -H "Content-Type: text/xml" --data-binary '' this give me this error: The system cannot find the file specified. I get the same error when I modify it to look like the following: >> c:\curl\bin\curl http://echo12:8983/solr/update/ '' >> c:\curl\bin\curl "http://echo12:8983/solr/update/"; -H "Content-Type: text/xml" --data-binary '' >> c:\curl\bin\curl http://echo12:8983/solr/update/ '' >> c:\curl\bin\curl "http://echo12:8983/solr/update/"; '' I am using the example configuration in Solr so my documents are found in the exampledocs folder also my curl program in located in the root directory which is the reason for the way the curl command is being executed. I would appreciate any information on where to look or how to get the commit command to execute after indexing multiple files. Kevin Miller Oklahoma Tax Commission Web Services
Backups using Replication
I'm trying to create data backups using the ReplicationHandler's built in functionality. I've configured my master as http://wiki.apache.org/solr/SolrReplication documented : ... optimize ... but I don't see any backups created on the master. Do I need the snapshooter script available? I did not deploy it on my master, I assumed it was part of the 'old' way of doing replication. If I invoke the backup command over HTTP (http://master_host:port/solr/replication?command=backup) then it seems to work - I get directories like "snapshot.20090908094423". Thanks, Wojtek -- View this message in context: http://www.nabble.com/Backups-using-Replication-tp25350083p25350083.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Whitespace in query
--- On Tue, 9/8/09, Dean Pullen wrote: > From: Dean Pullen > Subject: Whitespace in query > To: "solr-user@lucene.apache.org" > Date: Tuesday, September 8, 2009, 7:10 PM > Hi all, > > I'm trying to filter by a 'document name' which has spaces > in it. > (E.g. "docname:Struts 1" ) You can try q=docname:(Struts 1). What is the field type of docname? docname:Struts 1 is parsed to docname:Struts defaultField:1
Whitespace in query
Hi all, I'm trying to filter by a 'document name' which has spaces in it. (E.g. "docname:Struts 1" ) However, the query parameter throws an exception: [08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - >> "GET /solr/select/?q=((docname:Struts 1))&fl=id&start=0&rows=1 HTTP/1.1[\r][\n]"[08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - >> "User-Agent: Jakarta Commons-HttpClient/3.0.1[\r][\n]"[08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - >> "Host: localhost:8080[\r][\n]"[08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - >> "[\r][\n]" [08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - << "HTTP/1.1 505 HTTP Version Not Supported[\r][\n]" [08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - << "Server: Apache-Coyote/1.1[\r][\n]" [08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - << "Date: Tue, 08 Sep 2009 16:01:36 GMT[\r][\n]" [08 Sep 2009 17:01:36] DEBUG httpclient.wire.header - << "Connection: close[\r][\n]" [08 Sep 2009 17:01:36] ERROR com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor - Method failed: HTTP/1.1 505 HTTP Version Not Supported com.msp.solr.SolrException: Method failed: HTTP/1.1 505 HTTP Version NotSupported Even when I URLEncode the value, e.g. to docname:Struts+1 it doesn't work. How do I go about correcting this? Regards, Dean.
Re: Geographic clustering
there are clustering libraries like http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/cluster/, that have bindings to perl/python, you can preprocess your results and create clusters for each zoom level On Tue, Sep 8, 2009 at 8:08 AM, gwk wrote: > Hi, > > I just completed a simple proof-of-concept clusterer component which > naively clusters with a specified bounding box around each position, > similar to what the javascript MarkerClusterer does. It's currently very > slow as I loop over the entire docset and request the longitude and > latitude of each document (Not to mention that my unfamiliarity with > Lucene/Solr isn't helping the implementations performance any, most code > is copied from grep-ing the solr source). Clustering a set of about > 80.000 documents takes about 5-6 seconds. I'm currently looking into > storing the hilber curve mapping in Solr and clustering using facet > counts on numerical ranges of that mapping but I'm not sure it will pan out. > > Regards, > > gwk > > Grant Ingersoll wrote: >> >> Not directly related to geo clustering, but >> http://issues.apache.org/jira/browse/SOLR-769 is all about a pluggable >> interface to clustering implementations. It currently has Carrot2 >> implemented, but the APIs are marked as experimental. I would definitely be >> interested in hearing your experience with implementing your clustering >> algorithm in it. >> >> -Grant >> >> On Sep 8, 2009, at 4:00 AM, gwk wrote: >> >>> Hi, >>> >>> I'm working on a search-on-map interface for our website. I've created a >>> little proof of concept which uses the MarkerClusterer >>> (http://code.google.com/p/gmaps-utility-library-dev/) which clusters the >>> markers nicely. But because sending tens of thousands of markers over Ajax >>> is not quite as fast as I would like it to be, I'd prefer to do the >>> clustering on the server side. I've considered a few options like storing >>> the morton-order and throwing away precision to cluster, assigning all >>> locations to a grid position. Or simply cluster based on country/region/city >>> depending on zoom level by adding latitude on longitude fields for each zoom >>> level (so that for smaller countries you have to be zoomed in further to get >>> the next level of clustering). >>> >>> I was wondering if anybody else has worked on something similar and if so >>> what their solutions are. >>> >>> Regards, >>> >>> gwk >> >> -- >> Grant Ingersoll >> http://www.lucidimagination.com/ >> >> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using >> Solr/Lucene: >> http://www.lucidimagination.com/search >> > > >
Re: date field
Thanks for the answer. However we don't have strong performance issue (for now) and it that case, how do you face query where time part is missing ? On Tue, Sep 8, 2009 at 17:44, Silent Surfer wrote: > Hi, > > If you are still not went live already, I would suggest to use the long > instead of date field. According to our testing, search based on date fields > are very slow when compared to search based on long field. > > You can use System.getTimeInMillis() to get the time > When showing it to the user, apply a date formatter. > > When taking input from user, let him enter whatever the date he wants to > and then you can convert to "long" and do your searches based on it. > > Experts can pitch in with any other ideas.. > > Thanks, > sS >
Re: date field
Hi, If you are still not went live already, I would suggest to use the long instead of date field. According to our testing, search based on date fields are very slow when compared to search based on long field. You can use System.getTimeInMillis() to get the time When showing it to the user, apply a date formatter. When taking input from user, let him enter whatever the date he wants to and then you can convert to "long" and do your searches based on it. Experts can pitch in with any other ideas.. Thanks, sS --- On Tue, 9/8/09, Gérard Dupont wrote: > From: Gérard Dupont > Subject: date field > To: solr-user@lucene.apache.org > Cc: "Nicolas Bureau" > Date: Tuesday, September 8, 2009, 8:51 AM > Hi all, > > I'm currently facing a little difficulty to index and > search on date field. > The indexing is done in the right way (I guess) and I can > find valid date in > the field like "2009-05-01T12:45:32Z". However when I'm > searching the user > don't always give an exact date. for instance they give > "2008-05-01" to get > all documents related to that day. I can do a trick > using wildcard but is > there another way to do it ? Moreover if they give the full > date string (or > if I hack the query parser) I can have the full syntax, but > then the ":" > annoy me because the Lucene parser does not allow it > without quotes. Any > ideas ? > > -- > Gérard Dupont > Information Processing Control and Cognition (IPCC) - EADS > DS > http://weblab.forge.ow2.org > > Document & Learning team - LITIS Laboratory >
Re: SOLR-769 clustering
Hi there, I try to apply the stoplabels with the instructions that you given in the > solr clustering Wiki. But it didn't work. > > I am runing the patched solr on tomcat. So to enable the stop label. I add > "-cp " in to my system's CATALINA_OPTS. I > tried to change the file name from stoplabels.txt to stoplabel.en also . It > didn't work too. > > Then I also find out that in carrot manual page > ( > > http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-words > ). > It suggested to edit the stopwords files inside the carrot2-core.jar. I > tried this but it didn't work too. > > I am not sure what is wrong with my set up. will it be caused by any sort > of > caching? > A quick and dirty hack would be to simply replace the corresponding files (stoplabels.*) in carrot2-mini.jar. I know the packaging of the clustering contrib has changed a bit, so let me see how it currently works and correct the wiki if needed. Thanks, Staszek
Re: Geographic clustering
Hi, I just completed a simple proof-of-concept clusterer component which naively clusters with a specified bounding box around each position, similar to what the javascript MarkerClusterer does. It's currently very slow as I loop over the entire docset and request the longitude and latitude of each document (Not to mention that my unfamiliarity with Lucene/Solr isn't helping the implementations performance any, most code is copied from grep-ing the solr source). Clustering a set of about 80.000 documents takes about 5-6 seconds. I'm currently looking into storing the hilber curve mapping in Solr and clustering using facet counts on numerical ranges of that mapping but I'm not sure it will pan out. Regards, gwk Grant Ingersoll wrote: Not directly related to geo clustering, but http://issues.apache.org/jira/browse/SOLR-769 is all about a pluggable interface to clustering implementations. It currently has Carrot2 implemented, but the APIs are marked as experimental. I would definitely be interested in hearing your experience with implementing your clustering algorithm in it. -Grant On Sep 8, 2009, at 4:00 AM, gwk wrote: Hi, I'm working on a search-on-map interface for our website. I've created a little proof of concept which uses the MarkerClusterer (http://code.google.com/p/gmaps-utility-library-dev/) which clusters the markers nicely. But because sending tens of thousands of markers over Ajax is not quite as fast as I would like it to be, I'd prefer to do the clustering on the server side. I've considered a few options like storing the morton-order and throwing away precision to cluster, assigning all locations to a grid position. Or simply cluster based on country/region/city depending on zoom level by adding latitude on longitude fields for each zoom level (so that for smaller countries you have to be zoomed in further to get the next level of clustering). I was wondering if anybody else has worked on something similar and if so what their solutions are. Regards, gwk -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
getTextContent() caused my local solr build failed
All the follwing ".getTextContent()" failed , which caused my local solr build
failed, althogh it is just sync-ed with the SVN build.
list.add(nodeList.item(i).getTextContent());
assertEquals("prefix-proptwo-suffix", nl.item(0).getTextContent());
Node node = solrConfig.getNode("propTest", true);
assertEquals("prefix-proptwo-suffix", node.getTextContent());
Any hints on how to solve it are highly appreciated.
Thanks.
2009-09-08
Yue ZHANG
- Beijing
Re: LocalParams for faceting in nightly
The following works on the example data for me:
http://localhost:8983/solr/select?q=*:*&fq={!tag=foo}cat:memory&facet=true&facet.field={!ex=foo}cat
What's the line in the solr log after the exception (it should contain
the parameters solr thinks it got)?
-Yonik
http://www.lucidimagination.com
On Tue, Sep 8, 2009 at 9:58 AM, gareth
rushgrove wrote:
> Mmmm
>
> 2009/9/8 Yonik Seeley :
>> This should work, and there are unit tests. Make sure the server you
>> are talking to is a recent release of 1.4, and then get the full stack
>> trace from the solr server.
>>
>
> I'm running the release with the following in the CHANGES.txt file.
>
> $Id: CHANGES.txt 812246 2009-09-07 18:28:16Z yonik $
>
> The full exception from the response is:
>
> exception":"org.apache.solr.common.SolrException: undefined field
> {!ex=colour}material\n\tat
> org.apache.solr.schema.IndexSchema.getField(IndexSchema.java:994)\n\tat
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:152)\n\tat
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:182)\n\tat
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:96)\n\tat
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:70)\n\tat
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169)\n\tat
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)\n\tat
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)\n\tat
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)\n\tat
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)\n\tat
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)\n\tat
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)\n\tat
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)\n\tat
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)\n\tat
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)\n\tat
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)\n\tat
> org.mortbay.jetty.Server.handle(Server.java:285)\n\tat
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)\n\tat
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)\n\tat
> org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)\n\tat
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)\n\tat
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)\n\tat
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)\n\tat
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)\n"}
>
> I'm going to try with a brand new index build from scratch with a simple
> schema
>
> Thanks
>
> Gareth
>
>> -Yonik
>> http://www.lucidimagination.com
>>
>>
>>
>> On Tue, Sep 8, 2009 at 9:08 AM, gareth
>> rushgrove wrote:
>>> Thanks for the quick replies guys. Afraid that didn't work.
>>>
>>> 2009/9/8 gwk :
Try removing the space between de closing bracket } and the field name, I
think that should work.
>>>
>>> I tried:
>>>
>>> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}material&sort=popularity%20desc
>>>
>>> I'd actually tried this first and added the space to see if that was
>>> needed, then missed it when I pasted in the URL from the browser.
>>>
>>> Looking at the docs I thought this would be the correct syntax, but I
>>> still get the same exception:
>>>
>>> "exception":"org.apache.solr.common.SolrException: undefined field
>>> {!ex=colour}material\n\tat
>>>
>>> Thanks
>>>
>>> Gareth
>>>
Regards,
gwk
gareth rushgrove wrote:
>
> Hi All
>
> Hoping someone might be able to help me with a problem.
>
> I downloaded and got up and running with the latest nightly release of
> Solr:
> http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
>
> In order to try out the tagging and excluding filters which have a
> note saying they are only available in 1.4.
>
>
> http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
>
> I have a working index that I can query against, for instance the
> following returns what I would expect:
>
>
> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt
Re: LocalParams for faceting in nightly
Mmmm
2009/9/8 Yonik Seeley :
> This should work, and there are unit tests. Make sure the server you
> are talking to is a recent release of 1.4, and then get the full stack
> trace from the solr server.
>
I'm running the release with the following in the CHANGES.txt file.
$Id: CHANGES.txt 812246 2009-09-07 18:28:16Z yonik $
The full exception from the response is:
exception":"org.apache.solr.common.SolrException: undefined field
{!ex=colour}material\n\tat
org.apache.solr.schema.IndexSchema.getField(IndexSchema.java:994)\n\tat
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:152)\n\tat
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:182)\n\tat
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:96)\n\tat
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:70)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)\n\tat
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)\n\tat
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)\n\tat
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)\n\tat
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)\n\tat
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)\n\tat
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)\n\tat
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)\n\tat
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)\n\tat
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)\n\tat
org.mortbay.jetty.Server.handle(Server.java:285)\n\tat
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)\n\tat
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)\n\tat
org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)\n\tat
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)\n\tat
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)\n\tat
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)\n\tat
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)\n"}
I'm going to try with a brand new index build from scratch with a simple schema
Thanks
Gareth
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Tue, Sep 8, 2009 at 9:08 AM, gareth
> rushgrove wrote:
>> Thanks for the quick replies guys. Afraid that didn't work.
>>
>> 2009/9/8 gwk :
>>>
>>> Try removing the space between de closing bracket } and the field name, I
>>> think that should work.
>>>
>>
>> I tried:
>>
>> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}material&sort=popularity%20desc
>>
>> I'd actually tried this first and added the space to see if that was
>> needed, then missed it when I pasted in the URL from the browser.
>>
>> Looking at the docs I thought this would be the correct syntax, but I
>> still get the same exception:
>>
>> "exception":"org.apache.solr.common.SolrException: undefined field
>> {!ex=colour}material\n\tat
>>
>> Thanks
>>
>> Gareth
>>
>>> Regards,
>>>
>>> gwk
>>>
>>>
>>> gareth rushgrove wrote:
Hi All
Hoping someone might be able to help me with a problem.
I downloaded and got up and running with the latest nightly release of
Solr:
http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
In order to try out the tagging and excluding filters which have a
note saying they are only available in 1.4.
http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
I have a working index that I can query against, for instance the
following returns what I would expect:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field=material&sort=popularity%20desc
However, once I add the {!ex part it throws an exception:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}%20material&sort=popularity%20desc
specifically "exception":"org.apache.solr.common.SolrException:
undefined field
Re: fl query option limitations
On Sep 8, 2009, at 5:10 AM, Marcin Kuptel wrote: Hi, How can I make this kind of query work: ...&fl=Output Channels where "Output Channels" is the name of a field? Escaping the whitespace in the field's name does not seem to work. I don't think whitespace in a field name is ever recommended. While Lucene can handle it and Solr might, it is just easier to not do it. That being said, you might try URL Encoding it as Output%20Channels or whatever the appropriate syntax is. -Grant -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: SOLR-769 clustering
On Sep 8, 2009, at 5:11 AM, Wang Guangchen wrote: Hi Staszek, I try to apply the stoplabels with the instructions that you given in the solr clustering Wiki. But it didn't work. I am runing the patched solr on tomcat. So to enable the stop label. I add "-cp " in to my system's CATALINA_OPTS. I tried to change the file name from stoplabels.txt to stoplabel.en also . It didn't work too. Does it work if you add them to the Solr Home lib directory, which is where the other clustering files get loaded from? I haven't tried it. Then I also find out that in carrot manual page ( http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-words ). It suggested to edit the stopwords files inside the carrot2- core.jar. I tried this but it didn't work too. I am not sure what is wrong with my set up. will it be caused by any sort of caching? Please help. Thanks in advance. -GC On Fri, Apr 24, 2009 at 4:31 PM, Stanislaw Osinski wrote: How would we enable people via SOLR-769 to do this? Good point, Grant! To apply the modified stopwords.* and stoplabels.* files to Solr, simply make them available in the classpath. For the example Solr runner scripts that would be something like: java -cp -Dsolr.solr.home=./clustering/solr -jar start.jar I've documented the whole tuning procedure on the Wiki: http://wiki.apache.org/solr/ClusteringComponent Cheers, S. -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: Geographic clustering
Not directly related to geo clustering, but http://issues.apache.org/jira/browse/SOLR-769 is all about a pluggable interface to clustering implementations. It currently has Carrot2 implemented, but the APIs are marked as experimental. I would definitely be interested in hearing your experience with implementing your clustering algorithm in it. -Grant On Sep 8, 2009, at 4:00 AM, gwk wrote: Hi, I'm working on a search-on-map interface for our website. I've created a little proof of concept which uses the MarkerClusterer (http://code.google.com/p/gmaps-utility-library-dev/ ) which clusters the markers nicely. But because sending tens of thousands of markers over Ajax is not quite as fast as I would like it to be, I'd prefer to do the clustering on the server side. I've considered a few options like storing the morton-order and throwing away precision to cluster, assigning all locations to a grid position. Or simply cluster based on country/region/city depending on zoom level by adding latitude on longitude fields for each zoom level (so that for smaller countries you have to be zoomed in further to get the next level of clustering). I was wondering if anybody else has worked on something similar and if so what their solutions are. Regards, gwk -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: LocalParams for faceting in nightly
This should work, and there are unit tests. Make sure the server you
are talking to is a recent release of 1.4, and then get the full stack
trace from the solr server.
-Yonik
http://www.lucidimagination.com
On Tue, Sep 8, 2009 at 9:08 AM, gareth
rushgrove wrote:
> Thanks for the quick replies guys. Afraid that didn't work.
>
> 2009/9/8 gwk :
>>
>> Try removing the space between de closing bracket } and the field name, I
>> think that should work.
>>
>
> I tried:
>
> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}material&sort=popularity%20desc
>
> I'd actually tried this first and added the space to see if that was
> needed, then missed it when I pasted in the URL from the browser.
>
> Looking at the docs I thought this would be the correct syntax, but I
> still get the same exception:
>
> "exception":"org.apache.solr.common.SolrException: undefined field
> {!ex=colour}material\n\tat
>
> Thanks
>
> Gareth
>
>> Regards,
>>
>> gwk
>>
>>
>> gareth rushgrove wrote:
>>>
>>> Hi All
>>>
>>> Hoping someone might be able to help me with a problem.
>>>
>>> I downloaded and got up and running with the latest nightly release of
>>> Solr:
>>> http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
>>>
>>> In order to try out the tagging and excluding filters which have a
>>> note saying they are only available in 1.4.
>>>
>>>
>>> http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
>>>
>>> I have a working index that I can query against, for instance the
>>> following returns what I would expect:
>>>
>>>
>>> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field=material&sort=popularity%20desc
>>>
>>> However, once I add the {!ex part it throws an exception:
>>>
>>>
>>> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}%20material&sort=popularity%20desc
>>>
>>> specifically "exception":"org.apache.solr.common.SolrException:
>>> undefined field {!ex=colour} material\n\tat
>>>
>>> The schema I'm using was copied from a working solr 1.3 install and as
>>> mentioned works great with 1.4, except for this issue I'm having
>>>
>>> So:
>>>
>>> * Do I have to enable this feature somewhere?
>>> * Is the feature working in the latest release?
>>> * Is my syntax correct?
>>> * Do you have to define the tag name somewhere other than in the query?
>>>
>>> Any help much appreciated.
>>>
>>> Thanks
>>>
>>> Gareth
>>>
>>>
>>
>>
>
>
>
> --
> Gareth Rushgrove
>
> Web Geek
> Member WaSP Education Task Force
>
> morethanseven.net
> garethrushgrove.com
>
Re: LocalParams for faceting in nightly
Thanks for the quick replies guys. Afraid that didn't work.
2009/9/8 gwk :
>
> Try removing the space between de closing bracket } and the field name, I
> think that should work.
>
I tried:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}material&sort=popularity%20desc
I'd actually tried this first and added the space to see if that was
needed, then missed it when I pasted in the URL from the browser.
Looking at the docs I thought this would be the correct syntax, but I
still get the same exception:
"exception":"org.apache.solr.common.SolrException: undefined field
{!ex=colour}material\n\tat
Thanks
Gareth
> Regards,
>
> gwk
>
>
> gareth rushgrove wrote:
>>
>> Hi All
>>
>> Hoping someone might be able to help me with a problem.
>>
>> I downloaded and got up and running with the latest nightly release of
>> Solr:
>> http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
>>
>> In order to try out the tagging and excluding filters which have a
>> note saying they are only available in 1.4.
>>
>>
>> http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
>>
>> I have a working index that I can query against, for instance the
>> following returns what I would expect:
>>
>>
>> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field=material&sort=popularity%20desc
>>
>> However, once I add the {!ex part it throws an exception:
>>
>>
>> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}%20material&sort=popularity%20desc
>>
>> specifically "exception":"org.apache.solr.common.SolrException:
>> undefined field {!ex=colour} material\n\tat
>>
>> The schema I'm using was copied from a working solr 1.3 install and as
>> mentioned works great with 1.4, except for this issue I'm having
>>
>> So:
>>
>> * Do I have to enable this feature somewhere?
>> * Is the feature working in the latest release?
>> * Is my syntax correct?
>> * Do you have to define the tag name somewhere other than in the query?
>>
>> Any help much appreciated.
>>
>> Thanks
>>
>> Gareth
>>
>>
>
>
--
Gareth Rushgrove
Web Geek
Member WaSP Education Task Force
morethanseven.net
garethrushgrove.com
Re: LocalParams for faceting in nightly
Hi Gareth,
Try removing the space between de closing bracket } and the field name,
I think that should work.
Regards,
gwk
gareth rushgrove wrote:
Hi All
Hoping someone might be able to help me with a problem.
I downloaded and got up and running with the latest nightly release of Solr:
http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
In order to try out the tagging and excluding filters which have a
note saying they are only available in 1.4.
http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
I have a working index that I can query against, for instance the
following returns what I would expect:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field=material&sort=popularity%20desc
However, once I add the {!ex part it throws an exception:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}%20material&sort=popularity%20desc
specifically "exception":"org.apache.solr.common.SolrException:
undefined field {!ex=colour} material\n\tat
The schema I'm using was copied from a working solr 1.3 install and as
mentioned works great with 1.4, except for this issue I'm having
So:
* Do I have to enable this feature somewhere?
* Is the feature working in the latest release?
* Is my syntax correct?
* Do you have to define the tag name somewhere other than in the query?
Any help much appreciated.
Thanks
Gareth
Re: LocalParams for faceting in nightly
Try changing
{!ex=colour}%20material
to
{!ex=colour}material
local params syntax is a prefix to a parameter value - whitespace
might be significant so it is not removed.
-Yonik
http://www.lucidimagination.com
On Tue, Sep 8, 2009 at 8:53 AM, gareth
rushgrove wrote:
> Hi All
>
> Hoping someone might be able to help me with a problem.
>
> I downloaded and got up and running with the latest nightly release of Solr:
> http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
>
> In order to try out the tagging and excluding filters which have a
> note saying they are only available in 1.4.
>
> http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
>
> I have a working index that I can query against, for instance the
> following returns what I would expect:
>
> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field=material&sort=popularity%20desc
>
> However, once I add the {!ex part it throws an exception:
>
> http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}%20material&sort=popularity%20desc
>
> specifically "exception":"org.apache.solr.common.SolrException:
> undefined field {!ex=colour} material\n\tat
>
> The schema I'm using was copied from a working solr 1.3 install and as
> mentioned works great with 1.4, except for this issue I'm having
>
> So:
>
> * Do I have to enable this feature somewhere?
> * Is the feature working in the latest release?
> * Is my syntax correct?
> * Do you have to define the tag name somewhere other than in the query?
>
> Any help much appreciated.
>
> Thanks
>
> Gareth
>
> --
> Gareth Rushgrove
>
> Web Geek
> Member WaSP Education Task Force
>
> morethanseven.net
> garethrushgrove.com
>
LocalParams for faceting in nightly
Hi All
Hoping someone might be able to help me with a problem.
I downloaded and got up and running with the latest nightly release of Solr:
http://people.apache.org/builds/lucene/solr/nightly/solr-2009-09-08.zip
In order to try out the tagging and excluding filters which have a
note saying they are only available in 1.4.
http://wiki.apache.org/solr/SimpleFacetParameters#head-4ba81c89b265c3b5992e3292718a0d100f7251ef
I have a working index that I can query against, for instance the
following returns what I would expect:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=cl}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field=material&sort=popularity%20desc
However, once I add the {!ex part it throws an exception:
http://172.16.142.130:8983/solr/products/select/?q=material:metal&fq={!tag=colour}colour:Red&start=24&rows=25&indent=on&wt=json&facet=on&facet.sort=false&facet.field=colour&facet.field={!ex=colour}%20material&sort=popularity%20desc
specifically "exception":"org.apache.solr.common.SolrException:
undefined field {!ex=colour} material\n\tat
The schema I'm using was copied from a working solr 1.3 install and as
mentioned works great with 1.4, except for this issue I'm having
So:
* Do I have to enable this feature somewhere?
* Is the feature working in the latest release?
* Is my syntax correct?
* Do you have to define the tag name somewhere other than in the query?
Any help much appreciated.
Thanks
Gareth
--
Gareth Rushgrove
Web Geek
Member WaSP Education Task Force
morethanseven.net
garethrushgrove.com
Lucid's Solr Showcase, MLT?
Hi all, I'm currently trying to build something similar to Lucid imaginations's Solr showcase. There you can select a single message and get all the related messages from the same thread as a menu on the left. How is that done? Is that menu using Solr's More Like This feature? How is the initially requested message remembered and highlighted accordingly? http://www.lucidimagination.com/search/document/5225d37bc6671a12/uncommitted_documents_coutn_with_autocommit_enabled Currently I've indexed a couple of newsgroups and am storing a message's ID (like >) in a field called "messageId". termVectors="true" /> Then I also store all references, which are also message IDs in a field "references". (multiple space separated messageIds) positionIncrementGap="100"> termVectors="true" /> Now I'm trying to get the currently selected message through Solr using the accordant document's ID. When trying to use the messageId and references fields in MLT I can't get it to return the expected documents related to the same thread. In fact there're no documents returned at all, except for the initial message. To me it seems that Solr is unable to determine interestingTerms as they're empty. Here're the query parameters I'm using: qt=mlt q=id:0a904000c5dc7a72fa45ed0063b4389c/typo3.teams.core/0/5799/ nntpArticle mlt.fl=messageId,references mlt.match.include=true mlt.interestingTerms=details fl=*,score debugQuery=true I'm running a Solr 1.4 nightly in case that's of interest. Any insights would be interesting and highly appreciated. all the best Ingo -- Ingo Renner TYPO3 Core Developer, Release Manager TYPO3 4.2
Re: DIH applying variosu transformers to a field
On Tue, Sep 8, 2009 at 2:40 PM, Marc Sturlese wrote: > > I'm wondering if I write a second transformer and put it in > data-config.xml > after CustomTransformer1. Will the input value of the row in the second > transformer be the result of the transformed row in the CustomTransfomer1 > or > will be the original row value? > I would just need to index the result of transformer2 (whose input would be > the output of transformer1) > > config woul look like: > transformer="CustomTransformer1,CustomTransformer2" pk="id" > > Correct. Transformers in DIH are chained, so output of the first is the input of the second. > I have read something similar was done in february ( > https://issues.apache.org/jira/browse/SOLR-1033 ) but not sure if it's > what > I ask for > That was a different issue. That issue enabled us to use current entity's values as variables. Previously, only the parent entity's fields (and things like request params, evaluators etc.) were available as variables. -- Regards, Shalin Shekhar Mangar.
Re: SOLR-769 clustering
Hi Staszek, I try to apply the stoplabels with the instructions that you given in the solr clustering Wiki. But it didn't work. I am runing the patched solr on tomcat. So to enable the stop label. I add "-cp " in to my system's CATALINA_OPTS. I tried to change the file name from stoplabels.txt to stoplabel.en also . It didn't work too. Then I also find out that in carrot manual page ( http://download.carrot2.org/head/manual/#section.advanced-topics.fine-tuning.stop-words). It suggested to edit the stopwords files inside the carrot2-core.jar. I tried this but it didn't work too. I am not sure what is wrong with my set up. will it be caused by any sort of caching? Please help. Thanks in advance. -GC On Fri, Apr 24, 2009 at 4:31 PM, Stanislaw Osinski wrote: > > > > How would we enable people via SOLR-769 to do this? > > > Good point, Grant! To apply the modified stopwords.* and stoplabels.* files > to Solr, simply make them available in the classpath. For the example Solr > runner scripts that would be something like: > > java -cp > -Dsolr.solr.home=./clustering/solr -jar start.jar > > I've documented the whole tuning procedure on the Wiki: > > http://wiki.apache.org/solr/ClusteringComponent > > Cheers, > > S. >
fl query option limitations
Hi, How can I make this kind of query work: ...&fl=Output Channels where "Output Channels" is the name of a field? Escaping the whitespace in the field's name does not seem to work. Regards, Marcin Kuptel
DIH applying variosu transformers to a field
Hey there, I am using DIH to import a db table and and have writed a custom
transformer following the example:
package foo;
public class CustomTransformer1{
public Object transformRow(Map row) {
String artist = row.get("artist");
if (artist != null)
row.put("ar", artist.trim());
return row;
}
}
I'm wondering if I write a second transformer and put it in data-config.xml
after CustomTransformer1. Will the input value of the row in the second
transformer be the result of the transformed row in the CustomTransfomer1 or
will be the original row value?
I would just need to index the result of transformer2 (whose input would be
the output of transformer1)
config woul look like:
https://issues.apache.org/jira/browse/SOLR-1033 ) but not sure if it's what
I ask for
Thanks in advance
--
View this message in context:
http://www.nabble.com/DIH-applying-variosu-transformers-to-a-field-tp25342449p25342449.html
Sent from the Solr - User mailing list archive at Nabble.com.
date field
Hi all, I'm currently facing a little difficulty to index and search on date field. The indexing is done in the right way (I guess) and I can find valid date in the field like "2009-05-01T12:45:32Z". However when I'm searching the user don't always give an exact date. for instance they give "2008-05-01" to get all documents related to that day. I can do a trick using wildcard but is there another way to do it ? Moreover if they give the full date string (or if I hack the query parser) I can have the full syntax, but then the ":" annoy me because the Lucene parser does not allow it without quotes. Any ideas ? -- Gérard Dupont Information Processing Control and Cognition (IPCC) - EADS DS http://weblab.forge.ow2.org Document & Learning team - LITIS Laboratory
Geographic clustering
Hi, I'm working on a search-on-map interface for our website. I've created a little proof of concept which uses the MarkerClusterer (http://code.google.com/p/gmaps-utility-library-dev/) which clusters the markers nicely. But because sending tens of thousands of markers over Ajax is not quite as fast as I would like it to be, I'd prefer to do the clustering on the server side. I've considered a few options like storing the morton-order and throwing away precision to cluster, assigning all locations to a grid position. Or simply cluster based on country/region/city depending on zoom level by adding latitude on longitude fields for each zoom level (so that for smaller countries you have to be zoomed in further to get the next level of clustering). I was wondering if anybody else has worked on something similar and if so what their solutions are. Regards, gwk
