Re: Batch Update Fields

2010-12-03 Thread Markus Jelsma
You must reindex the complete document, even if you just want to update a single field. On Friday 03 December 2010 04:52:04 Adam Estrada wrote: OK part 2 of my previous question... Is there a way to batch update field values based on a certain criteria? For example, if thousands of

Re: Limit number of characters returned

2010-12-03 Thread Ahmet Arslan
--- On Fri, 12/3/10, Mark static.void@gmail.com wrote: From: Mark static.void@gmail.com Subject: Limit number of characters returned To: solr-user@lucene.apache.org Date: Friday, December 3, 2010, 5:39 AM Is there way to limit the number of characters returned from a stored field?

Re: [Wildcard query] Weird behaviour

2010-12-03 Thread Robert Muir
On Fri, Dec 3, 2010 at 6:28 AM, Tanguy Moal tanguy.m...@gmail.com wrote: However suddenly CPU usage simply doubles, and sometimes eventually start using all 16 cores of the server, whereas the number of handled request is pretty stable, and even starts decreasing because of degraded user

Re: [Wildcard query] Weird behaviour

2010-12-03 Thread Robert Muir
Actually, i took a look at the code again, the queries you mentioned: I send queries to that field in the form (*term1*term2*) I think the patch will not fix your problem... The only way i know you can fix this would be to upgrade to lucene/solr trunk, where wildcard comparison is linear to the

Re: [Wildcard query] Weird behaviour

2010-12-03 Thread Tanguy Moal
Thank you very much Robert for replying that fast and accurately. I have effectively an other idea in mind to provide similar suggestions less expansively, I was balancing between the work around and the report issue options. I don't regret it since you came with a possible fix. I'll give it a

Re: Solr Multi-thread Update Transaction Control

2010-12-03 Thread Erick Erickson
From Solr's perspective, the fact that multiple threads are sending data to be indexed is invisible, Solr is just reading http requests. So I don't think what you're asking for is possible. Could you outline the reason you want to do this? Perhaps there's another way to accomplish it. Best Erick

Re: Joining Fields in and Index

2010-12-03 Thread Jan Høydahl / Cominvent
Hi, I made a MappingUpdateRequestHandler which lets you map country codes to full country names with a config file. See https://issues.apache.org/jira/browse/SOLR-2151 -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 3. des. 2010, at 00.03, Adam Estrada wrote:

Facet same field with different preifx

2010-12-03 Thread Eric Grobler
Hi Everyone, Can I facet the same field twice with a different prefix as per example below? facet.field=myfield f.myfield.facet.prefix=*make* f.myfield.facet.sort=count facet.field=myfield f.myfield.facet.prefix=*model* f.myfield.facet.sort=count Thanks and Regards Ericz

Re: [Wildcard query] Weird behaviour

2010-12-03 Thread Robert Muir
On Fri, Dec 3, 2010 at 7:49 AM, Tanguy Moal tanguy.m...@gmail.com wrote: Thank you very much Robert for replying that fast and accurately. I have effectively an other idea in mind to provide similar suggestions less expansively, I was balancing between the work around and the report issue

Re: Batch Update Fields

2010-12-03 Thread Erick Erickson
No, there's no equivalent to SQL update for all values in a column. You'll have to reindex all the documents. On Thu, Dec 2, 2010 at 10:52 PM, Adam Estrada estrada.adam.gro...@gmail.com wrote: OK part 2 of my previous question... Is there a way to batch update field values based on a certain

Problem with dismax mm

2010-12-03 Thread Em
Hi list, I got a little problem with my mm definition: 2-1 450% 566% Here is what it *should* mean: If there are 2 clauses, at least one has to match. If there are more than 2 clauses, at least 50% should match (both rules seem to mean the same, don't they?). And if there are 5 or more than

Re: Problem with dismax mm

2010-12-03 Thread Shawn Heisey
On 12/3/2010 6:18 AM, Em wrote: I got a little problem with my mm definition: 2-1 450% 566% Are you defining this in a request handler in solrconfig.xml? If you have it entered just like that, I think it may not be understanding it. You need to encode the character. Here's an excerpt

Re: Problem with dismax mm

2010-12-03 Thread Erick Erickson
from: http://wiki.apache.org/solr/DisMaxQParserPlugin#mm_.28Minimum_.27Should.27_Match.29 If there are less than 3 optional clauses, they all must match; for 3 to 5 clauses, one less than the number of clauses must match, for 6 or more clauses, 80% must match, rounded down: 2-1 580% Personally,

Re: Problem with dismax mm

2010-12-03 Thread Em
Thank you both! Erick, what you said was absolutely correct. I missunderstood the definition completely. Now it works as intended. Thank you! Kind regards Erick Erickson wrote: from: http://wiki.apache.org/solr/DisMaxQParserPlugin#mm_.28Minimum_.27Should.27_Match.29 If there are less

Re: Limit number of characters returned

2010-12-03 Thread Mark
Correct me if I am wrong but I would like to return highlighted excerpts from the document so I would still need to index and store the whole document right (ie.. highlighting only works on stored fields)? On 12/3/10 3:51 AM, Ahmet Arslan wrote: --- On Fri, 12/3/10,

finding exact case insensitive matches on single and multiword values

2010-12-03 Thread PeterKerk
Users call this URL on my site: /?search=1city=den+haag or even /?search=1city=Den+Haag (casing of ctyname can be anything) Under water I call Solr: http://localhost:8983/solr/db/select/?indent=onfacet=truefq=city:den+haagq=*:*start=0rows=25fl=id,title,friendlyurl,cityfacet.field=city but

Negative fl param

2010-12-03 Thread Mark
When returning results is there a way I can say to return all fields except a certain one? So say I have stored fields foo, bar and baz but I only want to return foo and bar. Is it possible to do this without specifically listing out the fields I do want?

Re: Limit number of characters returned

2010-12-03 Thread Erick Erickson
Yep, you're correct. CopyField is probably your simplest option here as Ahmet suggested. A more complex solution would be your own response writer, but unless and until you index gets cumbersome, I'd avoid that. Plus, storing the copied contents only shouldn't impact search much, since this

Re: Limit number of characters returned

2010-12-03 Thread Mark
Thanks for the response. Couldn't I just use the highlighter and configure it to use the alternative field to return the first 200 characters? In cases where there is a highlighter match I would prefer to show the excerpts anyway.

Re: finding exact case insensitive matches on single and multiword values

2010-12-03 Thread Erick Erickson
The root of your problem, I think, is fq=city:den+haag which parses into city:den +defaultfield:haag Try parens, i.e. city:(den haag). Attaching debugQuery=on is often a way to see thing like this quickly Also, if you haven't seen the analysis page from the admin page, it's really valuable

Re: finding exact case insensitive matches on single and multiword values

2010-12-03 Thread PeterKerk
You are right, this is what I see when I append the debug query (very very useful btw!!!) in old situation: arr name=parsed_filter_queries strcity:den title:haag/str strPhraseQuery(themes:hotel en restaur)/str /arr I then changed the schema.xml to: fieldType name=myField

Re: spellchecker results not as desired

2010-12-03 Thread abhayd
Thanks, I was able to fix this issue with combination of EdgeNGrams and fuzzy query. here are details http://www.lucidimagination.com/blog/2009/09/08/auto-suggest-from-popular-queries-using-edgengrams/ I just added fuzzyquery operator and seems to be working so far -- View this message in

Re: finding exact case insensitive matches on single and multiword values

2010-12-03 Thread Geert-Jan Brits
when you went from strField to TextField in your config you enabled tokenizing (which I believe splits on spaces by default), which is why you see seperate 'words' / terms in the debugQuery-explanation. I believe you want to keep your old strField config and try quoting: fq=city:den+haag or

Re: solr 1.4 suggester component

2010-12-03 Thread abhayd
thanks .. i used http://www.lucidimagination.com/blog/2009/09/08/auto-suggest-from-popular-queries-using-edgengrams/ with fuzzy operator.. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-1-4-suggester-component-tp1766915p2012946.html Sent from the Solr - User mailing

boosting certain docs based on a filed value

2010-12-03 Thread abhayd
hi I was looking to boost certain docs based on some values in a indexed field. e.g. pType - post paid go phone Would like to have post paid docs first and then go phone. I checked the functional query but could not figure out. Any help? -- View this message in context:

Re: Limit number of characters returned

2010-12-03 Thread Ahmet Arslan
Couldn't I just use the highlighter and configure it to use the alternative field to return the first 200 characters?  In cases where there is a highlighter match I would prefer to show the excerpts anyway. http://wiki.apache.org/solr/HighlightingParameters#hl.alternateField

Re: Batch Update Fields

2010-12-03 Thread Adam Estrada
I wonder...I know that sed would work to find and replace the terms in all of the csv files that I am indexing but would it work to find and replace key terms in the index? find C:\\tmp\\index\\data -type f -exec sed -i 's/AF/AFGHANISTAN/g' {} \; That command would iterate through all the files

nexus of synonyms and stemming, take 2

2010-12-03 Thread Will Milspec
hi all, [This is a second attempt at emailing. The apache mailing list spam filter apparently did not like my synonyms entry, ie.. classified my email as spam. I have replaced phone with 'foo' , 'cell' with 'sell' and 'mobile' with 'nubile' ] This is a fairly basic synonyms question: how does

Re: Batch Update Fields

2010-12-03 Thread Markus Jelsma
On Friday 03 December 2010 18:20:44 Adam Estrada wrote: I wonder...I know that sed would work to find and replace the terms in all of the csv files that I am indexing but would it work to find and replace key terms in the index? It'll most likely corrupt your index. Offsets, positions etc

Re: Negative fl param

2010-12-03 Thread Ahmet Arslan
When returning results is there a way I can say to return all fields except a certain one? So say I have stored fields foo, bar and baz but I only want to return foo and bar. Is it possible to do this without specifically listing out the fields I do want? There were a similar discussion.

Re: finding exact case insensitive matches on single and multiword values

2010-12-03 Thread Erick Erickson
Arrrgh, Geert-Jan is right, that't the 15th time at least this has tripped me up. I'm pretty sure that text will work if you escape the space, e.g. city:(den\ haag). The debug output is a little confusing since it has a line like city:den haag which almost looks wrong... but it worked out OK on

Re: score from two cores

2010-12-03 Thread Erick Erickson
Uhhm, what are you trying to do? What do you want to do with the scores from two cores? Best Erick On Fri, Dec 3, 2010 at 11:21 AM, Ma, Xiaohui (NIH/NLM/LHC) [C] xiao...@mail.nlm.nih.gov wrote: I have multiple cores. How can I deal with score? Thanks so much for help! Xiaohui

Re: boosting certain docs based on a filed value

2010-12-03 Thread Ahmet Arslan
I was looking to boost certain docs based on some values in a indexed field. e.g. pType - post paid go phone Would like to have post paid docs first and then go phone. I checked the functional query but could not figure out. You can use

Re: Batch Update Fields

2010-12-03 Thread Erick Erickson
Have you consider defining synonyms for your code -country conversion at index time (or query time for that matter)? We may have an XY problem here. Could you state the high-level problem you're trying to solve? Maybe there's a better solution... Best Erick On Fri, Dec 3, 2010 at 12:20 PM, Adam

can solrj swap cores?

2010-12-03 Thread Will Milspec
hi all, Does solrj support swapping cores? One of our developers had initially tried swapping solr cores (e.g. core0 and core1) using the solrj api, but it failed. (don't have the exact error) He susequently replaced the call with straight http (i.e. http client). Unfortunately I don't have the

Re: Batch Update Fields

2010-12-03 Thread Adam Estrada
First off...I know enough about Solr to be VERY dangerous so please bare with me ;-) I am indexing the geonames database which only provides country codes. I can facet the codes but to the end user who may not know all 249 codes, it isn't really all that helpful. Therefore, I want to map the full

dataimports response returns before done?

2010-12-03 Thread Tri Nguyen
Hi,   After issueing a dataimport, I've noticed solr returns a response prior to finishing the import. Is this correct?   Is there anyway i can make solr not return until it finishes?   If not, how do I ping for the status whether it finished or not?   thanks,   tri

Question about Solr Fieldtypes, Chaining of Tokenizers

2010-12-03 Thread Matthew Hall
Hey folks, I'm working with a fairly specific set of requirements for our corpus that needs a somewhat tricky text type for both indexing and searching. The chain currently looks like this: tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.PatternReplaceFilterFactory

Re: dataimports response returns before done?

2010-12-03 Thread Ahmet Arslan
--- On Fri, 12/3/10, Tri Nguyen tringuye...@yahoo.com wrote: From: Tri Nguyen tringuye...@yahoo.com Subject: dataimports response returns before done? To: solr user solr-user@lucene.apache.org Date: Friday, December 3, 2010, 7:55 PM Hi,   After issueing a dataimport, I've noticed solr

RE: score from two cores

2010-12-03 Thread Ma, Xiaohui (NIH/NLM/LHC) [C]
Please correct me if I am doing something wrong. I really appreciate your help! I have a core for metadata (xml files) and a core for pdf documents. Sometimes I need search them separately, sometimes I need search both of them together. There is the same key which is related them for each item.

Re: Negative fl param

2010-12-03 Thread Mark
Ok simple enough. I just created a SearchComponent that removes values from the fl param. On 12/3/10 9:32 AM, Ahmet Arslan wrote: When returning results is there a way I can say to return all fields except a certain one? So say I have stored fields foo, bar and baz but I only want to return

Highlighting parameters

2010-12-03 Thread Mark
Is there a way I can specify separate configuration for 2 different fields. For field 1 I wan to display only 100 chars, Field 2 200 chars

Syncing 'delta-import' with 'select' query

2010-12-03 Thread Juan Manuel Alvarez
Hello everyone! I would like to ask you a question about DIH. I am using a database and DIH to sync against Solr, and a GUI to display and operate on the items retrieved from Solr. When I change the state of an item through the GUI, the following happens: a. The item is updated in the DB. b. A

Re: boosting certain docs based on a filed value

2010-12-03 Thread abhayd
thanks!! that worked.. Can i enter the sequence too like postpaid,free,costly? -- View this message in context: http://lucene.472066.n3.nabble.com/boosting-certain-docs-based-on-a-filed-value-tp2012962p2013895.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Highlighting parameters

2010-12-03 Thread Markus Jelsma
Yes Some parameters may be overriden on a per-field basis with the following syntax: f.fieldName.originalParam=value http://wiki.apache.org/solr/HighlightingParameters Is there a way I can specify separate configuration for 2 different fields. For field 1 I wan to display only 100

Re: Highlighting parameters

2010-12-03 Thread Ahmet Arslan
Is there a way I can specify separate configuration for 2 different fields. For field 1 I wan to display only 100 chars, Field 2 200 chars yes with the parameter accepts per-field overrides. the syntax is http://wiki.apache.org/solr/HighlightingParameters#HowToOverride

Re: boosting certain docs based on a filed value

2010-12-03 Thread Ahmet Arslan
thanks!! that worked.. Can i enter the sequence too like postpaid,free,costly? Does that mean you want to display first postpaid, after that free, and lastly costly? If thats you want, i think it is better to create a tint field using these types and then sort by this field.

Re: Batch Update Fields

2010-12-03 Thread Erick Erickson
That will certainly work. Another option, assuming the country codes are in their own field would be to put the transformations into a synonym file that was only used on that field. That way you'd get this without having to do the pre-process step of the raw data... That said, if you

Re: score from two cores

2010-12-03 Thread Erick Erickson
The scores will not be comparable. Scores are only relevant within one search on one core, so comparing them across two queries (even if it's the same query but against two different cores) is meaningless. So, given your setup I would just use the results from one of the cores and fill in data

Re: score from two cores

2010-12-03 Thread Paul
On Fri, Dec 3, 2010 at 4:47 PM, Erick Erickson erickerick...@gmail.com wrote: But why do you have two cores in the first place? Is it really necessary or is it just making things more complex? I don't know why the OP wants two cores, but I ran into this same problem and had to abandon using a

highlighting wiki confusion

2010-12-03 Thread Lance Norskog
http://wiki.apache.org/solr/HighlightingParameters?#hl.highlightMultiTerm If the SpanScorer is also being used, enables highlighting for range/wildcard/fuzzy/prefix queries. Default is false. Solr1.4. This parameter makes sense for Highlighter only. I think this meant 'for PhraseHighlighter

Re: Restrict access to localhost

2010-12-03 Thread Tom
If you are using another app to create the index, I think you can remove the update servlet mapping in the web.xml. -- View this message in context: http://lucene.472066.n3.nabble.com/Restrict-access-to-localhost-tp2004475p2014129.html Sent from the Solr - User mailing list archive at