Re: [External] Re: Query all fields

On Thu, Oct 25, 2012 at 1:34 AM, Greene, Daniel [USA] wrote: > Another option you'll find out there is to use a 'copy field ' to copy the > contents of multiple fields into a single indexed field for "universal " > searching... > Told with precision > > > - Reply message - > From: "Ahmet

SolrJ missing CollectionAdmin Api to create new collections dynamically

Hi, I can't find a good way to create a new Collection with SolrJ. I need to create my Collections dynamically and at the moment the only way I see is to call the CollectionAdmin with a HTTP Call directly to any of my SolrServers. I don't like this because I think its a better way only to com

org.apache.lucene.queryparser.classic.ParseException - a Bug?

Hi all, I was trying to provide spatial search via solrj client. but when i try to run it i got org.apache.solr.common.SolrException: org.apache.lucene.queryparser.classic.ParseException: Expected identifier at pos 9 str='{!geofilt+sfield=store}' I have tried to do the same search on browser

Re: MMapDirectory, demand paging, lazy evaluation, ramfs and the much maligned RAMDirectory (oh my!)

On 10/24/2012 6:29 PM, Aaron Daubman wrote: Let me be clear that that I am not interested in RAMDirectory. However, I would like to better understand the oft-recommended and currently-default MMapDirectory, and what the tradeoffs would be, when using a 64-bit linux server dedicated to this single

Re: MMapDirectory, demand paging, lazy evaluation, ramfs and the much maligned RAMDirectory (oh my!)

Was going to say the same thing. It's also usually a good idea to reduce paging (eg 0 swappiness in linux). - Mark On Oct 24, 2012, at 9:36 PM, François Schiettecatte wrote: > Aaron > > The best way to make sure the index is cached by the OS is to just cat it on > startup: > > cat `f

Re: MMapDirectory, demand paging, lazy evaluation, ramfs and the much maligned RAMDirectory (oh my!)

Aaron The best way to make sure the index is cached by the OS is to just cat it on startup: cat `find /path/to/solr/index` > /dev/null Just make sure your index is smaller than RAM otherwise data will be rotated out. Memory mapping is built on the virtual memory system, and I suspect

UnsupportedOperationException: ExternalFileField

(Solr4) I'm getting the following error trying to use ExternalFileField to maintain an inStock flag. Any idea what I'm doing wrong? schema.xml: indexed="false" class="solr.ExternalFileField" valType="float"/> -rw-r--r-- 1 tomcat tomcat 100434 Oct 24 20:07 external_inStock: YM0600=1 YM0544

Re: Improving performance for use-case where large (200) number of phrase queries are used?

Hi Peter, Thanks for the recommendation - I believe we are thinking along the same lines, but wanted to check to make sure. Are you suggesting something different than my #5 (below) or are we essentially suggesting the same thing? On Wed, Oct 24, 2012 at 1:20 PM, Peter Keegan wrote: > Could you

Re: Newbie - Setting up date and distance facets

: Thank you for the reply. The facet range gap loks good but it is too far down : the line to be of use, I wish it was implemented though. : : What I want is really a more simple question : : http://wiki.apache.org/solr/SimpleFacetParameters#facet.range : : Is it correct that to add facets on d

Re: [External] Re: Query all fields

Another option you'll find out there is to use a 'copy field ' to copy the contents of multiple fields into a single indexed field for "universal " searching... - Reply message - From: "Ahmet Arslan" To: "solr-user@lucene.apache.org" Subject: [External] Re: Query all fields Date: Wed

Re: Query all fields

Makes sense, thanks! Billy Sent from my iPhone On Oct 24, 2012, at 4:25 PM, Ahmet Arslan wrote: > >> Looking at the Solr tutorial I see >> queries like: >> >> q=video&fl=name,id (return only name and id fields) >> >> Does that query all fields for the word video? > > No query is executed

Re: Query all fields

> Looking at the Solr tutorial I see > queries like: > > q=video&fl=name,id (return only name and id fields) > > Does that query all fields for the word video?  No query is executed on default search field. If you add &debugQuery=on to your URL you can see which field is queried. > Is there

Seeking Use Cases: Boosting & Biasing to affect search scores

Hey folks, I'm giving a talk at ApacheCon in two weeks about how domain knowledge and/or knowledge of your user base can be used to boost/bias the scores of documents in Solr search results. Simple examples being things like: using function queries to boost by numeric fields like date or pop

Re: Search field affecting relevance

Sorry guys, I initially simplified the criteria. Actually, I use EDisMaxQueryParser with about dozen of fields configured with different boost values. If any of them is matched - it's ok to return such document. But I also have one (at the moment) field - that I would like to use only to "boost" r

Re: Search field affecting relevance

Hi, This is core lucene/solr functionality. +field1:foo field2:foo makes a match in field1 required. Otis -- Performance Monitoring - http://sematext.com/spm On Oct 24, 2012 4:39 AM, "Maxim Kuleshov" wrote: > Hi, > > For example, we have documents with two fields - field1 and field2. > Both fie

Re: is it possible to index

Do note that you _can_ join across cores. BUT the join capability is a fairly restricted use-case. And even if it was performant, it's not like a DB join, you can only return info from a single kind of doc. That is, if you were joining between customer documents and vendor documents, you could only

Re: is it possible to index

This is gold info for me! Thanks! 2012/10/24 Martin Koch > In my experience, about as fast as you can push the new data :) Depending > on the size of your records, this should be a matter of seconds. > > /Martin Koch > > On Wed, Oct 24, 2012 at 9:01 PM, Marcelo Elias Del Valle < > mvall...@gmail

Re: is it possible to index

In my experience, about as fast as you can push the new data :) Depending on the size of your records, this should be a matter of seconds. /Martin Koch On Wed, Oct 24, 2012 at 9:01 PM, Marcelo Elias Del Valle wrote: > Erick, > > Thanks for the help, it sure helps a lot to read that, as it

Re: is it possible to index

Erick, Thanks for the help, it sure helps a lot to read that, as it gives me more confidence I am not crazy about what I am thinking. The only problem I see by de-normalizing data as you said is that if any relation between customer and vendor changes, I will have to update the index for

Re: SolrCloud - loop in recovery mode

On a quick search, I didn't happen to see an open JIRA for this type of thing. Could you file one? - Mark On Oct 24, 2012, at 11:35 AM, AlexeyK wrote: > The situation can be replayed on solr 4 (solrcloud): > 1. Define the warmup query > 2. Add spell checker configuration to the /select search

Re: is it possible to index

One, take off your RDBMS cap ... DB folks regularly reject the idea of de-normalizing data to make best use of Solr, but that's what I would explore first. Yes, this repeats the, in your case, vendor information perhaps many times, but try that first, even though that causes you to update multiple

Re: Improving performance for use-case where large (200) number of phrase queries are used?

Thanks for the ideas - some followup questions in-line below: > * use shingles e.g. to turn two-word phrases into single terms (how > long is your average phrase?). Would this be different than what I was calling "common grams"? (other than shingling every two words, rather than just common ones

Re: Improving performance for use-case where large (200) number of phrase queries are used?

Could you index your 'phrase tags' as single tokens? Then your phrase queries become simple TermQuerys. On Wed, Oct 24, 2012 at 12:26 PM, Robert Muir wrote: > On Wed, Oct 24, 2012 at 11:09 AM, Aaron Daubman wrote: > > Greetings, > > > > We have a solr instance in use that gets some perhaps atyp

Re: solr 4.0 missing SolrPluginUtils addOrReplaceResults

Hi Solr-Users, Anyone has any work around for SolrPluginUtils.addOrReplaceResults in solr 4.0 ? Should be easy to migrate the code from 3.6 branch to 4.0 SolrPluginUtils. Is there any specific reason why this method is dropped in 4.0 ? Thanks Varun On Tue, Oct 23, 2012 at 11:14 AM, varun srivast

Re: Monitor Deleted Event

Since Lucene is a library there isn't much of a support for this since in theory the client application issuing the delete could also then do something else upon delete. solr on the other hand being a layer (a server layer) sitting on top of lucene, it makes sense for hooks to be configured there.

Re: Improving performance for use-case where large (200) number of phrase queries are used?

On Wed, Oct 24, 2012 at 11:09 AM, Aaron Daubman wrote: > Greetings, > > We have a solr instance in use that gets some perhaps atypical queries > and suffers from poor (>2 second) QTimes. > > Documents (~2,350,000) in this instance are mainly comprised of > various "descriptive fields", such as mul

Re: SolrJ CloudSolrServer throws ClassCastException

Hi Kevin, Solrj 4.0.0 is on Maven Central now, and has been since Oct. 11th: Steve On Oct 24, 2012, at 11:21 AM, Kevin Osborn wrote: > Thanks for that idea. The problem was that my Solr server wa

Re: SolrJ CloudSolrServer throws ClassCastException

Maven Central looks up to date now for SolrJ, with all three of 4.0.0-ALPHA, 4.0.0-BETA, and 4.0.0. The latter is dated 11-Oct-2012. See: http://search.maven.org/#browse%7C1147257723 http://search.maven.org/#browse%7C-591611598 -- Jack Krupansky -Original Message- From: Kevin Osborn

Re: How to import a part of index from main Solr server(based on a query) to another Solr server and then do incremental import at intervals later(the updated index)?

Hi, all: Sorry for the late response: ) Thanks for your reply. I think Solr Replication may not help in my case, as the central server would store all docs of all users(1000+), and in each client, I only want to copy index of his/her docs created or changed in last 2 weeks(for example), after the

is it possible to index

Hello, I am new to Solr and I have a scenario where I want to use it, but I might be misunderstanding some concepts. I will explain what I want here, if someone has a solution for this, I would gladly accept the help. I have a core indexing customers. I have another core indexing vendors.

Re: Monitor Deleted Event

I'm not 100% sure about this but looks like update processors may help? http://wiki.apache.org/solr/UpdateRequestProcessor It looks like you can put in custom code to execute when certain actions happen so sounds like this is what you are looking for. Cheers Amit On Wed, Oct 24, 2012 at 8:43 AM,

Re: Solr - Use Regex in the Query Phrase

Ok, now I have apache-solr-4.0.0 on windows. And I am able to use the plugin ComplexPhraseQuery as mentioned above. So I can search for: For example: "art(.*?)le" or "he sa*". Thanks for all help. What if I want to search a phrase like that: "he (.*?) that" in sentences like: he said that he is a

Re: help with solritas config

Hello again, Sorry, I took some time to process everything... I looked at some more documentation and realized I am confusing documents with cores. Actually, I was expecting to be able to have a USER core and a CITY core and be able to relate them somehow. Thanks for the clarification, I will

Monitor Deleted Event

When some docs are deleted from Solr server, I want to execute some code - for example, add an record such as {contentid, deletedat} to another solr server or database. How can I do this through Solr or Lucene? Thanks for any reply and help :) -- View this message in context: http://lucene.47

Re: SolrCloud - loop in recovery mode

The situation can be replayed on solr 4 (solrcloud): 1. Define the warmup query 2. Add spell checker configuration to the /select search handler 3. Set spellcheck.collation = true The server will stuck in init phase due to deadlock. Is there a bug open for this? Actually you cannot get collated sp

Re: help with solritas config

Shawn, First of all, thanks a lot for your answer, it was very useful. By the content of your email, it seemed to me the /browser is just something as a solr admin interface, so now I am confused. I am already using SOLRJ in my application and I am currently able to perform a query like fo

SolrJ and clustering with Carrot

Hi, At the moment, I run a Solr query in my Java program, using SolrJ, and get a QueryResponse object. But now I've just started using Carrot to do results clustering when I run a search in Solr, and although I can see that the clusters are now part of the response, the QueryResponse class doe

Re: SolrJ CloudSolrServer throws ClassCastException

Thanks for that idea. The problem was that my Solr server was on 4.0.0, but the latest version for SolrJ on Maven is 4.0.0-Beta. I downgraded my server to 4.0.0-beta and it worked. -Kevin On Wed, Oct 24, 2012 at 6:03 AM, Mark Miller wrote: > Did up upgrade your Solr instance from the beta or al

Re: Search field affecting relevance

First, documents are not matched "by field", but "by field value". So, make sure q.op=OR, mm=0%, and query: +field1:x field2:x^20 This means that "x" MUST be present in field1 of each document, but IF "x" happens to be present in field2 of the same document the score will be boosted (by howev

Improving performance for use-case where large (200) number of phrase queries are used?

Greetings, We have a solr instance in use that gets some perhaps atypical queries and suffers from poor (>2 second) QTimes. Documents (~2,350,000) in this instance are mainly comprised of various "descriptive fields", such as multi-word (phrase) tags - an average document contains 200-400 phrases

Re: help with solritas config

On 10/24/2012 8:05 AM, Marcelo Elias Del Valle wrote: I saw there is some documentation in solr wiki for SearchHandler and VelocityResponseWriter, which I am trying to digest. However, I saw there are some configuration fields that aren't there, like this QF field. I am not sure on how to cus

Re: Occasional Solr performance issues

Please consider never running "optimize". That should be called "force merge". wunder On Oct 24, 2012, at 3:28 AM, Dotan Cohen wrote: > On Tue, Oct 23, 2012 at 3:07 PM, Erick Erickson > wrote: >> Maybe you've been looking at it but one thing that I didn't see on a fast >> scan was that maybe

Re: Solr 4.0.0 - index version and generation not changed after delete by query on master

Sorry, I had copy/paste the wrong link before. Here is the correct one: https://issues.apache.org/jira/browse/SOLR-3986 Bill On Wed, Oct 24, 2012 at 10:26 AM, Bill Au wrote: > I just filed a bug with all the details: > > https://issues.apache.org/jira/browse/SOLR-3681 > > Bill > > > On Tue, O

Re: uniqueKey not enforced

Just do a query on one of the keys that appears to be duplicated and see if the "duplicates" are also returned. Also, look at all of the field values for the documents with "duplicated" keys - are they identical as well, or are there differences in specific field values. That might highlight w

Re: Solr 4.0.0 - index version and generation not changed after delete by query on master

I just filed a bug with all the details: https://issues.apache.org/jira/browse/SOLR-3681 Bill On Tue, Oct 23, 2012 at 2:47 PM, Chris Hostetter wrote: > : Just discovered that the replication admin REST API reports the correct > : index version and generation: > : > : http://master_host:port/sol

Re: SolrCloud - loop in recovery mode

It is actually connected to this: https://gist.github.com/2880527 Once you have collation = true + warmup, the init is stuck on wait -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-loop-in-recovery-mode-tp4015330p4015593.html Sent from the Solr - User mailing list

Re: SolrCloud - loop in recovery mode

After a little bit of investigation, it's about the searcher warmup that doesn't happen. I see the main thread waiting for the searcher. The warmup query handler is stuck in another thread on the very same lock in getSearcher(), and no notify() is called. If I set the useColdSearcher = true, this o

Re: uniqueKey not enforced

On Wed, Oct 24, 2012 at 2:03 PM, Erick Erickson wrote: > Robert: > > But you do have an index somewhere, so the alternative for > looking at it low-level would be > 1> get a copy of Luke and point it at your index. Very useful tool I will do that, next time I have that condition. Unfortunately I

Re: SolrJ CloudSolrServer throws ClassCastException

Did up upgrade your Solr instance from the beta or alpha to 4 at some point? - Mark On Wed, Oct 24, 2012 at 1:14 AM, Kevin Osborn wrote: > It looks like this is where the problem lies. Here is the JSON that SolrJ > is receiving from Zookeeper: > > "data":"{\\"manufacturer\\":{\\n\\"shard1\\"

Re: SolrCloud - loop in recovery mode

I only started learning the new features, so chances are it's about some misconfiguration. I removed the collection2 from the setup and indexed some files. Now there is another pattern that stucks the init, and it's about the overseer polling the queue: Oct 24, 2012 2:18:52 PM org.apache.solr.core

Re: uniqueKey not enforced

Robert: But you do have an index somewhere, so the alternative for looking at it low-level would be 1> get a copy of Luke and point it at your index. Very useful tool 2> just copy all your conf and data files somewhere and run the Jetty instance of Solr on that... FWIW, Erick On Wed, Oct 24

Re: solr 4.1 compression

i found this ticket: https://issues.apache.org/jira/browse/SOLR-3927 compression is currently lucene 4.1-branch only and not yet in solr4.1 branch?

RE: Failure to open existing log file (non fatal)

-Original message- > From:Mark Miller > Sent: Wed 24-Oct-2012 01:33 > To: solr-user@lucene.apache.org > Subject: Re: Failure to open existing log file (non fatal) > > Why the process died, I cannot say. Seems like the world of guesses is > just too large :) If there is nothing in the log

Re: Occasional Solr performance issues

On Tue, Oct 23, 2012 at 3:07 PM, Erick Erickson wrote: > Maybe you've been looking at it but one thing that I didn't see on a fast > scan was that maybe the commit bit is the problem. When you commit, > eventually the segments will be merged and a new searcher will be opened > (this is true even i

Re: DIH nested entities don't work

Hi, The views are very specific and every column contains relevant information, thus the '*'. But a strange thing happened, i ran the data-import again and for some reason products had features but still no synonyms. The only thing i changed was to use: processor="SqlEntityProcessor " cacheImpl="

Search field affecting relevance

Hi, For example, we have documents with two fields - field1 and field2. Both fields are indexed and both are used in search. Is there way to return only documents that are matched by field1, but taking in account that if field2 is matched - relevance should be higher? In other words, if document

Re: DIH nested entities don't work

On 24 October 2012 13:03, mroosendaal wrote: > Hi, > > Here's the relevant part of my schema: [...] > > pdt_id > ... > > the data is read from into another searchengine fine but i'll try the select > queries individually. The field-definitions need some tweaking. [...] As you are not specificall

Re: DIH nested entities don't work

Hi, Here's the relevant part of my schema: pdt_id ... the data is read from into another searchengine fine but i'll try the select queries individually. The field-definitions need some tweaking. As for the songtitle, this was a typo Cheers, Maarten

Re: uniqueKey not enforced

On Tue, Oct 23, 2012 at 2:37 PM, Erick Erickson wrote: > From left field: > > Try looking at your admin/schema browser page for the ID in question. > That actually > gets stuff out of your index (the actual indexed terms). See if you > have two values I'm running embedded, so I don't have that. Ho