Enabling realtime search in Solr 4.0

2011-12-29 Thread Avner Levy
Hi, I'm trying to enable realtime search in Solr 4.0 (So I can see new documents without committing). I've added: realtime visible=0 facet=truetrue/realtime updateLog class=solr.FSUpdateLog str name=dir${solr.data.dir:}/str /updateLog But documents aren't seen before commit (or

Re: Solr, SQL Server's LIKE

2011-12-29 Thread Shashi Kant
for a simple, hackish (albeit inefficient) approach look up wildcard searchers e,g foo*, *bar On Thu, Dec 29, 2011 at 12:38 PM, Devon Baumgarten dbaumgar...@nationalcorp.com wrote: I have been tinkering with Solr for a few weeks, and I am convinced that it could be very helpful in many of

distributed faceting: refineFacets()

2011-12-29 Thread Dmitry Kan
Hello list, In a distributed faceting search scenario, does SOLR frontend (the merger) expect shard facets to be pre-sorted (by count or by index) ? If so, when merging the results, is there some smart strategy for combining the shards results into a final sorted list? Can someone explain what

[Solr Event Listener plug-in] Execute query search from SolrCore - Java Code

2011-12-29 Thread Alessandro Benedetti
Hi guys, I'm developing a custom SolrEventListener, and inside the PostCommit() method I need to execute some queries and collect results. In my SolrEventListener class, I have a SolrCore Object( org.apache.solr.core.SolrCore) and a list of queries (Strings ). How can I use the SolrCore to

Re: Frequent Indexing of same Documents

2011-12-29 Thread Gora Mohanty
On Fri, Dec 30, 2011 at 12:25 AM, Avni, Itamar itamar.a...@verint.com wrote: [...] This electronic message may contain proprietary and confidential information of Verint Systems Inc., its affiliates and/or subsidiaries. The information is intended to be for the use of the individual(s) or

Frequent Indexing of same Documents

2011-12-29 Thread Avni, Itamar
Hi community, Say I have lots of documents to index, each with primary key in the index, and I index them frequently. They are not indexed all together (like in bulk), but each in a different time. 1) Is there a significant difference in performances between a freshly created core (the first

Re: Enabling realtime search in Solr 4.0

2011-12-29 Thread Mark Miller
On Dec 29, 2011, at 3:39 AM, Avner Levy wrote: Hi, I'm trying to enable realtime search in Solr 4.0 (So I can see new documents without committing). I've added: realtime visible=0 facet=truetrue/realtime updateLog class=solr.FSUpdateLog str name=dir${solr.data.dir:}/str /updateLog

Re: a question on jmx solr exposure

2011-12-29 Thread Dmitry Kan
Well, we don't use multicore feature of SOLR, so in our case SOLR instances are just separate web-apps. The web-app loading order probably then affects on which app gets hold of a jmx 'pipe'. We should probably start using the feature to collect stats from different cores at the same time. Thanks.

RE: Enabling realtime search in Solr 4.0

2011-12-29 Thread Avner Levy
Thanks Mark, I appreciate your help. I need the Solr index to be in sync with my database. This means that even if one record was added I need it to appear in the next search (including faceting). I've read in Solr-RA documentation that if you add realtimetrue/realtime you can add documents and

Re: a question on jmx solr exposure

2011-12-29 Thread Dmitry Kan
That's absolutely right. Thanks for the suggestion. On Thu, Dec 29, 2011 at 2:47 PM, Gora Mohanty g...@mimirtech.com wrote: On Thu, Dec 29, 2011 at 6:15 PM, Dmitry Kan dmitry@gmail.com wrote: Well, we don't use multicore feature of SOLR, so in our case SOLR instances are just separate

Re: solr keep old docs

2011-12-29 Thread Alexander Aristov
I have never developed for solr yet and don't know much internals but Today I have tried one approach with searcher. In my update processor I get searcher and search for ID. It works but I need to load test it. Will index traversal be faster (less resource consuming) than search? Best Regards

Re: a question on jmx solr exposure

2011-12-29 Thread Gora Mohanty
On Thu, Dec 29, 2011 at 6:15 PM, Dmitry Kan dmitry@gmail.com wrote: Well, we don't use multicore feature of SOLR, so in our case SOLR instances are just separate web-apps. The web-app loading order probably then affects on which app gets hold of a jmx 'pipe'. We should probably start using

Re: Large RDBMS dataset

2011-12-29 Thread Alexey Serba
The problem is that for each record in fd, Solr makes three distinct SELECT on the other three tables. Of course, this is absolutely inefficient. You can also try to use GROUP_CONCAT (it's MySQL function, but maybe there's something similar in MS SQL) to select all the nested 1-N entities in a

Re: 3.5 QueryResponseWriter

2011-12-29 Thread Erik Hatcher
Aleksander - Looks like you've experienced the issue described with fixes here: http://www.lucidimagination.com/search/document/48b9e75fe68be4b7 Erik On Dec 29, 2011, at 08:40 , Aleksander Akerø wrote: Hi! So I've decided try out Solr 3.5.0. What I have done this far is basicly

Re: Best practices for installing and maintaining Solr configuration

2011-12-29 Thread Erick Erickson
This should help: http://wiki.apache.org/solr/SolrTomcat The difference here is that you're not copying the example directory, you're copying the example/solr directory. And this is just basically to get the configuration files and directory structure right. You're not copying executables, jars,

Re: a question on jmx solr exposure

2011-12-29 Thread Alexey Serba
Which Solr version do you use? Maybe it has something to do with default collection? I do see separate jmx domain for every collection, i.e. solr/collection1 solr/collection2 solr/collection3 ... On Wed, Dec 21, 2011 at 1:56 PM, Dmitry Kan dmitry@gmail.com wrote: Hello list, This might

Re: Decimal Mapping problem

2011-12-29 Thread Alexey Serba
Try to cast MySQL decimal data type to string, i.e. CAST( IF(drt.discount IS NULL,'0',(drt.discount/100)) AS CHAR) as discount (or CAST AS TEXT) On Mon, Dec 19, 2011 at 1:24 PM, Niels Stevens ni...@kabisa.nl wrote: Hey everybody, I'm having an issue importing Decimal numbers from my Mysql DB

3.5 QueryResponseWriter

2011-12-29 Thread Aleksander Akerø
Hi! So I've decided try out Solr 3.5.0. What I have done this far is basicly just to copy the /example/solr folder, install the webapp .war file in a tomcat instance and start up. At first I complained about the VelocityResponseWriter, so i created a /lib folder in /$SOLR_HOME and added the

Re: solr keep old docs

2011-12-29 Thread Erick Erickson
Hmmm, we're not communicating G... The update processor wouldn't search in the classic sense. It would just use lower-level index traversal to determine if the doc (identified by your unique key) was already in the index and skip indexing that document if it was. No real *searching* involved (see

how to configure saxon xslt processor for solr

2011-12-29 Thread vrpar...@gmail.com
Hello all, i want to configure saxon xslt processor for solr; how to do that? its taking xalan as default processor. also if its needed to set classpath, please provide me path where can i set classpath? and also how can we check that which xslt processor default use. i am using solr 1.4

Re: Facet Ordering

2011-12-29 Thread Jamie Johnson
Thanks Hoss, I'll take a look at this and see if i can understand this. On Wed, Dec 28, 2011 at 9:44 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : I've seen in the solr faceting overview that it is possible to sort : either by count or lexicographically, but is there a way to sort so

Re: Grouping results after Sorting or vice-versa

2011-12-29 Thread Tomás Fernández Löbbe
Hi Vijayaragavan, did you apply a patch for grouping in Solr 3.1? It is available out of the box since 3.3. Also, the result from grouping will not look exactly like you are expecting, as results with the same value in the grouping field (in this case, thread_id) will be collapsed into one group.

Re: solr keep old docs

2011-12-29 Thread Alexander Aristov
well. The first results are ready. I have implemented custom update processor following your suggestion using low level index reader and termdocs. Launched scripts which add about 10 000 docs. Indexing took about 1 minute including commit that is quite good for me. I don't have larger datasets so

Re: solr keep old docs

2011-12-29 Thread Erick Erickson
I'd guess it would be much faster, assuming that the search savings wouldn't be swamped by the additional transmission time over the wire and parsing the request (although SolrJ uses a binary format, so parsing request probably isn't all that expensive). You could even do a hybrid approach. Pack

Solr, SQL Server's LIKE

2011-12-29 Thread Devon Baumgarten
I have been tinkering with Solr for a few weeks, and I am convinced that it could be very helpful in many of my upcoming projects. I am trying to decide whether Solr is appropriate for this one, and I haven't had luck looking for answers on Google. I need to search a list of names of companies

[SOLR 3.5] QueryResponseWriter

2011-12-29 Thread Aleksander Akerø
Hi! So I've decided try out Solr 3.5.0. What I have done this far is basicly just to copy the /example/solr folder, install the webapp .war file in a tomcat instance and start up. At first I complained about the VelocityResponseWriter, so i created a /lib folder in /$SOLR_HOME and added the

Re: Enabling realtime search in Solr 4.0

2011-12-29 Thread Yonik Seeley
On Thu, Dec 29, 2011 at 2:35 PM, Avner Levy av...@checkpoint.com wrote: Thanks Mark, I appreciate your help. I need the Solr index to be in sync with my database. This means that even if one record was added I need it to appear in the next search (including faceting). You could just add

Re: How can I check if a more complex query condition matched?

2011-12-29 Thread Chris Hostetter
: I have a more complex query condition like this: : : (city:15 AND country:60)^4 OR city:15^2 OR country:60^2 : : What I want to achive with this query is basically if a document has : city = 15 AND country = 60 it is more important then another document : which only has city = 15 OR country =

Re: Solr, SQL Server's LIKE

2011-12-29 Thread Erick Erickson
SQLs like is usually handled with ngrams if you want *stuff* kinds of searches. Wildcards are interesting in Solr. Things Solr handles that aren't easy in SQL Phrases, phrases with slop, stemming, synonyms. And, especially, some kind of relevance ranking. But Solr does NOT do the things SQL is

Re: Frequent Indexing of same Documents

2011-12-29 Thread Erick Erickson
See below 1) Is there a significant difference in performances between a freshly created core (the first time to index), to an old core (every document already exists is the core)? not really. Documents are indexed in segments, and a fresh one is usually opened after every commit (you only commit

NoClassDefFoundError: org/apache/solr/common/params/SolrParams

2011-12-29 Thread Bruno Adam Osiek
Hi, I'm trying to deploy a Solrj based application into JBoss AS 7 using Eclipse Indigo. When deploying it I get the following error message: ERROR [org.jboss.msc.service.fail] (MSC service thread

RE: NoClassDefFoundError: org/apache/solr/common/params/SolrParams

2011-12-29 Thread Dyer, James
The SolrParams class is in the solrj.jar file so you should verify that this is in the classpath. Also see if it is listed in the manifest.mf file in the war's META-INF dir. If you're running this on a server within Eclipse and letting Eclipse do the deploy, my experience is it can be

RE: Solr, SQL Server's LIKE

2011-12-29 Thread Devon Baumgarten
Erick, Thanks. I know I'll be able to utilize some of Solr's free text searching capabilities in other search types in this project. The product manager wants this particular search to exactly mimic LIKE%. N-Grams get me pretty great results in general, but I don't want the results for this

Re: Enabling realtime search in Solr 4.0

2011-12-29 Thread Mark Miller
On Thu, Dec 29, 2011 at 2:35 PM, Avner Levy av...@checkpoint.com wrote: I've read in Solr-RA documentation that if you add realtimetrue/realtime you can add documents and search for them without any commit at all (and I assumed it is functionality of Solr). In Solr 4 (trunk) you can either

Re: Solr, SQL Server's LIKE

2011-12-29 Thread Sujit Pal
Hi Devon, Have you considered using a permuterm index? Its workable, but depending on your requirements (size of fields that you want to create the index on), it may bloat your index. I've written about it here: http://sujitpal.blogspot.com/2011/10/lucene-wildcard-query-and-permuterm.html

RE: Solr, SQL Server's LIKE

2011-12-29 Thread Chris Hostetter
: Thanks. I know I'll be able to utilize some of Solr's free text : searching capabilities in other search types in this project. The : product manager wants this particular search to exactly mimic LIKE%. ... : Ex: If I search Albatross I want Albert to be excluded completely, : rather

Re: strange performance issue with many shards on one server

2011-12-29 Thread Ken Krugler
Hi Frederik, Did you figure out a solution to this problem? I'm asking because I recently ran into a similar problem, with a similar setup (8 shards on one server). Occasionally a query will take a very long time. Occasionally I see timeout exceptions with the HTTP requests. E.g. 348914

Re: 3.5 QueryResponseWriter

2011-12-29 Thread Chris Hostetter
: Looks like you've experienced the issue described with fixes here: : http://www.lucidimagination.com/search/document/48b9e75fe68be4b7 but specifically, since you've already copied the jar file in question, and are now getting a class not found for the *baseclass* it suggests you have a diff