Re: Solr http post performance seems slow - help?

2009-09-24 Thread Constantijn Visinescu
This may or may not help but here goes :) When i was running performance tests i look a look at the simple post tool that comes with the solr examples. First i changed my schema.xml to fit my needs and then i deleted the old index so solr created a blank one when i started up. Then i had a had a

Unsubscribe from this mailing-list

2009-09-24 Thread Rafeek Raja
Unsubscribe from this mailing-list

Re: Use cases for ReplicationHandler's backup facility?

2009-09-24 Thread Chris Harris
2009/9/24 Noble Paul നോബിള്‍ नोब्ळ् : > On Fri, Sep 25, 2009 at 4:57 AM, Chris Harris wrote: >> The ReplicationHandler (http://wiki.apache.org/solr/SolrReplication) >> has support for "backups", which can be triggered in one of two ways: >> >> 1. in response to startup/commit/optimize events (spe

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-24 Thread Silent Surfer
Hi, Thank you Michael and Chris for the response. Today after the mail from Michael, we tested with the dynamic loading of cores and it worked well. So we need to go with the hybrid approach of Multicore and Distributed searching. As per our testing, we found that a Solr instance with 20 GB o

Re: Use cases for ReplicationHandler's backup facility?

2009-09-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Fri, Sep 25, 2009 at 4:57 AM, Chris Harris wrote: > The ReplicationHandler (http://wiki.apache.org/solr/SolrReplication) > has support for "backups", which can be triggered in one of two ways: > > 1. in response to startup/commit/optimize events (specified through > the backupAfter tag specifie

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-24 Thread Chris Hostetter
: Using a multicore approach, you could send a "create a core named : 'core3weeksold' pointing to '/datadirs/3weeksold' " command to a live Solr, : which would spin it up on the fly. Then you query it, and maybe keep it : spun up until it's not queried for 60 seconds or something, then send a : "r

Re: Showcase: Facetted Search for Wine using Solr

2009-09-24 Thread Grant Ingersoll
Hi Marian, Looks great! Wish I could order some wine. When you get a chance, please add the site to http://wiki.apache.org/solr/PublicServers! Cheers, Grant On Sep 24, 2009, at 11:51 AM, marian.steinbach wrote: Hello everybody! The purpose of this mail is to say "thank you" to the creat

Re: Sorting/paging problem

2009-09-24 Thread Lance Norskog
Which version of Java are you using? Please try the standard tricks: Do a fresh checkout of the Solr trunk. Do 'ant clean dist' and use the newly built war & latest lucene libraries. Try changing the JVM startup parameters which control how incremental compilation works: -server and others. Also t

Re: Solrj possible deadlock

2009-09-24 Thread Chris Hostetter
: Well, in the same processes I am using a jdbc connection to get all the : relative paths to the documents I want to index, then I parse the documents : to plain text using tones of open source libraries like POI, PFDBox : etc.(which might account for java2d) then I add them to the index and comm

Re: solr caching problem

2009-09-24 Thread Lance Norskog
There are now two excellent books: "Lucene In Action 2" and "Solr 1.4 Enterprise Search Server" the describe the inners workings of these technologies and how they fit together. Otherwise Solr and Lucene knowledge are only available in a fragmented form across many wiki pages, bug reports and emai

Re: Solr http post performance seems slow - help?

2009-09-24 Thread Lance Norskog
In "top", press the '1' key. This will give a list of the CPUs and how much load is on each. The display is otherwise a little weird for multi-cpu machines. But don't be surprised when Solr is I/O bound. The biggest fanciest RAID is often a better investment than CPUs. On one project we bought low-

Re: Mixed field types and boolean searching

2009-09-24 Thread Lance Norskog
No- there are various analyzers. StandardAnalyzer is geared toward searching bodies of text for interesting words - punctuation is ripped out. Other analyzers are more useful for "concrete" text. You may have to work at finding one that leaves punctuation in. On Wed, Sep 23, 2009 at 2:14 PM, Ensd

Re: Very big numbers

2009-09-24 Thread Lance Norskog
There is no "bignum" support in Solr at this time. You can pick a fixed-length string with leading zeros. That is, if your other strings are the same length as the above. 99,999,999,999,999.99 00,000,999,999,999.99 You can do sorted queries, range queries, and facets from this format. Solr is gen

Re: Parallel requests to Tomcat

2009-09-24 Thread Lance Norskog
Are you on Java 5, 6 or 7? Each release sees some tweaking of the Java multithreading model as well as performance improvements (and bug fixes) in the Sun HotSpot runtime. You may be tripping over the TCP/IP multithreaded connection manager. You might wish to create each client thread with a separ

Re: Solrj possible deadlock

2009-09-24 Thread pof
Well, in the same processes I am using a jdbc connection to get all the relative paths to the documents I want to index, then I parse the documents to plain text using tones of open source libraries like POI, PFDBox etc.(which might account for java2d) then I add them to the index and commit every

Re: Solr highlighting doesn't respect quotes

2009-09-24 Thread Paul Tomblin
On Thu, Sep 24, 2009 at 7:04 PM, Koji Sekiguchi wrote: > Set hl.usePhraseHighlighter parameter to true: > > http://wiki.apache.org/solr/HighlightingParameters#hl.usePhraseHighlighter > > That seems to have done it. Thanks. -- http://www.linkedin.com/in/paultomblin

Use cases for ReplicationHandler's backup facility?

2009-09-24 Thread Chris Harris
The ReplicationHandler (http://wiki.apache.org/solr/SolrReplication) has support for "backups", which can be triggered in one of two ways: 1. in response to startup/commit/optimize events (specified through the backupAfter tag specified in the handler's requestHandler tag in solrconfig.xml) 2. by

Re: Seattle / PNW Hadoop/Lucene/HBase Meetup, Wed Sep 30th

2009-09-24 Thread Bradford Stephens
Friendly Reminder! One week to go. On Mon, Sep 14, 2009 at 11:35 AM, Bradford Stephens < bradfordsteph...@gmail.com> wrote: > Greetings, > > It's time for another Hadoop/Lucene/Apache"Cloud" Stack meetup! > This month it'll be on Wednesday, the 30th, at 6:45 pm. > > We should have a few interest

Re: Solr highlighting doesn't respect quotes

2009-09-24 Thread Koji Sekiguchi
Set hl.usePhraseHighlighter parameter to true: http://wiki.apache.org/solr/HighlightingParameters#hl.usePhraseHighlighter Koji Paul Tomblin wrote: If I do a query for a couple of words in quotes, Solr correctly only returns pages where those words appear exactly within the quotes. But the hig

RE: OutOfMemoryError due to auto-warming

2009-09-24 Thread Francis Yakin
I reduced the size of queryResultCache in solrconfig seems to fix the issue as well. 200 >From 500 500 Francis -Original Message- From: didier deshommes [mailto:dfdes...@gmail.com] Sent: Thursday, September 24, 2009 3:32 PM To: solr-user@lucene.apache.org Cc: Andrew Mont

Re: OutOfMemoryError due to auto-warming

2009-09-24 Thread didier deshommes
On Thu, Sep 24, 2009 at 5:40 PM, Francis Yakin wrote: > You also can increase the JVM HeapSize if you have enough physical memory, > like for example if you have 4GB physical, gives the JVM heapsize 2GB or > 2.5GB. Thanks, we can definitely do that (we have 4GB available). I also forgot to add

RE: OutOfMemoryError due to auto-warming

2009-09-24 Thread Francis Yakin
You also can increase the JVM HeapSize if you have enough physical memory, like for example if you have 4GB physical, gives the JVM heapsize 2GB or 2.5GB. Francis -Original Message- From: didier deshommes [mailto:dfdes...@gmail.com] Sent: Thursday, September 24, 2009 3:32 PM To: solr-u

OutOfMemoryError due to auto-warming

2009-09-24 Thread didier deshommes
Hi there, We are running solr and allocating 1GB to it and we keep having OutOfMemoryErrors. We get messages like this: Error during auto-warming of key:org.apache.solr.search.queryresult...@c785194d:java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3

Solr highlighting doesn't respect quotes

2009-09-24 Thread Paul Tomblin
If I do a query for a couple of words in quotes, Solr correctly only returns pages where those words appear exactly within the quotes. But the highlighting acts as if the words were given separately, and stems them and everything. For example, if I search for "knee pain", it returns a document th

Looking for suggestion of WordDelimiter filter config and 'ALMA awards'

2009-09-24 Thread michael8
Hi, I have this situation that I believe is very common but was curious if anyone knows the right way to go about solving it. I have a document with 'ALMA awards' in it. However, when user searches for 'aLMA awards', it ends up with no results found. However, when I search for 'alma awards'

Re: download pre-release nightly solr 1.4

2009-09-24 Thread Mark Miller
michael8 wrote: > Hi, > > I know Solr 1.4 is going to be released any day now pending Lucene 2.9 > release. Is there anywhere where one can download a pre-released nighly > build of Solr 1.4 just for getting familiar with new features (e.g. field > collapsing)? > > Thanks, > Michael > You can d

unsubcribe

2009-09-24 Thread Rafeek Raja
unsubcribe

download pre-release nightly solr 1.4

2009-09-24 Thread michael8
Hi, I know Solr 1.4 is going to be released any day now pending Lucene 2.9 release. Is there anywhere where one can download a pre-released nighly build of Solr 1.4 just for getting familiar with new features (e.g. field collapsing)? Thanks, Michael -- View this message in context: http://www

RE: Alphanumeric Wild Card Search Question

2009-09-24 Thread Ensdorf Ken
> Here's my question: > I have some products that I want to allow people to search for with > wild cards. For example, if my product is YBM354, I'd like for users to > be able to search on "YBM*", "YBM3*", "YBM35*" and for any of these > searches to return that product. I've found that I can search

Alphanumeric Wild Card Search Question

2009-09-24 Thread Carr, Adrian
Hello Solr Users, I've tried to find the answer to this question, and have tried changing my configuration several times, but to no avail. I think someone on this list will know the answer. Here's my question: I have some products that I want to allow people to search for with wild cards. For e

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-24 Thread Michael
Using a multicore approach, you could send a "create a core named 'core3weeksold' pointing to '/datadirs/3weeksold' " command to a live Solr, which would spin it up on the fly. Then you query it, and maybe keep it spun up until it's not queried for 60 seconds or something, then send a "remove core

Sorting/paging problem

2009-09-24 Thread Charlie Jackson
I've run into a strange issue with my Solr installation. I'm running queries that are sorting by a DateField field but from time to time, I'm seeing individual records very much out of order. What's more, they appear on multiple pages of my result set. Let me give an example. Starting with a basic

Showcase: Facetted Search for Wine using Solr

2009-09-24 Thread marian.steinbach
Hello everybody! The purpose of this mail is to say "thank you" to the creators of Solr and to the community that supports it. We released our first project using Solr several weeks ago, after having tested Solr for several months. The project I'm talking about is a product search for an online

Re: Finding near duplicates which searching Documents

2009-09-24 Thread Grant Ingersoll
On Sep 23, 2009, at 2:55 PM, Jason Rutherglen wrote: I think don't this handle near duplicates which would require some of the methods mentioned recently on the Mahout list. It's pluggable and I believe the TextProfileSignature is a fuzzy implementation in Solr that was brought over from Nu

Re: Multivalue Field Cache

2009-09-24 Thread Grant Ingersoll
Have a look at UninvertedField.java. I think that might help. On Sep 23, 2009, at 2:35 PM, Amit Nithian wrote: Are there any good implementations of a field cache that will return all values of a multivalued field? I am in the process of writing one for my immediate needs but I was wonderin

Re: define index at search time

2009-09-24 Thread DHast
well after looking at http://wiki.apache.org/solr/CoreAdmin perhaps multiple cores is what i want, DHast wrote: > > No, I am talking about having multiple indexes, i want to send the index > name to the searcher so it will search that index, rather than use the one > defined in the schema/solrc

Re: define index at search time

2009-09-24 Thread DHast
No, I am talking about having multiple indexes, i want to send the index name to the searcher so it will search that index, rather than use the one defined in the schema/solrconfig. nothing t do with multiple cores, i mean different indexes entirely with completely different content. Avlesh S

Re: Can solr build on top of HBase

2009-09-24 Thread Grant Ingersoll
I don't think using HDFS or HBase will perform for this kind of thing at all. If you are that large, you should look into distributing your index into shards and using Solr's distributed search capabilities. -Grant On Sep 24, 2009, at 3:25 AM, 梁景明 wrote: hi, thanks, and now i can index dat

Re: Can solr build on top of HBase

2009-09-24 Thread 梁景明
hi, thanks, and now i can index data from hbase to the solr server using nutch core. but the indexdata will be local storage,that 's what i worry about,to be too large in local. MountableHDFS i never use it ,i am not sure weather solr can write the index into HDFS,i doubt it can work without imple