Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-02 Thread Yonik Seeley
On Fri, Oct 1, 2010 at 5:42 PM, Renee Sun wrote: > Hi Yonik, > > I attached the solrconfig.xml to you in previous post, and we do have > firstSearch and newSearch hook ups. > > I commented them out, all 130 cores loaded up in 1 minute, same as in solr > 1.3.  total memory took about 1GB. Whereas i

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-01 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 4:52 PM, Renee Sun wrote: >  - do you have any warming queries configured? > > no, all autowarmingcount are set to 0 for all caches Any static warming requests though (newSearcher / firstSearcher hooks in solrconfig.xml)? Is anything at all querying these cores while y

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 10:41 AM, Renee Sun wrote: > > Hi - > I posted this problem but no response, I guess I need to post this in the > Solr-User forum. Hopefully you will help me on this. > > We were running Solr 1.3 for long time, with 130 cores. Just upgrade to Solr > 1.4, then when we start

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 1:48 PM, Yonik Seeley wrote: > Dynamic field types.  You can configure it such that anything ending > with _latlon is of type LatLonType. > Perhaps we should do this in the example schema. Looks like we already have it: So you should be able to add stuff li

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 1:40 PM, webdev1977 wrote: > Or.. do you mean each field must have a unique name, but both be of type > latLon(solr.LatLonType). > x,y > x,y Yes. > If the statement directly above is true (I hope that it is not), how does > one dynamically create fields when adding geota

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 1:09 PM, webdev1977 wrote: > 1.  I noticed that it said that the type of LatLongType can not be > mulitvalued. Does that mean that I can not have multiple lat/lon values for > one document. That means that if you want to have multiple points per document, each point must b

Re: Queries, Functions, and Params

2010-09-29 Thread Yonik Seeley
On Tue, Sep 28, 2010 at 6:08 PM, Robert Thayer wrote: > On the http://wiki.apache.org/solr/FunctionQuery page, the following query > function is listed: > > q={!func}add($v1,$v2)&v1=sqrt(popularity)&v2=100.0 > > When run against the default solr instance, server returns the error(400): > "undefi

Re: Conditional Function Queries

2010-09-28 Thread Yonik Seeley
On Tue, Sep 28, 2010 at 11:33 AM, Jan Høydahl / Cominvent wrote: > Have anyone written any conditional functions yet for use in Function Queries? Nope - but it makes sense and has been on my list of things to do for a long time. -Y http://lucenerevolution.org Lucene/Solr Conference, Boston Oct

Re: urgent SOLR query server request hangs

2010-09-27 Thread Yonik Seeley
On Mon, Sep 27, 2010 at 11:09 AM, Bharat Jain wrote: >   We are running into issues with SOLR queries. Our solr queries just hang. Are you perhaps using distributed search and accidentally set up an infinite loop? Do *not* configure a default "shards" param on your /select handler. Other than th

Re: bi-grams for common terms - any analyzers do that?

2010-09-25 Thread Yonik Seeley
On Sat, Sep 25, 2010 at 8:21 PM, Jonathan Rochkind wrote: > Huh, okay, I didn't know that #2 happened at all. Can you explain or point me > to documentation to explain when it happens?  I'm afraid I'm having trouble > understanding <<  if the analyzer returns more than one position back from a

Re: matches in result grouping

2010-09-25 Thread Yonik Seeley
On Sat, Sep 25, 2010 at 2:07 AM, Koji Sekiguchi wrote: > Thanks Yonik for the explanation. > One more question. I think SearchGroupDocs.matches is unused > (I think TopDocsCollector.totalHits is used for displaying numFound > in each group). > Will it be used in the future for some reasons (if so

Re: AbstractMethodError with lucid KStem

2010-09-24 Thread Yonik Seeley
On Fri, Sep 24, 2010 at 9:39 AM, Bernd Fehling wrote: > I tried using lucid KStem with solr trunk version but get AbstractMethodError. That hasn't been ported to trunk yet. -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8

Re: Range query not working

2010-09-23 Thread Yonik Seeley
On Thu, Sep 23, 2010 at 5:44 PM, Jonathan Rochkind wrote: > The field type in a standard schema.xml that's defined as "integer" is NOT > sortable. Right - before 1.4. There is no "integer" field type in 1.4 and beyond in the example schema. > You can not sort on this and get what you want. (Wha

Re: Range query not working

2010-09-23 Thread Yonik Seeley
On Thu, Sep 23, 2010 at 4:30 PM, PeterKerk wrote: > I have this in my query: > &q=*:*&facet.query=location_rating_total:[3 TO 100] > > And this document: > > - > > 1.0 > 1 > 2 > > > But still my total results equals 6 (total population) and not 0 as I would > expect > > Why? facet.query will

Re: Searches with a period (.) in the query

2010-09-23 Thread Yonik Seeley
On Wed, Sep 22, 2010 at 8:13 PM, Siddharth Powar wrote: > I am getting some weird output upon searching in solr. For certain searches > that have a period in the search term (e.g: q=ab.xyz) solr returns the > results perfectly, but for some other searches (e.g: q=ab.pqr) solr would > return 0 resu

Re: matches in result grouping

2010-09-23 Thread Yonik Seeley
2010/9/23 Koji Sekiguchi : >  (10/09/23 18:14), Koji Sekiguchi wrote: >>  I'm using recent committed field collapsing / result grouping >> feature in trunk. >> >> I'm confusing matches parameter in the result at the second >> sample output of Wiki: >> >> http://wiki.apache.org/solr/FieldCollapsing#

Re: multiple spatial values

2010-09-21 Thread Yonik Seeley
On Tue, Sep 21, 2010 at 12:12 PM, dan sutton wrote: > I was looking at the LatLonType and how it might represent multiple lon/lat > values ... it looks to me like the lat would go in {latlongfield}_0_LatLon > and the long in {latlongfield}_1_LatLon ... how then if we have multiple > lat/long point

Re: into

2010-09-17 Thread Yonik Seeley
On Fri, Sep 17, 2010 at 4:12 PM, facholi wrote: > > Hi, > > I would like a json result like that: > > { >   id:2342, >   name:"Abracadabra", >   metadatas: [ >      {type:"tag", name:"tutorial"}, >      {type:"value", name:"2323.434/434"}, >   ] > } Do you mean JSON with the tags not quoted (that

Re: Version stability [was: svn branch issues]

2010-09-17 Thread Yonik Seeley
On Fri, Sep 17, 2010 at 10:46 AM, Mark Miller wrote: > I agree it's mainly API wise, but there are other issues - largely due > to Lucene right now - consider the bugs that have been dug up this year > on the 4.x line because flex has been such a large rewrite deep in > Lucene. We wouldn't do flex

Re: Version stability [was: svn branch issues]

2010-09-17 Thread Yonik Seeley
I think we aim for a "stable" trunk (4.0-dev) too, as we always have (in the functional sense... i.e. operate correctly, don't crash, etc). The stability is more a reference to API stability - the Java APIs are much more likely to change on trunk. Solr's *external* APIs are much less likely to ch

Re: SOLR interface with PHP using javabin?

2010-09-16 Thread Yonik Seeley
On Thu, Sep 16, 2010 at 2:30 PM, onlinespend...@gmail.com wrote: >  I am planning on creating a website that has some SOLR search capabilities > for the users, and was also planning on using PHP for the server-side > scripting. > > My goal is to find the most efficient way to submit search queries

Re: Null Pointer Exception while indexing

2010-09-16 Thread Yonik Seeley
On Wed, Sep 15, 2010 at 2:01 PM, andrewdps wrote: > I still get the same error when I try to index the mrc file... If you get the exact same error, then you are still using GCJ. When you type "java" it's probably going to GCJ because of your path (i.e. change it or directly specify the path to th

Re: Null Pointer Exception while indexing

2010-09-15 Thread Yonik Seeley
On Wed, Sep 15, 2010 at 1:12 PM, andrewdps wrote: > > What could be possible error for > > 14-Sep-10 4:28:47 PM org.apache.solr.common.SolrException log > SEVERE: java.util.concurrent.ExecutionException: > java.lang.NullPointerException >   at java.util.concurrent.FutureTask$Sync.innerGet(libgcj.s

Re: Solr returning irrelevant results

2010-09-15 Thread Yonik Seeley
On Wed, Sep 15, 2010 at 11:29 AM, Nguyen, Vincent (CDC/OSELS/PHITPO) (CTR) wrote: > I was running a query on the word "mining" and got results from > documents that have nothing to do with mining.  I got results with a > score of 0.2997284 and less.  It looks like Solr was querying the > dsm.fullt

Re: solr.DateField: org.apache.solr.common.SolrException: Error while creating field

2010-09-14 Thread Yonik Seeley
On Tue, Sep 14, 2010 at 4:54 PM, h00kpub...@gmail.com wrote: > SEVERE: org.apache.solr.common.SolrException: Error while creating field > 'metadata_last_modified{type=date,properties=indexed,stored,omitNorms}' from > value '2010-09-14T22:29:24+0200' Different timezones are currently not allowed -

Re: Facet Field Value truncation

2010-09-14 Thread Yonik Seeley
On Tue, Sep 14, 2010 at 3:35 PM, Niall O'Connor wrote: > Has anyone come across a situation where they have seen their facet field > values wrap into a new facet entry when the value exceeds 256 characters? Yes, for indexed string fields, there currently is a limit of 256 chars per token. It's b

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

2010-09-10 Thread Yonik Seeley
On Fri, Sep 10, 2010 at 7:21 PM, Ron Mayer wrote: > Ron Mayer wrote: > Yes, looks good now. > Thanks! Great, thanks for the report! -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8

Re: SEVERE: java.io.IOException: The specified network name is no longer available

2010-09-10 Thread Yonik Seeley
On Fri, Sep 10, 2010 at 2:12 PM, brian519 wrote: > Once we see the error, it is persistent.  Restarting Tomcat makes the error > stop.  This is happening across a variety of deployments and networks, so I > don't think there is an actual network problem.  Many other apps operate > fine on the same

Re: solr / lucene engineering positions in Boston, MA USA @ the Echo Nest

2010-09-10 Thread Yonik Seeley
On Fri, Sep 10, 2010 at 9:18 AM, Brian Whitman wrote: > Hi all, brief message to let you know that we're in heavy hire mode at the > Echo Nest. As many of you know we are very heavy solr/lucene users (~1bn > documents across many many servers) and a lot of our staff have been working > with and co

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-09 Thread Yonik Seeley
On Thu, Sep 9, 2010 at 11:51 AM, Grant Ingersoll wrote: > On Sep 6, 2010, at 10:41 AM, Yonik Seeley wrote: >> For SolrCloud, I don't think we'll end up using consistent hashing - >> we don't need it (although some of the concepts may still be useful). > > Can y

Re: Solr, c/s type ?

2010-09-09 Thread Yonik Seeley
On Thu, Sep 9, 2010 at 1:20 AM, Jonathan Rochkind wrote: > You _could_ use SolrJ with EmbeddedSolrServer.  But personally I wouldn't > unless there's a reason to.  There's no automatic reason not to use the > ordinary Solr HTTP api, even for an in-house application which is not a web > applicat

[ANN] Webinar, Sep 15: Mastering the Power of Faceted Search

2010-09-08 Thread Yonik Seeley
Folks, here's an upcoming Solr webinar sponsored by my employer. It's Hoss on faceting, so it should be good! -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 --- Webinar Details Join us for a free webc

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

2010-09-08 Thread Yonik Seeley
? Can you try trunk again now? -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 On Wed, Sep 8, 2010 at 6:28 PM, Ron Mayer wrote: > Yonik Seeley wrote: >> On Tue, Sep 7, 2010 at 8:31 PM, Ron Mayer wrote: >>> Short summary: >>>  * Mixing

Re: Re: Invariants on a specific fq value

2010-09-08 Thread Yonik Seeley
2010 at 1:32 PM, Markus Jelsma wrote: > Interesting! I haven't met the appends method before and i'll be sure to give > it a try tomorrow. Try, the wiki [1] is not very clear on what it really does. Here's a comment from the example solrconfig.xml: -Yonik http://lucenerevolution.org Luce

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

2010-09-08 Thread Yonik Seeley
On Tue, Sep 7, 2010 at 8:31 PM, Ron Mayer wrote: > Short summary: >  * Mixing Facets and Shards give me a NullPointerException >    when not all docs have all facets. https://issues.apache.org/jira/browse/SOLR-2110 I believe the underlying real issue stemmed from your use of a complex key "invol

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

2010-09-07 Thread Yonik Seeley
Thanks for the report Ron, can you open a JIRA issue? What version of Solr is this? -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 On Tue, Sep 7, 2010 at 8:31 PM, Ron Mayer wrote: > Short summary: >  * Mixing Facets and Shards give me a NullPointerException >    when

Re: How to retrieve the full corpus

2010-09-06 Thread Yonik Seeley
On Mon, Sep 6, 2010 at 10:52 AM, Roland Villemoes wrote: > How can I retrieve all words from a Solr core? > I need a list of all the words and how often they occur in the index. http://wiki.apache.org/solr/TermsComponent It doesn't currently stream though, so requesting *all* at once might take

Re: How to enable Unicode Support in Solr

2010-09-06 Thread Yonik Seeley
On Mon, Sep 6, 2010 at 10:30 AM, Walter Underwood wrote: > On Sep 6, 2010, at 1:49 AM, Lance Norskog wrote: > >> 1) The XML file must include the UTF-8 encoding metadata in the first line. > > If it requires that, it isn't a legal XML parser. The encoding declaration is > optional and it defaults

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Yonik Seeley
On Mon, Sep 6, 2010 at 10:18 AM, MitchK wrote: [...consistent hashing...] > But it doesn't solve the problem at all, correct me if I am wrong, but: If > you add a new server, let's call him IP3-1, and IP3-1 is nearer to the > current ressource X, than doc x will be indexed at IP3-1 - even if IP2-1

Re: anyone use hadoop+solr?

2010-09-06 Thread Yonik Seeley
On Mon, Sep 6, 2010 at 9:47 AM, MitchK wrote: > are there any discussions about SolrCloud-indexing? Not recently - personally I've been sidetracked by other stuff. Mapping docs to shards is the easy part... take a hash of the id, and then I imagine the shard id (the label for the index) can just

Re: anyone use hadoop+solr?

2010-09-06 Thread Yonik Seeley
On Mon, Sep 6, 2010 at 8:37 AM, MitchK wrote: > 10 % numShards(10) ->  1 -> doc 10 will be indexed at shard 1... and what > about the older version at shard 2? I am no expert when it comes to > cloudComputing and the other stuff. > If you can point me to one or another reference where I can read a

Re: false matches with ReversedWildcardFilterFactory

2010-09-03 Thread Yonik Seeley
On Thu, Sep 2, 2010 at 1:10 PM, Landon Kuhn wrote: > Hello, I am using the ReversedWildcardFilterFactory, and I am > wondering if there is a way to prevent false matches when a query > token matches the reversed indexed token. For instance, the query > *zemog* matches documents that contain Gomez.

Re: Solr searching performance issues, using large documents (now 1MB documents)

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 2:34 PM, Peter Spam wrote: > This is a very small number of documents (7000), so I am surprised Solr is > having such a hard time with it!! > > I do facet on 3 terms. > > Subsequent "hello" searches are faster, but still well over a second.  This > is a very fast Mac Pro,

Re: Slow facet sorting - lex vs count

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 7:22 AM, Eric Grobler wrote: > Hi Solr experts, > > There is a huge difference doing facet sorting on lex vs count > The strange thing is that count sorting is fast when setting a small limit. > I realize I can do sorting in the client, but I am just curious why this is. >

Re: Slow facet sorting - lex vs count

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 2:50 PM, Yonik Seeley wrote: > On Wed, Aug 25, 2010 at 10:55 AM, Eric Grobler > wrote: >> Thanks for the technical explanation. >> I will in general try to use lex and sort by count in the client if there >> are not too many rows. > > I j

Re: Slow facet sorting - lex vs count

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 10:55 AM, Eric Grobler wrote: > Thanks for the technical explanation. > I will in general try to use lex and sort by count in the client if there > are not too many rows. I just developed a patch that may help this scenario: https://issues.apache.org/jira/browse/SOLR-2089

Re: Solr searching performance issues, using large documents (now 1MB documents)

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 11:29 AM, Peter Spam wrote: > So, I went through all the effort to break my documents into max 1 MB chunks, > and searching for hello still takes over 40 seconds (searching across 7433 > documents): > >        8 results (41980 ms) > > What is going on???  (scroll down for

Re: Slow facet sorting - lex vs count

2010-08-25 Thread Yonik Seeley
->string conversions. -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 > Regards > Eric > > On Wed, Aug 25, 2010 at 3:28 PM, Yonik Seeley > wrote: >> >> On Wed, Aug 25, 2010 at 7:22 AM, Eric Grobler >> wrote: >> > There is

Re: Regd WSTX EOFException

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 6:41 AM, Pooja Verlani wrote: > Hi, > Sometimes while indexing to solr, I am getting  the following exception. > "com.ctc.wstx.exc.WstxEOFException: Unexpected end of input block in end tag" > I think its some configuration issue. Kindly suggest. > > I have a solr working w

Re: Slow facet sorting - lex vs count

2010-08-25 Thread Yonik Seeley
On Wed, Aug 25, 2010 at 7:22 AM, Eric Grobler wrote: > There is a huge difference doing facet sorting on lex vs count > The strange thing is that count sorting is fast when setting a small limit. > I realize I can do sorting in the client, but I am just curious why this is. There are a lot of opt

Re: Proper Escaping of Ampersands

2010-08-23 Thread Yonik Seeley
ted - (The document could be found when I disabled the stopword > filter, details later in this mail...) > > On 20.08.2010 16:57, Yonik Seeley wrote >> On Thu, Aug 19, 2010 at 11:33 AM, Nikolas Tautenhahn >> wrote: >>> But when I search for q=at%26s (=at&s), I ge

Re: Fun with Spatial (Haversine formula)

2010-08-20 Thread Yonik Seeley
On Fri, Aug 20, 2010 at 11:05 PM, Lance Norskog wrote: > are latitudes equidistant on the surface of the sphere? Yes - each degree of latitude is ~69 miles. There is also a slight variation due to the earth not being a perfect sphere. -Yonik http://lucenerevolution.org Lucene/Solr Conference, B

Re: Fun with Spatial (Haversine formula)

2010-08-20 Thread Yonik Seeley
Lance, have you figured out what the issue is? Anyone know if this is a haversine limitation, or a bug? -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 On Wed, Aug 18, 2010 at 1:54 AM, Lance Norskog wrote: > The Haversine formula in o.a.s.s.f.d.DistanceUtils.java gives

Re: Deadlock in Server?

2010-08-20 Thread Yonik Seeley
This sounds like perhaps a major merge was triggered. You could do a nightly optimize - which will take just as long, but you control when it happens. The other option is to prevent too big of segments being greated (at the expense of search speed) with options such as maxMergeDocs. -Yonik http:/

Re: Proper Escaping of Ampersands

2010-08-20 Thread Yonik Seeley
On Thu, Aug 19, 2010 at 11:33 AM, Nikolas Tautenhahn wrote: > Hi, > > I have a problem with, for example, company names like "AT&S". > A Job is sending data to the solr 1.4 (also tested it with 1.4.1) index > via python in XML, everything is escaped properly ("&" becomes "&"). > > When I search fo

Re: tire fields and sortMissingLast

2010-08-19 Thread Yonik Seeley
On Thu, Aug 19, 2010 at 12:28 PM, harish.agarwal wrote: > Just curious if there has been any progress on implementing sortMissingLast > on TrieFields? Not yet - that info is not available from the lucene FieldCache. -Yonik http://www.lucidimagination.com

Re: queryResultCache has no hits for date boost function

2010-08-18 Thread Yonik Seeley
On Wed, Aug 18, 2010 at 4:34 PM, Peter Karich wrote: > Thanks a lot Yonik! Rounding makes sense. > Is there a date math for the 'LAST_COMMIT'? No - but it's an interesting idea! -Yonik http://www.lucidimagination.com

Re: Jetty rerturning HTTP error code 413

2010-08-18 Thread Yonik Seeley
Yep, or you can submit the query via POST, which has a much bigger limit on the size of the body. -Yonik http://www.lucidimagination.com On Wed, Aug 18, 2010 at 3:58 PM, didier deshommes wrote: > Hi Alexandre, > Have you tried setting a higher headerBufferSize?  Look in > etc/jetty.xml and sear

Re: sort order of "missing" items

2010-08-18 Thread Yonik Seeley
On Tue, Aug 17, 2010 at 4:10 PM, Brad Dewar wrote: > When items are sorted, are all the docs with the sort field missing > considered "tied" in terms of their sort order, or are they "indeterminate", > or do they have some arbitrary order imposed on them (e.g. _docid_)? If it's a numeric field,

Re: queryResultCache has no hits for date boost function

2010-08-18 Thread Yonik Seeley
On Tue, Aug 17, 2010 at 6:29 PM, Peter Karich wrote: > my queryResultCache has no hits. But if I am removing one line from the > bf section in my dismax handler all is fine. Here is the line: > recip(ms(NOW,date),3.16e-11,1,1) NOW has millisecond resolution, so it's actually a different query eac

Re: diacritics on query string

2010-08-15 Thread Yonik Seeley
Update: it looks like this (what I consider buggy) behavior is common to both firefox and IE. Both correctly encode the path part of the URL, but neither encode the query string part in UTF-8 (I believe for back compat with old buggy websites). Chrome does use UTF-8 for both. It's easy to verify

Re: diacritics on query string

2010-08-15 Thread Yonik Seeley
On Fri, Aug 13, 2010 at 3:20 AM, Andrea Gazzarini wrote: >  Hi, > I have a problem regarding a diacritic character on my query string : > > *q=intertestualità > * > which is encoded in > > *q=intertestualit%E0 The correct encoding is q=intertestualit%C3%A0 But I can see how you may have obtained

Re: edismax pf2 and ps

2010-08-13 Thread Yonik Seeley
On Fri, Aug 13, 2010 at 2:38 PM, Ron Mayer wrote: > Yonik Seeley wrote: >> Perhaps a ps2 parameter to match pf2? > > That might be nice. > > I could try to put together such a patch if people were interested. > > One more thing I've been contemplating is if my res

Re: edismax pf2 and ps

2010-08-13 Thread Yonik Seeley
Perhaps a ps2 parameter to match pf2? -Yonik http://www.lucidimagination.com On Fri, Aug 13, 2010 at 2:11 PM, Ron Mayer wrote: > Jayendra Patil wrote: >> We pretty much had the same issue, ended up customizing the ExtendedDismax >> code. >> >> In your case its just a change of a single line >>  

Re: how to update solr to older 1.5 builds instead of to trunk

2010-08-12 Thread Yonik Seeley
On Thu, Aug 12, 2010 at 12:24 PM, solr-user wrote: > Thanks Yonik but > http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/solr/CHANGES.txt > says that the lucene index has changed Right - but it will be able to read your older index. Do you need Solr 1.4 to be able to read the new ind

Re: how to update solr to older 1.5 builds instead of to trunk

2010-08-12 Thread Yonik Seeley
Another option is the 3x branch - that should still be able to read indexes from Solr 1.4/Lucene 2.9 I personally don't expect a 1.5 release to ever materialize. There will eventually be a Lucene/Solr 3.1 release off of the 3x branch, and a Lucene/Solr 4.0 release off of trunk. -Yonik http://www.l

Re: Solr 1.4 - stats page slow

2010-08-11 Thread Yonik Seeley
FYI, I opened https://issues.apache.org/jira/browse/SOLR-2036 for this. -Yonik http://www.lucidimagination.com On Tue, Aug 10, 2010 at 8:35 PM, entdeveloper wrote: > > Apologies if this was resolved, but we just deployed Solr 1.4.1 and the stats > page takes over a minute to load for us as well

Re: Scoring on multi-valued fields

2010-08-03 Thread Yonik Seeley
On Tue, Aug 3, 2010 at 3:16 PM, oleg.gnatovskiy wrote: > > Sorry guess I messed up my example query. The query should look like this: > > name:pizza AND id:(10 OR 20 OR 30) > > Thus if I do name:pizza^10 AND id:(10 OR 20 OR 30)^0 wouldn't a document > that has all the ids (10,20, and 30) still com

Re: Scoring on multi-valued fields

2010-08-03 Thread Yonik Seeley
On Tue, Aug 3, 2010 at 2:42 PM, oleg.gnatovskiy wrote: > > Oh sorry guys, I didn't correctly submit my original post to the mailing > list. The original message was this: > " > Hello all. We are having some trouble with queries similar to the type shown > below: > > name: pizza OR (id:10 OR id:20

Re: Good list of English words that get "butchered" by Porter Stemmer

2010-07-30 Thread Yonik Seeley
On Fri, Jul 30, 2010 at 4:41 PM, Otis Gospodnetic wrote: > I'm looking for a list of English  words that, when stemmed by Porter stemmer, > end up in the same stem as  some similar, but unrelated words.  Below are some > examples: > > # this gets stemmed to "iron", so if you search for "ironic", y

Re: myField:value does not seem to work

2010-07-29 Thread Yonik Seeley
Is membername an indexed field in the schema, and was it populated with something that would match "Khai"? If so, what is the fieldType in the schema for the membername field? -Yonik http://www.lucidimagination.com On Thu, Jul 29, 2010 at 3:17 PM, Khai Doan wrote: > Hello, > > My name is Khai.

Re: Querying throws java.util.ArrayList.RangeCheck

2010-07-29 Thread Yonik Seeley
On Thu, Jul 29, 2010 at 6:37 AM, Michael McCandless wrote: > Can you run CheckIndex on the index and post the output? One of these days we need to get around to adding support for this in Solr's admin interface. http://issues.apache.org/jira/browse/SOLR-566 -Yonik http://www.lucidimagination.com

Re: Querying throws java.util.ArrayList.RangeCheck

2010-07-27 Thread Yonik Seeley
I haven't been able to reproduce anything... But if you guys are sure you're not running any custom code, then there's definitely seems to be a bug somewhere. Can anyone reproduce this in something you can share? -Yonik http://www.lucidimagination.com

Re: Querying throws java.util.ArrayList.RangeCheck

2010-07-26 Thread Yonik Seeley
Do you have any custom code, or is this stock solr (and which version, and what is the request)? -Yonik http://www.lucidimagination.com On Tue, Jul 27, 2010 at 12:30 AM, Manepalli, Kalyan wrote: > Hi, >   I am stuck at this weird problem during querying. While querying the solr > index I am get

Re: java "GC overhead limit exceeded"

2010-07-26 Thread Yonik Seeley
On Mon, Jul 26, 2010 at 7:17 PM, Jonathan Rochkind wrote: > I am now occasionally getting a Java "GC overhead limit exceeded" error in > my Solr. This may or may not be related to recently adding much better (and > more) warming querries. When memory gets tight, the JVM kicks of a garbage collect

Re: a bug of solr distributed search

2010-07-23 Thread Yonik Seeley
On Fri, Jul 23, 2010 at 2:40 PM, MitchK wrote: > That only works if the docs are exactly the same - they may not be. > Ahm, what? Why? If the uniqueID is the same, the docs *should* be the same, > don't they? Documents aren't supposed to be duplicated across shards... so the presence of multiple

Re: a bug of solr distributed search

2010-07-23 Thread Yonik Seeley
On Fri, Jul 23, 2010 at 2:23 PM, MitchK wrote: > why do we do not send the output of TermsComponent of every node in the > cluster to a Hadoop instance? > Since TermsComponent does the map-part of the map-reduce concept, Hadoop > only needs to reduce the stuff. Maybe we even do not need Hadoop for

Re: Solr 3.1 dev

2010-07-23 Thread Yonik Seeley
On Fri, Jul 23, 2010 at 9:33 AM, robert mena wrote: > Hi, > is there any wiki/url of the proposed changes or new features that we should > expect with this new release? You can see what has already gone in by looking at the appropriate CHANGES.txt in subversion. http://svn.apache.org/viewvc/luce

Re: Solr 3.1 dev

2010-07-23 Thread Yonik Seeley
On Fri, Jul 23, 2010 at 6:09 AM, Eric Grobler wrote: > I have a few questions :-) > > a) Will the next release of solr be 3.0 (instead of 1.5)? The next release will be 3.1 (matching the next lucene version off of the 3x branch). Trunk is 4.0-dev > b) How stable/mature is the current 3x version?

Re: a bug of solr distributed search

2010-07-22 Thread Yonik Seeley
As the comments suggest, it's not a bug, but just the best we can do for now since our priority queues don't support removal of arbitrary elements. I guess we could rebuild the current priority queue if we detect a duplicate, but that will have an obvious performance impact. Any other suggestions?

Re: limiting the total number of documents matched

2010-07-16 Thread Yonik Seeley
On Wed, Jul 14, 2010 at 5:46 PM, Paul wrote: > I thought of another way to do it, but I still have one thing I don't > know how to do. I could do the search without sorting for the 50th > page, then look at the relevancy score on the first item on that page, > then repeat the search, but add score

Re: range query on TrieLongField - strange result?

2010-07-15 Thread Yonik Seeley
Yikes... confirmed! Something is very wrong here. -Yonik http://www.lucidimagination.com On Thu, Jul 15, 2010 at 8:47 PM, Yonik Seeley wrote: > Hmmm, I'll try and duplicate. > > -Yonik > http://www.lucidimagination.com > > 2010/7/15 Koji Sekiguchi : >> I

Re: range query on TrieLongField - strange result?

2010-07-15 Thread Yonik Seeley
Hmmm, I'll try and duplicate. -Yonik http://www.lucidimagination.com 2010/7/15 Koji Sekiguchi : > I see different results between SortableLongField and TrieLongField > when I try same range query. > > This is the test data: > > > > ZERO > 0 > 0 > > > Long.MAX_VALUE-1000 > 9223372036854774807

Re: Nested Function Query Syntax

2010-07-15 Thread Yonik Seeley
On Thu, Jul 15, 2010 at 12:49 PM, Rodrigo Rezende wrote: > Yeah, it is redundant, but I am using that to use the solr query > response as input of a plugin function: > > http://localhost:8983/solr/articles.0/select/?q={!func}myFunction(query({!query > v='the query string here'})) This might be ea

Re: Nested Function Query Syntax

2010-07-15 Thread Yonik Seeley
On Thu, Jul 15, 2010 at 11:51 AM, Rodrigo Rezende wrote: > I solved the problem. > The correct syntax is: > > http://localhost:8983/solr/articles.0/select/?q={!func}query({!query > v='hello'})&fl=Document.title,score,&debugQuery=on query() causes a new QParser to be created. so does {!query}...

Re: Problem with Wildcard searches in Solr

2010-07-12 Thread Yonik Seeley
On Mon, Jul 12, 2010 at 4:39 PM, imranak wrote: > A general search like 'computer' returns results but 'com*er' doesn't return > any results. This is due to issues with wildcards and stemming. "computer" is indexed and searched as "comput"... but it's not generally possible to stem wildcarded ter

Re: Indexing slowdowns

2010-07-08 Thread Yonik Seeley
Hmm, did the default number of background merge threads change sometime recently? I seem to recall so, but I can't find a reference to it. -Yonik http://www.lucidimagination.com

Re: Modifications to AbstractSubTypeFieldType

2010-07-07 Thread Yonik Seeley
On Wed, Jul 7, 2010 at 8:15 AM, Grant Ingersoll wrote: > Originally, I had intended that it was just for one Field Sub Type, thinking > that if we ever wanted multiple sub types, that a new, separate class would > be needed Right - this was my original thinking too. AbstractSubTypeFieldType i

Re: Modifications to AbstractSubTypeFieldType

2010-07-02 Thread Yonik Seeley
On Fri, Jul 2, 2010 at 9:51 AM, Mark Allan wrote: [...] > The changes to AbstractSubTypeFieldType do not have any adverse effects on > the solr.PointType class, so I'd quite like to suggest it gets included in > the main solr source code.  Where can I send a patch for someone to evaluate > or shou

Re: ValueSource/Function questions

2010-07-01 Thread Yonik Seeley
On Thu, Jul 1, 2010 at 1:02 PM, Blargy wrote: > Can someone explain what the createWeight methods should do? Its primary function is to add Searcher context to anything that needs it (such as weighting a query). If you're not dealing with relevancy-type queries, value sources should just delegate

Re: SolrJ: BinaryRequestWriter with StreamingUpdateSolrServer

2010-07-01 Thread Yonik Seeley
There's an issue open for this: https://issues.apache.org/jira/browse/SOLR-1565 I'm not sure off the top of my head how much is involved in making it happen though. -Yonik http://www.lucidimagination.com On Thu, Jul 1, 2010 at 9:25 AM, Jan Høydahl / Cominvent wrote: > Hi, > > I had the impressio

Re: OOM on uninvert field request

2010-06-30 Thread Yonik Seeley
method=fc -Yonik http://www.lucidimagination.com > -Original Message- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley > Sent: Wednesday, June 30, 2010 1:38 PM > To: solr-user@lucene.apache.org > Subject: Re: OOM on uninvert field request > >

Re: REST calls

2010-06-30 Thread Yonik Seeley
On Wed, Jun 30, 2010 at 4:55 PM, Lance Norskog wrote: > Apparently this is not ReStFuL It is IMVHO insane. Patches welcome... -Yonik http://www.lucidimagination.com

Re: OOM on uninvert field request

2010-06-30 Thread Yonik Seeley
On Tue, Jun 29, 2010 at 7:32 PM, Robert Petersen wrote: > Hello I am trying to find the right max and min settings for Java 1.6 on 20GB > index with 8 million docs, running 1.6_018 JVM with solr 1.4, and am > currently have java set to an even 4GB (export JAVA_OPTS="-Xmx4096m > -Xms4096m") for

Re: REST calls

2010-06-30 Thread Yonik Seeley
Solr's APIs are described as "REST-like", and probably do qualify as "restful" the way the term is commonly used. I'm personally much more interested in making our APIs more powerful and easier to use, regardless of any REST purity tests. -Yonik http://www.lucidimagination.com

Re: How to wait for StreamingUpdateSolrServer to finish?

2010-06-29 Thread Yonik Seeley
On Tue, Jun 22, 2010 at 9:38 AM, Stephen Duncan Jr wrote: > I'm prototyping using StreamingUpdateSolrServer.  I want to send a commit > (or optimize) after I'm done adding all of my docs, rather than wait for the > autoCommit to kick in.  However, since StreamingUpdateSolrServer is > multi-threade

Re: Similarity

2010-06-24 Thread Yonik Seeley
On Thu, Jun 24, 2010 at 4:20 PM, Blargy wrote: > Yonik Seeley-2-2 wrote: >> >> Depends on the larger context of what you are trying to do. >> Do you still want the idf and length norm relevancy factors?  If not, >> use a filter, or boost the particular clause with 0.

Re: Similarity

2010-06-24 Thread Yonik Seeley
On Thu, Jun 24, 2010 at 3:17 PM, Blargy wrote: > > Can someone explain how I can override the default behavior of the tf > contributing a higher score for documents with repeated words? > > For example: > > Query: "foo" > Doc1: "foo bar" score 1.0 > Doc2: "foo foo bar" score 1.1 > > Doc2 contains

Re: about function query

2010-06-23 Thread Yonik Seeley
See http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents for more info on how to do a multiplicative boost. -Yonik http://www.lucidimagination.com On Tue, Jun 22, 2010 at 11:13 PM, Li Li wrote: > I want to integrate document's timestamp into scoring of searc

Re: LocalParams, quotes, bug?

2010-06-16 Thread Yonik Seeley
On Wed, Jun 16, 2010 at 3:27 PM, Jonathan Rochkind wrote: > {!dismax qf=$some_qf}   => no problem, and debugQuery reveals it is indeed > using the qf I desire. > > {!dismax qf='$some_qf'}  => Solr throws "undefined field $some_qf". > > Is this a bug in Solr? Nope, it's by design. Parameter refere

<    7   8   9   10   11   12   13   14   15   16   >