Re: no response

2010-07-16 Thread Peter Karich
satya, sorry for being a bit harsh, but did you read the answer of Erick in the 'problem with storing??'-thread at all? just ask the same question again (and not answering old questions) might be a bit disappointing for people who want to help you. just my side-note ... Regards, Peter. > Hi all

Re: no response

2010-07-16 Thread satya swaroop
hi, i am sorry the mail u sent was in sent mail... I didnt look it I am going to check now.. I will definetely tell u the entire thing regards, satya

Re: Query help

2010-07-16 Thread Alejandro Marqués Rodríguez
I can't see a way of retrieving five results from one type and five from another in a single query. The only way I can think about that would have a similar behaviour would be: ?q=ContentType:(News+OR+Analysis)&sort=DatePublished+desc&start=0&rows=10 This way you'll have the first 10 results bein

Re: Solr Best Version

2010-07-16 Thread Tommaso Teofili
Hi all, I read in a previous thread [1] that also the branch3.x version could be a good choice, but I don't know what differences exist at the moment between the two versions and how stable branch3.x is. Maybe someone else could point these things out. My 0.0002 cents. Tommaso [1] : http://markmai

Finding distinct unique IDs in documents returned by fq -- Urgent Help Req

2010-07-16 Thread Ninad Raut
Hi, I have a scenario in which I have to find count of distinct unique IDs present in a field (rootId field in my case) for a particular query. I require this for pagination purpose. Is there a way in Solr to do something like this we do in SQL: select count(distinct(rootId)) from table where (

Re: Finding distinct unique IDs in documents returned by fq -- Urgent Help Req

2010-07-16 Thread Rebecca Watson
hi, would faceting work? http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Faceted-Search-Solr if you have a field for rootId that is multivalued + facet on it -- you'll get value+count pairs back (top 100 i think by default) bec :) On 16 July 2010 16:07, Ninad Raut wrot

Spatial Search - Best choice (if any)?

2010-07-16 Thread Saïd Radhouani
Hi, Using Solr 1.4, I'm now working on adding spatial search options, such as distance-based sorting, Bounding-box filter, etc. To the best of my knowledge, there are three possible points we can start from: 1. The http://blog.jteam.nl/2009/08/03/geo-location-search-with-solr-and-lucene/ 2. Th

Re: Securing Solr 1.4 in a glassfish container AS NEW THREAD

2010-07-16 Thread Bilgin Ibryam
Hi Jon, SolrJ (CommonsHttpSolrServer) internally uses apache http client to connect to solr. You can check there for some documentation. I secured solr also with BASIC auth-method and use the following snippet to access it from solrJ: //set username and password ((CommonsHttpSolrServe

Re: Custom comparator

2010-07-16 Thread dan sutton
Apologies I didn't make the requirement clear. I need to keep the best N documents - set A (chosen by some criteria - call them sponsored docs) in front of the natural scoring docs - set B so that I return (A,B). The set A docs need to all score above 1% of maxScore in B else they join the B set

Re: problem with storing??

2010-07-16 Thread satya swaroop
hi, I checked out the admin page and it is indexing for others.In the log files i dont get anything when i send the documents. I checked out the log in catalina(tomcat). I changed the dismax handler from q=*:* to q= . I atleast get the response when i send pdf/html files but dont even get for

Re: Using stored terms for faceting

2010-07-16 Thread Erik Hatcher
This is simple faceting, doesn't even have to be a multi-valued field. Just index your description field with the desired stop word removal and other analysis that you want done, and &facet.field=description Erik On Jul 15, 2010, at 3:26 AM, Peter Karich wrote: Dear Hoss, I wi

Getting facets count on multiple fields by doing a "Group By"

2010-07-16 Thread Rajinimaski
This is Condition example: I have employee with name Rajani and her ID 2 And another employee name also Rajani ID 3 and another also Rajani with id 4 When i make a facet on name:rajani and facets on ID ,Results will be like Name = rajani ID=1 ID= 2 ID =3 What I needed is Like (Name

DIH context fails to store in global scope

2010-07-16 Thread Marc Emery
Hi, I am writing a EventListener that put some data in the content on import start: ctx.setSessionAttribute( DOCTYPE_MAPPING, docTypeMap, Context.SCOPE_GLOBAL ); but it doesn't seem to work. looking at the trunk code of ContextImpl.java the globalSession is not called: private void putVal(St

RE: how to eliminating scoring from a query?

2010-07-16 Thread oferiko
that's actually what i already had in mind, just wasn't sure that specifying the sort order of indextime eliminates the work of scoring. -- View this message in context: http://lucene.472066.n3.nabble.com/how-to-eliminating-scoring-from-a-query-tp968581p972325.html Sent from the Solr - User mail

documents with known relevancy

2010-07-16 Thread fiedzia
I want to know if what i am trying to achieve is doable using solr. I have some objects that have tags assigned. Tag is as string with weight attached, so whole document that i want to index can look like that: { id: 123, tags: { tag1: 0.01, tag2: 0.3, ...

Re: Finding distinct unique IDs in documents returned by fq -- Urgent Help Req

2010-07-16 Thread kenf_nc
It may just be a mis-wording, but if you do distinct on 'unique' IDs, the count should be the same as response.numFound. But if you didn't mean 'unique', just count of some field in the results, Rebecca is correct, facets should do the job. Something like: ?q=content:query+text&facet=on&facet.fie

Re: documents with known relevancy

2010-07-16 Thread Peter Karich
Hi, Why do you need the weight for the tags? you could index it this way: { id: 123 tag:'tag1' weight: 0.01 uniqueKey: combine(id, tag) } { id: 123 tag:'tag2' weight: 0.3 uniqueKey: combine(id, tag) } and specify the query-time boost with the help of the weight. Retri

Re: documents with known relevancy

2010-07-16 Thread fiedzia
Peter Karich wrote: > > Hi, > > Why do you need the weight for the tags? > The only reason to include weights is to sort results by weights. So if there are multiple documents containing given tag, i want them to be sorted by weight. Also i would like to be able to seach by multiple tags at

Building maven artifacts

2010-07-16 Thread Pavel Minchenkov
Hi, I'm trying to run ant task "generate-maven-artifacts" in lucene-solr build.xml file. But getting this error: /home/chardex/lucene/dev/lucene/common-build.xml:312: Error deploying artifact 'org.apache.lucene:lucene-core:jar': Error deploying artifact: File /home/chardex/lucene/dev/lucene/build/$

Re: documents with known relevancy

2010-07-16 Thread fiedzia
I came up with another idea, which seem to do what i want. Any comments about better solutions or improving efficiency are welcome: for each document create multivalue text field "tags" with all tags, and multiple dynamic fields for each tag containging value, so we have: { id: 123 tags: tag1

RE: Securing Solr 1.4 in a glassfish container AS NEW THREAD

2010-07-16 Thread Sharp, Jonathan
Hi Bilgin, Thanks for the snippet -- that helps a lot. -Jon -Original Message- From: Bilgin Ibryam [mailto:bibr...@gmail.com] Sent: Friday, July 16, 2010 1:31 AM To: solr-user@lucene.apache.org Subject: Re: Securing Solr 1.4 in a glassfish container AS NEW THREAD Hi Jon, SolrJ (Common

Re: documents with known relevancy

2010-07-16 Thread Dennis Gearon
Seems to me that you are doing externally to Solr what you could be doing internally. If you had ONE field as and weighted those in your SOLR query, that is how I am guessing it is usually done. Dennis Gearon Signature Warning EARTH has a Right To Life, otherwise we all die.

Re: documents with known relevancy

2010-07-16 Thread Dennis Gearon
So does this mean that each document has a different weight for the same tag? Dennis Gearon Signature Warning EARTH has a Right To Life, otherwise we all die. Read 'Hot, Flat, and Crowded' Laugh at http://www.yert.com/film.php --- On Fri, 7/16/10, fiedzia wrote: > From: fie

Re: Spatial Search - Best choice (if any)?

2010-07-16 Thread Dennis Gearon
I hope that those who know will answer this. I am really interested in it also. TIA. Dennis Gearon Signature Warning EARTH has a Right To Life, otherwise we all die. Read 'Hot, Flat, and Crowded' Laugh at http://www.yert.com/film.php --- On Fri, 7/16/10, Saïd Radhouani wro

Re: Spatial Search - Best choice (if any)?

2010-07-16 Thread Dave Searle
I'm also just starting a project requiring spatial indexing so any info would be greatly appreciated. I had a quick look at the wiki last night and it appears solr has it built in in the latest version? Not sure if the patches need applying directly though My requirements are quite simple, I j

Re: documents with known relevancy

2010-07-16 Thread fiedzia
Dennis Gearon wrote: > > So does this mean that each document has a different weight for the same > tag? > Exactly. The weight is a weight of a given tag for specific document, not weight of the field as in weighted search. So one document may have tag1 with weight of 0.1, and another may have

Re: documents with known relevancy

2010-07-16 Thread fiedzia
Dennis Gearon wrote: > > Seems to me that you are doing externally to Solr what you could be doing > internally. If you had ONE field as and weighted those in your SOLR > query, that is how I am guessing it is usually done. > I guess i used confusing term for weight. The weight (value assigne

RE: documents with known relevancy

2010-07-16 Thread Jonathan Rochkind
> Exactly. The weight is a weight of a given tag for specific document, not > weight of the field as in weighted search. So one document may have tag1 > with weight of 0.1, and another may have the same tag1 with weight=0.8. I've never used it, but I think this is the use case that the Solr featur

Re: send to list

2010-07-16 Thread Mattmann, Chris A (388J)
Hi Joe, Take a look at the Cartesian Grid work from Patrick O'Leary here [1]. It's not fully integrated with Solr and they are moving away from it, but it'll give you a good idea of how to get started and to go about doing this... HTH, Chris [1] http://www.nsshutdown.com/projects/lucene/whitep

Re: Fwd: send to list

2010-07-16 Thread kenf_nc
If at all possible I like to do any processing work up front and not deal with extravagant queries. If your grid definitions don't change, or don't change often, just assign a cell number to each 100 square grid. Then in a pre-processing step assign the appropriate cell number to your document alo

Re:Re: How to speed up solr search speed

2010-07-16 Thread marship
Hi. Peter. Thanks for replying. >Hi Scott! > >> I am aware these cores on same server are interfering with each other. > >Thats not good. Try to use only one core per CPU. With more per CPU you >won't have any benefits over the single-core version, I think. I only have 2 servers, each CPU wi

indexing best practices

2010-07-16 Thread kenf_nc
I was curious if anyone has done work on finding what an optimal (or max) number of client processes are for indexing. That is, if I have the ability to spin up N number of processes that construct a POST to add/update a Solr document, is there a point at which the number of clients posting simult

Re: Tag generation

2010-07-16 Thread kenf_nc
Thanks for all the suggestions! I'm absorbing them as quickly as I can. -- View this message in context: http://lucene.472066.n3.nabble.com/Tag-generation-tp969888p973277.html Sent from the Solr - User mailing list archive at Nabble.com.

Re:Re:Re: How to speed up solr search speed

2010-07-16 Thread marship
Hi Tom Burton-West. Sorry looks my email ISP filtered out your replies. I checked web version of mailing list and saw your reply. My query string is always simple like "design", "principle of design", "tom" EG: URL: http://localhost:7550/solr/select/?q=design&version=2.2&start=0&rows=1

Re:indexing best practices

2010-07-16 Thread marship
Hi. I justed noticed when you add document to solr, turn the auto-commit flag off, after posting done, commit and optimize. The the speed is super fast. I was using 31 clients to post 31 solr cores at the same time. I think if you use 2 clients to post to same core, the question will be "how fa

Re: Re:Re: How to speed up solr search speed

2010-07-16 Thread Geert-Jan Brits
you mentioned that you have a lot of mem free, but your yetty containers only using between 40-60 mem. probably stating the obvious, but have you increased the -Xmx param like for instance: java -Xmx1024M -jar start.jar that way you're configuring the container to use a maximum of 1024 MB ram ins

Get both regular query and function query scores

2010-07-16 Thread Martynas Miliauskas
Hi, I am using a function query to tweak my regular query search score, so search query outputs regular query score modified by some function query. Is there a way to also obtain a score from regular query? Thanks!

Re:Re: How to speed up solr search speed

2010-07-16 Thread Dennis Gearon
Isn't it always one of these four? (from most likely to least likely, generally) Memory (as a ceiling limit) Disk Speed WebServer and it's code CPU. Memory and Disk are related, as swapping occurs between them. As long as memory is high enough, it becomes: Disk Speed WebServer and it's code CPU

Re: documents with known relevancy

2010-07-16 Thread Peter Karich
I didn't looked at payloads as mentioned by Jonathan, but another solution could be (similar to Dennis'): create a field 'tags' and then add the tag1 several times to it - depending on the weight. E.g. add it 10 times if the weight is 1.0 But add it only 2 times if the weight is 0.2 etc. Of cours

Re: documents with known relevancy

2010-07-16 Thread Dennis Gearon
Looks to me like a sort of way to get to 'categories', if one were interested in doing that, shudder. Dennis Gearon Signature Warning EARTH has a Right To Life, otherwise we all die. Read 'Hot, Flat, and Crowded' Laugh at http://www.yert.com/film.php --- On Fri, 7/16/10, P

Re: How to speed up solr search speed

2010-07-16 Thread Peter Karich
> > Each solr(jetty) instance on consume 40M-60M memory. > java -Xmx1024M -jar start.jar That's a good suggestion! Please, double check that you are using the -server version of the jvm and the latest 1.6.0_20 or so. Additionally you can start jvisualvm (shipped with the jdk) and hook into jetty

RE: documents with known relevancy

2010-07-16 Thread fiedzia
Jonathan Rochkind wrote: > > I've never used it, but I think this is the use case that the Solr feature > to use Lucene 'payloads' is meant for? > http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payloads/ > This is it, thanks for this link. -- View this message in conte

JSON and DataImportHandler

2010-07-16 Thread P Williams
Hi All, Has anyone gotten the DataImportHandler to work with json as input? Is there an even easier alternative to DIH? Could you show me an example? Many thanks, Tricia

Re: indexing rich documents

2010-07-16 Thread Lance Norskog
The libraries are searched in the solr/llib directory, not solr home. If using multicore, solr/core/lib. These are searched automatically. You can also tell Solr to search in other directories with the directive in solrconfig.xml. On Tue, Jul 13, 2010 at 11:48 PM, satya swaroop wrote: > > here

Re: limiting the total number of documents matched

2010-07-16 Thread Lance Norskog
Yes, multiple (radix) sorts work and you can use the score value. The sort parameters come in order, most important to least important. This sorts first by score, and then documents with the same score are sorted by field f: sort=score+desc,f+asc On Wed, Jul 14, 2010 at 2:46 PM, Paul wrote: >

Re: SOLR Search Query : Exception : Software caused connection abort

2010-07-16 Thread Lance Norskog
How big is "very big"? Tomcat has to be configured for the maximum length of the parameter field in a POST. Is your query string longer than that? If much of the query string is repeated across queries, you can make a in solrconfig.xml that adds extra parameters in the file with a clause. It i

Re: limiting the total number of documents matched

2010-07-16 Thread Yonik Seeley
On Wed, Jul 14, 2010 at 5:46 PM, Paul wrote: > I thought of another way to do it, but I still have one thing I don't > know how to do. I could do the search without sorting for the 50th > page, then look at the relevancy score on the first item on that page, > then repeat the search, but add score

HTTP ERROR: 500 - java.lang.ArrayIndexOutOfBoundsException

2010-07-16 Thread Girish Pandit
Hi, As soon as I add "sort=first+desc" parameter to the select clause, it throws ArrayIndexOutOfBound exception. Please suggest if I am missing anything. http://localhost:8983/solr/select?q=girish&start=0&indent=on&wt=json&sort=first+desc I have close to 1 million records indexed. Thanks Gi

Re: HTTP ERROR: 500 - java.lang.ArrayIndexOutOfBoundsException

2010-07-16 Thread Lance Norskog
This can happen when there are multiple values in a field. Is 'first' a multi-valued field? Sorting only works on single-valued fields. After all, if there are multiple values, it can only sort on one field and there is no way to decide which one. So, make sure that 'field' has multiValued='false'