Re: If statements in DataImportHandler?

2011-03-11 Thread Stefan Matheis
Jason, have a look on http://wiki.apache.org/solr/DataImportHandler#Accessing_request_parameters , this might help you out. If not, you'd give us a quick example of what you plan to do? Regards Stefan

Re: Solr and Permissions

2011-03-11 Thread Jan Høydahl
Hi, Talk to the ManifoldCF guys - they have successfully implemented support for document level security for many repositories including CMC/ECMs and may have some hints for you to write your own Authority connector against your system, which will fetch the ACL for the document and index it

Re: Possible to sort in .xml file?

2011-03-11 Thread Andy Newby
Hi, Ah cool - missed that bit! Will give that a go (as it will be handy for passing along other paramaters too) Cheers Andy On Thu, Mar 10, 2011 at 9:13 PM, Chris Hostetter hossman_luc...@fucit.orgwrote: : I know its possible to do via adding sort= , but the Perl module :

Re: Problem with copyfield

2011-03-11 Thread Grijesh
Which field you are copping id to ng_text or description to ng_text - Thanx: Grijesh http://lucidimagination.com -- View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-copyfield-tp2663873p2663998.html Sent from the Solr - User mailing list archive at Nabble.com.

Query on facet field’s count

2011-03-11 Thread rajini maski
Query on facet field results... When I run a facet query on some field say : facet=on facet.field=StudyID I get list of distinct StudyID list with the count that tells that how many times did this study occur in the search query. But I also needed the count of these distinct StudyID

Getting Category ID (primary key)

2011-03-11 Thread Prav Buz
Hi, I need to show a facets on Category and then I need the category id in the href link. For this what I 'm trying to do is create a field which will store ID|Category in the schema and split it in the UI. Also I have Category and category id 's indexed . Categories are multivalued as well,

Re: Getting Category ID (primary key)

2011-03-11 Thread Gora Mohanty
On Fri, Mar 11, 2011 at 2:32 PM, Prav Buz buz.p...@gmail.com wrote: [...] I need to show a facets on Category and then I need the category id in the href link. For this what I 'm trying to do is create a field which will store ID|Category in the schema and split it in the UI. Also I have

Re: Getting Category ID (primary key)

2011-03-11 Thread Prav Buz
Hi, Yes I already have different fields for category and category Id , and they are in same order when retrieved from solr for eg: IDs 1 3 4 5 names a b c d e id 1 is of name a and id 5 is of name e. but when I sort the category names , looses this order as they are not related in any manner in

I send a email to lucene-dev solr-dev lucene-user but always failed

2011-03-11 Thread Li Li
hi it seems my mail is judged as spam. Technical details of permanent failure: Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other

Re: I send a email to lucene-dev solr-dev lucene-user but always failed

2011-03-11 Thread Li Li
Problem of Replication Reservation Durationhi all, I tried to send this mail to solr dev mail list but it tells me this is a spam. So I send it again and to lucene dev too. The replication handler in solr 1.4 which we used seems to be a little problematic in some extreme situation. The

Re: disquery - difference qf qs / pf ps

2011-03-11 Thread Gastone Penzo
ok thank you!!! 2011/3/10 Jonathan Rochkind rochk...@jhu.edu On 3/10/2011 8:15 AM, Gastone Penzo wrote: Thank you very much. i understand the difference beetween qs and ps but not what pf is...is it necessary to use ps? It's not neccesary to use anything, including Solr. pf: Will take

Solr insert error

2011-03-11 Thread rahul
Hi, I have received the following error, when I try to insert a document into solr, SEVERE: org.apache.solr.common.SolrException: ERROR: multiple values encountered for non multiValued copy field id: 272327_1 In my schema.xml, I have specified, id In the query, I have passed as

Re: Problem with copyfield

2011-03-11 Thread Erick Erickson
You're copying the id field rather than description into ng_text. Try: copyField source=description dest=ng_text/ Best Erick On Fri, Mar 11, 2011 at 1:11 AM, nidhi gupta wdnidhigu...@yahoo.co.in wrote: I want to implement type ahead styling feature for description field.For that I defined

Re: Query on facet field’s count

2011-03-11 Thread Erick Erickson
There's nothing that I know of that gives you this, but it's simple to count the members of the list yourself... Best Erick On Fri, Mar 11, 2011 at 3:34 AM, rajini maski rajinima...@gmail.com wrote: Query on facet field results...       When I run a facet query on some field say : facet=on

Re: Getting Category ID (primary key)

2011-03-11 Thread Erick Erickson
Thinking out loud here, but would it work to just have ugly categories? Instead of splitting them up, just encode them like 1|a 2|b 3|c or some such. Then split them back up again and display the name to the user and use the ID in the URL Best Erick On Fri, Mar 11, 2011 at 4:17 AM, Prav

Re: Solr insert error

2011-03-11 Thread Erick Erickson
Show us the input please? But what that means is that you have a field in your input document twice, something like field1val1field1 field1val2field1 where field1 is single valued. Or, more correctly, a field for which you have NOT specified multiValued=true. Note that multiValued is case

Re: Solr insert error

2011-03-11 Thread rahul
Hi, thanks for your reply. I have post that value only one time. The following are the list of values that I have posted,

Index Defaults Section and main index section that is in solrconfig.xml

2011-03-11 Thread rajini maski
Any documentation on index Defaults Section and main index section that is in solrconfig.xml -- Solr1.4.1 I want to understand the terminology of these parameters and how are they inter connected?mergeFactor10/mergeFactorramBufferSizeMB32/ramBufferSizeMB

RE: Solr and Permissions

2011-03-11 Thread Tim Gilbert
What about using the BitwiseQueryParserPlugin? https://issues.apache.org/jira/browse/SOLR-1913 You could encode your documents with a series of permissions based on Bit flags and then OR them on query. Tim -Original Message- From: r...@intelligencebank.com

Re: Solr insert error

2011-03-11 Thread rahul
hi,, seems I have identified the issue. In the code I am using ContentStreamBase.StringStream stream = new ContentStreamBase.StringStream(streamData); If the streamData contains name=ID , ie, ID value then already I set copyfield for uniqueid as id. Hence, It throws error. Seems, it check

Re: Solr and Permissions

2011-03-11 Thread go canal
Thank you Jan, I will take a look at the MainfoldCF. So it seems that the solution is basically to implement something outside of Solr for permission control. thanks, canal From: Jan Høydahl jan@cominvent.com To: solr-user@lucene.apache.org Sent: Fri,

Multiple Japanese Alphabets in Solr

2011-03-11 Thread Tomás Fernández Löbbe
This question is probably not a completely Solr question but it's related to it. I'm dealing with a Japanese Solr application in which I would like to be able to search in any of the Japanese Alphabets. The content can also be in any Japanese Alphabet. I've been thinking in this solution: Convert

Re: disquery - difference qf qs / pf ps

2011-03-11 Thread Bill Bell
What exactly is phrase and query slop? What technically is a slop? Bill Bell Sent from mobile On Mar 11, 2011, at 4:15 AM, Gastone Penzo gastone.pe...@gmail.com wrote: ok thank you!!! 2011/3/10 Jonathan Rochkind rochk...@jhu.edu On 3/10/2011 8:15 AM, Gastone Penzo wrote: Thank you

Re: Query on facet field’s count

2011-03-11 Thread Bill Bell
There is my patch to do that. SOLR-2242 Bill Bell Sent from mobile On Mar 11, 2011, at 1:34 AM, rajini maski rajinima...@gmail.com wrote: Query on facet field results... When I run a facet query on some field say : facet=on facet.field=StudyID I get list of distinct StudyID list

Re: disquery - difference qf qs / pf ps

2011-03-11 Thread Tomás Fernández Löbbe
AFAIK, the difference is that the phrase slop applies to the phrase queries generated automatically by dismax and applied to the fields in pf. Query Slop (qs) is the phrase slop applied to a phrase explicitly specified by the user, which will be applied to all the Query Fields (qf). An easy way

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread François Schiettecatte
Tomás That wont really work, transliteration to Romaji works for individual terms only so you would need to tokenize the Japanese prior to transliteration. I am not sure what tool you plan to use for transliteration, I have used ICU in the past and from what I can tell it does not

[GSoC] Apache Lucene @ Google Summer of Code 2011 [STUDENTS READ THIS]

2011-03-11 Thread Simon Willnauer
Hey folks, Google Summer of Code 2011 is very close and the Project Applications Period has started recently. Now it's time to get some excited students on board for this year's GSoC. I encourage students to submit an application to the Google Summer of Code web-application. Lucene Solr are

Re: Solr insert error

2011-03-11 Thread Erick Erickson
We need to see the relevant portions of your schema file, particularly the uniqueKey definition and any copyfields directives. Best Erick On Fri, Mar 11, 2011 at 9:29 AM, rahul asharud...@gmail.com wrote: hi,, seems I have identified the issue. In the code I am using

Re: Getting Category ID (primary key)

2011-03-11 Thread Prav Buz
Hi, Thanks Erik, yes that's what I've done for now, but was wondering if it's the best way :) thanks Praveen On Fri, Mar 11, 2011 at 6:06 PM, Erick Erickson erickerick...@gmail.comwrote: Thinking out loud here, but would it work to just have ugly categories? Instead of splitting them up,

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread Walter Underwood
Why not index it as-is? Solr can handle Unicode. Transliterating hiragana to katakana is a very weird idea. I cannot imagine how that would help. You will need some sort of tokenization to find word boundaries. N-grams work OK for search, but are really ugly for highlighting. As far as I

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread François Schiettecatte
Good question about transliteration, the issue has to do with recall, for example, I can write 'Toyota' as 'トヨタ' or 'とよた' (Katakana and Hiragana respectively), not doing the transliteration will miss results. You will find that the big search engines do the transliteration for you

Re: Solr and Permissions

2011-03-11 Thread Walter Underwood
On Mar 10, 2011, at 10:48 PM, go canal wrote: But in real world, any content management systems need full text search; so the question is to how to support search with permission control. I have yet to see a Search Engine that provides some sort of Content Management features like we

Re: Getting Category ID (primary key)

2011-03-11 Thread Geert-Jan Brits
If it works, it's performant and not too messy it's a good way :-) . You can also consider just faceting on Id, and use the id to fetch the categoryname through sql / nosql. That way your logic is seperated from your presentation, which makes extending (think internationalizing, etc.) easier. Not

Re: Solr and Permissions

2011-03-11 Thread Bill Bell
Why not just add a security field in Solr and use fq to limit to the users permissions? Bill Bell Sent from mobile On Mar 11, 2011, at 10:27 AM, Walter Underwood wun...@wunderwood.org wrote: On Mar 10, 2011, at 10:48 PM, go canal wrote: But in real world, any content management systems

Re: Solr and Permissions

2011-03-11 Thread Walter Underwood
On Mar 11, 2011, at 9:32 AM, Bill Bell wrote: Why not just add a security field in Solr and use fq to limit to the users permissions? You can. When permissions change, you need to reload every affected document. You also need to build the whole security filtering from scratch instead of

Re: Solr and Permissions

2011-03-11 Thread Geert-Jan Brits
About the 'having to reindex when permissions change'-problem: have a look at ExternalFileField http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.htmlwhich enables you to reload a file without

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread Tomás Fernández Löbbe
the issue has to do with recall, for example, I can write 'Toyota' as 'トヨタ' or 'とよた' (Katakana and Hiragana respectively), not doing the transliteration will miss results. Exactly, that's my problem, searching on a different alphabet than the one on which it was indexed a document. François, thank

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread Walter Underwood
Sounds more like generating synonyms than conflating everything to one set of kana. Why not a filter that does that transliteration and adds a token at the some position? wunder On Mar 11, 2011, at 9:49 AM, Tomás Fernández Löbbe wrote: the issue has to do with recall, for example, I can

Re: Solr and Permissions

2011-03-11 Thread Sujit Pal
Yes there can be cases where user is allowed a subset of a content type, or a combination of content type groups and individual documents, where this would break down. And yes, afaik, if you want to update the permissions in the document (seems slightly strange, since you would potentially many

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread François Schiettecatte
Tomás The ICU code base is used by a *lot* so I think it is safe to say that it works ok :) François On Mar 11, 2011, at 12:49 PM, Tomás Fernández Löbbe wrote: the issue has to do with recall, for example, I can write 'Toyota' as 'トヨタ' or 'とよた' (Katakana and Hiragana respectively), not

Re: Multiple Japanese Alphabets in Solr

2011-03-11 Thread François Schiettecatte
You could certainly do it that way if you wanted. The one point I would make here is that from a linguistic POV these are not synonyms but are the same term written in a different alphabet. François On Mar 11, 2011, at 12:51 PM, Walter Underwood wrote: Sounds more like generating synonyms

Re: Solr Hanging all of sudden with update/csv

2011-03-11 Thread danomano
Sweet, those links very very useful :). and should most definitely help :) One overriding concern I have: 1) if I were to simply update the config to use a different mergeFactor, and restart the solr server, (would it then adjust the segments accordingly?) or would I need to start from

Using Solr over Lucene effects performance?

2011-03-11 Thread sivaram
Hello All, I searched for this but couldn't find a convincing answer. I'm planning to use Lucene/Solr in a tool for indexing and searching documents. I'm thinking of if I use Lucene directly instead of Solr, will it improves the performance of the search?(in terms of time taken for indexing or

Re: Using Solr over Lucene effects performance?

2011-03-11 Thread Yonik Seeley
On Fri, Mar 11, 2011 at 4:21 PM, sivaram yogendra.bopp...@gmail.com wrote: I searched for this but couldn't find a convincing answer. I'm planning to use Lucene/Solr in a tool for indexing and searching documents. I'm thinking of if I use Lucene directly instead of Solr, will it improves the

Re: Using Solr over Lucene effects performance?

2011-03-11 Thread sivaram
Thanks for the quick reply Yonik, So you are saying that it all depends on how we setup the Solr? In a performance perspective, does Solr lags behind Lucene because it's a layer above Lucene to our application or will it have a better indexing and searching techniques than Lucene? (when talking

Re: Using Solr over Lucene effects performance?

2011-03-11 Thread Yonik Seeley
On Fri, Mar 11, 2011 at 5:07 PM, sivaram yogendra.bopp...@gmail.com wrote: So you are saying that it all depends on how we setup the Solr? In a performance perspective, does Solr lags behind Lucene because it's a layer above Lucene to our application or will it have a better indexing and

Re: Using Solr over Lucene effects performance?

2011-03-11 Thread Glen Newton
I have seen little repeatable empirical evidence for the usual answer mostly no. With respect: everyone in the Solr universe seems to answer this question in the way Yonik has. However, with a large number of requests the XML serialization/deserialization must have some, likely significant,

access document by primary key

2011-03-11 Thread onlinespend...@gmail.com
what's the quickest and most efficient way to access a doc by its primary key? suppose I already know a document's unique id and simply want to fetch it without issuing a sophisticated query. Thanks, Ben

Any way to do payload queries in Luke?

2011-03-11 Thread Sujit Pal
Hello, I am denormalizing a map of string,float into a single lucene document by storing it as key1|score1 key2|score2 In Solr, I pull this in using the following analyzer definition. fieldtype name=payloads stored=false indexed=true class=solr.TextField analyzer

Re: access document by primary key

2011-03-11 Thread Yonik Seeley
On Fri, Mar 11, 2011 at 5:58 PM, onlinespend...@gmail.com onlinespend...@gmail.com wrote: what's the quickest and most efficient way to access a doc by its primary key? suppose I already know a document's unique id and simply want to fetch it without issuing a sophisticated query. Bypassing

Re: Using Solr over Lucene effects performance?

2011-03-11 Thread Glen Newton
On Fri, Mar 11, 2011 at 5:26 PM, Yonik Seeley yo...@lucidimagination.com wrote: That's an apples to oranges comparison - lucene is a library and solr is a server. I partially agree ;-) Lucene is a library and Solr is an http server wrapper-plus around Lucene. Solr also adds (all sorts of

Re: Solr and Permissions

2011-03-11 Thread go canal
Looking at the API doc, it seems that only floating value is currently supported, is it true? thanks, canal From: Geert-Jan Brits gbr...@gmail.com To: solr-user@lucene.apache.org Sent: Sat, March 12, 2011 1:42:38 AM Subject: Re: Solr and Permissions About

Re: disquery - difference qf qs / pf ps

2011-03-11 Thread Bill Bell
Thank you, that is the best explanation I have ever heard. We should add to the Wiki (given that is is correct). On 3/11/11 9:05 AM, Tomás Fernández Löbbe tomasflo...@gmail.com wrote: AFAIK, the difference is that the phrase slop applies to the phrase queries generated automatically by dismax

Re: Query on facet field¹s count

2011-03-11 Thread Bill Bell
http://localhost:8983/solr/select?q=*:*facet=truefacet.field=StudyIDface t.mincount=1facet.limit=-1f.StudyID.facet.namedistinct=1 Would do what you want I believe... On 3/11/11 8:51 AM, Bill Bell billnb...@gmail.com wrote: There is my patch to do that. SOLR-2242 Bill Bell Sent from mobile

Re: Solr and Permissions

2011-03-11 Thread Koji Sekiguchi
(11/03/12 10:28), go canal wrote: Looking at the API doc, it seems that only floating value is currently supported, is it true? Right. And it is just for changing score by using float values in the file, so it cannot be used for filtering. Koji -- http://www.rondhuit.com/en/

Re: Query on facet field¹s count

2011-03-11 Thread rajini maski
Thanks Bill Bell . .This query works after applying the patch you refered to, is it? Please can you let me know how do I need to update the current war (apache solr 1.4.1 )file with this new patch? Thanks a lot. Thanks, Rajani On Sat, Mar 12, 2011 at 8:56 AM, Bill Bell billnb...@gmail.com wrote:

Re: Getting Category ID (primary key)

2011-03-11 Thread Prav Buz
Hi, Thanks for your suggestions, thanks to the great community too :) regards Praveen On Fri, Mar 11, 2011 at 11:01 PM, Geert-Jan Brits gbr...@gmail.com wrote: If it works, it's performant and not too messy it's a good way :-) . You can also consider just faceting on Id, and use the id to