Re: Solr 5.2.1 versus Solr 4.7.0 performance

2015-08-27 Thread Shawn Heisey
On 8/27/2015 3:26 AM, Toke Eskildsen wrote: On Thu, 2015-08-27 at 11:23 +0300, Esther Goldbraich wrote: We are using GC tuning options: Xgcpolicy:gencon , verbose:gc. RAM: 64GB Solr heap: -Xms512m -Xmx32768m Index per server: 500G Is this in use for Solr 5.2.1 as well? The start script for

Re: Solr relevancy score order

2015-08-27 Thread Steven White
Thanks Erick. Your summary about doc IDs is much helpful. I tested the second level sort with a small set of data (10K records) and didn't see much of a significant impact. I will test with a 10m records at some time later. Steve On Mon, Aug 24, 2015 at 11:03 PM, Erick Erickson

Looking for Traditional Chinese support

2015-08-27 Thread Steven White
Hi Everyone Per https://cwiki.apache.org/confluence/display/solr/Language+Analysis#LanguageAnalysis-Language-SpecificFactories I see the languages Solr supports. Where is Traditional Chinese? Is CJK the one? Thanks Steve

Re: Looking for Traditional Chinese support

2015-08-27 Thread Jeanne Wang
Chinese instead of Simplified Chinese should be Traditional Chinese. Jeanne On Thu, Aug 27, 2015 at 12:51 PM, Steven White swhite4...@gmail.com wrote: Hi Everyone Per https://cwiki.apache.org/confluence/display/solr/Language+Analysis#LanguageAnalysis-Language-SpecificFactories I see the

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Yonik Seeley
UnInvertingReader makes indexed fields look like docvalues fields. The caching itself is still done in FieldCache/FieldCacheImpl but you could perhaps wrap what is cached there to either screen out stuff or construct a new entry based on the user. -Yonik On Thu, Aug 27, 2015 at 12:55 PM, Jamie

FW: Issue while setting Solr on Slider / YARN

2015-08-27 Thread Vijay Bhoomireddy
Hi Tim, For some reason, I was not receiving messages from Solr mailing list, though I could post it to the list. Now I got that sorted. For my below query, I saw your response on the mailing list. Below is the snippet of your response: Hi Vijay, Verify the ResourceManager URL and

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Jamie Johnson
I think a custom UnInvertingReader would work as I could skip the process of putting things in the cache. Right now in Solr 4.x though I am caching based but including the users authorities in the key of the cache so we're not rebuilding the UnivertedField on every request. Where in 5.x is the

Re: StrDocValues

2015-08-27 Thread Jamie Johnson
Actually I should have just tried this before asking but I'll say what I'm seeing and maybe someone can confirm. Faceting looks like it took this into account, i.e. the counts were 0 for values that were in documents that I removed using my AnalyticQuery. I had expected that the AnalyticsQuery

Re: StrDocValues

2015-08-27 Thread Jamie Johnson
Thanks Yonik. I currently am using this to negate the score of a document given the value of a particular field within the document, then using a custom AnalyticQuery to only collect documents with a score 0. Will this also impact the faceting counts? On Wed, Aug 26, 2015 at 8:32 PM, Yonik

Re: Search to Ignore ,

2015-08-27 Thread EXTERNAL Taminidi Ravi (ETI, AA-AS/PAS-PTS)
Hi Team, Can someone help on implementing filter to ignore , in string? e.g. I have Technical, Specification If I search Technical' or Technical, Specification I have to get this results, but current I am not . Simply Ignore the , I can't index the existing Data as it is very huge , is

Re: Search to Ignore ,

2015-08-27 Thread Alexandre Rafalovitch
This is both very specific and very general question at the same time. The way indexing and search are both done is via analyzer chains, as defined in your schema. So, you need to check what the definition is for the field you search and then play with that. There is Analysis screen in the Web

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Jamie Johnson
Also in this vein I think that Lucene should support factories for the cache creation as described @ https://issues.apache.org/jira/browse/LUCENE-2394. I'm not endorsing the patch that is provided (I haven't even looked at it) just the concept in general. On Thu, Aug 27, 2015 at 12:01 PM, Jamie

Re: Not receicing mails from the mailing list

2015-08-27 Thread Upayavira
You were subscribed to the allow list, meaning you could post, but would not receive messages. This is almost certainly because you started sending mails to the list without subscribing first. Moderators then moderated through your request in such a way as to allow you to post in future - as in,

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Yonik Seeley
On Thu, Aug 27, 2015 at 11:59 AM, Tomás Fernández Löbbe tomasflo...@gmail.com wrote: I don't think there is a way to do this now. Maybe we should separate the logic of creating the SolrIndexSearcher to a factory. That should probably be extended down to where lucene creates searchers as well

Not receicing mails from the mailing list

2015-08-27 Thread Vijaya Narayana Reddy Bhoomi Reddy
Hi, Sorry to spam everyone with this email. I am not able to get emails into my inbox from Solr mailing list. However, I am able to send mails to the mailing list. I am able to see them on the mail archive on the website. In case anyone replies to my mail, it appears in the archive, but I don't

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Tomás Fernández Löbbe
I don't think there is a way to do this now. Maybe we should separate the logic of creating the SolrIndexSearcher to a factory. Moving this logic away from SolrCore is already a win, plus it will make it easier to unit test and extend for advanced use cases. Tomás On Wed, Aug 26, 2015 at 8:10

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Yonik Seeley
The FieldCache has become implementation rather than interface, so I don't think you're going to see plugins at that level (it's all package protected now). One could either subclass or re-implement UnInvertingReader though. -Yonik On Thu, Aug 27, 2015 at 12:09 PM, Jamie Johnson

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Jamie Johnson
That makes sense, then I could extend the SolrIndexSearcher by creating a different factory class that did whatever magic I needed. If you create a Jira ticket for this please link it here so I can track it! Again thanks On Thu, Aug 27, 2015 at 11:59 AM, Tomás Fernández Löbbe

Re: StrDocValues

2015-08-27 Thread Erick Erickson
Right, when scoring any document that scores 0 is removed from the results and facets etc. are calculated afterwards. Best, Erick On Thu, Aug 27, 2015 at 10:24 AM, Jamie Johnson jej2...@gmail.com wrote: Actually I should have just tried this before asking but I'll say what I'm seeing and maybe

Re: StrDocValues

2015-08-27 Thread Yonik Seeley
On Thu, Aug 27, 2015 at 2:43 PM, Erick Erickson erickerick...@gmail.com wrote: Right, when scoring any document that scores 0 is removed from the results Just to clarify, I think Jamie removed 0 scoring documents himself. Solr has never done this itself. Lucene used to a long time ago and

Re: no default request handler is registered

2015-08-27 Thread Shawn Heisey
On 8/27/2015 1:10 PM, Scott Hollenbeck wrote: I'm doing some experimenting with Solr 5.3 and the 7.x-1.x-dev version of the Apache Solr Search module for Drupal. Things seem to be working fine, except that this warning message appears in the Solr admin logging window and in the server log:

Re: no default request handler is registered

2015-08-27 Thread Chris Hostetter
Thats... strange. Looking at hte code it appears to be a totally bogus and missleading warning -- but it also shouldn't affect anything. You can feel free to ignore it for now... https://issues.apache.org/jira/browse/SOLR-7984 : Date: Thu, 27 Aug 2015 15:10:18 -0400 : From: Scott

no default request handler is registered

2015-08-27 Thread Scott Hollenbeck
I'm doing some experimenting with Solr 5.3 and the 7.x-1.x-dev version of the Apache Solr Search module for Drupal. Things seem to be working fine, except that this warning message appears in the Solr admin logging window and in the server log: no default request handler is registered (either

Re: Looking for Traditional Chinese support

2015-08-27 Thread Steven White
Hi Jeanne, I don't understand. Are you saying Chinese Tokenizer per https://cwiki.apache.org/confluence/display/solr/Language+Analysis#LanguageAnalysis-Chinese is Traditional Chinese? If so, then it is deprecated as of Solr 3.4 and I just tried it with Solr 5.2 and could not get Solr started

Re: no default request handler is registered

2015-08-27 Thread Chris Hostetter
I just want to clarify: all of Shawn's points below are valid and good -- but they stll don't explain the warning messgae you are getting. it makes no sense as the code is currently written, and doesn't do anything to help encourage people to transition to path based handler names. : Date:

Re: StrDocValues

2015-08-27 Thread Jamie Johnson
Right, I am removing them myself. Another feature which would be great would be the ability to specify a custom collector like the positive score only collector in this case to avoid having to do an extra pass over all of the scores, but I don't believe there is a way to do that now right? On

Re: FW: Issue while setting Solr on Slider / YARN

2015-08-27 Thread Timothy Potter
Hi Vijay, I'm not sure what's wrong here ... have you posted to the Slider mailing list? Also, which version of Java are you using when interacting with Slider? I know it had some issues with Java 8 at one point. Which version of Slider so I can try to reproduce ... Cheers, Tim On Thu, Aug 27,

RE: FW: Issue while setting Solr on Slider / YARN

2015-08-27 Thread Vijay Bhoomireddy
Tim, Here is the complete content of the appConfig-default.json file. I haven’t worked with Slider so far, so not very sure if some mistake has crept into this file while modifying it as per the changes mentioned by Lucidworks on the Github page. However, I tried to simulate the steps

Re: Exact substring search with ngrams

2015-08-27 Thread Christian Ramseyer
On 26/08/15 18:05, Erick Erickson wrote: bq: my dog has fleas I wouldn't want some variant of og ha to match, Here's where the mysterious positionIncrementGap comes in. If you make this field multiValued, and index this like this: doc field name=blahmy dog/field field name=blahhas

Re: Solr 5.2.1 versus Solr 4.7.0 performance

2015-08-27 Thread Esther Goldbraich
:) Yes, 64 collections / 1 shard is compared to 1 collection / 64 shards (with router.name=compositeId) on Solr 5. Quering with _route_ should not eliminate distributed search overhead? What is the difference in distribution mechanism between Solr 4 Solr 5? Especially is there any change in

Re: Solr 5.2.1 versus Solr 4.7.0 performance

2015-08-27 Thread Toke Eskildsen
On Thu, 2015-08-27 at 13:16 +0300, Esther Goldbraich wrote: Yes, 64 collections / 1 shard is compared to 1 collection / 64 shards (with router.name=compositeId) on Solr 5. Quering with _route_ should not eliminate distributed search overhead? Caveat: I am guessing a bit here. When you

Re: Solr 5.2.1 versus Solr 4.7.0 performance

2015-08-27 Thread Esther Goldbraich
Found the reason for many evictions (bug in our code), please ignore the specific question on filter cache. All other questions (in bold) are still very relevant. From: Esther Goldbraich/Haifa/IBM To: solr-user@lucene.apache.org Date: 27/08/2015 01:13 PM Subject:Re: Solr 5.2.1

SolrCloud: collection creation: There are duplicate coreNodeName in core.properties in a same collection.

2015-08-27 Thread forest_soup
https://issues.apache.org/jira/browse/SOLR-7982 We have a 3 Zookeeper 5 solr server Solrcloud. We created collection1 and collection2 with 80 shards respectively in the cloud, replicateFactor is 2. But after created, we found in a same collection, the coreNodeName has some duplicate in

Re: Solr 5.2.1 versus Solr 4.7.0 performance

2015-08-27 Thread Toke Eskildsen
On Thu, 2015-08-27 at 11:23 +0300, Esther Goldbraich wrote: We are using GC tuning options: Xgcpolicy:gencon , verbose:gc. RAM: 64GB Solr heap: -Xms512m -Xmx32768m Index per server: 500G Expecting Your RAM size should be equal to index size-posts to arrive in 3, 2, 1... Surprisingly,

Re: Solr 5.2.1 versus Solr 4.7.0 performance

2015-08-27 Thread Esther Goldbraich
We are using GC tuning options: Xgcpolicy:gencon , verbose:gc. RAM: 64GB Solr heap: -Xms512m -Xmx32768m Index per server: 500G Surprisingly, running different setup on same machines, 64 collections / 1 shard per collection gives significantly better results. Any ideas? Thank you, Esther

Problem with JSON sub-faceting

2015-08-27 Thread Pritam Kute
Do result grouping and tagging and excluding filters feature works with JSON sub-faceting? If yes, it will be a great help if someone point me to some documentation for the same. Thanks in advance. Thanks Regards, -- *Pritam Kute*