Re: Solr Cloud: Duplicate documents in multiple shards

2015-07-20 Thread mesenthil1
Thanks Erick for clarifying .. We are not explicitly setting the compositeId. We are using numShards=5 alone as part of the server start up. We are using uuid as unique field. One sample id is : possting.mongo-v2.services.com-intl-staging-c2d2a376-5e4a-11e2-8963-0026b9414f30 Not sure how it wou

Re: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

2015-07-20 Thread Ali Nazemian
Dear Erick, Actually faceting on this field is not a user wanted application. I did that for the purpose of testing the customized normalizer and charfilter which I used. Therefore it just used for the purpose of testing. Anyway I did some googling on this error and It seems that changing facet me

Re: SOLR nrt read writes

2015-07-20 Thread Bhawna Asnani
Thanks, I tried turning off auto softCommits but that didn't help much. Still seeing stale results every now and then. Also load on the server very light. We are running this just on a test server with one or two users. I don't see any warning in logs whole doing softCommits and it says it succe

Re: Issue with using createNodeSet in Solr Cloud

2015-07-20 Thread Erick Erickson
Glad you found a solution Best, Erick On Mon, Jul 20, 2015 at 3:21 AM, Savvas Andreas Moysidis wrote: > Erick, spot on! > > The nodes had been registered in zookeeper under my network interface's IP > address...after specifying those the command worked just fine. > > It was indeed the thing

Re: Solr Cloud: Duplicate documents in multiple shards

2015-07-20 Thread Erick Erickson
bq: We have 130 million documents in our set up and the routing key is set as "compositeId". The most likely explanation is that somehow you've sent the same document out with different routing keys. So what is the ID field (or, more generally, your field) for a pair of duplicated documents? My b

Re: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

2015-07-20 Thread Erick Erickson
This really seems like an XY problem. _Why_ are you faceting on a tokenized field? What are you really trying to accomplish? Because faceting on a generalized content field that's an analyzed field is often A Bad Thing. Try going into the admin UI>> Schema Browser for that field, and you'll see how

Re: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

2015-07-20 Thread Ali Nazemian
Dear Toke and Davidphilip, Hi, The fieldtype text_fa has some custom language specific normalizer and charfilter, here is the schema.xml value related for this field: I did t

Re: solr blocking and client timeout issue

2015-07-20 Thread Erick Erickson
bq: the config is set up per the NRT suggestions in the docs. autoSoftCommit every 2 seconds and autoCommit every 10 minutes. 2 second soft commit is very aggressive, no matter what the NRT suggestions are. My first question is whether that's really needed. The soft commits should be as long as yo

Re: Use REST API URL to update field

2015-07-20 Thread Zheng Lin Edwin Yeo
Hi Shawn, So it means that if my following is in a text file called update.txt, {"id":"testing_0001", "popularity":{"inc":1} This text file must still exist if I use the URL? Or can this information in the text file be put directly onto the URL? Regards, Edwin On 20 July 2015 at 22:04, Shawn

Re: Installing Banana on Solr 5.2.1

2015-07-20 Thread Shawn Heisey
On 7/20/2015 5:45 PM, Vineeth Dasaraju wrote: > I am trying to install Banana on top of solr but haven't been able to do > so. All the procedures that I get are for an earlier version of solr. Since > the directory structure has changed in the new version, inspite of me > placing the banana folder

WordDelimiterFilter Leading & Trailing Special Character

2015-07-20 Thread Sathiya N Sundararajan
Question about WordDelimiterFilter. The search behavior that we experience with WordDelimiterFilter satisfies well, except for the case where there is a special character either at the leading or trailing end of the term. For instance: *‘d&b’ * —> Works as expected. Finds all docs with ‘d&b’. *

Re: shareSchema property unknown in new solr.xml format

2015-07-20 Thread Yago Riveiro
Thank Hoss this was the example that I saw to configure my solr.xml I have more than 250 cores and any resource that I can optimize is always welcome but if in 5.x this feature is deprecated I will try to upgrade as soon as possible.

Installing Banana on Solr 5.2.1

2015-07-20 Thread Vineeth Dasaraju
Hi, I am trying to install Banana on top of solr but haven't been able to do so. All the procedures that I get are for an earlier version of solr. Since the directory structure has changed in the new version, inspite of me placing the banana folder under the server/solr-webapp/webapp folder, I am

Re: Data Import Handler Stays Idle

2015-07-20 Thread Raja Pothuganti
>Yes the number of unimported matches (with IOExceptions) What is the IOException about? On 7/20/15, 5:10 PM, "Paden" wrote: >Yes the number of unimported matches. No I did not specify "false" to >commit >on any of my dataimporthandler. Since it defaults to true I really didn't >take it into ac

Re: Multiple boost queries on a specific field

2015-07-20 Thread Chris Hostetter
: /?q=&wt=json&defType=dismax&q.alt=*:*&bq=provider:A^2.0/ : My first results have provider A. : ?q=&wt=json&defType=dismax&q.alt=*:*&bq=provider:​B​^​1.5 : My​ first results have provider B. Good! : /?q=&wt=json&defType=dismax&q.alt=*:*&bq=provider:​(​A^2.0​ B^1.5)​/ : Then my first results h

Re: Data Import Handler Stays Idle

2015-07-20 Thread Paden
Yes the number of unimported matches. No I did not specify "false" to commit on any of my dataimporthandler. Since it defaults to true I really didn't take it into account though. -- View this message in context: http://lucene.472066.n3.nabble.com/Data-Import-Handler-Stays-Idle-tp4218250p42182

Re: Data Import Handler Stays Idle

2015-07-20 Thread Raja Pothuganti
Number of Ioexceptions , are they equal to un-imported/un processed documents? By any chance commit set to false in import request example: http://localhost:8983/solr/db/dataimport?command=full-import&commit=false Thanks Raja On 7/20/15, 4:51 PM, "Paden" wrote: >I was consistently checking th

Re: Data Import Handler Stays Idle

2015-07-20 Thread Paden
I was consistently checking the logs to see if there were any errors that would give me any idling. There were no errors except for a few skipped documents due to some Illegal IOexceptions from Tika but none of those occurred around the time that solr began idling. A lot of font warnings. But again

Re: set the param [facet.offset] for EVERY [facet.pivot]

2015-07-20 Thread Chris Hostetter
: HI All:I need a pagenigation with facet offset. : There are two or more fields in [facet.pivot], but only one value : for [facet.offset], eg: facet.offset=10&facet.pivot=field_1,field_2. : In this condition, field_2 is 10's offset and then field_1 is 10's : offset. Bu

Re: Data Import Handler Stays Idle

2015-07-20 Thread Shawn Heisey
On 7/20/2015 3:03 PM, Paden wrote: > I'm currently trying to index about 54,000 files with the Solr Data Import > Handler and I've got a small problem. It fetches about half (28,289) of the > 54,000 files and it process about 14,146 documents before it stops and just > stands idle. Here's the statu

Data Import Handler Stays Idle

2015-07-20 Thread Paden
Hello, I'm currently trying to index about 54,000 files with the Solr Data Import Handler and I've got a small problem. It fetches about half (28,289) of the 54,000 files and it process about 14,146 documents before it stops and just stands idle. Here's the status output { "responseHeader": {

Re: solr blocking and client timeout issue

2015-07-20 Thread Jeremy Ashcraft
heap is already at 5GB On 07/20/2015 12:29 PM, Jeremy Ashcraft wrote: no swapping that I'm seeing, although we are noticing a lot higher number of hard commits than usual. the config is set up per the NRT suggestions in the docs. autoSoftCommit every 2 seconds and autoCommit every 10 minutes

Re: solr blocking and client timeout issue

2015-07-20 Thread Jeremy Ashcraft
no swapping that I'm seeing, although we are noticing a lot higher number of hard commits than usual. the config is set up per the NRT suggestions in the docs. autoSoftCommit every 2 seconds and autoCommit every 10 minutes. there have been 463 updates in the past 2 hours, all followed by ha

Re: solr blocking and client timeout issue

2015-07-20 Thread Shawn Heisey
On 7/20/2015 11:54 AM, Jeremy Ashcraft wrote: > I'm ugrading to the 1.8 JDK on our dev VM now and testing. Hopefully i > can get production upgraded tonight. > > still getting the big GC pauses this morning, even after applying the > GC tuning options. Everything was fine throughout the weekend. >

SOLR 5.1.0 FileListEntity exclude

2015-07-20 Thread Joe Fidanza
Very new to SOLR and haven't been able to find an answer to this issue: Using SOLR 5.1.0 we have a successful FileListEntityProcessor setup but would like to exclude several directories that live below the baseDir. The data-config.xml file looks like: baseDir="/abc/def/" recurs

Re: shareSchema property unknown in new solr.xml format

2015-07-20 Thread Chris Hostetter
: > I’m getting this error on startup: : > : > section of solr.xml contains 1 unknown config parameter(s): [shareSchema] Pretty sure that's because it was never a supported property of the section -- even in the old format of solr.xml. it's just a top level property -- ie: create a child no

Re: solr blocking and client timeout issue

2015-07-20 Thread Jeremy Ashcraft
I'm ugrading to the 1.8 JDK on our dev VM now and testing. Hopefully i can get production upgraded tonight. still getting the big GC pauses this morning, even after applying the GC tuning options. Everything was fine throughout the weekend. My biggest concern is that this instance had been r

Re: SOLR nrt read writes

2015-07-20 Thread Shawn Heisey
On 7/20/2015 9:29 AM, Bhawna Asnani wrote: > Thanks for your suggestions. The requirement is still the same , to be > able to make a change to some solr documents and be able to see it on > subsequent search/facet calls. > I am using softCommit with waitSearcher=true. > > Also I am sending reads/wr

Re: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

2015-07-20 Thread Toke Eskildsen
Ali Nazemian wrote: > I have a collection of 1.6m documents in Solr 5.2.1. > [...] > Caused by: java.lang.IllegalStateException: Too many values for > UnInvertedField faceting on field content > [...] > default="noval" termVectors="true" termPositions="true" > termOffsets="true"/> You are hittin

Re: SOLR nrt read writes

2015-07-20 Thread Bhawna Asnani
Hi, Thanks for your suggestions. The requirement is still the same , to be able to make a change to some solr documents and be able to see it on subsequent search/facet calls. I am using softCommit with waitSearcher=true. Also I am sending reads/writes to a single solr node only. I have tried dis

Re: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

2015-07-20 Thread davidphilip cherian
I think you should just make docValues=true and reindex. But be warned that faceting is generally not performed on field that are of type text and tokenized. They should be string if they are not numeric. What is analysis chain of 'text_fa'? On Mon, Jul 20, 2015 at 8:16 PM, Ali Nazemian wrote:

java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

2015-07-20 Thread Ali Nazemian
Dears, Hi, I have a collection of 1.6m documents in Solr 5.2.1. When I use facet on field of content this error will appear after around 30s of trying to return the results: null:org.apache.solr.common.SolrException: Exception during facet.field: content at org.apache.solr.request.SimpleFa

RE: Rule for Score

2015-07-20 Thread EXTERNAL Taminidi Ravi (ETI, AA-AS/PAS-PTS)
Thanks Shawn, I will Check that. Ravi -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, July 20, 2015 10:23 AM To: solr-user@lucene.apache.org Subject: Re: Rule for Score On 7/20/2015 8:16 AM, EXTERNAL Taminidi Ravi (ETI, AA-AS/PAS-PTS) wrote: > Hi All, C

Re: Basic auth

2015-07-20 Thread Steven White
Thanks for updating the wiki page. However, my issue remains, I cannot get Basic auth working. Has anyone got it working, on Windows? Steve On Mon, Jul 20, 2015 at 9:09 AM, Shawn Heisey wrote: > On 7/20/2015 6:06 AM, Steven White wrote: > > Just to be clear, the example at > > https://wiki.ap

Re: Rule for Score

2015-07-20 Thread Shawn Heisey
On 7/20/2015 8:16 AM, EXTERNAL Taminidi Ravi (ETI, AA-AS/PAS-PTS) wrote: > Hi All, Can you someone explain or refer me right place to know more on how > the Score is calculated, I am seeing it has few attribute like, > termfrequency, document frequeuncy, weight, boost also it is say sum of , > p

Rule for Score

2015-07-20 Thread EXTERNAL Taminidi Ravi (ETI, AA-AS/PAS-PTS)
Hi All, Can you someone explain or refer me right place to know more on how the Score is calculated, I am seeing it has few attribute like, termfrequency, document frequeuncy, weight, boost also it is say sum of , product of.. Is there any example of understanding the basics of these attributes

Re: Use REST API URL to update field

2015-07-20 Thread Shawn Heisey
On 7/20/2015 2:06 AM, Zheng Lin Edwin Yeo wrote: > I'm using Solr 5.2.1, and I would like to check, is there a way to update > certain field by using REST API URL directly instead of using curl? > > For example, I would like to increase the "popularity" field in my index > each time a user click o

Re: shareSchema property unknown in new solr.xml format

2015-07-20 Thread Shawn Heisey
On 7/20/2015 6:40 AM, Yago Riveiro wrote: > I’m trying to move from legacy solr.xml to new solr.xml format (version > 4.10.4) > > I’m getting this error on startup: > > section of solr.xml contains 1 unknown config parameter(s): > [shareSchema] There's somewhat confusing information in Jira o

Re: Basic auth

2015-07-20 Thread Shawn Heisey
On 7/20/2015 6:06 AM, Steven White wrote: > Just to be clear, the example at > https://wiki.apache.org/solr/SolrSecurity#Jetty_realm_example states to > modify the file in /example/etc/webdefault.xml and in > /example/etc/jetty.xml, but with Solr 5.2.1, those two files are in > C:\solr-5.2.1\server

shareSchema property unknown in new solr.xml format

2015-07-20 Thread Yago Riveiro
Hi, I’m trying to move from legacy solr.xml to new solr.xml format (version 4.10.4) I’m getting this error on startup: section of solr.xml contains 1 unknown config parameter(s): [shareSchema] The las documentation has an entry for this property … I have the property configured as: ${

Re: Basic auth

2015-07-20 Thread Steven White
Hi Everyone, I don't mean to hijack this thread, but I have auth issues that maybe related to this topic. I'm on 5.2.1 and trying to setup basic auth using jetty realm per https://wiki.apache.org/solr/SolrSecurity#Jetty_realm_example And I found other examples on the web which are very similar t

Solr Cloud: Duplicate documents in multiple shards

2015-07-20 Thread mesenthil1
Hi All, We are using solr 4.2.1 cloud with 5 shards set up ( 1 leader & 1 replica for each shard). We are seeing the following issue in our set up. Few of the documents are getting returned from more than one shard for queries. When we try to update the document, it is not updating the document

Re: Issue with using createNodeSet in Solr Cloud

2015-07-20 Thread Savvas Andreas Moysidis
Erick, spot on! The nodes had been registered in zookeeper under my network interface's IP address...after specifying those the command worked just fine. It was indeed the thing I thought was true that wasn't... :) Many thanks, Savvas On 18 July 2015 at 20:47, Erick Erickson wrote: > P.S. > >

Use REST API URL to update field

2015-07-20 Thread Zheng Lin Edwin Yeo
Hi, I'm using Solr 5.2.1, and I would like to check, is there a way to update certain field by using REST API URL directly instead of using curl? For example, I would like to increase the "popularity" field in my index each time a user click on the record. Currently, it can work with the curl co

Reload cause core to become not functional

2015-07-20 Thread Nir Barel
Hi, We are using RELOAD command with parameters as described in patch: ( BTW, why isn't part of a version? ) https://issues.apache.org/jira/browse/SOLR-6063 http://localhost:port/solr/admin/cores?action=RELOAD&core=CORE_NAME&transient=true&loadOnStartup=false We run this every midnight to trans