OOM on solr cloud 5.2.1, does not trigger oom_solr.sh

2015-10-22 Thread Raja Pothuganti
Hi, Some times I see OOM happening on replicas,but does not trigger script oom_solr.sh which was passed in as -XX:OnOutOfMemoryError=/actualLocation/solr/bin/oom_solr.sh 8091. These OOM happened while DIH importing data from database. Is this known issue? is there any quick fix? Here are stack t

Re: Help on Out of memory when using Cursor with sort on Unique Key

2015-09-08 Thread Raja Pothuganti
Hi Naresh 1) For 'sort by' fields, have you considered using DocValue=true for in schema definition. If you are changing schema definition, you would need redo full reindex after backing up & deleting current index from dataDir. Also note that, adding docValue=true would increase size of index.

Re: spread index not equally each sharding

2015-07-31 Thread Raja Pothuganti
As far as I know sharding is done on basis of unique key hash(by default). So most of the time, each shard will have almost equal number of documents. But each of document my have different size which can show up as different index size per shard. Thanks On 7/31/15, 5:49 AM, "wilanjar ." wrote:

Re: Data Import Handler Stays Idle

2015-07-20 Thread Raja Pothuganti
>Yes the number of unimported matches (with IOExceptions) What is the IOException about? On 7/20/15, 5:10 PM, "Paden" wrote: >Yes the number of unimported matches. No I did not specify "false" to >commit >on any of my dataimporthandler. Since it defaults to true I really didn't >take it into ac

Re: Data Import Handler Stays Idle

2015-07-20 Thread Raja Pothuganti
Number of Ioexceptions , are they equal to un-imported/un processed documents? By any chance commit set to false in import request example: http://localhost:8983/solr/db/dataimport?command=full-import&commit=false Thanks Raja On 7/20/15, 4:51 PM, "Paden" wrote: >I was consistently checking th

Re: How to create indexes for files in hdfs using tika embeded in solr?

2015-07-18 Thread Raja Pothuganti
would MapReduceIndexerTool option ? http://www.cloudera.com/content/cloudera/en/documentation/cloudera-search/v 1-latest/Cloudera-Search-User-Guide/csug_mapreduceindexertool.html On 7/18/15, 9:38 AM, "步青云" wrote: >I need help. I have several hundreds of GB files in hdfs and I want to >creat

Re: copying data from one collection to another collection (solr cloud 521)

2015-07-15 Thread Raja Pothuganti
g". You can create an entirely >new collection, index to it however you want then switch to using that >new collection. > >bq: Any updates to EXISTING document in the LIVE collection should NOT be >replicated to the previous week(s) snapshot(s) > >then give it a new ID ma

Re: copying data from one collection to another collection (solr cloud 521)

2015-07-13 Thread Raja Pothuganti
p to date then switch over when necessary. > >I'd really like to know this isn't an XY problem though, what's the >high-level problem you're trying to solve? > >Best, >Erick > >On Mon, Jul 13, 2015 at 12:49 PM, Raja Pothuganti > wrote: >> >> Hi,

copying data from one collection to another collection (solr cloud 521)

2015-07-13 Thread Raja Pothuganti
Hi, We are setting up a new SolrCloud environment with 5.2.1 on Ubuntu boxes. We currently ingest data into a large collection, call it LIVE. After the full ingest is done we then trigger a delta delta ingestion every 15 minutes to get the documents & data that have changed into this LIVE insta