Re: Disk Usage anomoly across shards/replicas

2016-03-05 Thread Varun Thacker
Hi Robert, Within the shard directory there should be multiple directories - "tlog" "index." . Do you see multiple "index.*" directories in there for the shard which has more data on disk? On Sat, Mar 5, 2016 at 6:39 PM, Robert Brown wrote: > Hi, > > I have an index with

High Cpu sys usage

2016-03-05 Thread YouPeng Yang
Hi We are using Solr Cloud 4.6 in our production for searching service since 2 years ago.And now it has 700GB in one cluster which is comprised of 3 machines with ssd. At beginning ,everything go well,while more and more business services interfered with our searching service .And a problem

Re: Field exclusion from fl and hl.fl

2016-03-05 Thread William Bell
it used to support fl=*,-field Does that not work now? On Sat, Mar 5, 2016 at 7:37 PM, Zheng Lin Edwin Yeo wrote: > I have yet to find any workaround so far.Still have to list out all the > remaining fields one by one. > > Does anyone else has any suggestions? > >

Re: Field exclusion from fl and hl.fl

2016-03-05 Thread Zheng Lin Edwin Yeo
I have yet to find any workaround so far.Still have to list out all the remaining fields one by one. Does anyone else has any suggestions? Regards, Edwin On 18 February 2016 at 17:07, Anil wrote: > I am looking for the same. please do let me know just in case you find >

Re: How to use geospatial search to find the locations within polygon

2016-03-05 Thread Jack Krupansky
The doc does indeed say "JTS... It's a JAR file that you need to put on Solr's classpath (but not via the standard solrconfig.xml mechanisms)", but that is a little vague and nonspecific. It should probably be a labeled section in the doc, like "Configuring JTS for Polygon Search", and have the

Re: How to use geospatial search to find the locations within polygon

2016-03-05 Thread david.w.smi...@gmail.com
A Java NoClassDefFoundError of something in com.vividsolutions.jts means you don't have JTS on your classpath. You should put the JTS jar file in server/lib/. You can download it from maven-central. Here's a search for JTS with the 1.14 version:

Re: Disk Usage anomoly across shards/replicas

2016-03-05 Thread Robert Brown
Thanks Shawn, I'm just about to remove that node and rebuild it, at least there won't be any actual downtime. On 05/03/16 14:44, Shawn Heisey wrote: On 3/5/2016 6:09 AM, Robert Brown wrote: I have an index with 65m docs spread across 2 shards, each with 1 replica. The replica1 of shard2

Re: Disk Usage anomoly across shards/replicas

2016-03-05 Thread Shawn Heisey
On 3/5/2016 6:09 AM, Robert Brown wrote: > I have an index with 65m docs spread across 2 shards, each with 1 > replica. > > The replica1 of shard2 is using up nearly double the amount of disk > space as the other shards/replicas. I *very* occasionally see some of the shards in my non-SolrCloud

Re: Question about Solr logs

2016-03-05 Thread Shawn Heisey
On 3/5/2016 5:33 AM, Steven White wrote: > Thanks Shawn. > > To make sure I get this right, I see two methods on UpdateResponse class, > is getElapsedTime the client time and getQTime Solr's time? If so, than > getElapsedTime is how long my call was blocked, right? And getQTime will > have the

Re: How to use geospatial search to find the locations within polygon

2016-03-05 Thread david.w.smi...@gmail.com
Yes you're right Jack. The ref guide page on this needs more. On Fri, Mar 4, 2016 at 11:36 AM Jack Krupansky wrote: > It would be nice for the doc to say that - describe when IsWithin is and > isn't appropriate. And give some examples as well for people to copy/mimic.

Re: Spatial Search on Postal Code

2016-03-05 Thread david.w.smi...@gmail.com
Another path to consider is doing this point-in-zipcode-poly lookup at index time and enriching the document with a zipcode field (possibly multi-valued if there is doubt). On Sat, Mar 5, 2016 at 4:05 AM steve shepard wrote: > re: Postal Codes and polygons. I've heard of

Re: Disk Usage anomoly across shards/replicas

2016-03-05 Thread Robert Brown
Nope, we never run optimise. Would there be some tell-tale files in the index dir to indicate if someone else had ran an optimise? On 05/03/16 13:11, Binoy Dalal wrote: Have you executed an optimize across that particular shard? On Sat, 5 Mar 2016, 18:39 Robert Brown,

Re: Disk Usage anomoly across shards/replicas

2016-03-05 Thread Binoy Dalal
Have you executed an optimize across that particular shard? On Sat, 5 Mar 2016, 18:39 Robert Brown, wrote: > Hi, > > I have an index with 65m docs spread across 2 shards, each with 1 replica. > > The replica1 of shard2 is using up nearly double the amount of disk > space

Disk Usage anomoly across shards/replicas

2016-03-05 Thread Robert Brown
Hi, I have an index with 65m docs spread across 2 shards, each with 1 replica. The replica1 of shard2 is using up nearly double the amount of disk space as the other shards/replicas. Could there be a reason/fix for this? /home/s123/solr/data/de_shard1_replica1 = 72G numDocs:34,786,026

Re: Question about Solr logs

2016-03-05 Thread Steven White
Thanks Shawn. To make sure I get this right, I see two methods on UpdateResponse class, is getElapsedTime the client time and getQTime Solr's time? If so, than getElapsedTime is how long my call was blocked, right? And getQTime will have the value of 1174 (per the log of my example), right?

Re: What is the best way to index 15 million documents of total size 425 GB?

2016-03-05 Thread Aneesh Mon N
Thanks Toke, Charlie and Evert, I will reconsider using a light weight program to read and index data from database to Solr..! Hi Walter, no we have not yet started with atomic updates; just the plan is in progress. Current issue is with respect to indexing the document as such in bulk. On Fri,

RE: Spatial Search on Postal Code

2016-03-05 Thread steve shepard
re: Postal Codes and polygons. I've heard of basic techniques that use Commerce Department (or was it Census within Commerce??) that give the basic points, but the real run is deciding what the "center" of that polygon is. There is likely a commercial solution available, and certainly you can