Query on Solr Caches, OOM Errors and - how q and fq affect Solr Cache memory consumption

2014-10-11 Thread IJ
This question is related to Out of Memory Errors that I am seeing on my Solr Cloud Setup - I am running Solr 4.5.1. Here is how my setup looks: 1. Have 6 Solr Tomcat Nodes distributed across 3 Servers - i.e. 2 nodes per server 2. Each tomcat node has been allocated 2 GB RAM - XmX setting Have two

Re: solr always loading and not any response

2014-07-27 Thread IJ
I always get the "Loading" message on the Solr Admin Console if I use IE. However - the page loads perfectly fine when I use Google Chrome or Mozilla Firefox. Could you check if your problem resolves itself if you use a different browser ??? -- View this message in context: http://lucene.47206

Re: shards as subset of All Shards

2014-07-19 Thread IJ
Here is one potential design approach: 1. Create a single collection (instead of two collections). Let your schema have a "RecordType" field which can take the values of either "initial" or "follow-up" for documents that are indexed into this collection. 2. Let there be 30 shards - just like you

Re: clearing fieldValueCache in solr 4.6

2014-07-16 Thread IJ
One thing you could do is: 1. If you current index is called A1, then you can create a new index called A2 with the correct schema.xml / solrconfig.xml 2. Index your 18,000 documents into A2 afresh 3. Then delete A1 (the bad index) 4. Then quickly create an Alias with the name of A1 pointng to A2 -

Re: Solr irregularly having QTime > 50000ms, stracing solr cures the problem

2014-07-16 Thread IJ
I know u mentioned you have a single machine at play - but do you have multiple nodes on the machine that talk to one another ?? Does your problem recur when the load on the system is low ? Also faced a similar problem wherein the "5 second delay" (described in detail on my other post) kept happe

Re: Solr irregularly having QTime > 50000ms, stracing solr cures the problem

2014-07-12 Thread IJ
GUess - I had the same issues as you. Was resolved http://lucene.472066.n3.nabble.com/Slow-QTimes-5-seconds-for-Small-sized-Collections-td4143681.html was resolved by adding an explicit host mapping entry on /etc/hosts for inter node solr communication - thereby bypassing DNS Lookups. -- View

Re: Slow QTimes - 5 seconds for Small sized Collections

2014-07-02 Thread IJ
This issue was finally resolved. Adding an explicit Host - IP address mapping on /etc/host file seemed to do the trick. The one strange thing is - before the host file entry was made - we were unable to simulate the 5 second delay from the linux shell by performing a simple nslookup . In any case -

Re: Does Solr move documents between shards when the value of the shard key is updated ?

2014-07-02 Thread IJ
So - we do end up with two copies / versions of the same document (uniqueid) - one in each of the two shards - Is this a BUG or a FEATURE in Solr ? Have a follow up question - In case one were to attempt to delete the document -lets say usng the CloudSolrServer - deleteById() API - would that atte

Does Solr move documents between shards when the value of the shard key is updated ?

2014-07-01 Thread IJ
Lets say I create a Solr Collection with multiple shards (say 2 shards) and set the value of "router.field" to a field called "CompanyName". Now - we all know that during Indexing Solr would compute a hash on the value indexed into the "CompanyName" and route to an appropriate shard. Lets say I in

Re: Out of Memory when i downdload 5 Million records from sqlserver to solr

2014-07-01 Thread IJ
We faced similar problems on our side. We found it more reliable to have a mechanism to extract all data from the Database into a flat file - and then use a JAVA program to bulk index into Solr from the file via SolrJ API. -- View this message in context: http://lucene.472066.n3.nabble.com/Out-

Re: How to sort value that numeric mix alpha

2014-06-29 Thread IJ
Try indexing your data as follows: C01,C02,C03,C04,C09,C12,C23,C50 instead of C1,C2,C3,C4,C9,C12,C23,C50 and the sort order would work correctly. BTW, what you are describing as an issue is NOT unique to Solr. The same happens on regular Databases as well. Google up how database type systems

Re: While creating collection in SolrCloud can we manually select machines(nodes)

2014-06-29 Thread IJ
Yes, the Solr Collections API allows you to pass in a set of explicit nodes (subset of the complete list of nodes in your cluster) to setup your Collection. This the "createNodeSet" input parameter in the CREATE COLLECTION API - described as follows in the documentation: Allows defining the nodes

Re: Slow QTimes - 5 seconds for Small sized Collections

2014-06-27 Thread IJ
I am a colleague of the person who posted the original question. We have done some more analysis and have more information to provide. Here are the responses to Toke's questions: >> * Do they (slow performing queries) occur under heavy network load? No, they don't. This happens even when there