Hi Furkan, Just curious what was the index rate that you were able to achieve? Regards,
Hien On Thursday, December 5, 2013 3:06 PM, Furkan KAMACI <furkankam...@gmail.com> wrote: Hi; Erick and Shawn have explained that we need more information about your infrastructure. I should add that: I had test data at my SolrCloud nearly as much as yours and I did not have any problems except for when indexing at a huge index rate and it can be solved with turning. You should optimize your parameters according to your system. So you should give use more information about your system. Thanks; Furkan KAMACI 4 Aralık 2013 Çarşamba tarihinde Shawn Heisey <s...@elyograg.org> adlı kullanıcı şöyle yazdı: > On 12/4/2013 6:31 AM, kumar wrote: >> I am having almost 5 to 6 crores of indexed documents in solr. And when i am >> going to change anything in the configuration file solr server is going >> down. > > If you mean crore and not core, then you are talking about 50 to 60 > million documents. That's a lot. Solr is perfectly capable of handling > that many documents, but you do need to have very good hardware. > > Even if they are small, your index is likely to be many gigabytes in > size. If the documents are large, that might be measured in terabytes. > Large indexes require a lot of memory for good performance. This will > be discussed in more detail below. > >> As a new user to solr i can't able to find the exact reason for going server >> down. >> >> I am using cache's in the following way : >> >> <filterCache class="solr.FastLRUCache" >> size="16384" >> initialSize="4096" >> autowarmCount="4096"/> >> <queryResultCache class="solr.FastLRUCache" >> size="16384" >> initialSize="4096" >> autowarmCount="1024"/> >> >> and i am not using any documentCache, fieldValueCahe's > > As Erick said, these cache sizes are HUGE. In particular, your > autowarmCount values are extremely high. > >> Whether this can lead any performance issue means going server down. > > Another thing that Erick pointed out is that you haven't really told us > what's happening. When you say that the server goes down, what EXACTLY > do you mean? > >> And i am seeing logging in the server it is showing exception in the >> following way >> >> >> Servlet.service() for servlet [default] in context with path [/solr] threw >> exception [java.lang.IllegalStateException: Cannot call sendError() after >> the response has been committed] with root cause > > This message comes from your servlet container, not Solr. You're > probably using Tomcat, not the included Jetty. There is some indirect > evidence that this can be fixed by increasing the servlet container's > setting for the maximum number of request parameters. > > http://forums.adobe.com/message/4590864 > > Here's what I can say without further information: > > You're likely having performance issues. One potential problem is your > insanely high autowarmCount values. Your cache configuration tells Solr > that every time you have a soft commit or a hard commit with > openSearcher=true, you're going to execute up to 1024 queries and up to > 4096 filters from the old caches, in order to warm the new caches. Even > if you have an optimal setup, this takes a lot of time. I suspect that > you don't have an optimal setup. > > Another potential problem is that you don't have enough memory for the > size of your index. A number of potential performance problems are > discussed on this wiki page: > > http://wiki.apache.org/solr/SolrPerformanceProblems > > A lot more details are required. Here's some things that will be > helpful, and more is always better: > > * Exact symptoms. > * Excerpts from the Solr logfile that include entire stacktraces. > * Operating system and version. > * Total server index size on disk. > * Total machine memory. > * Java heap size for your servlet container. > * Which servlet container you are using to run Solr. > * Solr version. > * Server hardware details. > > Thanks, > Shawn > >
