Re: Commit Issue in Solr 3.4

2014-02-08 Thread Roman Chyla
Thanks for the links. I think it would be worth getting more detailed info. Because it could be the performance threshold, or it could be st else /such as updated java version or st else, loosely related to ram, eg what is held in memory before the commit, what is cached, leaked custom query object

Re: Commit Issue in Solr 3.4

2014-02-08 Thread Shawn Heisey
On 2/8/2014 11:02 AM, Roman Chyla wrote: > I would be curious what the cause is. Samarth says that it worked for over > a year /and supposedly docs were being added all the time/. Did the index > grew considerably in the last period? Perhaps he could attach visualvm > while it is in the 'black hole

Re: Commit Issue in Solr 3.4

2014-02-08 Thread Roman Chyla
I would be curious what the cause is. Samarth says that it worked for over a year /and supposedly docs were being added all the time/. Did the index grew considerably in the last period? Perhaps he could attach visualvm while it is in the 'black hole' state to see what is actually going on. I don't

Re: Commit Issue in Solr 3.4

2014-02-08 Thread Shawn Heisey
On 2/8/2014 10:22 AM, Shawn Heisey wrote: > Can you share your solrconfig.xml file? I may be able to confirm a > couple of things I suspect, and depending on what's there, may be able > to offer some ideas to help a little bit. It's best if you use a file > sharing site like dropbox - the list do

Re: Commit Issue in Solr 3.4

2014-02-08 Thread Shawn Heisey
On 2/8/2014 1:40 AM, samarth s wrote: > Yes it is amazon ec2 indeed. > > To expqnd on that, > This solr deployment was working fine, handling the same load, on a 34 GB > instance on ebs storage for quite some time. To reduce the time taken by a > commit, I shifted this to a 30 GB SSD instance. It

Re: Commit Issue in Solr 3.4

2014-02-08 Thread samarth s
Yes it is amazon ec2 indeed. To expqnd on that, This solr deployment was working fine, handling the same load, on a 34 GB instance on ebs storage for quite some time. To reduce the time taken by a commit, I shifted this to a 30 GB SSD instance. It performed better in writes and commits for sure. B

Re: Commit Issue in Solr 3.4

2014-02-06 Thread Shawn Heisey
On 2/6/2014 9:56 AM, samarth s wrote: > Size of index = 260 GB > Total Docs = 100mn > Usual writing speed = 50K per hour > autoCommit-maxDocs = 400,000 > autoCommit-maxTime = 1500,000 (25 mins) > merge factor = 10 > > M/c memory = 30 GB, Xmx = 20 GB > Server - Jetty > OS - Cent OS 6 With 30GB of

Commit Issue in Solr 3.4

2014-02-06 Thread samarth s
Hi, I have been using the solr version 3.4 in a project for about more than a year. It is only now that I have started facing a weird problem of never ending back to back commit cycles. I can say this looking at the InfoStream logs, that, as soon as one commit cycle is done with another one almost