Re: Cant commit on 125 GB index
What is timing out? The external HTTP request? Commit times are a sawtooth and slowly increase. My record is 59 minutes, but I was doing benchmarking. On Thu, Mar 11, 2010 at 1:46 AM, Frederico Azeiteiro frederico.azeite...@cision.com wrote: Hi, I'm having timeouts commiting on a 125 GB index with about 2200 docs. I'm inserting new docs every 5m and commiting after that. I would like to try the autocommit option and see if I can get better results. I need the docs indexed available for searching in about 10 minutes after the insert. I was thinking of using something like autoCommit maxDocs5000/maxDocs maxTime86000/maxTime /autoCommit I update about 4000 docs every 15m. Can you share your thoughts on this config? Do you think this will solve my commits timeout problem? Thanks, Frederico -- Lance Norskog goks...@gmail.com
RE: Cant commit on 125 GB index
Yes, the http request is timing out even when using values of 10m. Normally the commit takes about 10s. I did an optimize (it took 6h) and it looks good for now... 59m? well i didn't wait that long, i restarted the solr instance and tried again. I'll try to use autocommit on a near future. Using autocommit how can i check how many commits are happening at the moment, when they started to? Is there a way to control and konw what is happening behind the scenes in real time? I'm using solr 1.4 with jetty. De: Lance Norskog [mailto:goks...@gmail.com] Enviada: sáb 13-03-2010 23:31 Para: solr-user@lucene.apache.org Assunto: Re: Cant commit on 125 GB index What is timing out? The external HTTP request? Commit times are a sawtooth and slowly increase. My record is 59 minutes, but I was doing benchmarking. On Thu, Mar 11, 2010 at 1:46 AM, Frederico Azeiteiro frederico.azeite...@cision.com wrote: Hi, I'm having timeouts commiting on a 125 GB index with about 2200 docs. I'm inserting new docs every 5m and commiting after that. I would like to try the autocommit option and see if I can get better results. I need the docs indexed available for searching in about 10 minutes after the insert. I was thinking of using something like autoCommit maxDocs5000/maxDocs maxTime86000/maxTime /autoCommit I update about 4000 docs every 15m. Can you share your thoughts on this config? Do you think this will solve my commits timeout problem? Thanks, Frederico -- Lance Norskog goks...@gmail.com
Re: Cant commit on 125 GB index
Commit actions are in the jetty log. I don't have a script to pull them out in a spread-sheet-able form, but that would be useful. On 3/13/10, Frederico Azeiteiro frederico.azeite...@cision.com wrote: Yes, the http request is timing out even when using values of 10m. Normally the commit takes about 10s. I did an optimize (it took 6h) and it looks good for now... 59m? well i didn't wait that long, i restarted the solr instance and tried again. I'll try to use autocommit on a near future. Using autocommit how can i check how many commits are happening at the moment, when they started to? Is there a way to control and konw what is happening behind the scenes in real time? I'm using solr 1.4 with jetty. De: Lance Norskog [mailto:goks...@gmail.com] Enviada: sáb 13-03-2010 23:31 Para: solr-user@lucene.apache.org Assunto: Re: Cant commit on 125 GB index What is timing out? The external HTTP request? Commit times are a sawtooth and slowly increase. My record is 59 minutes, but I was doing benchmarking. On Thu, Mar 11, 2010 at 1:46 AM, Frederico Azeiteiro frederico.azeite...@cision.com wrote: Hi, I'm having timeouts commiting on a 125 GB index with about 2200 docs. I'm inserting new docs every 5m and commiting after that. I would like to try the autocommit option and see if I can get better results. I need the docs indexed available for searching in about 10 minutes after the insert. I was thinking of using something like autoCommit maxDocs5000/maxDocs maxTime86000/maxTime /autoCommit I update about 4000 docs every 15m. Can you share your thoughts on this config? Do you think this will solve my commits timeout problem? Thanks, Frederico -- Lance Norskog goks...@gmail.com -- Lance Norskog goks...@gmail.com
Cant commit on 125 GB index
Hi, I'm having timeouts commiting on a 125 GB index with about 2200 docs. I'm inserting new docs every 5m and commiting after that. I would like to try the autocommit option and see if I can get better results. I need the docs indexed available for searching in about 10 minutes after the insert. I was thinking of using something like autoCommit maxDocs5000/maxDocs maxTime86000/maxTime /autoCommit I update about 4000 docs every 15m. Can you share your thoughts on this config? Do you think this will solve my commits timeout problem? Thanks, Frederico
