Re: Cant commit on 125 GB index
Commit actions are in the jetty log. I don't have a script to pull them out in a spread-sheet-able form, but that would be useful. On 3/13/10, Frederico Azeiteiro wrote: > Yes, the http request is timing out even when using values of 10m. > > Normally the commit takes about 10s. I did an optimize (it took 6h) and it > looks good for now... > > 59m? well i didn't wait that long, i restarted the solr instance and tried > again. > > I'll try to use autocommit on a near future. > > Using autocommit how can i check how many commits are happening at the > moment, when they started to? Is there a way to control and konw what is > happening behind the scenes in "real time"? > > I'm using solr 1.4 with jetty. > > > > De: Lance Norskog [mailto:goks...@gmail.com] > Enviada: sáb 13-03-2010 23:31 > Para: solr-user@lucene.apache.org > Assunto: Re: Cant commit on 125 GB index > > > > What is timing out? The external HTTP request? Commit times are a > sawtooth and slowly increase. My record is 59 minutes, but I was doing > benchmarking. > > On Thu, Mar 11, 2010 at 1:46 AM, Frederico Azeiteiro > wrote: >> Hi, >> >> I'm having timeouts commiting on a 125 GB index with about 2200 >> docs. >> >> >> >> I'm inserting new docs every 5m and commiting after that. >> >> >> >> I would like to try the autocommit option and see if I can get better >> results. I need the docs indexed available for searching in about 10 >> minutes after the insert. >> >> >> >> I was thinking of using something like >> >> >> >> >> >> 5000 >> >> 86000 >> >> >> >> >> >> I update about 4000 docs every 15m. >> >> >> >> Can you share your thoughts on this config? >> >> Do you think this will solve my commits timeout problem? >> >> >> >> Thanks, >> >> Frederico >> >> > > > > -- > Lance Norskog > goks...@gmail.com > > > -- Lance Norskog goks...@gmail.com
RE: Cant commit on 125 GB index
Yes, the http request is timing out even when using values of 10m. Normally the commit takes about 10s. I did an optimize (it took 6h) and it looks good for now... 59m? well i didn't wait that long, i restarted the solr instance and tried again. I'll try to use autocommit on a near future. Using autocommit how can i check how many commits are happening at the moment, when they started to? Is there a way to control and konw what is happening behind the scenes in "real time"? I'm using solr 1.4 with jetty. De: Lance Norskog [mailto:goks...@gmail.com] Enviada: sáb 13-03-2010 23:31 Para: solr-user@lucene.apache.org Assunto: Re: Cant commit on 125 GB index What is timing out? The external HTTP request? Commit times are a sawtooth and slowly increase. My record is 59 minutes, but I was doing benchmarking. On Thu, Mar 11, 2010 at 1:46 AM, Frederico Azeiteiro wrote: > Hi, > > I'm having timeouts commiting on a 125 GB index with about 2200 > docs. > > > > I'm inserting new docs every 5m and commiting after that. > > > > I would like to try the autocommit option and see if I can get better > results. I need the docs indexed available for searching in about 10 > minutes after the insert. > > > > I was thinking of using something like > > > > > > 5000 > > 86000 > > > > > > I update about 4000 docs every 15m. > > > > Can you share your thoughts on this config? > > Do you think this will solve my commits timeout problem? > > > > Thanks, > > Frederico > > -- Lance Norskog goks...@gmail.com
Re: Cant commit on 125 GB index
What is timing out? The external HTTP request? Commit times are a sawtooth and slowly increase. My record is 59 minutes, but I was doing benchmarking. On Thu, Mar 11, 2010 at 1:46 AM, Frederico Azeiteiro wrote: > Hi, > > I'm having timeouts commiting on a 125 GB index with about 2200 > docs. > > > > I'm inserting new docs every 5m and commiting after that. > > > > I would like to try the autocommit option and see if I can get better > results. I need the docs indexed available for searching in about 10 > minutes after the insert. > > > > I was thinking of using something like > > > > > > 5000 > > 86000 > > > > > > I update about 4000 docs every 15m. > > > > Can you share your thoughts on this config? > > Do you think this will solve my commits timeout problem? > > > > Thanks, > > Frederico > > -- Lance Norskog goks...@gmail.com
