If you're trying to do a bulk ingest of data, I recommend committing less frequently. Don't soft commit at all until the end of the batch, and hard commit every 60 seconds.
Michael Della Bitta Senior Software Engineer o: +1 646 532 3062 appinions inc. “The Science of Influence Marketing” 18 East 41st Street New York, NY 10017 t: @appinions <https://twitter.com/Appinions> | g+: plus.google.com/appinions <https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts> w: appinions.com <http://www.appinions.com/> On Tue, Feb 3, 2015 at 12:51 AM, Nitin Solanki <nitinml...@gmail.com> wrote: > Hi Michael Della and Michael Sokolov, > > *size of tlog :-* > 56K /mnt/nitin/solr/node1/solr/wikingram_shard3_replica1/data/tlog/ > 56K /mnt/nitin/solr/node1/solr/wikingram_shard7_replica1/data/tlog/ > 56K /mnt/nitin/solr/node2/solr/wikingram_shard4_replica1/data/tlog/ > 52K /mnt/nitin/solr/node2/solr/wikingram_shard8_replica1/data/tlog/ > 52K /mnt/nitin/solr/node3/solr/wikingram_shard1_replica1/data/tlog/ > 52K /mnt/nitin/solr/node3/solr/wikingram_shard5_replica1/data/tlog/ > 56K /mnt/nitin/solr/node4/solr/wikingram_shard2_replica1/data/tlog/ > 48K /mnt/nitin/solr/node4/solr/wikingram_shard6_replica1/data/tlog/ > > *Size of logs :-* > 755M /mnt/nitin/solr/node1/logs/ > 729M /mnt/nitin/solr/node2/logs/ > 729M /mnt/nitin/solr/node3/logs/ > 729M /mnt/nitin/solr/node4/logs/ > > Which log is reducing performance? I am committing more frequent hard > commits. After 1 second , I am performing soft commit and after 15 seconds, > I am performing hard commit. I indexed 2 GB of data and you can see the > size of tlog that I pasted above. Is this tlog is good for 2GB indexed > data? Or is it high? The main question is that size of log will harm > performance of Solr? > > > > > On Mon, Feb 2, 2015 at 10:27 PM, Michael Della Bitta < > michael.della.bi...@appinions.com> wrote: > > > Good call, it could easily be the tlog Nitin is talking about. > > > > As for which definition of high, I was making assumptions as well. :) > > > > Michael Della Bitta > > > > Senior Software Engineer > > > > o: +1 646 532 3062 > > > > appinions inc. > > > > “The Science of Influence Marketing” > > > > 18 East 41st Street > > > > New York, NY 10017 > > > > t: @appinions <https://twitter.com/Appinions> | g+: > > plus.google.com/appinions > > < > > > https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts > > > > > w: appinions.com <http://www.appinions.com/> > > > > On Mon, Feb 2, 2015 at 11:51 AM, Michael Sokolov < > > msoko...@safaribooksonline.com> wrote: > > > > > I was tempted to suggest rehab -- but seriously it wasn't clear if > Nitin > > > meant the log files Michael is referring to, or the transaction log > > > (tlog). If it's the transaction log, the solution is more frequent > hard > > > commits. > > > > > > -Mike > > > > > > On 2/2/2015 11:48 AM, Michael Della Bitta wrote: > > > > > >> If you'd like to reduce the amount of lines Solr logs, you need to > edit > > >> the > > >> file example/resources/log4j.properties in Solr's home directory. > Change > > >> lines that say INFO to WARN. > > >> > > >> Michael Della Bitta > > >> > > >> Senior Software Engineer > > >> > > >> o: +1 646 532 3062 > > >> > > >> appinions inc. > > >> > > >> “The Science of Influence Marketing” > > >> > > >> 18 East 41st Street > > >> > > >> New York, NY 10017 > > >> > > >> t: @appinions <https://twitter.com/Appinions> | g+: > > >> plus.google.com/appinions > > >> <https://plus.google.com/u/0/b/112002776285509593336/ > > >> 112002776285509593336/posts> > > >> w: appinions.com <http://www.appinions.com/> > > >> > > >> On Mon, Feb 2, 2015 at 7:42 AM, Nitin Solanki <nitinml...@gmail.com> > > >> wrote: > > >> > > >> Hi, > > >>> My solr logs directory has been get high. It is seriously > > >>> problem > > >>> or It harms my solr performance in both cases indexing as well as > > >>> searching. > > >>> > > >>> > > > > > >
