Just to pile on, transaction logs do use up some memory, but it does NOT store the whole document in memory, docs are flushed to the tlog on disk. What is kept in memory is some basic doc info (unique id?) and a pointer to that doc in memory, so not much really unless you're keeping a boatload of docs.
But hard commits with openSearcher=false won't do much of anything. They don't open new searcher, invalidate caches etc. What that does do is close the current segment and start a new one. But that segment isn't used until the next time a searcher is opened. FWIW, Erick On Tue, Jun 11, 2013 at 8:39 PM, Chris Hostetter <hossman_luc...@fucit.org> wrote: > > : > However, we are wondering how to best setup autoCommit/autoSoftCommit on > : > masters to preserve the old behavior. It seems that setting autoCommit to > : > 1hr (openSearcher=true) without any autoSoftCommit preserves our previous > : > setup - is this correct? Wil the transaction log make masters use much > more > : > heap due to 1hr periods between commits? This can be a problem for us > : > because we put many master cores on one solr JVM > : > : If you want to completely preserve your previous setup, then you've got it > : correct. Depending on how much you index over the course of that hour, you > : might want to go a different way. > > if you want to *exactly* recreate teh behavior you had before, use the > same autoCommit settings you had before, and don't add any updateLog > config. > > however: if the only reason you were using 1 hour autocommits was the > minimize the searcher re-opening on slaves to improve query performance > via caching, then i would suggest that you switch to using autocommit more > frequently, and instead change your slave polling interval to only be once > an hour -- that recomendation is independent of which version of solr you > use, it will just previde you better durability of updates regardless of > wether you use the update log. > > If you want to take advantage of updateLog features (ie: doing atomic > updates or optimistic concurrency updates against your master) or want > improved durability w/o needing to block on every doc update to wait for a > hard commit, then enable updateLog, set a reasonable autocommit -- and > still continue to use that long polling interval on your slaves. > > concerns about tradeoffs between updateLog size, autoCommit, > autoSoftCOmmit vs openSearcher=true & caches are really only a big deal in > a SolrCloud type setup -- in classic replication the snappull frequency > acts as a mitigator between the former and the later. > > > > -Hoss
