Thanks Erick. Couple of Questions : Our transaction logs are huge as we have disabled auto commit. The biggest one is 6.1 GB.
*567M* autosuggest/data/tlog *22M* avmediaCore/data/tlog *388M* booksCore/data/tlog *4.9G * books/data/tlog *6.1G * mp3-downloads/data/tlog ( 150 % of index Size) 1*.5G* next-5/data/tlog 690M queries/data/tlog ( 25 % of Index Size ) 207M queryProduct/data/tlog (100 % of Index Size) Btw, I am surprised by the size of transaction log, because that is a significant amount of index size itself 2.6G autosuggest/data/index 992M avmediaCore/data/index 12G booksCore/data/index 4.2G mp3-downloads-new/data/index 45G next-5/data/index 2.9G queries/data/index *220M* queryProduct/data/index We use DIH and have turned off the Auto commit because we have to sometimes build index from Scratch (clean=true) and we not want to Our master server sees a lot of restarts, sometimes 2-3 times a day. It polls other Data Sources for updates which are quite a few. Master maintains a version of last committed version and can handle uncommitted changes. Given the frequent restarts, We can't really afford a huge start up at this point. In the worst case, does Solr allow for disabling transactional log ? Here is our Index Config <indexConfig> <!-- Values here affect all index writers and act as a default unless overridden. --> <useCompoundFile>false</useCompoundFile> <mergeFactor>10</mergeFactor> <ramBufferSizeMB>32</ramBufferSizeMB> <maxMergeDocs>2147483647</maxMergeDocs> <maxFieldLength>50000</maxFieldLength> <writeLockTimeout>1000</writeLockTimeout> <commitLockTimeout>10000</commitLockTimeout> <lockType>single</lockType> <!-- options specific to the main on-disk lucene index --> <useCompoundFile>false</useCompoundFile> <ramBufferSizeMB>32</ramBufferSizeMB> <mergeFactor>5</mergeFactor> <!-- Deprecated --> <!--<maxBufferedDocs>1000</maxBufferedDocs>--> <maxMergeDocs>2147483647</maxMergeDocs> <maxFieldLength>50000</maxFieldLength> <unlockOnStartup>false</unlockOnStartup> <deletionPolicy class="solr.SolrDeletionPolicy"> <!-- The number of commit points to be kept --> <str name="maxCommitsToKeep">5</str> <!-- The number of optimized commit points to be kept --> <str name="maxOptimizedCommitsToKeep">0</str> <str name="maxCommitAge">2HOUR</str> </deletionPolicy> </indexConfig> Thanks & Regards Umesh Prasad On Wed, Apr 17, 2013 at 4:57 PM, Erick Erickson <erickerick...@gmail.com>wrote: > How big are you transaction logs? They can be replayed on startup. > They are truncated and a new one started when you do a hard commit > (openSearcher true or false doesn't matter). > > So a quick test of this theory would be to just stop your indexing > process, issue a hard commit on all your cores and _then_ try to > restart. If it comes up immediately, you've identified your problem. > > Best > Erick > > On Tue, Apr 16, 2013 at 8:33 AM, Umesh Prasad <umesh.i...@gmail.com> > wrote: > > Hi, > > We are migrating to Solr 4.2 from Solr 3.6 and Solr 4.2 is throwing > > Exception on Restart. What is More, it take a hell lot of Time ( More > than > > one hour to get Up and Running) > > > > > > THE exception After Restart ... > > ========================= > > "Apr 16, 2013 4:47:31 PM org.apache.solr.update.UpdateLog$RecentUpdates > > update > > WARNING: Unexpected log entry or corrupt log. Entry=11 > > java.lang.ClassCastException: java.lang.Long cannot be cast to > > java.util.List > > at > > org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:929) > > at > > > org.apache.solr.update.UpdateLog$RecentUpdates.access$000(UpdateLog.java:863) > > at > > org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1014) > > at org.apache.solr.update.UpdateLog.init(UpdateLog.java:253) > > at > > org.apache.solr.update.UpdateHandler.initLog(UpdateHandler.java:82) > > at > > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137) > > at > > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:123) > > at > > > org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:95) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > Method) > > at > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > > at > org.apache.solr.core.SolrCore.createInstance(SolrCore.java:525) > > at > > org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:596) > > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:806) > > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:618) > > at > > > org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1021) > > at > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051) > > at > org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634) > > at > org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629) > > at > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > > at > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > Apr 16, 2013 4:47:31 PM org.apache.solr.update.UpdateLog$RecentUpdates > > update > > WARNING: Unexpected log entry or corrupt log. Entry=8120?785879438123 > > java.lang.ClassCastException: java.lang.String cannot be cast to > > java.util.List > > at > > org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:929) > > at > > > org.apache.solr.update.UpdateLog$RecentUpdates.access$000(UpdateLog.java:863) > > at > > org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1014) > > at org.apache.solr.update.UpdateLog.init(UpdateLog.java:253) > > at > > org.apache.solr.update.UpdateHandler.initLog(UpdateHandler.java:82) > > at > > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137) > > at > > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:123) > > at > > > org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:95) > > > > ========================= > > > > And Once Restarted, I start getting replication errors > > > > > > Apr 16, 2013 5:20:30 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > SEVERE: Master at: http://localhost:25280/solr/accessories is not > > available. Index fetch failed. Exception: > > org.apache.solr.client.solrj.SolrServerException: IOException occured > when > > talking to server at: http://localhost:25280/solr/accessories > > Apr 16, 2013 5:20:30 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > SEVERE: Master at: http://localhost:25280/solr/newQueries is not > available. > > Index fetch failed. Exception: > > org.apache.solr.client.solrj.SolrServerException: IOException occured > when > > talking to server at: http://localhost:25280/solr/newQueries > > Apr 16, 2013 5:21:00 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > SEVERE: Master at: http://localhost:25280/solr/phcare is not available. > > Index fetch failed. Exception: > > org.apache.solr.client.solrj.SolrServerException: Timeout occured while > > waiting response from server at: http://localhost:25280/solr/phcare > > Apr 16, 2013 5:21:00 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > SEVERE: Master at: http://localhost:25280/solr/audioplayersCore is not > > available. Index fetch failed. Exception: > > org.apache.solr.client.solrj.SolrServerException: Timeout occured while > > waiting response from server at: > > http://localhost:25280/solr/audioplayersCore > > Apr 16, 2013 5:21:24 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > SEVERE: Master at: http://localhost:25280/solr/cameras is not available. > > Index fetch failed. Exception: > > org.apache.solr.client.solrj.SolrServerException: Timeout occured while > > waiting response from server at: http://localhost:25280/solr/cameras > > > > > > Before Restart : Server was running Incremental Indexing ( Triggered by > a > > Cron). The cron triggers every 5 mins for each of about 40 Cores. This > was > > the same with Solr 3.5 also. But we never faced any issues. > > > > > > > > -- > > --- > > Thanks & Regards > > Umesh Prasad > -- --- Thanks & Regards Umesh Prasad